Skip to content

Commit a082e4a

Browse files
slavapestovslavapestov
authored
Merge pull request #214 from rxwei/main-integration-rdar89791117
[Integration] main (39c54d3) -> swift/main
2 parents 8144b3a + 39c54d3 commit a082e4a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+2096
-916
lines changed

‎CMakeLists.txt

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
2+
cmake_minimum_required(VERSION 3.18)
3+
project(SwiftExperimentalStringProcessing
4+
LANGUAGES Swift)
5+
6+
if(CMAKE_SYSTEM_NAME STREQUAL Windows OR CMAKE_SYSTEM_NAME STREQUAL Darwin)
7+
option(BUILD_SHARED_LIBS "Build shared libraries by default" YES)
8+
endif()
9+
10+
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
11+
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
12+
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
13+
set(CMAKE_Swift_MODULE_DIRECTORY ${CMAKE_BINARY_DIR}/swift)
14+
15+
find_package(ArgumentParser CONFIG)
16+
17+
add_subdirectory(Sources)

‎Documentation/Evolution/RegexSyntax.md

Lines changed: 845 additions & 0 deletions
Large diffs are not rendered by default.

‎README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,65 @@ See [Declarative String Processing Overview][decl-string]
99
## Requirements
1010

1111
- [Swift Trunk Development Snapshot](https://www.swift.org/download/#snapshots) DEVELOPMENT-SNAPSHOT-2022-02-03 or later.
12+
13+
## Integration with Swift
14+
15+
`_MatchingEngine`, `_CUnicode` and `_StringProcessing` are specially integrated modules that are built as part of apple/swift.
16+
17+
Specifically, `_MatchingEngine` contains the parser for regular expression literals and is built both as part of the compiler and as a core library. `_CUnicode` and `_StringProcessing` are built together as a core library named `_StringProcessing`.
18+
19+
| Module | Swift toolchain component |
20+
| ------------------- | ------------------------------------------------------------------------------------ |
21+
| `_MatchingEngine` | `SwiftCompilerSources/Sources/ExperimentalRegex` and `stdlib/public/_MatchingEngine` |
22+
| `_CUnicode` | `stdlib/public/_StringProcessing` |
23+
| `_StringProcessing` | `stdlib/public/_StringProcessing` |
24+
25+
### Branching scheme
26+
27+
#### Development branch
28+
29+
The `main` branch is the branch for day-to-day development. Generally, you should create PRs against this branch.
30+
31+
#### Swift integration branches
32+
33+
Branches whose name starts with `swift/` are Swift integration branches similar to those in [apple/llvm-project](https://github.com/apple/llvm-project). For each branch, dropping the `swift/` prefix is the corresponding branch in [apple/swift](https://github.com/apple/swift).
34+
35+
| apple/swift branch | apple/swift-experimental-string-processing branch |
36+
| ------------------- | ----------------------------------------------------- |
37+
| main | swift/main |
38+
| release/5.7 | swift/release/5.7 |
39+
| ... | swift/... |
40+
41+
A pair of corresponding branches are expected to build successfully together and pass all tests.
42+
43+
### Integration workflow
44+
45+
To integrate the latest changes in apple/swift-experimental-string-processing to apple/swift, carefully follow the workflow:
46+
47+
- Create pull requests.
48+
- Create a pull request in apple/swift-experimental-string-processing from `main` to `swift/main`, e.g. "[Integration] main -> swift/main".
49+
- If apple/swift needs to be modified to work with the latest `main` in apple/swift-experimental-string-processing, create a pull request in apple/swift.
50+
- Trigger CI.
51+
- In the apple/swift-experimental-string-processing pull request, trigger CI using the following command (replacing `<PR NUMBER>` with the apple/swift pull request number, if any):
52+
```
53+
apple/swift#<PR NUMBER> # use this line only if there is an corresponding apple/swift PR
54+
@swift-ci please test
55+
```
56+
- In the apple/swift pull request (if any), trigger CI using the following command (replacing `<PR NUMBER>` with the apple/swift-experimental-string-processing pull request number):
57+
```
58+
apple/swift-experimental-string-processing#<PR NUMBER>
59+
@swift-ci please test
60+
```
61+
- Merge when approved.
62+
- Merge the pull request in apple/swift-experimental-string-processing as a **merge commit**.
63+
- Merge the pull request in apple/swift (if any).
64+
65+
### Development notes
66+
67+
Compiler integration can be tricky. Use special caution when developing `_MatchingEngine`, `_CUnicode` and `_StringProcessing` modules.
68+
69+
- Do not change the names of these modules without due approval from compiler and infrastructure teams.
70+
- Do not modify the existing ABI (e.g. C API, serialization format) between the regular expression parser and the Swift compiler unless absolutely necessary.
71+
- Always minimize the number of lockstep integrations, i.e. when apple/swift-experimental-string-processing and apple/swift have to change together. Whenever possible, introduce new API first, migrate Swift compiler onto it, and then deprecate old API. Use versioning if helpful.
72+
- In `_StringProcessing`, do not write fully qualified references to symbols in `_CUnicode`, and always wrap `import _CUnicode` in a `#if canImport(_CUnicode)`. This is because `_CUnicode` is built as part of `_StringProcessing` with CMake.
73+
- In `_MatchingEngine`, do not write fully qualified references to `_MatchingEngine` itself. This is because `_MatchingEngine` is built as `ExperimentalRegex` in `SwiftCompilerSources/` with CMake.

‎Sources/CMakeLists.txt

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
add_subdirectory(_Unicode)
3+
add_subdirectory(_MatchingEngine)
4+
add_subdirectory(_StringProcessing)
5+
add_subdirectory(Prototypes)
6+
add_subdirectory(VariadicsGenerator)

‎Sources/Prototypes/CMakeLists.txt

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
2+
add_library(Prototypes
3+
Combinators/Combinators.swift
4+
PEG/PEG.swift
5+
PEG/PEGCode.swift
6+
PEG/PEGCompile.swift
7+
PEG/PEGCore.swift
8+
PEG/PEGInterpreter.swift
9+
PEG/PEGTranspile.swift
10+
PEG/PEGVM.swift
11+
PEG/PEGVMExecute.swift
12+
PEG/Printing.swift
13+
PTCaRet/Interpreter.swift
14+
PTCaRet/PTCaRet.swift
15+
TourOfTypes/CharacterClass.swift
16+
TourOfTypes/Literal.swift)
17+
target_link_libraries(Prototypes PUBLIC
18+
_MatchingEngine)

‎Sources/Prototypes/PEG/PEGCode.swift

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
//
1010
//===----------------------------------------------------------------------===//
1111

12-
import _StringProcessing
12+
@testable import _StringProcessing
1313

1414
extension PEG.VM {
1515
struct Code {

‎Sources/Prototypes/PEG/PEGCompile.swift

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
//
1010
//===----------------------------------------------------------------------===//
1111

12-
import _StringProcessing
12+
@testable import _StringProcessing
1313

1414
extension PEG.VM {
1515
typealias InIndex = Input.Index

‎Sources/Prototypes/PEG/PEGCore.swift

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
//
1010
//===----------------------------------------------------------------------===//
1111

12-
import _StringProcessing
12+
@testable import _StringProcessing
1313
let emitComments = true
1414

1515
struct PEGCore<

‎Sources/Prototypes/PEG/PEGTranspile.swift

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,7 @@
99
//
1010
//===----------------------------------------------------------------------===//
1111

12-
import _MatchingEngine
13-
import _StringProcessing
12+
@testable import _StringProcessing
1413

1514
extension PEG.VM where Input == String {
1615
typealias MEProg = MEProgram<String>

‎Sources/Prototypes/PEG/PEGVM.swift

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@
99
//
1010
//===----------------------------------------------------------------------===//
1111

12-
import _StringProcessing
12+
13+
@testable import _StringProcessing
1314

1415
extension PEG {
1516

‎Sources/Prototypes/PEG/Printing.swift

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
//
1010
//===----------------------------------------------------------------------===//
1111

12-
import _StringProcessing
12+
@testable import _StringProcessing
1313

1414
extension PEGCore.Instruction: InstructionProtocol {
1515
var operandPC: InstructionAddress? { self.pc }

‎Sources/VariadicsGenerator/CMakeLists.txt

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
2+
add_executable(VariadicsGenerator
3+
VariadicsGenerator.swift)
4+
target_compile_options(VariadicsGenerator PRIVATE
5+
-parse-as-library)
6+
target_link_libraries(VariadicsGenerator PUBLIC
7+
ArgumentParser)

‎Sources/_MatchingEngine/CMakeLists.txt

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
2+
add_library(_MatchingEngine
3+
Engine/Backtracking.swift
4+
Engine/Builder.swift
5+
Engine/Capture.swift
6+
Engine/Consume.swift
7+
Engine/Engine.swift
8+
Engine/InstPayload.swift
9+
Engine/Instruction.swift
10+
Engine/Processor.swift
11+
Engine/Program.swift
12+
Engine/Registers.swift
13+
Engine/Tracing.swift
14+
Regex/AST/AST.swift
15+
Regex/AST/ASTAction.swift
16+
Regex/AST/ASTProtocols.swift
17+
Regex/AST/Atom.swift
18+
Regex/AST/Conditional.swift
19+
Regex/AST/CustomCharClass.swift
20+
Regex/AST/Group.swift
21+
Regex/AST/MatchingOptions.swift
22+
Regex/AST/Quantification.swift
23+
Regex/Parse/CaptureStructure.swift
24+
Regex/Parse/CharacterPropertyClassification.swift
25+
Regex/Parse/Diagnostics.swift
26+
Regex/Parse/LexicalAnalysis.swift
27+
Regex/Parse/Mocking.swift
28+
Regex/Parse/Parse.swift
29+
Regex/Parse/Source.swift
30+
Regex/Parse/SourceLocation.swift
31+
Regex/Parse/SyntaxOptions.swift
32+
Regex/Printing/DumpAST.swift
33+
Regex/Printing/PrettyPrinter.swift
34+
Regex/Printing/PrintAsCanonical.swift
35+
Regex/Printing/PrintAsPattern.swift
36+
Regex/Printing/RenderRanges.swift
37+
Utility/AllScalars.swift
38+
Utility/Formatting.swift
39+
Utility/Misc.swift
40+
Utility/MissingUnicode.swift
41+
Utility/Protocols.swift
42+
Utility/TypeConstruction.swift
43+
Utility/TypedIndex.swift
44+
Utility/TypedInt.swift)
45+
target_compile_options(_MatchingEngine PRIVATE
46+
-enable-library-evolution)

‎Sources/_MatchingEngine/Regex/Parse/CharacterPropertyClassification.swift

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -381,7 +381,7 @@ extension Source {
381381
return .generalCategory(cat)
382382
}
383383
if let script = classifyScriptProperty(value) {
384-
return .script(script)
384+
return .scriptExtension(script)
385385
}
386386
if let posix = classifyPOSIX(value) {
387387
return .posix(posix)

0 commit comments

Comments
 (0)