Skip to content

Document specially integrated modules and integration process. #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 2, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,65 @@ See [Declarative String Processing Overview][decl-string]
## Requirements

- [Swift Trunk Development Snapshot](https://www.swift.org/download/#snapshots) DEVELOPMENT-SNAPSHOT-2022-02-03 or later.

## Integration with Swift

`_MatchingEngine`, `_CUnicode` and `_StringProcessing` are specially integrated modules that are built as part of apple/swift.

Specifically, `_MatchingEngine` contains the parser for regular expression literals and is built both as part of the compiler and as a core library. `_CUnicode` and `_StringProcessing` are built together as a core library named `_StringProcessing`.

| Module | Swift toolchain component |
| ------------------- | ------------------------------------------------------------------------------------ |
| `_MatchingEngine` | `SwiftCompilerSources/Sources/ExperimentalRegex` and `stdlib/public/_MatchingEngine` |
| `_CUnicode` | `stdlib/public/_StringProcessing` |
| `_StringProcessing` | `stdlib/public/_StringProcessing` |

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good for an initial integration doc, but I do think we'll want some CMake, perhaps in the Swift repo, that would add or remove certain sub-folders for the stdlib, like the engine itself.

### Branching scheme

#### Development branch

The `main` branch is the branch for day-to-day development. Generally, you should create PRs against this branch.

#### Swift integration branches

Branches whose name starts with `swift/` are Swift integration branches similar to those in [apple/llvm-project](https://github.com/apple/llvm-project). For each branch, dropping the `swift/` prefix is the corresponding branch in [apple/swift](https://github.com/apple/swift).

| apple/swift branch | apple/swift-experimental-string-processing branch |
| ------------------- | ----------------------------------------------------- |
| main | swift/main |
| release/5.7 | swift/release/5.7 |
| ... | swift/... |

A pair of corresponding branches are expected to build successfully together and pass all tests.

### Integration workflow

To integrate the latest changes in apple/swift-experimental-string-processing to apple/swift, carefully follow the workflow:

- Create pull requests.
- Create a pull request in apple/swift-experimental-string-processing from `main` to `swift/main`, e.g. "[Integration] main -> swift/main".
- If apple/swift needs to be modified to work with the latest `main` in apple/swift-experimental-string-processing, create a pull request in apple/swift.
- Trigger CI.
- In the apple/swift-experimental-string-processing pull request, trigger CI using the following command (replacing `<PR NUMBER>` with the apple/swift pull request number, if any):
```
apple/swift#<PR NUMBER> # use this line only if there is an corresponding apple/swift PR
@swift-ci please test
```
- In the apple/swift pull request (if any), trigger CI using the following command (replacing `<PR NUMBER>` with the apple/swift-experimental-string-processing pull request number):
```
apple/swift-experimental-string-processing#<PR NUMBER>
@swift-ci please test
```
- Merge when approved.
- Merge the pull request in apple/swift-experimental-string-processing as a **merge commit**.
- Merge the pull request in apple/swift (if any).

### Development notes

Compiler integration can be tricky. Use special caution when developing `_MatchingEngine`, `_CUnicode` and `_StringProcessing` modules.

- Do not change the names of these modules without due approval from compiler and infrastructure teams.
- Do not modify the existing ABI (e.g. C API, serialization format) between the regular expression parser and the Swift compiler unless absolutely necessary.
- Always minimize the number of lockstep integrations, i.e. when apple/swift-experimental-string-processing and apple/swift have to change together. Whenever possible, introduce new API first, migrate Swift compiler onto it, and then deprecate old API. Use versioning if helpful.
- In `_StringProcessing`, do not write fully qualified references to symbols in `_CUnicode`, and always wrap `import _CUnicode` in a `#if canImport(_CUnicode)`. This is because `_CUnicode` is built as part of `_StringProcessing` with CMake.
- In `_MatchingEngine`, do not write fully qualified references to `_MatchingEngine` itself. This is because `_MatchingEngine` is built as `ExperimentalRegex` in `SwiftCompilerSources/` with CMake.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And related, we should rename it to RegexParse or something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but I think it is better for this document to describe the current state. We can patch it once the name has changed.