Update RegexSyntax.md

Author: hamishknight

Documentation/Evolution/RegexSyntax.md

@@ -136,11 +136,11 @@ PCRE2 defines explicitly spelled out versions of the syntax, e.g `(*negative_loo
 
 #### Atomic groups
 
-**TODO: Add description**
+An atomic group e.g `(?>...)` specifies that its contents should not be re-evaluated for backtracking. This the same semantics as a possessive quantifier, but applies more generally to any regex pattern.
 
 #### Script runs
 
-**TODO: Add description**
+A script run e.g `(*script_run:...)` specifies that the contents must match against a sequence of characters from the same Unicode script, e.g Latin or Greek.
 
 #### Balancing groups
 
@@ -203,15 +203,16 @@ Anchor -> '^' | '$' | '\A' | '\b' | '\B' | '\G' | '\y' | '\Y' | '\z' | '\Z'
 
 Anchors match against a certain position in the input rather than on a particular character of the input.
 
-#### Start and end of line
-
-`^` matches against the start of a line of input, `$` matches against the end of a line.
-
-#### Word boundaries
-
-`\b` matches a word boundary, which is [...]
-
-**TODO: List these out with a very brief description of what they mean.**
+- `^`: Matches at the start of a line.
+- `$`: Matches at the end of a line.
+- `\A`: Matches at the very start of the input string.
+- `\Z`: Matches at the very end of the input string, in addition to before a newline at the very end of the input string.
+- `\z`: Like `\Z`, but only matches at the very end of the input string.
+- `\G`: Like `\A`, but also matches against the start position of where matching resumes in global matching mode (e.g `\Gab` matches twice in `abab`, `\Aab` would only match once).
+- `\b` matches a boundary between a word character and a non-word character. The definitions of which vary depending on matching engine.
+- `\B` matches a non-word-boundary. 
+- `\y` matches a text segment boundary, the definition of which varies based on the `y{w}` and `y{g}` matching option.
+- `\Y` matches a non-text-segment-boundary.
 
 ### Unicode scalars
 
@@ -224,8 +225,8 @@ UniScalar -> '\u{' HexDigit{1...} '}'
            | '\o{' OctalDigit{1...} '}'
            | '\' OctalDigit{1...3}
 
-HexDigit    -> [0-9a-zA-Z]
-OctalDigit  -> [0-7]
+HexDigit   -> [0-9a-zA-Z]
+OctalDigit -> [0-7]
 ```
 
 These sequences define a unicode scalar value to be matched against.
@@ -311,12 +312,16 @@ Allows a specific Unicode scalar to be specified by name or code point.
 ### Trivia
 
 ```
-Trivia     -> Comment | Whitespace
-Comment    -> '(?#' (!')')* ')'
+Trivia  -> Comment | Whitespace
+Comment -> InlineComment | EndOfLineComment
+
+InlineComment    -> '(?#' (!')')* ')'
+EndOfLineComment -> '#' .*$
+
 Whitespace -> \s+
 ```
 
-Trivia is consumed by the regular expression parser, but has no semantic meaning. Non-semantic whitespace may only occur when the either of the extended syntax matching options `(?x)`, `(?xx)` are enabled.
+Trivia is consumed by the regular expression parser, but has no semantic meaning. This includes inline PCRE-style comments e.g `(?#comment)`. It also includes non-semantic whitespace and end-of-line comments which may only occur when either of the extended syntax matching options `(?x)`, `(?xx)` are enabled.
 
 **TODO: Differences between PCRE extended syntax and our syntax**
 
@@ -406,14 +411,10 @@ This is syntax specific to PCRE, and is used to control backtracking behavior. A
 - `ACCEPT`: Causes matching to terminate immediately as a successful match. If used within a subpattern, only that level of recursion is terminated.
 - `FAIL`, `F`: Causes matching to fail, forcing backtracking to occur if possible.
 - `MARK`: Assigns a label to the current matching path, which is passed back to the caller on success. Subsequent `MARK` directives overwrite the label assigned, so only the last is passed back.
-- `COMMIT`: Prevents backtracking from reaching any point prior to this directive.
-
-
-**TODO:**
-
-- `PRUNE`: 
-- `SKIP`:
-- `THEN`:
+- `COMMIT`: Prevents backtracking from reaching any point prior to this directive, causing the match to fail. This does not allow advancing the input to try a different starting match position.
+- `PRUNE`: Similar to `COMMIT`, but allows advancing the input to try and find a different starting match position.
+- `SKIP`: Similar to `PRUNE`, but skips ahead to the position of `SKIP` to try again as the starting position.
+- `THEN`: Similar to `PRUNE`, but when used inside an alternation will try to match in the subsequent branch before attempting to advance the input to find a different starting position.
 
 ### PCRE global matching options
 
@@ -437,17 +438,14 @@ This is syntax specific to PCRE, and allows a set of global options to appear at
 - `LIMIT_DEPTH`, `LIMIT_HEAP`, `LIMIT_MATCH`: These place certain limits on the resources the matching engine may consume, and matches it may make.
 - `CRLF`, `CR`, `ANYCRLF`, `ANY`, `LF`, `NUL`: These control the definition of a newline character, which is used when matching e.g the `.` character class, and evaluating where a line ends in multi-line mode.
 - `BSR_ANYCRLF`, `BSR_UNICODE`: These change the definition of `\R`.
-
-**TODO:**
-
-- `NOTEMPTY_ATSTART`:
-- `NOTEMPTY`:
-- `NO_AUTO_POSSESS`:
-- `NO_DOTSTAR_ANCHOR`:
-- `NO_JIT`:
-- `NO_START_OPT`:
-- `UTF`:
-- `UCP`:
+- `NOTEMPTY`: Does not consider the empty string to be a valid match.
+- `NOTEMPTY_ATSTART`: Like `NOT_EMPTY`, but only applies to the first matching position in the input.
+- `NO_AUTO_POSSESS`: Disables an optimization that treats a quantifier as possessive if the following construct clearly cannot be part of the match. In other words, disables the short-circuiting of backtracks in cases where the engine knows it will not produce a match. This is useful for debugging, or for ensuring a callout gets invoked.
+- `NO_DOTSTAR_ANCHOR`: Disables an optimization that tries to automatically anchor `.*` at the start of a regex. Like `NO_AUTO_POSSESS`, this is mainly used for debugging or ensuring a callout gets invoked.
+- `NO_JIT`: Disables JIT compilation
+- `NO_START_OPT`: Disables various optimizations performed at the start of matching. Like `NO_DOTSTAR_ANCHOR`, is mainly used for debugging or ensuring a callout gets invoked.
+- `UTF`: Enables UTF pattern support.
+- `UCP`: Enables Unicode property support.
 
 ### Callouts