Upgrade to Oniguruma 6.9.4

Oniguruma 6.9.4 fixes several CVEs.

Author: cmb69

NEWS

@@ -11,6 +11,9 @@ PHP                                                                        NEWS
 - GD:
   . Fixed bug #78849 (GD build broken with -D SIGNED_COMPARE_SLOW). (cmb)
 
+- MBString:
+  . Upgraded bundled Oniguruma to 6.9.4. (cmb)
+
 - OPcache:
   . Fixed potential ASLR related invalid opline handler issues. (cmb)
   . Fixed $x = (bool)$x; with opcache (should emit undeclared variable notice).

ext/mbstring/oniguruma/HISTORY

@@ -1,8 +1,33 @@
 History
 
+2019/11/29: Version 6.9.4
+
+2019/11/22: Release Candidate 3 for Version 6.9.4
+
+2019/11/20: fix a problem found by libFuzzer test
+2019/11/14: Release Candidate 2 for Version 6.9.4
+2019/11/12: fix integer overflow by nested quantifier
+2019/11/11: fix CVE-2019-19012: Integer overflow related to reg->dmax in search_in_range()
+2019/11/07: fix CVE-2019-19203: heap-buffer-overflow in gb18030_mbc_enc_len()
+2019/11/06: fix CVE-2019-19204: heap-buffer-overflow in fetch_interval_quantifier()
+2019/11/06: add HAVE_INTTYPES_H into config.h.windows.in and config.h.win{32,64}
+2019/11/06: add HAVE_STDINT_H into config.h.win{32,64}
+2019/11/05: Release Candidate 1 for Version 6.9.4
+2019/10/31: Update Unicode Emoji version to 12.1 (Nothing data changed)
+2019/10/29: implement USE_REPEAT_AND_EMPTY_CHECK_LOCAL_VAR configuration
+2019/10/18: re-implement case fold conversion
+2019/10/04: fix #156: Heap buffer overflow in match_at() with case-insensitive match
+2019/09/30: NEW API: add onig_regset_replace()
+2019/09/30: change Unicode VERSION value format
+2019/09/20: NEW API: add regset functions
+2019/09/20: add data ensure check before peek string value in OP_PUSH_IF_PEEK_NEXT
+2019/09/20: fix loose code in encode-harness.c
+2019/08/13: fix heap-buffer-overflow
+2019/08/13: Add a macro to disable direct threading in the match engine (PR#149)
+
 2019/08/06: Version 6.9.3 (secirity fix release)
 
-2019/07/30: add ONIG_SYN_ALLOW_INVALID_CODE_END_OF_RANGE
+2019/07/30: add ONIG_SYN_ALLOW_INVALID_CODE_END_OF_RANGE_IN_CC
 2019/07/29: add STK_PREC_READ_START/END stack type
 2019/07/29: Fix #147: Stack Exhaustion Problem caused by some parsing functions
 2019/07/11: add a dictionary file for libfuzzer

ext/mbstring/oniguruma/README.md

@@ -27,25 +27,34 @@ Supported character encodings:
 * doc/SYNTAX.md: contributed by seanofw
 
 
+Version 6.9.4
+-------------
+
+* NEW API: RegSet (set of regexes)
+* Fixed CVE-2019-19012
+* Fixed CVE-2019-19203 (Does not affect UTF-8, UTF-16 and UTF-32 encodings)
+* Fixed CVE-2019-19204 (Affects only PosixBasic, Emacs and Grep syntaxes)
+* Fixed CVE-2019-19246
+* Fixed some problems (found by libFuzzer test)
+
+
 Version 6.9.3 (security fix release)
 ------------------------------------
 
 * Fixed CVE-2019-13224
 * Fixed CVE-2019-13225
-* Fixed many problems (found by libfuzzer programs)
+* Fixed CVE-2019-16163
+* Fixed many problems (found by libFuzzer test)
 
 
 Version 6.9.2 (Reiwa)
 ---------------------
 
 * add doc/SYNTAX.md
+* Direct threaded code (for GCC and Clang)
 * Update Unicode version 12.1.0
 * NEW: Unicode Text Segment mode option (?y{g}) (?y{w})  (*original)
 
-  g: Extended Grapheme Cluster mode / w: Word mode
-
-  (Unicode Standard Annex #29 [http://unicode.org/reports/tr29/])
-
 
 Version 6.9.1
 -------------
@@ -118,7 +127,7 @@ Version 6.5.0
 * NEW: \O (true anychar)
 * NEW: if-then-else   (?(...)...\|...)
 * NEW: Backreference validity checker (?(xxx)) (*original)
-* NEW: Absent repeater (?~absent)  \[is equal to (?\~\|absent|\O*)]
+* NEW: Absent repeater (?~absent)  \[is equal to (?\~\|(?:absent)|\O*)]
 * NEW: Absent expression   (?~|absent|expr)  (*original)
 * NEW: Absent stopper (?~|absent)     (*original)
 
@@ -244,15 +253,18 @@ Sample Programs
 
 |File                  |Description                               |
 |:---------------------|:-----------------------------------------|
+|sample/callout.c      |example of callouts                       |
+|sample/count.c        |example of built-in callout *COUNT        |
+|sample/echo.c         |example of user defined callouts of name  |
+|sample/encode.c       |example of some encodings                 |
+|sample/listcap.c      |example of the capture history            |
+|sample/names.c        |example of the named group callback       |
+|sample/posix.c        |POSIX API sample                          |
+|sample/regset.c       |example of using RegSet API               |
+|sample/scan.c         |example of using onig_scan()              |
 |sample/simple.c       |example of the minimum (Oniguruma API)    |
-|sample/names.c        |example of the named group callback.      |
-|sample/encode.c       |example of some encodings.                |
-|sample/listcap.c      |example of the capture history.           |
-|sample/posix.c        |POSIX API sample.                         |
-|sample/scan.c         |example of using onig_scan().             |
-|sample/sql.c          |example of the variable meta characters.  |
-|sample/user_property.c|example of user defined Unicode property. |
-|sample/callout.c      |example of callouts.                      |
+|sample/sql.c          |example of the variable meta characters   |
+|sample/user_property.c|example of user defined Unicode property  |
 
 
 Test Programs

ext/mbstring/oniguruma/doc/API

@@ -1,4 +1,4 @@
-Oniguruma API  Version 6.9.3  2019/07/06
+Oniguruma API  Version 6.9.4  2019/09/30
 
 #include <oniguruma.h>
 
@@ -168,7 +168,7 @@ Oniguruma API  Version 6.9.3  2019/07/06
 # int onig_new_deluxe(regex_t** reg, const UChar* pattern, const UChar* pattern_end,
                       OnigCompileInfo* ci, OnigErrorInfo* einfo)
 
-  This function is deprecate, and it does not allow the case where
+  This function is deprecated, and it does not allow the case where
   the encoding of pattern and target is different.
 
   Create a regex object.
@@ -306,6 +306,7 @@ Oniguruma API  Version 6.9.3  2019/07/06
 
   normal return: match position offset (i.e.  p - str >= 0)
   not found:     ONIG_MISMATCH (< 0)
+  error:         error code    (< 0)
 
   arguments
   1 reg:    regex object
@@ -342,7 +343,8 @@ Oniguruma API  Version 6.9.3  2019/07/06
   Do not pass invalid byte string in the regex character encoding.
 
   normal return: match length  (>= 0)
-  not match:     ONIG_MISMATCH ( < 0)
+  not match:     ONIG_MISMATCH (< 0)
+  error:         error code    (< 0)
 
   arguments
   1 reg:    regex object
@@ -391,6 +393,136 @@ Oniguruma API  Version 6.9.3  2019/07/06
   7 callback_arg:  optional argument passed to callback
 
 
+# int onig_regset_new(OnigRegSet** rset, int n, regex_t* regs[])
+
+  Create a regset object.
+  All regex objects must have the same character encoding.
+  All regex objects are prohibited from having the ONIG_OPTION_FIND_LONGEST option.
+
+  arguments
+  1 rset: return address of regset object
+  2 n:    number of regex in regs
+  3 regs: array of regex
+
+  normal return: ONIG_NORMAL
+
+
+# int onig_regset_add(OnigRegSet* set, regex_t* reg)
+
+  Add a regex into regset.
+  The regex object must have the same character encoding with the regset.
+  The regex object is prohibited from having the ONIG_OPTION_FIND_LONGEST option.
+
+  arguments
+  1 set: regset object
+  2 reg: regex object
+
+  normal return: ONIG_NORMAL
+
+
+# int onig_regset_replace(OnigRegSet* set, int at, regex_t* reg)
+
+  Replace a regex in regset with another one.
+  If the reg argument value is NULL, then remove at-th regex. (and indexes of other regexes are changed)
+
+  arguments
+  1 set: regset object
+  2 at:  index of regex (zero origin)
+  3 reg: regex object
+
+  normal return: ONIG_NORMAL
+
+
+# void onig_regset_free(OnigRegSet* set)
+
+  Free memory used by regset object and regex objects in the regset.
+  If the same regex object is registered twice, the situation becomes destructive.
+
+  arguments
+  1 set: regset object
+
+
+# int onig_regset_number_of_regex(OnigRegSet* set)
+
+  Returns number of regex objects in the regset.
+
+  arguments
+  1 set: regset object
+
+
+# regex_t* onig_regset_get_regex(OnigRegSet* set, int at)
+
+  Returns the regex object corresponding to the at-th regex.
+
+  arguments
+  1 set: regset object
+  2 at:  index of regex array (zero origin)
+
+
+# OnigRegion* onig_regset_get_region(OnigRegSet* set, int at)
+
+  Returns the region object corresponding to the at-th regex.
+
+  arguments
+  1 set: regset object
+  2 at:  index of regex array (zero origin)
+
+
+# int onig_regset_search(OnigRegSet* set, const OnigUChar* str, const OnigUChar* end, const OnigUChar* start, const OnigUChar* range, OnigRegSetLead lead, OnigOptionType option, int* rmatch_pos)
+
+  Perform a search with regset.
+
+  return value:
+    normal return: index of match regex (zero origin)
+    not found:     ONIG_MISMATCH (< 0)
+    error:         error code    (< 0)
+
+  arguments
+  1 set:    regset object
+  2 str:    target string
+  3 end:    terminate address of target string
+  4 start:  search start address of target string
+  5 range:  search terminate address of target string
+  6 lead:   outer loop element
+      ONIG_REGSET_POSITION_LEAD (returns most left position)
+      ONIG_REGSET_REGEX_LEAD    (returns most left position)
+      ONIG_REGSET_PRIORITY_TO_REGEX_ORDER (returns first match regex)
+  7 option: search time option
+      ONIG_OPTION_NOTBOL    string head(str) isn't considered as begin of line
+      ONIG_OPTION_NOTEOL    string end (end) isn't considered as end of line
+  8 rmatch_pos: return address of match position (match_address - str)
+
+  * ONIG_REGSET_POSITION_LEAD and ONIG_REGSET_REGEX_LEAD return the same result.
+    These differences only appear in search time.
+    In most cases, ONIG_REGSET_POSITION_LEAD seems to be faster.
+
+
+# int onig_regset_search_with_param(OnigRegSet* set, const OnigUChar* str, const OnigUChar* end, const OnigUChar* start, const OnigUChar* range,  OnigRegSetLead lead, OnigOptionType option, OnigMatchParam* mps[], int* rmatch_pos)
+
+  Perform a search with regset and match-params.
+
+  return value:
+    normal return: index of match regex (zero origin)
+    not found:     ONIG_MISMATCH (< 0)
+    error:         error code    (< 0)
+
+  arguments
+  1 set:    regset object
+  2 str:    target string
+  3 end:    terminate address of target string
+  4 start:  search start address of target string
+  5 range:  search terminate address of target string
+  6 lead:   outer loop element
+      ONIG_REGSET_POSITION_LEAD (returns most left position)
+      ONIG_REGSET_REGEX_LEAD    (returns most left position)
+      ONIG_REGSET_PRIORITY_TO_REGEX_ORDER (returns first match regex)
+  7 option: search time option
+      ONIG_OPTION_NOTBOL    string head(str) isn't considered as begin of line
+      ONIG_OPTION_NOTEOL    string end (end) isn't considered as end of line
+  8 mps:    array of match-params
+  9 rmatch_pos: return address of match position (match_address - str)
+
+
 # OnigRegion* onig_region_new(void)
 
   Create a region.