Upgrade oniguruma lib to 6.9.3

Author: smalyshev

ext/mbstring/oniguruma/COPYING

@@ -1,7 +1,7 @@
 Oniguruma LICENSE
 -----------------
 
-Copyright (c) 2002-2018  K.Kosako  <kkosako0@gmail.com>
+Copyright (c) 2002-2019  K.Kosako  <kkosako0@gmail.com>
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

ext/mbstring/oniguruma/HISTORY

@@ -1,5 +1,42 @@
 History
 
+2019/08/06: Version 6.9.3 (secirity fix release)
+
+2019/07/30: add ONIG_SYN_ALLOW_INVALID_CODE_END_OF_RANGE
+2019/07/29: add STK_PREC_READ_START/END stack type
+2019/07/29: Fix #147: Stack Exhaustion Problem caused by some parsing functions
+2019/07/11: add a dictionary file for libfuzzer
+2019/07/07: add harnesses directory
+2019/07/05-2019/07/29: fix many problems found by libfuzzer programs
+2019/06/27: deprecate onig_new_deluxe()
+2019/06/27: Fix CVE-2019-13224: don't allow different encodings for onig_new_deluxe()
+2019/06/27: Fix CVE-2019-13225: problem in converting if-then-else pattern
+
+2019/05/07: Version 6.9.2 (same as Release Candidate 3)
+
+2019/04/23: Release Candidate 3 for 6.9.2
+2019/04/23: add doc/SYNTAX.md into distribution file
+2019/04/09: Release Candidate 2 for 6.9.2
+2019/04/09: fix #139: UAF in match_at()
+2019/04/01: Release Candidate 1 for 6.9.2
+2019/04/01: update Unicode version to 12.1.0 (draft)
+2019/03/29: allow {n,m} (n>m) as possessive interval
+2019/03/25: add ONIG_SYN_OP2_OPTION_ONIGURUMA
+2019/03/22: add new options ONIG_OPTION_TEXT_SEGMENT_EXTENDED_GRAPHEME_CLUSTER and
+            ONIG_OPTION_TEXT_SEGMENT_WORD
+2019/03/21: PR #137: fix cross-compilation
+2019/03/20: update Unicode version to 12.0.0
+2019/03/17: add doc/SYNTAX.md
+2019/03/13: {n,m}+ and {n,m}? are possessive and reluctant range operator
+            in Perl syntax
+2019/03/04: fix #132: don't execute testp if ENABLE_POSIX_API == no
+2019/02/28: re-implement bytecode by using Operation struct
+2019/02/26: fix #130: Build error on UWP with VS2017
+2019/02/03: PR #128: regerror/toascii: do not attempt to serialize NULL pointer
+2019/01/30: Build breaks without autoreconf #73
+2019/01/02: fix #127: Windows VS 2008 build errors
+2018/12/19: fix #126: Unable to compile when USE_CALLOUT is not defined
+
 2018/12/11: Version 6.9.1
 
 2018/10/08: use ENC_FLAG_SKIP_OFFSET_XXX values

ext/mbstring/oniguruma/README.md

@@ -24,37 +24,58 @@ Supported character encodings:
 
 * GB18030: contributed by KUBO Takehiro
 * CP1251:  contributed by Byte
+* doc/SYNTAX.md: contributed by seanofw
 
 
-New feature of version 6.9.1
---------------------------
+Version 6.9.3 (security fix release)
+------------------------------------
+
+* Fixed CVE-2019-13224
+* Fixed CVE-2019-13225
+* Fixed many problems (found by libfuzzer programs)
+
+
+Version 6.9.2 (Reiwa)
+---------------------
+
+* add doc/SYNTAX.md
+* Update Unicode version 12.1.0
+* NEW: Unicode Text Segment mode option (?y{g}) (?y{w})  (*original)
+
+  g: Extended Grapheme Cluster mode / w: Word mode
+
+  (Unicode Standard Annex #29 [http://unicode.org/reports/tr29/])
+
+
+Version 6.9.1
+-------------
 
 * Speed improvement (* especially UTF-8)
 
 
-New feature of version 6.9.0
---------------------------
+Version 6.9.0
+-------------
 
 * Update Unicode version 11.0.0
 * NEW: add Emoji properties
 
 
-New feature of version 6.8.2
---------------------------
+Version 6.8.2
+-------------
 
 * Fix: #80 UChar in header causes issue
 * NEW API: onig_set_callout_user_data_of_match_param()  (* omission in 6.8.0)
 * add doc/CALLOUTS.API and doc/CALLOUTS.API.ja
 
 
-New feature of version 6.8.1
---------------------------
+Version 6.8.1
+-------------
 
 * Update shared library version to 5.0.0 for API incompatible changes from 6.7.1
 
 
-New feature of version 6.8.0
---------------------------
+Version 6.8.0
+-------------
 
 * Retry-limit-in-match function enabled by default
 * NEW: configure option --enable-posix-api=no  (* enabled by default)
@@ -65,32 +86,32 @@ New feature of version 6.8.0
 * Examples of Callouts program: [callout.c](sample/callout.c), [count.c](sample/count.c), [echo.c](sample/echo.c)
 
 
-New feature of version 6.7.1
---------------------------
+Version 6.7.1
+-------------
 
 * NEW: Mechanism of retry-limit-in-match (* disabled by default)
 
 
-New feature of version 6.7.0
---------------------------
+Version 6.7.0
+-------------
 
 * NEW: hexadecimal codepoint \uHHHH
 * NEW: add ONIG_SYNTAX_ONIGURUMA (== ONIG_SYNTAX_DEFAULT)
 * Disabled \N and \O on ONIG_SYNTAX_RUBY
 * Reduced size of object file
 
 
-New feature of version 6.6.0
---------------------------
+Version 6.6.0
+-------------
 
 * NEW: ASCII only mode options for character type/property (?WDSP)
-* NEW: Extended Grapheme Cluster boundary \y, \Y (*original)
+* NEW: Extended Grapheme Cluster boundary \y, \Y
 * NEW: Extended Grapheme Cluster \X
 * Range-clear (Absent-clear) operator restores previous range in retractions.
 
 
-New feature of version 6.5.0
---------------------------
+Version 6.5.0
+-------------
 
 * NEW: \K (keep)
 * NEW: \R (general newline) \N (no newline)
@@ -102,16 +123,16 @@ New feature of version 6.5.0
 * NEW: Absent stopper (?~|absent)     (*original)
 
 
-New feature of version 6.4.0
---------------------------
+Version 6.4.0
+-------------
 
 * Fix fatal problem of endless repeat on Windows
 * NEW: call zero (call the total regexp) \g<0>
 * NEW: relative backref/call by positive number \k<+n>, \g<+n>
 
 
-New feature of version 6.3.0
---------------------------
+Version 6.3.0
+-------------
 
 * NEW: octal codepoint \o{.....}
 * Fixed CVE-2017-9224
@@ -122,20 +143,20 @@ New feature of version 6.3.0
 * Fixed CVE-2017-9229
 
 
-New feature of version 6.1.2
---------------------------
+Version 6.1.2
+-------------
 
 * allow word bound, word begin and word end in look-behind.
 * NEW option: ONIG_OPTION_CHECK_VALIDITY_OF_STRING
 
-New feature of version 6.1
---------------------------
+Version 6.1
+-----------
 
 * improved doc/RE
 * NEW API: onig_scan()
 
-New feature of version 6.0
---------------------------
+Version 6.0
+-----------
 
 * Update Unicode 8.0 Property/Case-folding
 * NEW API: onig_unicode_define_user_property()

ext/mbstring/oniguruma/doc/API

@@ -1,4 +1,4 @@
-Oniguruma API  Version 6.8.0  2018/03/13
+Oniguruma API  Version 6.9.3  2019/07/06
 
 #include <oniguruma.h>
 
@@ -92,6 +92,8 @@ Oniguruma API  Version 6.8.0  2018/03/13
                                      (alnum, alpha, blank, cntrl, digit, graph,
                                       lower, print, punct, space, upper, xdigit,
                                       word)
+      ONIG_OPTION_TEXT_SEGMENT_EXTENDED_GRAPHEME_CLUSTER  Extended Grapheme Cluster mode
+      ONIG_OPTION_TEXT_SEGMENT_WORD                       Word mode
 
   5 enc:        character encoding.
 
@@ -166,6 +168,9 @@ Oniguruma API  Version 6.8.0  2018/03/13
 # int onig_new_deluxe(regex_t** reg, const UChar* pattern, const UChar* pattern_end,
                       OnigCompileInfo* ci, OnigErrorInfo* einfo)
 
+  This function is deprecate, and it does not allow the case where
+  the encoding of pattern and target is different.
+
   Create a regex object.
   This function is deluxe version of onig_new().
 
@@ -297,6 +302,7 @@ Oniguruma API  Version 6.8.0  2018/03/13
                    const UChar* range, OnigRegion* region, OnigOptionType option)
 
   Search string and return search result and matching region.
+  Do not pass invalid byte string in the regex character encoding.
 
   normal return: match position offset (i.e.  p - str >= 0)
   not found:     ONIG_MISMATCH (< 0)
@@ -321,15 +327,19 @@ Oniguruma API  Version 6.8.0  2018/03/13
                    const UChar* start, const UChar* range, OnigRegion* region,
                    OnigOptionType option, OnigMatchParam* mp)
 
-   arguments
-   1-7:  same as onig_search()
-   8 mp: match parameter values (match_stack_limit, retry_limit_in_match)
+  Search string and return search result and matching region.
+  Do not pass invalid byte string in the regex character encoding.
+
+  arguments
+  1-7:  same as onig_search()
+  8 mp: match parameter values (match_stack_limit, retry_limit_in_match)
 
 
 # int onig_match(regex_t* reg, const UChar* str, const UChar* end, const UChar* at,
                  OnigRegion* region, OnigOptionType option)
 
   Match string and return result and matching region.
+  Do not pass invalid byte string in the regex character encoding.
 
   normal return: match length  (>= 0)
   not match:     ONIG_MISMATCH ( < 0)
@@ -351,6 +361,9 @@ Oniguruma API  Version 6.8.0  2018/03/13
                             const UChar* at, OnigRegion* region,
                             OnigOptionType option, OnigMatchParam* mp)
 
+  Match string and return result and matching region.
+  Do not pass invalid byte string in the regex character encoding.
+
    arguments
    1-6:  same as onig_match()
    7 mp: match parameter values (match_stack_limit, retry_limit_in_match)
@@ -362,6 +375,7 @@ Oniguruma API  Version 6.8.0  2018/03/13
                 void* callback_arg)
 
   Scan string and callback with matching region.
+  Do not pass invalid byte string in the regex character encoding.
 
   normal return: number of matching times
   error:         error code
@@ -609,14 +623,20 @@ Oniguruma API  Version 6.8.0  2018/03/13
 
 
 # int onigenc_strlen(OnigEncoding enc, const UChar* s, const UChar* end)
+
+  Return number of characters in the string.
+
+
 # int onigenc_strlen_null(OnigEncoding enc, const UChar* s)
 
   Return number of characters in the string.
+  Do not pass invalid byte string in the character encoding.
 
 
 # int onigenc_str_bytelen_null(OnigEncoding enc, const UChar* s)
 
   Return number of bytes in the string.
+  Do not pass invalid byte string in the character encoding.
 
 
 # int onig_set_default_syntax(OnigSyntaxType* syntax)

ext/mbstring/oniguruma/doc/API.ja

@@ -1,4 +1,4 @@
-鬼車インターフェース Version 6.8.0   2018/03/13
+鬼車インターフェース Version 6.9.3   2019/07/06
 
 #include <oniguruma.h>
 
@@ -91,6 +91,8 @@
                                      (alnum, alpha, blank, cntrl, digit, graph,
                                       lower, print, punct, space, upper, xdigit,
                                       word)
+      ONIG_OPTION_TEXT_SEGMENT_EXTENDED_GRAPHEME_CLUSTER  拡張書記素房モード
+      ONIG_OPTION_TEXT_SEGMENT_WORD                       単語モード
 
   5 enc:        文字エンコーディング
 
@@ -165,6 +167,9 @@
 # int onig_new_deluxe(regex_t** reg, const UChar* pattern, const UChar* pattern_end,
                       OnigCompileInfo* ci, OnigErrorInfo* einfo)
 
+  この関数は廃止予定。
+  パターンと対象文字列の文字エンコーディングが異なる場合を許さなくなった。
+
   正規表現オブジェクト(regex)を作成する。
   この関数は、onig_new()のデラックス版。
 
@@ -296,6 +301,7 @@
                    const UChar* range, OnigRegion* region, OnigOptionType option)
 
   正規表現で文字列を検索し、検索結果とマッチ領域を返す。
+  正規表現オブジェクトの文字エンコーディングで、検索文字列として不正な文字列を渡してはいけない。
 
   正常終了戻り値: マッチ位置 (p - str >= 0)
   検索失敗:       ONIG_MISMATCH (< 0)
@@ -320,15 +326,19 @@
                    const UChar* start, const UChar* range, OnigRegion* region,
                    OnigOptionType option, OnigMatchParam* mp)
 
+  正規表現で文字列を検索し、検索結果とマッチ領域を返す。
+  正規表現オブジェクトの文字エンコーディングで、検索文字列として不正な文字列を渡してはいけない。
+
   引数
   1-7:  onig_search()と同じ
   8 mp: マッチパラメタ値 (match_stack_limit, retry_limit_in_match)
 
 
-# int onig_match(regex_t* reg, const UChar* str, const UChar* end, const UChar* at,
-		 OnigRegion* region, OnigOptionType option)
+# int onig_match(regex_t* reg, const UChar* str, const UChar* end,
+                 const UChar* at, OnigRegion* region, OnigOptionType option)
 
   文字列の指定位置でマッチングを行い、結果とマッチ領域を返す。
+  正規表現オブジェクトの文字エンコーディングで、検索文字列として不正な文字列を渡してはいけない。
 
   正常終了戻り値: マッチしたバイト長 (>= 0)
   not match:      ONIG_MISMATCH      ( < 0)
@@ -350,6 +360,9 @@
                             const UChar* at, OnigRegion* region,
                             OnigOptionType option, OnigMatchParam* mp)
 
+  文字列の指定位置でマッチングを行い、結果とマッチ領域を返す。
+  正規表現オブジェクトの文字エンコーディングで、検索文字列として不正な文字列を渡してはいけない。
+
   引数
   1-6:  onig_match()と同じ
   7 mp: マッチパラメタ値 (match_stack_limit, retry_limit_in_match)
@@ -361,6 +374,7 @@
                 void* callback_arg)
 
   正規表現で文字列をスキャンして、マッチングする毎にコールバック関数を呼び出す。
+  正規表現オブジェクトの文字エンコーディングで、検索文字列として不正な文字列を渡してはいけない。
 
   正常終了: マッチ回数 (0回も含める)
   エラー:   エラーコード (< 0)
@@ -614,14 +628,20 @@
 
 
 # int onigenc_strlen(OnigEncoding enc, const UChar* s, const UChar* end)
+
+  文字列の文字数を返す。
+
+
 # int onigenc_strlen_null(OnigEncoding enc, const UChar* s)
 
   文字列の文字数を返す。
+  文字エンコーディングに対して、不正な文字列を渡してはいけない。
 
 
 # int onigenc_str_bytelen_null(OnigEncoding enc, const UChar* s)
 
   文字列のバイト数を返す。
+  文字エンコーディングに対して、不正な文字列を渡してはいけない。
 
 
 # int onig_set_default_syntax(OnigSyntaxType* syntax)