JsonChecker tests broken

[baylesj]

After the recent trailing commas change, json checker tests aren't broken.

A fix means making these tests run as part of checkin again, and resolving the actual test failures as well.

[LocutusOfBorg]

Hello, I have some test failures on Debian experimental, version 1.9.2
first error is:

Testing EscapeSequenceTest/writeEscapeSequence: FAILED
* Detail of EscapeSequenceTest/writeEscapeSequence test failure:
/<<PKGBUILDDIR>>/src/test_lib_json/main.cpp(3370): expected == result
  Expected: '["\"","\\","\b","\f","\n","\r","\t","\u0278","\ud852\udf62"]
  '
  Actual  : '["\"","\\","\b","\f","\n","\r","\t","ɸ","𤭢"]
  '
101/102 tests passed (1 failure(s))

And happens on arm64, armhf, armel. ppc64el, s390x (and others)
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=arm64&ver=1.9.2-2&stamp=1580476809&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=armhf&ver=1.9.2-2&stamp=1580476979&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=armel&ver=1.9.2-2&stamp=1580477812&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=ppc64el&ver=1.9.2-2&stamp=1580476937&raw=0
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=s390x&ver=1.9.2-2&stamp=1580476882&raw=0

[LocutusOfBorg]

The second kind of error is on mips64el:
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=mips64el&ver=1.9.2-2&stamp=1580514060&raw=0

2/3 Test #1: jsoncpp_readerwriter ................***Failed    2.54 sec
Unhandled exception:
Exceeded stackLimit in readValue().
Failed to parse input file: 
* Line 1, Column 5
  Missing ',' or ']' in array declaration

TESTING: /<<PKGBUILDDIR>>/test/data/legacy_test_array_03.json FAILED


Failure details:
* Test /<<PKGBUILDDIR>>/test/data/legacy_test_array_03.json
  Difference in input at line 1:
  Expected: '.=[]'
  Actual:   ''


Test results: 57 passed, 1 failed.

3/3 Test #2: jsoncpp_readerwriter_json_checker ...***Failed    3.25 sec
Unhandled exception:
Exceeded stackLimit in readValue().
Failed to parse input file: 
* Line 1, Column 5
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 22
  Missing '}' or object member name

Failed to parse input file: 
* Line 1, Column 5
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 23
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 16
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 12
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 41
  Missing '}' or object member name

Failed to parse input file: 
* Line 1, Column 2
  '0e+' is not a number.

Failed to parse input file: 
* Line 1, Column 2
  '0e+' is not a number.

Failed to parse input file: 
* Line 1, Column 2
  Missing '}' or object member name

Failed to parse input file: 
* Line 1, Column 2
  '0e' is not a number.

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 2, Column 1 for detail.

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 1, Column 8 for detail.

Failed to parse input file: 
* Line 1, Column 2
  Syntax error: value, object or array expected.
/<<PKGBUILDDIR>>/obj-mips64el-linux-gnuabi64/src/jsontestrunner/jsontestrunner_exe  --json-writer StyledWriter "/<<PKGBUILDDIR>>/test/data/legacy_test_comment_02.json"Failed to parse input file: 
* Line 1, Column 15
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 26
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 26
  Missing ':' after object member name

Failed to parse input file: 
* Line 1, Column 17
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 18
  Missing ',' or ']' in array declaration

Failed to parse input file: 
* Line 1, Column 18
  Missing ':' after object member name

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 1, Column 31 for detail.

Failed to parse input file: 
* Line 1, Column 2
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 2
  Bad escape sequence in string
See Line 1, Column 31 for detail.

Failed to parse input file: 
* Line 1, Column 28
  Missing ',' or '}' in object declaration

Failed to parse input file: 
* Line 1, Column 24
  Syntax error: value, object or array expected.

Failed to parse input file: 
* Line 1, Column 26
  Missing ',' or '}' in object declaration

Failed to parse input file: 
* Line 1, Column 1
  A valid JSON document must be either an array or an object value.


Failure details:
* Test /<<PKGBUILDDIR>>/test/data/legacy_test_basic_05.json
  Difference in rewrite at line 1:
  Expected: '.=null'
  Actual:   ''


Test results: 57 passed, 1 failed.


33% tests passed, 2 tests failed out of 3

Total Test time (real) =   3.27 sec

The following tests FAILED:
	  1 - jsoncpp_readerwriter (Failed)
	  2 - jsoncpp_readerwriter_json_checker (Failed)

Probably similar to the failure in mipsel
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=mipsel&ver=1.9.2-2&stamp=1580514054&raw=0

I'm uploading in Debian the patch for this issue you proposed, I hope it fixes also the above kind of failures!

[LocutusOfBorg]

nope, looks like the patch didn't work for my failures:
http://debomatic-arm64.debian.net/distribution#experimental/libjsoncpp/1.9.2-4/buildlog
at least on arm64 it didn't make differences

* Detail of EscapeSequenceTest/writeEscapeSequence test failure:
/<<PKGBUILDDIR>>/src/test_lib_json/main.cpp(3370): expected == result
  Expected: '["\"","\\","\b","\f","\n","\r","\t","\u0278","\ud852\udf62"]
  '
  Actual  : '["\"","\\","\b","\f","\n","\r","\t","ɸ","𤭢"]
  '
101/102 tests passed (1 failure(s))

Do you have any clue?

[ggardet]

And happens on arm64, armhf, armel. ppc64el, s390x (and others)

It seems to be all non-x86 archs.

[ggardet]

It should be fixed by #1120

[LocutusOfBorg]

After applying #1103 and #1120 I got no failures left, except from when I run tests with parallel!=1

if you do make test -j 8 for example you can get lots of failures...

[LocutusOfBorg]

https://buildd.debian.org/status/package.php?p=libjsoncpp&suite=experimental
look e.g.
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=amd64&ver=1.9.2-5&stamp=1582121093&raw=0
and
https://buildd.debian.org/status/fetch.php?pkg=libjsoncpp&arch=amd64&ver=1.9.2-6&stamp=1582137147&raw=0

the only difference is the parallel in tests...

[baylesj]

I'm working on actually getting #1103 closed out, hopefully will be soon

[cdunn2001]

@LocutusOfBorg, it's definitely important for tests to pass in parallel. We'll try to get to that for the upcoming release.