Paper suggests that the test suite is lacking

[Relequestual]

http://www2016.net/proceedings/proceedings/p263.pdf

Specifcially section 2...
"Table 1 shows the outcome of this process, It is impor-
tant to mention that all validators successfully validate the
JSON Schema test-suite [4]. As we can see, no two valida-
tors behave the same on all inputs, which is clearly not the
desired behaviour. This illustrates the need for a formal
de nition of JSON Schema which will either disallow am-
biguous schemas, or formally specify how these should be
evaluated."

I'm not sure on their methods, and I'm not especially up for downloading random code from academic URLs.
I've contacted them regarding something else. If I get a reply, I'll push them to publish the code in a repo or as a gist, and see if they deliver. It's a shame they didn't feel the need to put it on github.

[Relequestual]

They replied! I'll keep this issue posted.

[Julian]

Please do, thanks! If they can offer any specifics would be great to fill
in some gaps.
On May 3, 2016 17:38, "Ben Hutton" notifications@github.com wrote:

They replied! I'll keep this issue posted.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#115 (comment)

[seagreen]

Any luck? That part of the paper was a hilarious tease -- it gave us just enough info to know they've got something very useful for the test suite, but not enough to know what it is.

[Relequestual]

@seagreen Nope. Had no further update. I'll chase them via email to ask if they ever looked into providing further info. =]

[iainbeeston]

I've taken a look at the paper and the code released with it, and reproduced the test cases in #128

[iainbeeston]

FYI they explained the tests they used in section 2, with the full code available online in the appendices under "available validator tests code". In fact the paper describes 4 tests but the source code actually has a 5th test, which I've also included.

[Relequestual]

@iainbeeston It's great you've been able to find time to review the paper properly and the associated code. Super thankyou and really appreciated. There's still a lot I don't really understand about what they were getting at with regards JSON Schema needing a formal syntax, but that might be more academic than practical (or I may miss the point completely). I'm very much in favour of having practical examples when anyone calls out a probelm.

[Julian]

Gonna close this in favor of #128 which @iainbeeston was so kind to submit.

[Relequestual]

I'm happy with this issue being closed in this way, with the new issue superseeding this one.