All values fail against the false schema when additionalProperties is set to false

[fredjeck]

Environment

• Nuget Package: . JsonSchema.Net
• Nuget Version: 2.0.3
• OS: Linux & Windows
• .Net Target: 6

Describe the bug
First and before : Kudos for the awesome job 😍
Potentially related to #216
When validating a faulty schema with additionalProperties set to false, validation yields extraneous error messages ("All values fail against the false schema") those extraenous messages are not present when additionalProperties is set to true

To Reproduce
Schema

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "additionalProperties": false,
    "properties": {
        "firstname": {"type": "string"},
        "phone": {"type": "string"},
        "email": {"$ref": "#/$defs/emailAddress"}
    },
    "required": ["firstname","phone","email"],
    "$defs": {
        "emailAddress": {
                "type": "object",
                "additionalProperties": false,
                "properties": {
                    "email": {"type": "string"},
                    "isConfirmed": {"type": "boolean"}
                },
                "required": [ "email","isConfirmed"]
        }
    }
}

The following sample is missing the isConfirmed property

{
    "firstname": "firstnameTest",
    "phone": "123456",
    "email": {
        "email": "test@test.com"
    }
}

This generates the following validation errors :

• Required properties [isConfirmed] were not present found at #/email
• All values fail against the false schema found at #/firstname
• All values fail against the false schema found at #/phone
• All values fail against the false schema found at #/email

Setting additionalProperties is set to true only one validation error is raised

• Required properties [isConfirmed] were not present found at #/email

Expected behavior
Behavior should be consistent independently from the additionalProperty property and should only report the first error

[gregsdennis]

I don't see how this is a bug. A true schema passes all values, so no errors would arise. They show up explicitly because the subschema is false.

The way the spec is worded, any properties not validated by another "properties" keyword are validated against the subschema in additionalProperties. This means that these errors should be present when that subschema is false because those properties are failing both their associates subschema in properties and the false subschema in additionalProperties.

We are currently looking at removing the "annotation" behavior that drives this and returning to the draft 7 language where the keyword instead looks at its sibling keywords directly to determine which properties to assess.

Please see json-schema-org/json-schema-spec#1154 for more info.

[fredjeck]

Thanks for the quick reply. However when additionalProperties is set to false (default setup in my case) why are all the "All values fail against the false schema" errors being reported for each property of the top level schema whereas the validation error is linked to just one property missing in the subschema ?

In the provided sample this is trivial, however in the case of complex schemas, it is almost impossible to locate the source of the validation failure due to all the noise generated (over 200 errors for 1 missing field) - errors are being reported on fields which are correct which is misleading for the users.

Or maybe is it possible to fine tune the validation output format ?

[gregsdennis]

This is due to the aforementioned annotation behavior.

• When properties succeeds (all of the properties pass their associated subschemas), it produces an annotation with a value of a list of all of the properties it validates.
• But when any keyword fails to validate, it doesn't produce annotations.

When properties fails, it produces no annotation, and when additionalProperties is looking for an annotation from properties to indicate what properties were validated, it finds nothing, so it attempts to validate everything.

I agree: this is a weird side effect of the annotation behavior, and I'm not sure that we (the JSON Schema team) fully understood all of this when we put this behavior in. It was intended to make things simpler (for implementers) by providing a model to describe how keywords could interact. Instead it made failures harder to debug for schema authors (who should be the real focus, IMO).

[gregsdennis]

Related: #39 & #89