Defining "evaluation"

[handrews]

From @Relequestual:

I've been thinking about this section and its phrasing.
It's the first time in the whole document we mention "evaluation", and we don't really define what it means.

People (including myself previously) incorrectly assume that if a location has been evaluated (had something applied to it), then it counts as having been evaluated.

I'm going to riff some ideas...

We say here "successful evaluations", we mean if an application doesn't result in validation errors, and therefore the applied schema or keyword is correctly applicable to the instance location.

For example, if you had a schema...
anyOf [ { title> unicycle, props > wheels > const > 1 }, { title: bicycle, props > wheels > const > 2 } ]

If your instance was a unicycle, the first subschema item in the anyOf would be applied and successfully evaluated. Because it only has one wheel, we know it's a unicycle. It is not a bicycle, so that whole subschema is not successfully evaluated.

The instance is successfully evaluated as (or determined to be) a unicycle.

We can further justify this terminology by the fact that evaluation can be short-circuited, and therefore there's no expectation for a schema to be fully evaluated (although that would be a weak argument, and probably confuse things).

We need to clearly define what successful evaluation means, and what unsuccessful evaluation means, with justification (IMHO). I'm not currently convinced I can do that.

[karenetheridge]

One place in which this came up was in the context of unevaluatedItems + unevaluatedProperties -- that is, these keywords apply to items/properties that were not successfully evaluated by other keywords.

for example, this instance data and schema:

  { "fruit": 1 }

  {
    "type": "object",
    "properties": {
      "fruit": {
        "type": "string"
      }
    },
    unevaluatedProperties: false
  }

..should fail evaluation, because while there is a definition for the "fruit" property under "properties", when the instance is applied against that keyword it fails evaluation (because the type is wrong), so there is no annotation generated from "properties" for "fruit", therefore "unevaluatedProperties" will consider it, and therefore generate a failure (via false).

(But maybe this doesn't really matter because evaluation fails at the "properties" keyword anyway, so who cares if unevaluatedProperties looks at "fruit" or not?)