From ff02e882ba0075144209c5cdd495f556c2425e3e Mon Sep 17 00:00:00 2001
From: Henry Andrews <andrews_henry@yahoo.com>
Date: Sat, 24 Aug 2019 00:15:24 -0700
Subject: [PATCH] More consistent concept of schema resources

This further clarifies the schema resource concept and
how it is used.  In particular, schema resources are
crucial to determining when it is safe to embed a
reference target into its referring schema.

This also requires allowing "$ref" to take a schema value.

The bundling use case should be more clear given this
guidance.  While many reference target embedding transformations
are likely to break things and are therefor to be discouraged,
when framed in terms of schema resources a clear use case emerges.

This also relaxes a restriction on "$schema" that has no functional
impact but avoids requiring implementations to detect and handle
a rather complex case (embedding schema resources with different
"$schema" values).  And also means that embedding can work without
having to change the embedded value by removing the "$schema"
keyword.
---
 jsonschema-core.xml | 151 +++++++++++++++++++++++++++++++-------------
 1 file changed, 108 insertions(+), 43 deletions(-)

diff --git a/jsonschema-core.xml b/jsonschema-core.xml
index e292e275..bfe5a8e9 100644
--- a/jsonschema-core.xml
+++ b/jsonschema-core.xml
@@ -100,7 +100,7 @@
             <t>
                 This specification defines JSON Schema core terminology and mechanisms, including
                 pointing to another JSON Schema by reference,
-                dereferencing a JSON Schema reference,
+                dereferencing a JSON Schema reference or embedding its target,
                 specifying the vocabulary being used,
                 and defining the expected output.
             </t>
@@ -409,11 +409,6 @@
                         <xref target="RFC6596">canonically</xref> identified by an
                         <xref target="RFC3986">absolute URI</xref>.
                     </t>
-                    <t>
-                        As discussed in section
-                        <xref target="id-keyword" format="counter"></xref>, a JSON Schema document
-                        can contain multiple JSON Schema resources.
-                    </t>
                     <t>
                         The root schema is the schema that comprises the entire JSON document
                         in question.  The root schema is always a schema resource, where the
@@ -442,6 +437,15 @@
                     <t>
                         As with the root schema, a subschema is either an object or a boolean.
                     </t>
+                    <t>
+                        As discussed in section
+                        <xref target="id-keyword" format="counter"></xref>, a JSON Schema document
+                        can contain multiple JSON Schema resources.  When used without qualification,
+                        the term "root schema" refers to the document's root schema.  In some
+                        cases, resource root schemas are discussed.  A resource's root schema
+                        is its top-level schema object, which would also be a document root schema
+                        if the resource were to be extracted to a standalone JSON Schema document.
+                    </t>
                 </section>
             </section>
 
@@ -624,7 +628,7 @@
                 <t>
                     Note that some keywords, such as "$schema", apply to the lexical
                     scope of the entire schema document, and therefore MUST only
-                    appear in the document's root schema.
+                    appear in a schema resource's root schema.
                 </t>
                 <t>
                     Other keywords may take into account the dynamic scope that
@@ -1131,11 +1135,15 @@
                         media type "application/schema+json".
                     </t>
                     <t>
-                        The "$schema" keyword SHOULD be used in a root schema.
-                        It MUST NOT appear in subschemas.  If absent from the root schema, the
+                        The "$schema" keyword SHOULD be used in a resource root schema.
+                        It MUST NOT appear in resource subschemas.  If absent from the root schema, the
                         resulting behavior is implementation-defined.
                     </t>
                     <t>
+                        If multiple schema resources are present in a single document, then all
+                        schema resources SHOULD Have the same value for "$schema".  The result of
+                        differing values for "$schema" within the same schema document is
+                        implementation-defined.
                         <cref>
                             Using multiple "$schema" keywords in the same document would imply that the
                             feature set and therefore behavior can change within a document.  This would
@@ -1145,6 +1153,12 @@
                             implementation behavior is subject to be revised or liberalized in
                             future drafts.
                         </cref>
+                        <cref>
+                            The exception made for embedded schema resources is to
+                            allow bundling multiple schema resources into a single schema document
+                            without needing to change their contents, as described later in this
+                            specification.
+                        </cref>
                         <!--
                             In particular, the process of validating an instance, including validating a
                             schema as an instance against its meta-schema, only allows for a single set
@@ -1566,25 +1580,11 @@
                             canonical URI, or relative to the containing resource's URI.
                         </t>
                         <t>
-                            Conceptually, a set of linked schemas should behave identically whether
-                            each schema is a separate document connected with
+                            Conceptually, a set of linked schema resources should behave
+                            identically whether each resource is a separate document connected with
                             <xref target="references">schema references</xref>, or is structured as
                             a single document with one or more schema resources embedded as
                             subschemas.
-                            <cref>
-                                Note that when using schema references, the reference keyword
-                                appears in the runtime path in the standard output format for
-                                errors and annotations.  This means that while the validation
-                                outcome is unchanged when switching between an embedded schema
-                                resource and a referenced one, the runtime paths of annotations
-                                do change.  A future draft may allow directly replacing the value
-                                of the reference keyword with its target while leaving the keyword
-                                itself in place in order to make embedding vs referencing transparent
-                                to annotation collection.  This would allow replacing
-                                {"$ref": "/foo"} with {"$ref": {"type": "string"}}, assuming
-                                the schema at "verb">"/foo" consists of just a "type" keyword with
-                                value "string".  Feedback on this topic is highly encouraged.
-                            </cref>
                         </t>
                         <t>
                             Since URIs involving JSON Pointer fragments relative to the parent
@@ -1593,22 +1593,65 @@
                             SHOULD NOT use such URIs to identify embedded schema resources or
                             locations within them.
                         </t>
+                        <figure>
+                            <preamble>
+                                Consider the following schema document that contains another
+                                schema resource embedded within it (the usage of "$ref" with
+                                a schema value is explained in section
+                                <xref target="ref" format="counter" />):
+                            </preamble>
+                            <artwork>
+<![CDATA[
+{
+  "$id": "https://example.com/foo",
+  "items": {
+    "$ref": {
+      "$id": "https://example.com/bar",
+      "additionalProperties": { }
+    }
+  }
+}
+]]>
+                            </artwork>
+                            <postamble>
+                                The URI "https://example.com/foo#/items/$ref/additionalProperties"
+                                points to the schema of the "additionalProperties" keyword in
+                                the embedded resource.  The canonical URI of that schema, however,
+                                is "https://example.com/bar#/additionalProperties".
+                            </postamble>
+                        </figure>
+                        <figure>
+                            <preamble>
+                                Now consider the following two schema resources linked by reference
+                                using a URI value for "$ref":
+                            </preamble>
+                            <artwork>
+<![CDATA[
+{
+  "$id": "https://example.com/foo",
+  "items": {
+    "$ref": "bar"
+  }
+}
+
+{
+  "$id": "https://example.com/bar",
+  "additionalProperties": { }
+}
+]]>
+                            </artwork>
+                            <postamble>
+                                Here we see that the canonical URI for that "additionalProperties"
+                                subschema is still valid, while the non-canonical URI with the fragment
+                                beginning with "#/items/$ref" now resolves to nothing.
+                            </postamble>
+                        </figure>
                         <t>
-                            Using such URIs is unreliable, and an implementation MAY choose not to
-                            support them.  If an embedded schema were to be replaced with a reference,
-                            then the JSON Pointer fragment URI for that schema relative to its
-                            parent's base URI would then identify the reference, while JSON Pointers
-                            to locations within the formerly embedded resource would become invalid.
-                            <cref>
-                                If the change regarding reference replacement noted in the previous
-                                CREF were to be implemented, the pointer behavior would be more
-                                consistent.  Really it is the pointers to deeper locations within
-                                embedded schemas which should be strongly discouraged and
-                                need not be supported.
-                            </cref>
+                            An implementation MAY choose not to support addressing schemas
+                            by non-canonical URIs.
                         </t>
                         <t>
-                            Examples of such non-canonical URIs, as well as the appropriate
+                            Further examples of such non-canonical URIs, as well as the appropriate
                             canonical URIs to use instead, are provided in section
                             <xref target="idExamples" format="counter"></xref>.
                         </t>
@@ -1779,7 +1822,10 @@
                         As the value of "$ref" and "$recursiveRef" are URI References, this allows
                         the possibility to externalise or divide a schema across multiple files,
                         and provides the ability to validate recursive structures through
-                        self-reference.
+                        self-reference.  For static references with "$ref", the syntax also
+                        allows bundling multiple schema resources into a single document
+                        for ease of distribution, while still indicating which subschema
+                        objects were bundled through a reference.
                     </t>
                     <t>
                         The resolved URI produced by these keywords is not necessarily a network
@@ -1791,12 +1837,31 @@
 
                     <section title='Direct References with "$ref"' anchor="ref">
                         <t>
-                            The "$ref" keyword is used to reference a statically identified schema.
+                            The "$ref" keyword is used to reference a statically identified schema,
+                            or to indicate that its subschema is an embedded resource that has been
+                            logically substituted for a reference URI.
+                        </t>
+                        <t>
+                            The value of the "$ref" property MUST be either a string which is
+                            a URI-Reference, or a schema.
+                        </t>
+                        <t>
+                            If it is a URI-reference, then when resolved against the current URI base,
+                            it produces the URI of the schema object (or boolean) to be applied.
+                        </t>
+                        <t>
+                            When the URI-reference is an absolute-URI, or the schema contains
+                            an "$id" with an absolute-URI, the two forms can be interchanged
+                            without affecting any results (validation or annotation), and
+                            without changing any schema object's canonical URI.  This allows
+                            structuring schema resources either in separate documents or
+                            in a single schema document, with no change in behavior.
                         </t>
                         <t>
-                            The value of the "$ref" property MUST be a string which is a URI Reference.
-                            Resolved against the current URI base, it identifies the URI of a schema
-                            to use.
+                            A reference using a non-absolute-URI value can be replaced with
+                            its target, but preserving results and canonical URIs across
+                            such a transformation is not guaranteed.  Therefore it is
+                            NOT RECOMMENDED for implementations to perform such operations.
                         </t>
                     </section>