ReferenceGrant from Gateway to Secret

conformance/Makefile

@@ -2,7 +2,6 @@ NKG_TAG = edge
 NKG_PREFIX = nginx-kubernetes-gateway
 GATEWAY_CLASS = nginx
 SUPPORTED_FEATURES = HTTPRoute,HTTPRouteQueryParamMatching,HTTPRouteMethodMatching,HTTPRoutePortRedirect,HTTPRouteSchemeRedirect
-EXEMPT_FEATURES = ReferenceGrant
 KIND_KUBE_CONFIG_FOLDER = $${HOME}/.kube/kind
 TAG = latest
 PREFIX = conformance-test-runner
@@ -62,7 +61,7 @@ run-conformance-tests: ## Run conformance tests
 		--image=$(PREFIX):$(TAG) --image-pull-policy=Never \
 		--overrides='{ "spec": { "serviceAccountName": "conformance" }  }' \
 		--restart=Never -- go test -v . -tags conformance -args --gateway-class=$(GATEWAY_CLASS) --debug \
-						        --supported-features=$(SUPPORTED_FEATURES) --exempt-features=$(EXEMPT_FEATURES)
+						        --supported-features=$(SUPPORTED_FEATURES)
 
 .PHONY: cleanup-conformance-tests
 cleanup-conformance-tests: ## Clean up conformance tests fixtures

conformance/tests/conformance-rbac.yaml

@@ -30,18 +30,13 @@ rules:
   - delete
   - get
   - list
-- apiGroups:
-  - gateway.networking.k8s.io
-  resources:
-  - gatewayclasses
-  verbs:
-  - get
-  - list
 - apiGroups:
   - gateway.networking.k8s.io
   resources:
   - gateways
   - httproutes
+  - referencegrants
+  - gatewayclasses
   verbs:
   - create
   - delete

deploy/manifests/rbac.yaml

@@ -38,6 +38,7 @@ rules:
   - gatewayclasses
   - gateways
   - httproutes
+  - referencegrants
   verbs:
   - list
   - watch

docs/gateway-api-compatibility.md

@@ -12,7 +12,7 @@ This document describes which Gateway API resources NGINX Kubernetes Gateway sup
 | [TLSRoute](#tlsroute) | Not supported |
 | [TCPRoute](#tcproute) | Not supported |
 | [UDPRoute](#udproute) | Not supported |
-| [ReferenceGrant](#referencegrant) |  Not supported |
+| [ReferenceGrant](#referencegrant) |  Partially supported |
 | [Custom policies](#custom-policies) | Not supported |
 
 ## Terminology
@@ -148,7 +148,20 @@ Fields:
 
 ### ReferenceGrant
 
-> Status: Not supported.
+> Status: Partially supported.
+
+NKG only supports ReferenceGrants that permit Gateways to reference Secrets. 
+
+Fields:
+* `spec`
+  * `to`
+    * `group` - supported.
+    * `kind` - partially supported. Only `Secret`.
+    * `name`- supported.
+  * `from`
+    * `group` - supported.
+    * `kind` - partially supported. Only `Gateway`.
+    * `namespace`- supported.
 
 ### Custom Policies
 

examples/https-termination/README.md

@@ -1,6 +1,8 @@
 # HTTPS Termination Example
 
-In this example, we expand on the simple [cafe-example](../cafe-example) by adding HTTPS termination to our routes and an HTTPS redirect from port 80 to 443.
+In this example, we expand on the simple [cafe-example](../cafe-example) by adding HTTPS termination to our routes and
+an HTTPS redirect from port 80 to 443. We will also show how you can use a ReferenceGrant to permit your Gateway to
+reference a Secret in a different Namespace.
 
 ## Running the Example
 
@@ -40,37 +42,50 @@ In this example, we expand on the simple [cafe-example](../cafe-example) by addi
 
 ## 3. Configure HTTPS Termination and Routing
 
-1. Create a Secret with a TLS certificate and key:
+1. Create the Namespace `certificate` and a Secret with a TLS certificate and key:
    ```
-   kubectl apply -f cafe-secret.yaml
+   kubectl apply -f certificate-ns-and-cafe-secret.yaml
    ```
 
    The TLS certificate and key in this Secret are used to terminate the TLS connections for the cafe application.
-   **Important**: This certificate and key are for demo purposes only.
+   > **Important**: This certificate and key are for demo purposes only.
+
+1. Create the `ReferenceGrant`:
+   ```
+   kubectl apply -f reference-grant.yaml
+   ```
+
+   This ReferenceGrant allows all Gateways in the `default` namespace to reference the `cafe-secret` Secret in
+   the `certificate` namespace.
 
 1. Create the `Gateway` resource:
    ```
    kubectl apply -f gateway.yaml
    ```
 
    This [Gateway](./gateway.yaml) configures:
-   * `http` listener for HTTP traffic
-   * `https` listener for HTTPS traffic. It terminates TLS connections using the `cafe-secret` we created in step 1.
+    * `http` listener for HTTP traffic
+    * `https` listener for HTTPS traffic. It terminates TLS connections using the `cafe-secret` we created in step 1.
 
 1. Create the `HTTPRoute` resources:
    ```
    kubectl apply -f cafe-routes.yaml
    ```
 
-   To configure HTTPS termination for our cafe application, we will bind our `coffee` and `tea` HTTPRoutes to the `https` listener in [cafe-routes.yaml](./cafe-routes.yaml) using the [`parentReference`](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.ParentReference) field:
+   To configure HTTPS termination for our cafe application, we will bind our `coffee` and `tea` HTTPRoutes to
+   the `https` listener in [cafe-routes.yaml](./cafe-routes.yaml) using
+   the [`parentReference`](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.ParentReference)
+   field:
 
    ```yaml
    parentRefs:
    - name: gateway
      sectionName: https
    ```
 
-   To configure an HTTPS redirect from port 80 to 443, we will bind the special `cafe-tls-redirect` HTTPRoute with a [`HTTPRequestRedirectFilter`](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRequestRedirectFilter) to the `http` listener:
+   To configure an HTTPS redirect from port 80 to 443, we will bind the special `cafe-tls-redirect` HTTPRoute with
+   a [`HTTPRequestRedirectFilter`](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRequestRedirectFilter)
+   to the `http` listener:
 
    ```yaml
    parentRefs:
@@ -80,13 +95,16 @@ In this example, we expand on the simple [cafe-example](../cafe-example) by addi
 
 ## 4. Test the Application
 
-To access the application, we will use `curl` to send requests to the `coffee` and `tea` Services. First, we will access the application over HTTP to test that the HTTPS redirect works. Then we will use HTTPS.
+To access the application, we will use `curl` to send requests to the `coffee` and `tea` Services. First, we will access
+the application over HTTP to test that the HTTPS redirect works. Then we will use HTTPS.
 
 ### 4.1 Test HTTPS Redirect
 
-To test that NGINX sends an HTTPS redirect, we will send requests to the `coffee` and `tea` Services on HTTP port. We will use curl's `--include` option to print the response headers (we are interested in the `Location` header).
+To test that NGINX sends an HTTPS redirect, we will send requests to the `coffee` and `tea` Services on HTTP port. We
+will use curl's `--include` option to print the response headers (we are interested in the `Location` header).
 
 To get a redirect for coffee:
+
 ```
 curl --resolve cafe.example.com:$GW_HTTP_PORT:$GW_IP http://cafe.example.com:$GW_HTTP_PORT/coffee --include
 HTTP/1.1 302 Moved Temporarily
@@ -96,6 +114,7 @@ Location: https://cafe.example.com:443/coffee
 ```
 
 To get a redirect for tea:
+
 ```
 curl --resolve cafe.example.com:$GW_HTTP_PORT:$GW_IP http://cafe.example.com:$GW_HTTP_PORT/tea --include
 HTTP/1.1 302 Moved Temporarily
@@ -104,9 +123,10 @@ Location: https://cafe.example.com:443/tea
 ...
 ```
 
-### 4.2 Access Coffee and Tea 
+### 4.2 Access Coffee and Tea
 
-Now we will access the application over HTTPS. Since our certificate is self-signed, we will use curl's `--insecure` option to turn off certificate verification.
+Now we will access the application over HTTPS. Since our certificate is self-signed, we will use curl's `--insecure`
+option to turn off certificate verification.
 
 To get coffee:
 
@@ -123,3 +143,45 @@ curl --resolve cafe.example.com:$GW_HTTPS_PORT:$GW_IP https://cafe.example.com:$
 Server address: 10.12.0.19:80
 Server name: tea-7cd44fcb4d-xfw2x
 ```
+
+### 4.3 Remove the ReferenceGrant
+
+To restrict access to the `cafe-secret` in the `certificate` Namespace, we can delete the ReferenceGrant we created in
+Step 3:
+
+```
+kubectl delete -f reference-grant.yaml
+```
+
+Now, if we try to access the application over HTTPS, we will get a connection refused error:
+```
+curl --resolve cafe.example.com:$GW_HTTPS_PORT:$GW_IP https://cafe.example.com:$GW_HTTPS_PORT/coffee --insecure -vvv
+...
+curl: (7) Failed to connect to cafe.example.com port 443 after 0 ms: Connection refused
+```
+
+
+You can also check the conditions of the Gateway `https` Listener to verify the that the reference is not permitted:
+
+```
+ Name:                    https
+ Conditions:
+   Last Transition Time:  2023-06-26T20:23:56Z
+   Message:               Certificate ref to secret certificate/cafe-secret not permitted by any ReferenceGrant
+   Observed Generation:   2
+   Reason:                RefNotPermitted
+   Status:                False
+   Type:                  Accepted
+   Last Transition Time:  2023-06-26T20:23:56Z
+   Message:               Certificate ref to secret certificate/cafe-secret not permitted by any ReferenceGrant
+   Observed Generation:   2
+   Reason:                RefNotPermitted
+   Status:                False
+   Type:                  ResolvedRefs
+   Last Transition Time:  2023-06-26T20:23:56Z
+   Message:               Certificate ref to secret certificate/cafe-secret not permitted by any ReferenceGrant
+   Observed Generation:   2
+   Reason:                Invalid
+   Status:                False
+   Type:                  Programmed
+```

examples/https-termination/cafe-secret.yaml → examples/https-termination/certificate-ns-and-cafe-secret.yaml

@@ -1,7 +1,13 @@
 apiVersion: v1
+kind: Namespace
+metadata:
+  name: certificate
+---
+apiVersion: v1
 kind: Secret
 metadata:
   name: cafe-secret
+  namespace: certificate
 type: kubernetes.io/tls
 data:
   tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNzakNDQVpvQ0NRQzdCdVdXdWRtRkNEQU5CZ2txaGtpRzl3MEJBUXNGQURBYk1Sa3dGd1lEVlFRRERCQmoKWVdabExtVjRZVzF3YkdVdVkyOXRNQjRYRFRJeU1EY3hOREl4TlRJek9Wb1hEVEl6TURjeE5ESXhOVEl6T1ZvdwpHekVaTUJjR0ExVUVBd3dRWTJGbVpTNWxlR0Z0Y0d4bExtTnZiVENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFECmdnRVBBRENDQVFvQ2dnRUJBTHFZMnRHNFc5aStFYzJhdnV4Q2prb2tnUUx1ek10U1Rnc1RNaEhuK3ZRUmxIam8KVzFLRnMvQVdlS25UUStyTWVKVWNseis4M3QwRGtyRThwUisxR2NKSE50WlNMb0NEYUlRN0Nhck5nY1daS0o4Qgo1WDNnVS9YeVJHZjI2c1REd2xzU3NkSEQ1U2U3K2Vab3NPcTdHTVF3K25HR2NVZ0VtL1Q1UEMvY05PWE0zZWxGClRPL051MStoMzROVG9BbDNQdTF2QlpMcDNQVERtQ0thaEROV0NWbUJQUWpNNFI4VERsbFhhMHQ5Z1o1MTRSRzUKWHlZWTNtdzZpUzIrR1dYVXllMjFuWVV4UEhZbDV4RHY0c0FXaGRXbElweHlZQlNCRURjczN6QlI2bFF1OWkxZAp0R1k4dGJ3blVmcUVUR3NZdWxzc05qcU95V1VEcFdJelhibHhJZVVDQXdFQUFUQU5CZ2txaGtpRzl3MEJBUXNGCkFBT0NBUUVBcjkrZWJ0U1dzSnhLTGtLZlRkek1ISFhOd2Y5ZXFVbHNtTXZmMGdBdWVKTUpUR215dG1iWjlpbXQKL2RnWlpYVE9hTElHUG9oZ3BpS0l5eVVRZVdGQ2F0NHRxWkNPVWRhbUloOGk0Q1h6QVJYVHNvcUNOenNNLzZMRQphM25XbFZyS2lmZHYrWkxyRi8vblc0VVNvOEoxaCtQeDljY0tpRDZZU0RVUERDRGh1RUtFWXcvbHpoUDJVOXNmCnl6cEJKVGQ4enFyM3paTjNGWWlITmgzYlRhQS82di9jU2lyamNTK1EwQXg4RWpzQzYxRjRVMTc4QzdWNWRCKzQKcmtPTy9QNlA0UFlWNTRZZHMvRjE2WkZJTHFBNENCYnExRExuYWRxamxyN3NPbzl2ZzNnWFNMYXBVVkdtZ2todAp6VlZPWG1mU0Z4OS90MDBHUi95bUdPbERJbWlXMGc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==

examples/https-termination/gateway.yaml

@@ -18,4 +18,4 @@ spec:
       certificateRefs:
       - kind: Secret
         name: cafe-secret
-        namespace: default
+        namespace: certificate

examples/https-termination/reference-grant.yaml

@@ -0,0 +1,14 @@
+apiVersion: gateway.networking.k8s.io/v1beta1
+kind: ReferenceGrant
+metadata:
+  name: allow-default-to-cafe-secret
+  namespace: certificate
+spec:
+  to:
+  - group: ""
+    kind: Secret
+    name: cafe-secret # if you omit this name, then Gateways in default ns can access all Secrets in the certificate ns
+  from:
+  - group: gateway.networking.k8s.io
+    kind: Gateway
+    namespace: default

internal/events/handler.go

@@ -126,6 +126,8 @@ func (h *EventHandlerImpl) propagateUpsert(e *UpsertEvent) {
 		h.cfg.Processor.CaptureUpsertChange(r)
 	case *v1beta1.HTTPRoute:
 		h.cfg.Processor.CaptureUpsertChange(r)
+	case *v1beta1.ReferenceGrant:
+		h.cfg.Processor.CaptureUpsertChange(r)
 	case *apiv1.Service:
 		h.cfg.Processor.CaptureUpsertChange(r)
 	case *apiv1.Namespace:
@@ -149,6 +151,8 @@ func (h *EventHandlerImpl) propagateDelete(e *DeleteEvent) {
 		h.cfg.Processor.CaptureDeleteChange(e.Type, e.NamespacedName)
 	case *v1beta1.HTTPRoute:
 		h.cfg.Processor.CaptureDeleteChange(e.Type, e.NamespacedName)
+	case *v1beta1.ReferenceGrant:
+		h.cfg.Processor.CaptureDeleteChange(e.Type, e.NamespacedName)
 	case *apiv1.Service:
 		h.cfg.Processor.CaptureDeleteChange(e.Type, e.NamespacedName)
 	case *apiv1.Namespace:

internal/manager/manager.go

@@ -121,6 +121,9 @@ func Start(cfg config.Config) error {
 				controller.WithK8sPredicate(k8spredicate.LabelChangedPredicate{}),
 			},
 		},
+		{
+			objectType: &gatewayv1beta1.ReferenceGrant{},
+		},
 	}
 
 	ctx := ctlr.SetupSignalHandler()
@@ -207,6 +210,7 @@ func prepareFirstEventBatchPreparerArgs(
 		&apiv1.NamespaceList{},
 		&discoveryV1.EndpointSliceList{},
 		&gatewayv1beta1.HTTPRouteList{},
+		&gatewayv1beta1.ReferenceGrantList{},
 	}
 
 	if gwNsName == nil {

internal/manager/manager_test.go

@@ -34,6 +34,7 @@ func TestPrepareFirstEventBatchPreparerArgs(t *testing.T) {
 				&discoveryV1.EndpointSliceList{},
 				&gatewayv1beta1.HTTPRouteList{},
 				&gatewayv1beta1.GatewayList{},
+				&gatewayv1beta1.ReferenceGrantList{},
 			},
 		},
 		{
@@ -52,6 +53,7 @@ func TestPrepareFirstEventBatchPreparerArgs(t *testing.T) {
 				&apiv1.NamespaceList{},
 				&discoveryV1.EndpointSliceList{},
 				&gatewayv1beta1.HTTPRouteList{},
+				&gatewayv1beta1.ReferenceGrantList{},
 			},
 		},
 	}

internal/state/change_processor.go

@@ -85,11 +85,12 @@ type ChangeProcessorImpl struct {
 // NewChangeProcessorImpl creates a new ChangeProcessorImpl for the Gateway resource with the configured namespace name.
 func NewChangeProcessorImpl(cfg ChangeProcessorConfig) *ChangeProcessorImpl {
 	clusterStore := graph.ClusterState{
-		GatewayClasses: make(map[types.NamespacedName]*v1beta1.GatewayClass),
-		Gateways:       make(map[types.NamespacedName]*v1beta1.Gateway),
-		HTTPRoutes:     make(map[types.NamespacedName]*v1beta1.HTTPRoute),
-		Services:       make(map[types.NamespacedName]*apiv1.Service),
-		Namespaces:     make(map[types.NamespacedName]*apiv1.Namespace),
+		GatewayClasses:  make(map[types.NamespacedName]*v1beta1.GatewayClass),
+		Gateways:        make(map[types.NamespacedName]*v1beta1.Gateway),
+		HTTPRoutes:      make(map[types.NamespacedName]*v1beta1.HTTPRoute),
+		Services:        make(map[types.NamespacedName]*apiv1.Service),
+		Namespaces:      make(map[types.NamespacedName]*apiv1.Namespace),
+		ReferenceGrants: make(map[types.NamespacedName]*v1beta1.ReferenceGrant),
 	}
 
 	extractGVK := func(obj client.Object) schema.GroupVersionKind {
@@ -119,6 +120,11 @@ func NewChangeProcessorImpl(cfg ChangeProcessorConfig) *ChangeProcessorImpl {
 				store:             newObjectStoreMapAdapter(clusterStore.HTTPRoutes),
 				trackUpsertDelete: true,
 			},
+			{
+				gvk:               extractGVK(&v1beta1.ReferenceGrant{}),
+				store:             newObjectStoreMapAdapter(clusterStore.ReferenceGrants),
+				trackUpsertDelete: true,
+			},
 			{
 				gvk:               extractGVK(&apiv1.Namespace{}),
 				store:             newObjectStoreMapAdapter(clusterStore.Namespaces),
@@ -145,8 +151,8 @@ func NewChangeProcessorImpl(cfg ChangeProcessorConfig) *ChangeProcessorImpl {
 
 			var err error
 			switch o := obj.(type) {
-			// We don't validate GatewayClass, because as of 0.7.1, the webhook doesn't validate it (it only
-			// validates an update that requires the previous version of the resource,
+			// We don't validate GatewayClass or ReferenceGrant, because as of 0.7.1, the webhook doesn't validate them.
+			// It only validates a GatewayClass update that requires the previous version of the resource,
 			// which NKG cannot reliably provide - for example, after NKG restarts).
 			// https://github.com/kubernetes-sigs/gateway-api/blob/v0.7.1/apis/v1beta1/validation/gatewayclass.go#L28
 			case *v1beta1.Gateway: