Skip to content

Commit b2d2635

Browse files
miledxzkate-osborn
miledxz
and
kate-osborn
committed
Document how Gateway API resources are validated (nginx#520)
This commits documents how Gateway API resources are validated. Fixes nginx#364 Co-authored-by: Kate Osborn <50597707+kate-osborn@users.noreply.github.com>
1 parent 65a4325 commit b2d2635

File tree

1 file changed

+138
-0
lines changed

1 file changed

+138
-0
lines changed

‎docs/resource-validation.md

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# Gateway API Resource Validation
2+
3+
This document describes how NGINX Kubernetes Gateway (NKG) validates Gateway API resources.
4+
5+
## Overview
6+
7+
There are several reasons why NKG validates Gateway API resources:
8+
9+
- *Robustness*: to gracefully handle invalid resources.
10+
- *Security*: to prevent malicious input from propagating to the NGINX configuration.
11+
- *Correctness*: to conform to the Gateway API specification for handling invalid resources.
12+
13+
Ultimately, the goal is to ensure that NGINX continues to handle traffic even if invalid Gateway API resources were
14+
created.
15+
16+
A Gateway API resource (a new resource or an update for the existing one) is validated by the following steps:
17+
18+
1. OpenAPI schema validation by the Kubernetes API server.
19+
2. Webhook validation by the Gateway API webhook.
20+
3. Webhook validation by NKG.
21+
4. Validation by NKG.
22+
23+
To confirm that a resource is valid and accepted by NKG, check that the `Accepted` condition in the resource status
24+
has the Status field set to `True`. For example, in a status of a valid HTTPRoute, if NKG accepts a parentRef,
25+
the status of that parentRef will look like this:
26+
```
27+
Status:
28+
Parents:
29+
Conditions:
30+
Last Transition Time: 2023-03-30T23:18:00Z
31+
Message: The route is accepted
32+
Observed Generation: 2
33+
Reason: Accepted
34+
Status: True
35+
Type: Accepted
36+
Controller Name: k8s-gateway.nginx.org/nginx-gateway-controller
37+
Parent Ref:
38+
Group: gateway.networking.k8s.io
39+
Kind: Gateway
40+
Name: gateway
41+
Namespace: default
42+
Section Name: http
43+
```
44+
45+
> Make sure the reported observed generation is the same as the resource generation.
46+
47+
The remaining part of this document describes each step in detail with examples of how validation errors are reported.
48+
49+
### Step 1 - OpenAPI Scheme Validation by Kubernetes API Server
50+
51+
The Kubernetes API server validates Gateway API resources against the OpenAPI schema embedded in the Gateway API CRDs.
52+
For example, if you create an HTTPRoute with an invalid hostname `cafe.!@#$%example.com`, the API server will reject it
53+
with the following error:
54+
55+
```
56+
kubectl apply -f coffee-route.yaml
57+
The HTTPRoute "coffee" is invalid: spec.hostnames[0]: Invalid value: "cafe.!@#$%example.com": spec.hostnames[0] in body should match '^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$'
58+
```
59+
60+
> While unlikely, bypassing this validation step is possible if the Gateway API CRDs are modified to remove the validation.
61+
> If this happens, Step 4 will reject any invalid values (from NGINX perspective).
62+
63+
### Step 2 - Webhook Validation by Gateway API Webhook
64+
65+
The Gateway API comes with a validating webhook which is enabled by default in the Gateway API installation manifests.
66+
It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation. For example, if
67+
you create a Gateway resource with a TCP listener that configures a hostname, the webhook will reject it with the
68+
following error:
69+
70+
```
71+
kubectl apply -f prod-gateway.yaml
72+
Error from server: error when creating "prod-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
73+
```
74+
75+
> Bypassing this validation step is possible if the webhook is not running in the cluster.
76+
> If this happens, Step 3 will reject the invalid values.
77+
78+
### Step 3 - Webhook validation by NKG
79+
80+
The previous step relies on the Gateway API webhook running in the cluster. To ensure that the resources are validated
81+
with the webhook validation rules, even if the webhook is not running, NKG performs the same validation. However, NKG
82+
performs the validation *after* the Kubernetes API server accepts the resource.
83+
84+
Below is an example of how NKG rejects an invalid resource (a Gateway resource with a TCP listener that configures a
85+
hostname) with a Kubernetes event:
86+
87+
```
88+
kubectl describe gateway prod-gateway
89+
. . .
90+
Events:
91+
Type Reason Age From Message
92+
---- ------ ---- ---- -------
93+
Warning Rejected 6s nginx-kubernetes-gateway-nginx the resource failed webhook validation, however the Gateway API webhook failed to reject it with the error; make sure the webhook is installed and running correctly; validation error: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP; NKG will delete any existing NGINX configuration that corresponds to the resource
94+
```
95+
96+
> This validation step always runs and cannot be bypassed.
97+
98+
> NKG will ignore any resources that fail the webhook validation, like in the example above.
99+
> If the resource previously existed, NKG will remove any existing NGINX configuration for that resource.
100+
101+
### Step 4 - Validation by NKG
102+
103+
This step catches the following cases of invalid values:
104+
105+
* Valid values from the Gateway API perspective but not supported by NKG yet. For example, a feature in an
106+
HTTPRoute routing rule. Note: for the list of supported features,
107+
see [Gateway API Compatibility](gateway-api-compatibility.md) doc.
108+
* Valid values from the Gateway API perspective, but invalid for NGINX, because NGINX has stricter validation
109+
requirements for certain fields. Such values will cause NGINX to fail to reload or operate erroneously.
110+
* Invalid values (both from the Gateway API and NGINX perspectives) that were not rejected because Step 1 was bypassed.
111+
Similarly to the previous case, such values will cause NGINX to fail to reload or operate erroneously.
112+
* Malicious values that inject unrestricted NGINX config into the NGINX configuration (similar to an SQL injection
113+
attack).
114+
115+
Below is an example of how NGK rejects an invalid resource. The validation error is reported via the status:
116+
117+
```
118+
kubectl describe httproutes.gateway.networking.k8s.io coffee
119+
. . .
120+
Status:
121+
Parents:
122+
Conditions:
123+
Last Transition Time: 2023-03-30T22:37:53Z
124+
Message: All rules are invalid: spec.rules[0].matches[0].method: Unsupported value: "CONNECT": supported values: "DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"
125+
Observed Generation: 1
126+
Reason: UnsupportedValue
127+
Status: False
128+
Type: Accepted
129+
Controller Name: k8s-gateway.nginx.org/nginx-gateway-controller
130+
Parent Ref:
131+
Group: gateway.networking.k8s.io
132+
Kind: Gateway
133+
Name: prod-gateway
134+
Namespace: default
135+
Section Name: http
136+
```
137+
138+
> This validation step always runs and cannot be bypassed.

0 commit comments

Comments
 (0)