Skip to content

feat: Add BackendTLS Policy support #1487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Feb 15, 2024
Merged

feat: Add BackendTLS Policy support #1487

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
871ffc1
Add BackendTLS Policy support
ciarams87 Jan 18, 2024
955feeb
Add status updater for backendtlspolicy, move logic
ciarams87 Jan 19, 2024
a2ae465
Review feedback 1
ciarams87 Jan 22, 2024
e726354
Add more validation; handle conflicts
ciarams87 Jan 31, 2024
46d2bbf
Add status updater unit tests
ciarams87 Feb 1, 2024
498c43a
Move example to docs
ciarams87 Feb 1, 2024
105c349
Improve how-to guide
ciarams87 Feb 2, 2024
549cb1d
Review feedback
ciarams87 Feb 7, 2024
df21907
Add change processor test cases
ciarams87 Feb 9, 2024
96269ae
Fix linter
ciarams87 Feb 13, 2024
42af02a
Tidy code, fix panic, update for plus manifest
ciarams87 Feb 14, 2024
949c08e
Add additional test case
ciarams87 Feb 14, 2024
244391d
Add conditional to RBAC and generate new manifests
ciarams87 Feb 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/conformance.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
matrix:
k8s-version: ["1.23.17", "latest"]
nginx-image: [nginx, nginx-plus]
enable-experimental: [true, false]
permissions:
contents: write # needed for uploading release artifacts
steps:
Expand Down Expand Up @@ -148,6 +149,7 @@ jobs:
ngf_tag=${{ steps.ngf-meta.outputs.version }}
if [ ${{ github.event_name }} == "schedule" ]; then export GW_API_VERSION=main; fi
if [ ${{ startsWith(matrix.k8s-version, '1.23') || startsWith(matrix.k8s-version, '1.24') }} == "true" ]; then export INSTALL_WEBHOOK=true; fi
if [ ${{ matrix.enable-experimental }} == "true" ]; then export ENABLE_EXPERIMENTAL=true; fi
make install-ngf-local-no-build${{ matrix.nginx-image == 'nginx-plus' && '-with-plus' || ''}} PREFIX=${ngf_prefix} TAG=${ngf_tag}
working-directory: ./conformance

Expand Down
1 change: 1 addition & 0 deletions .gitleaksignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
68d1f6eb80d23c8650c11629459dd6a06c986ca1:internal/state/graph/graph_test.go:private-key:44
890fddb787ff3560b9b743647a36b649d498ae51:internal/state/graph/secret_test.go:private-key:35
890fddb787ff3560b9b743647a36b649d498ae51:internal/state/change_processor_test.go:private-key:211
internal/mode/static/state/graph/config_maps_test.go:private-key:35
9 changes: 4 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,18 +81,17 @@ generate-crds: ## Generate CRDs and Go types using kubebuilder
go run sigs.k8s.io/controller-tools/cmd/controller-gen object paths=./apis/...

.PHONY: generate-manifests
generate-manifests: generate-manifests-plus ## Generate manifests using Helm.
generate-manifests: ## Generate manifests using Helm.
cp $(CHART_DIR)/crds/* $(MANIFEST_DIR)/crds/
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) $(HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE) -n nginx-gateway | cat $(strip $(MANIFEST_DIR))/namespace.yaml - > $(strip $(MANIFEST_DIR))/nginx-gateway.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) $(HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE) --set nginx.plus=true --set nginx.image.repository=$(NGINX_PLUS_PREFIX) -n nginx-gateway | cat $(strip $(MANIFEST_DIR))/namespace.yaml - > $(strip $(MANIFEST_DIR))/nginx-plus-gateway.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) $(HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE) --set nginxGateway.gwAPIExperimentalFeatures.enable=true -n nginx-gateway | cat $(strip $(MANIFEST_DIR))/namespace.yaml - > $(strip $(MANIFEST_DIR))/nginx-gateway-experimental.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) $(HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE) --set nginxGateway.gwAPIExperimentalFeatures.enable=true --set nginx.plus=true --set nginx.image.repository=$(NGINX_PLUS_PREFIX) -n nginx-gateway | cat $(strip $(MANIFEST_DIR))/namespace.yaml - > $(strip $(MANIFEST_DIR))/nginx-plus-gateway-experimental.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) --set metrics.enable=false -n nginx-gateway -s templates/deployment.yaml > conformance/provisioner/static-deployment.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) -n nginx-gateway -s templates/service.yaml > $(strip $(MANIFEST_DIR))/service/loadbalancer.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) --set service.annotations.'service\.beta\.kubernetes\.io\/aws-load-balancer-type'="nlb" -n nginx-gateway -s templates/service.yaml > $(strip $(MANIFEST_DIR))/service/loadbalancer-aws-nlb.yaml
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) --set service.type=NodePort --set service.externalTrafficPolicy="" -n nginx-gateway -s templates/service.yaml > $(strip $(MANIFEST_DIR))/service/nodeport.yaml

.PHONY: generate-manifests-plus
generate-manifests-plus: ## Generate manifests using Helm for NGINX Plus.
helm template nginx-gateway $(CHART_DIR) $(HELM_TEMPLATE_COMMON_ARGS) $(HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE) --set nginx.plus=true --set nginx.image.repository=$(NGINX_PLUS_PREFIX) -n nginx-gateway | cat $(strip $(MANIFEST_DIR))/namespace.yaml - > $(strip $(MANIFEST_DIR))/nginx-plus-gateway.yaml

.PHONY: crds-release-file
crds-release-file: ## Generate combined crds file for releases
scripts/combine-crds.sh
Expand Down
12 changes: 12 additions & 0 deletions cmd/gateway/commands.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ func createStaticModeCommand() *cobra.Command {
leaderElectionDisableFlag = "leader-election-disable"
leaderElectionLockNameFlag = "leader-election-lock-name"
plusFlag = "nginx-plus"
gwAPIExperimentalFlag = "gateway-api-experimental-features"
)

// flag values
Expand Down Expand Up @@ -95,6 +96,8 @@ func createStaticModeCommand() *cobra.Command {
}

plus bool

gwExperimentalFeatures bool
)

cmd := &cobra.Command{
Expand Down Expand Up @@ -172,6 +175,7 @@ func createStaticModeCommand() *cobra.Command {
Plus: plus,
TelemetryReportPeriod: period,
Version: version,
ExperimentalFeatures: gwExperimentalFeatures,
}

if err := static.StartManager(conf); err != nil {
Expand Down Expand Up @@ -285,6 +289,14 @@ func createStaticModeCommand() *cobra.Command {
"Use NGINX Plus",
)

cmd.Flags().BoolVar(
&gwExperimentalFeatures,
gwAPIExperimentalFlag,
false,
"Enable the experimental features of Gateway API which are supported by NGINX Gateway Fabric. "+
"Requires the Gateway APIs installed from the experimental channel.",
)

return cmd
}

Expand Down
7 changes: 4 additions & 3 deletions conformance/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ CRDS=../deploy/manifests/crds/
STATIC_MANIFEST=provisioner/static-deployment.yaml
PROVISIONER_MANIFEST=provisioner/provisioner.yaml
INSTALL_WEBHOOK ?= false
ENABLE_EXPERIMENTAL ?= false
.DEFAULT_GOAL := help

.PHONY: help
Expand All @@ -37,7 +38,7 @@ create-kind-cluster: ## Create a kind cluster

.PHONY: update-ngf-manifest
update-ngf-manifest: ## Update the NGF deployment manifest image names and imagePullPolicies
cd .. && make generate-manifests HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE="--set nginxGateway.kind=skip" HELM_TEMPLATE_COMMON_ARGS="--set nginxGateway.image.repository=$(PREFIX) --set nginxGateway.image.tag=$(TAG) --set nginxGateway.image.pullPolicy=Never --set nginx.image.repository=$(NGINX_PREFIX) --set nginx.image.tag=$(TAG) --set nginx.image.pullPolicy=Never" && cd -
cd .. && make generate-manifests HELM_TEMPLATE_EXTRA_ARGS_FOR_ALL_MANIFESTS_FILE="--set nginxGateway.kind=skip" HELM_TEMPLATE_COMMON_ARGS="--set nginxGateway.image.repository=$(PREFIX) --set nginxGateway.image.tag=$(TAG) --set nginxGateway.image.pullPolicy=Never --set nginx.image.repository=$(NGINX_PREFIX) --set nginx.image.tag=$(TAG) --set nginx.image.pullPolicy=Never --set nginxGateway.experimentalFeatures.enable=$(ENABLE_EXPERIMENTAL)" && cd -

.PHONY: update-ngf-manifest-with-plus
update-ngf-manifest-with-plus: ## Update the NGF deployment manifest image names and imagePullPolicies including nginx-plus
Expand All @@ -61,7 +62,7 @@ load-images-with-plus: ## Load NGF and NGINX Plus images on configured kind clus

.PHONY: prepare-ngf-dependencies
prepare-ngf-dependencies: update-ngf-manifest ## Install NGF dependencies on configured kind cluster
./scripts/install-gateway.sh $(GW_API_VERSION) $(INSTALL_WEBHOOK)
./scripts/install-gateway.sh $(GW_API_VERSION) $(INSTALL_WEBHOOK) $(ENABLE_EXPERIMENTAL)
kubectl apply -f $(CRDS)
kubectl apply -f $(NGF_MANIFEST)

Expand Down Expand Up @@ -118,7 +119,7 @@ uninstall-ngf: uninstall-k8s-components undo-manifests-update ## Uninstall NGF o
.PHONY: uninstall-k8s-components
uninstall-k8s-components: ## Uninstall installed components on configured kind cluster
-kubectl delete -f $(NGF_MANIFEST)
./scripts/uninstall-gateway.sh $(GW_API_VERSION) $(INSTALL_WEBHOOK)
./scripts/uninstall-gateway.sh $(GW_API_VERSION) $(INSTALL_WEBHOOK) $(ENABLE_EXPERIMENTAL)
kubectl delete clusterrole nginx-gateway-provisioner
kubectl delete clusterrolebinding nginx-gateway-provisioner

Expand Down
41 changes: 24 additions & 17 deletions conformance/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,23 +44,24 @@ update-ngf-manifest Update the NGF deployment manifest image na

**Note:** The following variables are configurable when running the below `make` commands:

| Variable | Default | Description |
|----------------------|---------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
| CONFORMANCE_TAG | latest | The tag for the conformance test image |
| CONFORMANCE_PREFIX | conformance-test-runner | The prefix for the conformance test image |
| TAG | edge | The tag for the locally built NGF image |
| PREFIX | nginx-gateway-fabric | The prefix for the locally built NGF image |
| GW_API_VERSION | 1.0.0 | Tag for the Gateway API version to check out. Set to `main` to get the latest version |
| KIND_IMAGE | Latest kind image, as defined in the tests/Dockerfile | The kind image to use |
| KIND_KUBE_CONFIG | ~/.kube/kind/config | The location of the kubeconfig |
| GATEWAY_CLASS | nginx | The gateway class that should be used for the tests |
| SUPPORTED_FEATURES | HTTPRoute,HTTPRouteQueryParamMatching, HTTPRouteMethodMatching,HTTPRoutePortRedirect, HTTPRouteSchemeRedirect | The supported features that should be tested by the conformance tests. Ensure the list is comma separated with no spaces. |
| EXEMPT_FEATURES | ReferenceGrant | The features that should not be tested by the conformance tests |
| NGF_MANIFEST | ../deploy/manifests/nginx-gateway.yaml | The location of the NGF manifest |
| SERVICE_MANIFEST | ../deploy/manifests/service/nodeport.yaml | The location of the NGF Service manifest |
| STATIC_MANIFEST | provisioner/static-deployment.yaml | The location of the NGF static deployment manifest |
| PROVISIONER_MANIFEST | provisioner/provisioner.yaml | The location of the NGF provisioner manifest |
| INSTALL_WEBHOOK | false | Install the Gateway API Validating Webhook. Necessary for Kubernetes versions < 1.25. |
| Variable | Default | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| CONFORMANCE_TAG | latest | The tag for the conformance test image |
| CONFORMANCE_PREFIX | conformance-test-runner | The prefix for the conformance test image |
| TAG | edge | The tag for the locally built NGF image |
| PREFIX | nginx-gateway-fabric | The prefix for the locally built NGF image |
| GW_API_VERSION | 1.0.0 | Tag for the Gateway API version to check out. Set to `main` to get the latest version |
| KIND_IMAGE | Latest kind image, as defined in the tests/Dockerfile | The kind image to use |
| KIND_KUBE_CONFIG | ~/.kube/kind/config | The location of the kubeconfig |
| GATEWAY_CLASS | nginx | The gateway class that should be used for the tests |
| SUPPORTED_FEATURES | HTTPRoute,HTTPRouteQueryParamMatching, HTTPRouteMethodMatching,HTTPRoutePortRedirect, HTTPRouteSchemeRedirect | The supported features that should be tested by the conformance tests. Ensure the list is comma separated with no spaces. |
| EXEMPT_FEATURES | ReferenceGrant | The features that should not be tested by the conformance tests |
| NGF_MANIFEST | ../deploy/manifests/nginx-gateway.yaml | The location of the NGF manifest |
| SERVICE_MANIFEST | ../deploy/manifests/service/nodeport.yaml | The location of the NGF Service manifest |
| STATIC_MANIFEST | provisioner/static-deployment.yaml | The location of the NGF static deployment manifest |
| PROVISIONER_MANIFEST | provisioner/provisioner.yaml | The location of the NGF provisioner manifest |
| INSTALL_WEBHOOK | false | Install the Gateway API Validating Webhook. Necessary for Kubernetes versions < 1.25. |
| ENABLE_EXPERIMENTAL | false | Enable experimental features. Installs the Gateway APIs from the experimental channel and enables any supported experimental features in NGF. |

### Step 1 - Create a kind Cluster

Expand All @@ -85,6 +86,12 @@ make create-kind-cluster KIND_IMAGE=kindest/node:v1.27.3
```

> Otherwise, the latest stable version will be used by default.
> Additionally, if you want to run conformance tests with experimental features enabled, set the following
> environment variable before deploying NGF:

```bash
export ENABLE_EXPERIMENTAL=true
```

#### *Option 1* Build and install NGINX Gateway Fabric from local to configured kind cluster

Expand Down
12 changes: 10 additions & 2 deletions conformance/scripts/install-gateway.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,22 @@ if [ $1 == "main" ]; then
temp_dir=$(mktemp -d)
cd ${temp_dir}
curl -s https://codeload.github.com/kubernetes-sigs/gateway-api/tar.gz/main | tar -xz --strip=2 gateway-api-main/config
kubectl apply -f crd/standard
if [ $3 == "true" ]; then
kubectl apply -f crd/experimental
else
kubectl apply -f crd/standard
fi
if [ $2 == "true" ]; then
kubectl apply -f webhook
kubectl wait --for=condition=available --timeout=60s deployment gateway-api-admission-server -n gateway-system
fi
rm -rf ${temp_dir}
else
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v$1/standard-install.yaml
if [ $3 == "true" ]; then
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v$1/experimental-install.yaml
else
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v$1/standard-install.yaml
fi
if [ $2 == "true" ]; then
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v$1/webhook-install.yaml
kubectl wait --for=condition=available --timeout=60s deployment gateway-api-admission-server -n gateway-system
Expand Down
Loading