From 7efd24a4d9b074f40a845bd2a5429aa2bc199975 Mon Sep 17 00:00:00 2001 From: dervoeti Date: Mon, 15 Jan 2024 23:41:43 +0100 Subject: [PATCH 1/6] wip --- ...nning_stackable_in_an_airgapped_environment.adoc | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc diff --git a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc new file mode 100644 index 000000000..028efc6b1 --- /dev/null +++ b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc @@ -0,0 +1,13 @@ += Running Stackable in an air-gapped environment + +The main challenge with running Stackable in an air-gapped environment is how to get the artifacts (container images and Helm charts) into the environment. There are a few ways to do this: + +* Mirror our images and Helm charts to a registry (e.g. https://goharbor.io/[Harbor]) in the air-gapped environment. This is the recommended approach. If you use Harbor, you can setup https://goharbor.io/docs/latest/administration/configuring-replication/[replication] to replicate our project `sdp` from our registry `oci.stackable.tech` (and maybe `sdp-charts` as well, if you also need access to Stackable's Helm charts from within your air-gapped environment). +* If possible, setup a reverse proxy to Stackable's registry with internet connection that is reachable from all nodes in your air-gapped environment. +* Download our images (e.g. using `docker save`) on a machine with internet access, copy them onto the nodes in your air-gapped environment and load them (e.g. using `ctr images import`). Then render the Helm charts using the `--template` flag, copy the rendered YAML files your air-gapped environment and apply them. + +In the first two scenarios, you also need to make sure that the nodes load the images from the registry (or the registry mirror). Again, there are a several ways to do this: + +* Provide the URL to your registry in the CRDs (see https://docs.stackable.tech/home/nightly/concepts/product-image-selection#_custom_docker_registry["Custom docker registry"]) and in the values of the Helm charts of the operators (`helm template --set image.repository="my.custom.registry" ...`). !! TODO: explain how to do this for stackablectl !! +* If you use `containerd` as your container runtime, patch the `containerd` config on every node to use the mirrored registry instead of `oci.stackable.tech`, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. !! TODO: provide example !! +* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (e.g. `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:concepts:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file From 063975fecc104d1e3ed75582c609ceaf8a9cc309 Mon Sep 17 00:00:00 2001 From: dervoeti Date: Tue, 16 Jan 2024 15:08:47 +0100 Subject: [PATCH 2/6] wip --- ...unning_stackable_in_an_airgapped_environment.adoc | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc index 028efc6b1..8260a41b8 100644 --- a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc +++ b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc @@ -2,12 +2,12 @@ The main challenge with running Stackable in an air-gapped environment is how to get the artifacts (container images and Helm charts) into the environment. There are a few ways to do this: -* Mirror our images and Helm charts to a registry (e.g. https://goharbor.io/[Harbor]) in the air-gapped environment. This is the recommended approach. If you use Harbor, you can setup https://goharbor.io/docs/latest/administration/configuring-replication/[replication] to replicate our project `sdp` from our registry `oci.stackable.tech` (and maybe `sdp-charts` as well, if you also need access to Stackable's Helm charts from within your air-gapped environment). -* If possible, setup a reverse proxy to Stackable's registry with internet connection that is reachable from all nodes in your air-gapped environment. -* Download our images (e.g. using `docker save`) on a machine with internet access, copy them onto the nodes in your air-gapped environment and load them (e.g. using `ctr images import`). Then render the Helm charts using the `--template` flag, copy the rendered YAML files your air-gapped environment and apply them. +* Mirror our images and Helm charts to a registry (e.g. https://goharbor.io/[Harbor]) in the air-gapped environment. This is the recommended approach. If you use Harbor and the instance running Harbor has internet connection, you can setup https://goharbor.io/docs/latest/administration/configuring-replication/[replication] to replicate our project `sdp` from our registry `oci.stackable.tech` (and optionally `sdp-charts` as well, if you also need access to Stackable's Helm charts from within your air-gapped environment). Otherwise you need to transfer the relevant images from the `sdp` project in `oci.stackable.tech` to your registry manually. +* If possible, setup a reverse proxy to Stackable's registry on a node with internet connection that is reachable from all nodes in your air-gapped environment. +* Download our images (e.g. using `docker save`) on a machine with internet access, copy them onto the nodes in your air-gapped environment and load them (e.g. using `ctr images import`). Then render the Helm charts using the `helm template` subcommand, copy the rendered YAML files your air-gapped environment and apply them. -In the first two scenarios, you also need to make sure that the nodes load the images from the registry (or the registry mirror). Again, there are a several ways to do this: +In the first two scenarios, you need to make sure that the nodes load the images from the registry mirror. Again, there are a several ways to do this: -* Provide the URL to your registry in the CRDs (see https://docs.stackable.tech/home/nightly/concepts/product-image-selection#_custom_docker_registry["Custom docker registry"]) and in the values of the Helm charts of the operators (`helm template --set image.repository="my.custom.registry" ...`). !! TODO: explain how to do this for stackablectl !! -* If you use `containerd` as your container runtime, patch the `containerd` config on every node to use the mirrored registry instead of `oci.stackable.tech`, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. !! TODO: provide example !! +* Specify the image repository in the CRDs (see https://docs.stackable.tech/home/nightly/concepts/product-image-selection#_custom_docker_registry["Custom docker registry"]) and in the values of the Helm charts of the operators (`helm install --set image.repository="my.custom.registry/stackable/nifi-operator" ...`). +* If you use `containerd` as your container runtime, patch the `containerd` config on every node to use the mirrored registry instead of `oci.stackable.tech`, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. !! TODO provide example !! * Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (e.g. `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:concepts:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file From d6863107b5f1d69bbd0f57f02a555a6553bfac3e Mon Sep 17 00:00:00 2001 From: dervoeti Date: Thu, 18 Jan 2024 12:09:51 +0100 Subject: [PATCH 3/6] Provide more details --- ...stackable_in_an_airgapped_environment.adoc | 34 +++++++++++++++---- 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc index 8260a41b8..6da0bde38 100644 --- a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc +++ b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc @@ -2,12 +2,34 @@ The main challenge with running Stackable in an air-gapped environment is how to get the artifacts (container images and Helm charts) into the environment. There are a few ways to do this: -* Mirror our images and Helm charts to a registry (e.g. https://goharbor.io/[Harbor]) in the air-gapped environment. This is the recommended approach. If you use Harbor and the instance running Harbor has internet connection, you can setup https://goharbor.io/docs/latest/administration/configuring-replication/[replication] to replicate our project `sdp` from our registry `oci.stackable.tech` (and optionally `sdp-charts` as well, if you also need access to Stackable's Helm charts from within your air-gapped environment). Otherwise you need to transfer the relevant images from the `sdp` project in `oci.stackable.tech` to your registry manually. -* If possible, setup a reverse proxy to Stackable's registry on a node with internet connection that is reachable from all nodes in your air-gapped environment. +* Mirror our images and Helm charts to a registry (e.g. https://goharbor.io/[Harbor]) in the air-gapped environment. This is the recommended approach. If you use Harbor and the instance running Harbor has internet connection, you can setup https://goharbor.io/docs/latest/administration/configuring-replication/[replication] to replicate our project `sdp` from our registry `oci.stackable.tech` (and optionally `sdp-charts` as well, if you also need access to Stackable's Helm charts from within your air-gapped environment). Otherwise, you need to transfer the images that are relevant to you from the `sdp` project in `oci.stackable.tech` to your registry manually. +* If possible, setup a reverse proxy to Stackable's registry on a node with internet connection that is reachable from all nodes within your air-gapped environment. You could, for example, use https://distribution.github.io/distribution/[distribution] for this. Here's a command to spin up a pull-through cache to `oci.stackable.tech` on port 5001: `docker run -d --name proxy-stackable -p 5001:5000 --restart=always -e REGISTRY_PROXY_REMOTEURL=https://oci.stackable.tech registry:2`. The registry is now available on localhost:5001 via HTTP. Once an image has been loaded, it will be cached by the proxy. * Download our images (e.g. using `docker save`) on a machine with internet access, copy them onto the nodes in your air-gapped environment and load them (e.g. using `ctr images import`). Then render the Helm charts using the `helm template` subcommand, copy the rendered YAML files your air-gapped environment and apply them. -In the first two scenarios, you need to make sure that the nodes load the images from the registry mirror. Again, there are a several ways to do this: +In the first two scenarios, you need to make sure that the nodes load the images from your local registry mirror. Again, there are a several ways to do this: -* Specify the image repository in the CRDs (see https://docs.stackable.tech/home/nightly/concepts/product-image-selection#_custom_docker_registry["Custom docker registry"]) and in the values of the Helm charts of the operators (`helm install --set image.repository="my.custom.registry/stackable/nifi-operator" ...`). -* If you use `containerd` as your container runtime, patch the `containerd` config on every node to use the mirrored registry instead of `oci.stackable.tech`, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. !! TODO provide example !! -* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (e.g. `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:concepts:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file +* Specify the image repository in the CRDs (see https://docs.stackable.tech/home/nightly/concepts/product-image-selection#_custom_docker_registry["Custom docker registry"]) and in the values of the Helm charts when installing the operators (`helm install --set image.repository="my.custom.registry/stackable/nifi-operator" ...`). +* If you use `containerd` as your container runtime, you can patch the `containerd` config on every node to use the mirrored registry instead of `oci.stackable.tech`. + +Example: Create the file `/etc/containerd/certs.d/oci.stackable.tech/hosts.toml` with the following contents: ++ +[source,toml] +---- +server = "http://10.7.228.12" + +[host."10.7.228.12"] + ca = "/etc/pki/tls/certs/mycert.crt" +---- ++ +Note that `/etc/pki/tls/certs/mycert.crt` has to contain the IP (`10.7.228.12` in the example) as SAN. + +Modify your containerd config (usually located at `/etc/containerd/config.toml`): ++ +[source,toml] +---- +[plugins."io.containerd.grpc.v1.cri".registry] + config_path = "/etc/containerd/certs.d" +---- +Then restart the `containerd` service. ++ +For further information, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. + +* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:concepts:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file From e1c8cbb96bfda1978c1212726ae140691b097fbc Mon Sep 17 00:00:00 2001 From: dervoeti Date: Thu, 18 Jan 2024 13:22:14 +0100 Subject: [PATCH 4/6] Fixed links / formatting --- modules/tutorials/nav.adoc | 1 + .../running_stackable_in_an_airgapped_environment.adoc | 6 ++---- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/modules/tutorials/nav.adoc b/modules/tutorials/nav.adoc index 669405578..e36877831 100644 --- a/modules/tutorials/nav.adoc +++ b/modules/tutorials/nav.adoc @@ -2,3 +2,4 @@ ** xref:authentication_with_openldap.adoc[] ** xref:logging-vector-aggregator.adoc[] ** xref:enabling_verification_of_image_signatures.adoc[] +** xref:running_stackable_in_an_airgapped_environment.adoc[] diff --git a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc index 6da0bde38..e64670d11 100644 --- a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc +++ b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc @@ -28,8 +28,6 @@ Modify your containerd config (usually located at `/etc/containerd/config.toml`) [plugins."io.containerd.grpc.v1.cri".registry] config_path = "/etc/containerd/certs.d" ---- -Then restart the `containerd` service. -+ -For further information, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. +Then restart the `containerd` service. For further information, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. -* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:concepts:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file +* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:tutorials:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file From ea99e1342c971fd64b07aed60de1248040284a2a Mon Sep 17 00:00:00 2001 From: dervoeti Date: Thu, 18 Jan 2024 17:14:18 +0100 Subject: [PATCH 5/6] Clarified containerd config example --- .../running_stackable_in_an_airgapped_environment.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc index e64670d11..d5bfa32fd 100644 --- a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc +++ b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc @@ -10,17 +10,17 @@ In the first two scenarios, you need to make sure that the nodes load the images * Specify the image repository in the CRDs (see https://docs.stackable.tech/home/nightly/concepts/product-image-selection#_custom_docker_registry["Custom docker registry"]) and in the values of the Helm charts when installing the operators (`helm install --set image.repository="my.custom.registry/stackable/nifi-operator" ...`). * If you use `containerd` as your container runtime, you can patch the `containerd` config on every node to use the mirrored registry instead of `oci.stackable.tech`. + -Example: Create the file `/etc/containerd/certs.d/oci.stackable.tech/hosts.toml` with the following contents: +Example: Let's assume you have a registry mirror running on `10.7.228.12`, reachable via HTTPS on port 443 using a self signed certificate. Now copy the certificate over to your Kubernetes node, in this example we'll place it in the `/etc/pki/tls/certs` folder. +Create the file `/etc/containerd/certs.d/oci.stackable.tech/hosts.toml` on the node, with the following contents: + [source,toml] ---- -server = "http://10.7.228.12" - [host."10.7.228.12"] ca = "/etc/pki/tls/certs/mycert.crt" + capabilities = ["pull", "resolve"] ---- + -Note that `/etc/pki/tls/certs/mycert.crt` has to contain the IP (`10.7.228.12` in the example) as SAN. + +Since `containerd` defaults to using HTTPS on port 443, we don't have to specify the protocol and port and can just provide the IP. Note that the certificate (`mycert.crt`) has to contain the IP (`10.7.228.12`) as SAN. + Modify your containerd config (usually located at `/etc/containerd/config.toml`): + [source,toml] @@ -28,6 +28,6 @@ Modify your containerd config (usually located at `/etc/containerd/config.toml`) [plugins."io.containerd.grpc.v1.cri".registry] config_path = "/etc/containerd/certs.d" ---- -Then restart the `containerd` service. For further information, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. +Then restart the `containerd` service. Now `containerd` will fetch all images that would normally be fetched from `oci.stackable.tech` from `10.7.228.12` instead. The registry host name is determined by the path `hosts.toml` is located in, so other registry hosts are not affected. For further information, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. * Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:tutorials:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file From 4f2251f69e3fd732d99093dd8e00e307fb9edb21 Mon Sep 17 00:00:00 2001 From: dervoeti Date: Tue, 23 Jan 2024 16:57:03 +0100 Subject: [PATCH 6/6] Made link to signature verification tutorial more prominent --- .../pages/running_stackable_in_an_airgapped_environment.adoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc index d5bfa32fd..6ba54f0fe 100644 --- a/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc +++ b/modules/tutorials/pages/running_stackable_in_an_airgapped_environment.adoc @@ -30,4 +30,6 @@ Modify your containerd config (usually located at `/etc/containerd/config.toml`) ---- Then restart the `containerd` service. Now `containerd` will fetch all images that would normally be fetched from `oci.stackable.tech` from `10.7.228.12` instead. The registry host name is determined by the path `hosts.toml` is located in, so other registry hosts are not affected. For further information, see https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration["Registry configuration"]. -* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). Learn more about image signature verification xref:tutorials:enabling_verification_of_image_signatures.adoc[here]. \ No newline at end of file +* Add an alias for `oci.stackable.tech` to the `/etc/hosts` file on every node (like `10.7.228.12 oci.stackable.tech`), issue a self-signed certificate for `oci.stackable.tech` to your registry and add the certificate to the trusted certificates on every node. Note that if you also want to enforce signature checks for Stackable's images via a policy controller, you will need to add this host alias to the Pod of the policy controller as well (and make it trust the certificate). + +If you want to know how to verify image signatures in an air-gapped environment, check out our documentation about xref:tutorials:enabling_verification_of_image_signatures.adoc[image signature verification]. \ No newline at end of file