Skip to content

chore: update documentation to oci registry #716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 7 additions & 12 deletions modules/ROOT/pages/getting-started.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -44,23 +44,18 @@ stackablectl release install -i commons -i secret -i listener -i zookeeper -i ka
====
NOTE: These examples assume Helm version 3. They will not work with Helm version 2.

Add the stackable-stable Helm Chart repository:

[source,bash]
----
helm repo add stackable-stable https://repo.stackable.tech/repository/helm-stable/
----
`helm repo` subcommands are not supported for OCI registries. The operators are installed directly, without adding the Helm Chart repository first.

Install the operators:

[source,bash]
----
helm install zookeeper-operator stackable-stable/zookeeper-operator --version=24.11.1
helm install kafka-operator stackable-stable/kafka-operator --version=24.11.1
helm install secret-operator stackable-stable/secret-operator --version=24.11.1
helm install listener-operator stackable-stable/listener-operator --version=24.11.1
helm install commons-operator stackable-stable/commons-operator --version=24.11.1
helm install nifi-operator stackable-stable/nifi-operator --version=24.11.1
helm install zookeeper-operator oci://oci.stackable.tech/sdp-charts/zookeeper-operator --version=24.11.1
helm install kafka-operator oci://oci.stackable.tech/sdp-charts/kafka-operator --version=24.11.1
helm install secret-operator oci://oci.stackable.tech/sdp-charts/secret-operator --version=24.11.1
helm install listener-operator oci://oci.stackable.tech/sdp-charts/listener-operator --version=24.11.1
helm install commons-operator oci://oci.stackable.tech/sdp-charts/commons-operator --version=24.11.1
helm install nifi-operator oci://oci.stackable.tech/sdp-charts/nifi-operator --version=24.11.1
----
====

Expand Down
6 changes: 3 additions & 3 deletions modules/ROOT/pages/product-information.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -54,15 +54,15 @@ Stackable components.
=== Operators and products

All operators are supplied in container images. The products are also deployed in container images. The docker images
are available for download here: https://repo.stackable.tech/#browse/browse:docker[]
are available for download here: https://oci.stackable.tech/[]. Information on how to browse the images can be found xref:contributor:project-overview.adoc#docker-images[here].

Stackable supports installing the Operators via https://helm.sh/[Helm] or with <<stackablectl>>. Every Operator includes
installation instructions in the Getting started guide.

==== Helm Charts

The Helm Charts can be found here: https://repo.stackable.tech/#browse/browse:helm-stable[] Using the Helm Charts
requires Helm version 3 or above.
The Helm Charts can be found here: https://oci.stackable.tech/api/v2.0/projects/sdp-charts[] Using the Helm Charts
requires Helm version 3 or above. Information on how to browse the OCI repository for Helm Charts is described xref:contributor:project-overview.adoc#product-artifacts[here].


[#stackablectl-installation]
Expand Down
2 changes: 1 addition & 1 deletion modules/concepts/pages/container-images.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
= Container images
:ubi: https://catalog.redhat.com/software/base-images
:stackable-image-registry: https://repo.stackable.tech/#browse/browse
:stackable-image-registry: https://oci.stackable.tech/
:stackable-sboms: https://sboms.stackable.tech/
:description: Overview of Stackable’s container images, including structure, multi-platform support, and why upstream images are not used.

Expand Down
8 changes: 4 additions & 4 deletions modules/concepts/pages/product-image-selection.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ At the bottom of this page in the <<_common_scenarios, common scenarios>> sectio

== Stackable provided images

If your Kubernetes cluster has internet access, the easiest way is to use the publicly available images from the https://docker.stackable.tech/[Image registry hosted by Stackable] (as has been noted at the beginning of the page):
If your Kubernetes cluster has internet access, the easiest way is to use the publicly available images from the https://oci.stackable.tech/[Image registry hosted by Stackable] (as has been noted at the beginning of the page):

[source,yaml]
----
Expand All @@ -99,7 +99,7 @@ Security updates within a release line will result in patch version bumps in the
If you don't specify the Stackable version, the operator will use its own version, e.g. `23.7.0`.
When using a nightly operator or a `pr` version, it will use the nightly `0.0.0-dev` image.

All the available images (with their product and stackable version) can be found in our https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable[docker repository].
All the available images (with their product and stackable version) can be found in our https://oci.stackable.tech/api/v2.0/projects/sdp[OCI registry]. Information on how to browse the registry can be found xref:contributor:project-overview.adoc#docker-images[here]).

== Custom docker registry

Expand All @@ -117,7 +117,7 @@ spec:
repo: my.corp/myteam/stackable
----

This will change the image from the default Stackable repository `docker.stackable.tech/stackable/kafka:3.3.1-stackable23.7.0` to `my.corp/myteam/stackable/kafka:3.3.1-stackable23.7.0`.
This will change the image from the default Stackable repository `oci.stackable.tech/sdp/kafka:3.3.1-stackable23.7.0` to `my.corp/myteam/stackable/kafka:3.3.1-stackable23.7.0`.

== [[customimages]] Custom images

Expand All @@ -140,7 +140,7 @@ Only when the correct product version is given to the operator, the product will
Using custom images has a few limitations that users should be aware of:

* The images will *have* to have the same structures that Stackable operators expect.
This should usually be ensured by specifying a Stackable image in the `FROM` clause of the Dockerfile (all the available images can be found in our https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable[docker repository] - the schema is typically: `docker.stackable.tech/stackable/<product>:<product-version>-stackable<stackable-version>`).
This should usually be ensured by specifying a Stackable image in the `FROM` clause of the Dockerfile (all the available images can be found in our https://oci.stackable.tech/api/v2.0/projects/sdp[OCI registry] - the schema is typically: `oci.stackable.tech/sdp/<product>:<product-version>-stackable<stackable-version>`. Information on how to browse the registry can be found xref:contributor:project-overview.adoc#docker-images[here]).

* Images will have to be upgraded for every new Stackable release to follow structural changes that Stackable may have made to their images.
When deriving images from official Stackable images this will mean updating the version of the image in the `FROM` clause to the correct Stackable release.
Expand Down
4 changes: 2 additions & 2 deletions modules/contributor/pages/contributing-code.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ For `VisualStudio Code` we also recommend the following extensions:
+
[source,bash]
----
docker build --file docker/Dockerfile --tag docker.stackable.tech/stackable/<operator>:<version>-dev .
kind load docker-image docker.stackable.tech/stackable/<operator>:<version>-dev --name=integration-tests
docker build --file docker/Dockerfile --tag oci.stackable.tech/sdp/<operator>:<version>-dev .
kind load docker-image oci.stackable.tech/sdp/<operator>:<version>-dev --name=integration-tests
helm install <operator> deploy/helm/<operator>/
----
. Run the tests from the repository root with `./scripts/run-tests`.
Expand Down
106 changes: 103 additions & 3 deletions modules/contributor/pages/project-overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -85,10 +85,110 @@ Where are binaries, Helm Charts and Docker images stored?
[[product-artifacts]]
=== Product artifacts

A lot of artifacts are stored in the https://repo.stackable.tech/#browse/browse[Nexus repo].
Under `packages` you find product binaries, the different `helm-*` stores contain the Helm Charts and Docker images are stored there too.
A lot of artifacts are stored in the https://oci.stackable.tech[OCI registry]. Currently, those artifacts can be browsed by calling the API.

The following command lists all the different projects the artifacts are distributed across:

[source,console]
----
$ curl -X GET --header 'Accept: application/json' 'https://oci.stackable.tech/api/v2.0/projects/' | jq --raw-output '.[] | .name'
library
sandbox
sdp
sdp-charts
stackable
stackable-charts
----

`sdp` contains the product and operator Docker images. The Helm Charts for the operators are found under `sdp-charts`. Some artifacts like the
product binaries are stored in the https://repo.stackable.tech/#browse/browse[Nexus repo] under `packages`.

List the Helm Charts in `sdp-charts`:

[source,console]
----
$ curl -X GET --header 'Accept: application/json' 'https://oci.stackable.tech/api/v2.0/projects/sdp-charts/repositories?page_size=20' | jq --raw-output '.[] | .name'
----

List all available versions of a specific Helm Chart, e.g. the zookeeper-operator:

[source,console]
----
$ curl -X GET --header 'Accept: application/json' 'https://oci.stackable.tech/api/v2.0/projects/sdp-charts/repositories/zookeeper-operator/artifacts' | jq --raw-output '.[] | .tags[].name'
0.0.0-dev
0.0.0-pr920
0.0.0-pr919
0.0.0-pr917
24.11.1
24.11.1-rc2
24.11.1-rc1
24.11.0
24.7.0
24.3.0
----

[[docker-images]]
=== Docker images

Docker images are stored in Nexus as mentioned above, but also in https://oci.stackable.tech - which is where we want to move to in the future.
Docker images are stored in https://oci.stackable.tech as mentioned above. To list all the available repositories in a project, for example in
the `sdp` project, run this command:

[source,console]
----
$ curl -X GET --header 'Accept: application/json' 'https://oci.stackable.tech/api/v2.0/projects/sdp/repositories?page=1' | jq --raw-output '.[] | .name'
sdp/statsd_exporter
sdp/stackable-cockpit
sdp/product1
sdp/git-sync/git-sync
sdp/sig-storage/csi-provisioner
sdp/sig-storage/csi-node-driver-registrar
sdp/kcat
sdp/java-devel
sdp/vector
sdp/hello-world
----

Since the API is using pagination it might be necessary to increment the `page` in the command to see additional results.

It is also possible to list all the available image tags for a repository of choice. The following command shows the tags of the `trino` repository:

[source,console]
----
$ curl -X GET --header 'Accept: application/json' 'https://oci.stackable.tech/api/v2.0/projects/sdp/repositories/trino/artifacts?page=1&page_size=25' | jq --raw-output '.[] | select(.extra_attrs.config != {}) | .tags[]?.name'
455-stackable0.0.0-dev
451-stackable0.0.0-dev
470-stackable0.0.0-dev
470-stackable0.0.0-dev-arm64
455-stackable0.0.0-dev-arm64
451-stackable0.0.0-dev-arm64
470-stackable0.0.0-dev-amd64
455-stackable0.0.0-dev-amd64
451-stackable0.0.0-dev-amd64
469-stackable0.0.0-dev
469-stackable0.0.0-dev-arm64
469-stackable0.0.0-dev-amd64
451-stackable24.11.1
455-stackable24.11.1
----

Similar to the previous command, the API call uses pagination again. So the `page` value in the command can be incremented to see more results. Here the `page_size`
parameter was also used to increase the results per page.

Another possibility, instead of using `curl`, would be the https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md[crane tool], which can also be used
to browse the tags when given the path to a repository.

[source,console]
----
$ crane ls -O oci.stackable.tech/sdp/trino
414-stackable0.0.0-dev
414-stackable0.0.0-dev-amd64
414-stackable0.0.0-dev-arm64
414-stackable24.3.0
414-stackable24.7.0
414-stackable24.7.0-amd64
414-stackable24.7.0-arm64
428-stackable0.0.0-dev
428-stackable0.0.0-dev-amd64
428-stackable0.0.0-dev-arm64
...
----
2 changes: 1 addition & 1 deletion modules/contributor/pages/testing-on-kubernetes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ The easiest way that we found to do this is by using https://github.com/tilt-dev
Tilt should then automatically discover this registry from the cluster config and push images there.

If you are using a remote cluster, Tilt will push the generated container images to a remote registry, in order for your Kubernetes to be able to access them.
We have configured `docker.stackable.tech/sandbox` as the default registry, as this is what all our developers use.
We have configured `oci.stackable.tech/sandbox` as the default registry, as this is what all our developers use.
External contributors will not have access to this registry and need to override this to a registry of their choice.

Overriding the default registry can be done by providing a file called `tilt_options.json` in the same directory as the Tiltfile.
Expand Down
11 changes: 6 additions & 5 deletions modules/guides/pages/custom-images.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
= Using customized product images
:stackable-docker-registry: https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable
:stackable-oci-registry: https://oci.stackable.tech/api/v2.0/projects/sdp
:kind: https://kind.sigs.k8s.io/
:kind-load-image: https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster
:description: Customize Stackable product images by modifying base images, deploying to a registry or Kubernetes cluster, and configuring your Stacklet to use them.
Expand All @@ -21,12 +21,12 @@ To use a customized image you have to:

The Stackable operators rely on the structure and contents of the product images, so any modifications need to be done using the Stackable images as base images.

You can find the Stackable Docker images in the {stackable-docker-registry}[Stackable Docker registry].
Images follow a naming schema: `docker.stackable.tech/stackable/<product>:<product-version>-stackable<stackable-version>` where `<product>` are the products like `druid`, `trino` and `opa`, `<product-version>` are product versions like `28.0.1`, `414` or `0.61.0` and `<stackable-version>` is a Stackable platform version like `23.7` or `24.3.0`.
You can find the Stackable Docker images in the {stackable-oci-registry}[Stackable OCI registry].
Images follow a naming schema: `oci.stackable.tech/sdp/<product>:<product-version>-stackable<stackable-version>` where `<product>` are the products like `druid`, `trino` and `opa`, `<product-version>` are product versions like `28.0.1`, `414` or `0.61.0` and `<stackable-version>` is a Stackable platform version like `23.7` or `24.3.0`.
The Stackable version can also be `0.0.0-dev` for nightly images.
You can use this naming schema together with the xref:operators:supported_versions.adoc[] list to quickly find the base image you need.

For example, you might use the `docker.stackable.tech/stackable/hive:3.1.3-stackable24.3.0` image as a base for a modified Apache Hive image where you add a custom database driver.
For example, you might use the `oci.stackable.tech/sdp/hive:3.1.3-stackable24.3.0` image as a base for a modified Apache Hive image where you add a custom database driver.
Use the `FROM` clause in your Dockerfile to reference the base image.

When tagging the image, there are no requirements by the operator, any registry and tag is possible.
Expand Down Expand Up @@ -60,4 +60,5 @@ With this configuration the operator deploys your Stacklet using your custom ima
== Further reading and useful links

* Read about xref:concepts:product-image-selection.adoc[] to learn about other ways of specifying a product version or images, for example how to use a custom registry when mirroring Stackable images.
* Have a look at the {stackable-docker-registry}[Stackable Docker registry] to find out which images are available to use as a base.
* Have a look at the {stackable-oci-registry}[Stackable OCI registry] to find out which images are available to use as a base. Information on how to browse the registry
can be found xref:contributor:project-overview.adoc#docker-images[here].
Loading