Skip to content

Commit d74e091

Browse files
chore(tracking): Release Notes for SDP 25.3.0 (#723)
* chore: Update release template * chore: Add 25.3.0 release notes partial * chore: Add link to 25.3.0 release notes partial * chore: Update release template * chore(25.3.0): Add and link product version changes * chore(25.3.0): Add upgrade intructions using stackablectl * chore(25.3.0): Add upgrade intructions using helm * chore: Add upgrade instructions to release template * chore(25.3.0): Add supported K8s and OpenShift versions * chore: Add supported K8s and OpenShift section to release template * chore: Update ui submodule * chore(25.3.0): Add Trino breaking change * chore(25.3.0): Add platform feature for Additional trust roots * chore(25.3.0): Add s3 region breaking change * chore(25.3.0): Add OPA rego changes * chore(25.3.0): Add Druid changes * chore(25.3.0): Add Listener volume bug fix * chore(25.3.0): Add experimentalCertManager key length feature * chore(25.3.0): Add TLS certificate configurable lifetime feature * chore(25.3.0): Add containerdebug diagnostics feature * chore(25.3.0): Add airflow breaking change for dagsGitSync[].wait Duration * chore(25.3.0): Add Hive healthcheck known issue * chore(25.3.0): Add Hive memory reservation increase * chore(25.3.0): Add OCI registry migration section * chore(25.3.0): Add checksum/config annotation section * chore(25.3.0): Add Kafka -nodeport CM deprecation section * chore(25.3.0): Add NiFi Hadoop library change * chore(25.3.0): Add DNS lookup perfomance improvements * chore: Update headlines * chore(25.3.0): Add stackablectl and demo section * chore(25.3.0): Add fixed vulnerabilities * chore(25.3.0): Add jmx_exporter bug fix * chore: Sigh, Antora... * chore(25.3.0): Add Airflow and Superset OPA auth integration * chore(25.3.0): Add JVM argument override support * fix(25.3.0): Add .adoc extension to xref * chore: Use updated heading sizes * chore(25.3.0): Add more links * chore(25.3.0): Remove leftover section from template * chore(25.3.0): Improve druid-opa-authorizer xref * chore: Update release and product versions in getting_started page * chore(25.3.0): Put each sentence on a new line * chore: Remove mention of airflowdbs crd * chore(25.3.0): Fix spelling errors --------- Co-authored-by: Techassi <git@techassi.dev>
1 parent a54e225 commit d74e091

15 files changed

+663
-102
lines changed

modules/ROOT/pages/getting-started.adoc

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,11 @@ Install the Stackable command line utility xref:management:stackablectl:index.ad
3232

3333
The Stackable operators are components that translate the service definitions deployed via Kubernetes into deploy services on the worker nodes. These can be installed on any node that has access to the Kubernetes control plane. In this example we will install them on the controller node.
3434

35-
Stackable operators can be installed using `stackablectl`. Run the following commands to install ZooKeeper, Kafka and NiFi from the Stackable 24.11 release.
35+
Stackable operators can be installed using `stackablectl`. Run the following commands to install ZooKeeper, Kafka and NiFi from the Stackable 25.3 release.
3636

3737
[source,bash]
3838
----
39-
stackablectl release install -i commons -i secret -i listener -i zookeeper -i kafka -i nifi 24.11
39+
stackablectl release install -i commons -i secret -i listener -i zookeeper -i kafka -i nifi 25.3
4040
----
4141

4242
.Using Helm instead
@@ -50,12 +50,12 @@ Install the operators:
5050
5151
[source,bash]
5252
----
53-
helm install zookeeper-operator oci://oci.stackable.tech/sdp-charts/zookeeper-operator --version=24.11.1
54-
helm install kafka-operator oci://oci.stackable.tech/sdp-charts/kafka-operator --version=24.11.1
55-
helm install secret-operator oci://oci.stackable.tech/sdp-charts/secret-operator --version=24.11.1
56-
helm install listener-operator oci://oci.stackable.tech/sdp-charts/listener-operator --version=24.11.1
57-
helm install commons-operator oci://oci.stackable.tech/sdp-charts/commons-operator --version=24.11.1
58-
helm install nifi-operator oci://oci.stackable.tech/sdp-charts/nifi-operator --version=24.11.1
53+
helm install zookeeper-operator oci://oci.stackable.tech/sdp-charts/zookeeper-operator --version=25.3.0
54+
helm install kafka-operator oci://oci.stackable.tech/sdp-charts/kafka-operator --version=25.3.0
55+
helm install secret-operator oci://oci.stackable.tech/sdp-charts/secret-operator --version=25.3.0
56+
helm install listener-operator oci://oci.stackable.tech/sdp-charts/listener-operator --version=25.3.0
57+
helm install commons-operator oci://oci.stackable.tech/sdp-charts/commons-operator --version=25.3.0
58+
helm install nifi-operator oci://oci.stackable.tech/sdp-charts/nifi-operator --version=25.3.0
5959
----
6060
====
6161

@@ -64,12 +64,12 @@ You can check which operators are installed using `stackablectl operator install
6464
[source,console]
6565
----
6666
OPERATOR VERSION NAMESPACE STATUS LAST UPDATED
67-
commons 24.11.1 default deployed 2024-11-30 17:58:32.916032854 +0100 CET
68-
kafka 24.11.1 default deployed 2024-11-30 17:58:55.036115353 +0100 CET
69-
listener 24.11.1 default deployed 2024-11-30 17:59:18.136775259 +0100 CET
70-
nifi 24.11.1 default deployed 2024-11-30 17:59:51.927081648 +0100 CET
71-
secret 24.11.1 default deployed 2024-11-30 18:00:05.060241771 +0100 CET
72-
zookeeper 24.11.1 default deployed 2024-11-30 18:00:08.425686918 +0100 CET
67+
commons 25.3.0 default deployed 2024-11-30 17:58:32.916032854 +0100 CET
68+
kafka 25.3.0 default deployed 2024-11-30 17:58:55.036115353 +0100 CET
69+
listener 25.3.0 default deployed 2024-11-30 17:59:18.136775259 +0100 CET
70+
nifi 25.3.0 default deployed 2024-11-30 17:59:51.927081648 +0100 CET
71+
secret 25.3.0 default deployed 2024-11-30 18:00:05.060241771 +0100 CET
72+
zookeeper 25.3.0 default deployed 2024-11-30 18:00:08.425686918 +0100 CET
7373
----
7474

7575
== Deploying Stackable Services
@@ -90,7 +90,7 @@ metadata:
9090
name: simple-zk
9191
spec:
9292
image:
93-
productVersion: 3.9.2
93+
productVersion: 3.9.3
9494
clusterConfig:
9595
tls:
9696
serverSecretClass: null
@@ -278,7 +278,7 @@ To get the IP address we need to connect to (in this case `172.18.0.2`), run:
278278
----
279279
$ kubectl get nodes -o wide
280280
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
281-
quickstart-control-plane Ready control-plane 4d18h v1.30.0 172.18.0.2 <none> Debian GNU/Linux 12 (bookworm) 6.11.3 containerd://1.7.15
281+
quickstart-control-plane Ready control-plane 4d18h v1.32.0 172.18.0.2 <none> Debian GNU/Linux 12 (bookworm) 6.13.2 containerd://1.7.24
282282
----
283283

284284

modules/ROOT/pages/release-notes.adoc

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,25 @@
66
The Stackable platform consists of multiple operators that work together. Periodically a platform release is made,
77
including all components of the platform at a specific version.
88

9+
// WARNING: Please keep the empty newlines, otherwise headings are broken.
10+
include::partial$release-notes/release-25.3.adoc[]
11+
912
include::partial$release-notes/release-24.11.adoc[]
13+
1014
include::partial$release-notes/release-24.7.adoc[]
15+
1116
include::partial$release-notes/release-24.3.adoc[]
17+
1218
include::partial$release-notes/release-23.11.adoc[]
19+
1320
include::partial$release-notes/release-23.7.adoc[]
21+
1422
include::partial$release-notes/release-23.4.adoc[]
23+
1524
include::partial$release-notes/release-23.1.adoc[]
25+
1626
include::partial$release-notes/release-22.11.adoc[]
27+
1728
include::partial$release-notes/release-22.9.adoc[]
29+
1830
include::partial$release-notes/release-22.6.adoc[]

modules/ROOT/partials/release-notes/release-22.11.adoc

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ This is the third release of the Stackable Data Platform, which this time focuse
99

1010
The following new major platform features were added:
1111

12-
CPU and memory limits configurable::
12+
===== CPU and memory limits configurable
1313

1414
The operators now https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/[request] resources from Kubernetes for the products and required CPU and memory can now also be configured for all products.
1515
If your product instances are less performant after the update, the new defaults might be set too low and we recommend to xref:kafka:usage-guide/storage-resources.adoc[set custom requests] for your cluster.
@@ -24,7 +24,7 @@ If your product instances are less performant after the update, the new defaults
2424
* https://github.com/stackabletech/airflow-operator/pull/167[Apache Airflow]
2525
* https://github.com/stackabletech/superset-operator/pull/273[Apache Superset]
2626

27-
Orphaned Resources::
27+
===== Orphaned Resources
2828

2929
The operators now properly clean up after scaling down products.
3030
This means for example deleting StatefulSets that were left over after scaling down.
@@ -37,7 +37,7 @@ This means for example deleting StatefulSets that were left over after scaling d
3737
* https://github.com/stackabletech/trino-operator/pull/310[Trino]
3838
* https://github.com/stackabletech/airflow-operator/pull/174[Apache Airflow]
3939

40-
New Versions::
40+
===== New Versions
4141

4242
New product versions are supported.
4343

@@ -47,7 +47,7 @@ New product versions are supported.
4747
* https://github.com/stackabletech/druid-operator/pull/317[Apache Druid 24.0.0]
4848
* https://github.com/stackabletech/airflow-operator/pull/179[Apache Airflow 2.4.1]
4949

50-
Product features::
50+
===== Product features
5151

5252
Additionally there are some individual product features that are noteworthy
5353

modules/ROOT/partials/release-notes/release-22.6.adoc

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,37 +13,37 @@ While we are very proud of this release it is our first one and we'll add new fe
1313

1414
==== Platform features
1515

16-
Easily install production ready data applications::
16+
===== Easily install production ready data applications
1717

1818
Using a familiar declarative approach, users can easily install data applications such as Apache Kafka or Trino across multiple cloud Kubernetes providers or on their own data centers.
1919
The installation process is fully automated while also providing the flexibility for the user to tune relevant aspects of each application.
2020

21-
Monitoring::
21+
===== Monitoring
2222

2323
All products have monitoring with prometheus enabled.
2424
xref:operators:monitoring.adoc[Learn more]
2525

26-
Service discovery::
26+
===== Service discovery
2727

2828
Products on the Stackable platform use service discovery to easily interconnect with each other.
2929
xref:concepts:service_discovery.adoc[Learn more]
3030

31-
Configuration overrides::
31+
===== Configuration overrides
3232

3333
All operators support configuration overrides, these are documented in the specific operator documentation pages.
3434

35-
Common S3 configuration::
35+
===== Common S3 configuration
3636

3737
Many products support connecting to S3 to load and/or store data.
3838
There is a common resource for S3 connections and buckets across all operators that can be reused.
3939
xref:concepts:s3.adoc[Learn more]
4040

41-
Roles and role groups::
41+
===== Roles and role groups
4242

4343
To support hybrid hardware clusters, the Stackable platform uses the concept of role groups.
4444
Services and applications can be configured to maximize hardware efficiency.
4545

46-
Standardized::
46+
===== Standardized
4747

4848
Learn once reuse everywhere.
4949
We use the same conventions in all our operators.

modules/ROOT/partials/release-notes/release-22.9.adoc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ The main features focus on OpenShift support and security.
1010

1111
The following new major platform features were added:
1212

13-
OpenShift compatibility::
13+
===== OpenShift compatibility
1414

1515
We have made continued progress towards OpenShift compability, and the following operators can now be previewed on OpenShift.
1616
Further improvements are expected in future releases, but no stability or compatibility guarantees are currently made for OpenShift clusters.
@@ -20,15 +20,15 @@ Further improvements are expected in future releases, but no stability or compat
2020
* https://github.com/stackabletech/hdfs-operator/pull/225[Apache HDFS]
2121
* https://github.com/stackabletech/spark-k8s-operator/pull/126[Apache Spark on K8s]
2222

23-
Support for internal and external TLS::
23+
===== Support for internal and external TLS
2424

2525
The following operators support operating the products at a maximal level of transport security by using TLS certificates to secure internal and external communication:
2626

2727
* https://github.com/stackabletech/trino-operator/pull/244[Trino]
2828
* https://github.com/stackabletech/kafka-operator/pull/442[Apache Kafka]
2929
* https://github.com/stackabletech/zookeeper-operator/pull/479[Apache ZooKeeper]
3030

31-
LDAP authentication::
31+
===== LDAP authentication
3232

3333
Use a central LDAP server to manage all of your user identities in a single place.
3434
The following operators added support for LDAP authentication:

modules/ROOT/partials/release-notes/release-23.1.adoc

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ The focus in this platform release is on the support of offline (or on-premise)
2020

2121
The following new major platform features were added:
2222

23-
Product image selection::
23+
===== Product image selection
2424

2525
Product image selection has been expanded to cover different scenarios:
2626

@@ -32,7 +32,7 @@ These options are described in more detail xref:contributor:adr/ADR023-product-i
3232

3333
*N.B.* this is a breaking change across all operators as `spec.version` has been replaced by `spec.image`.
3434

35-
Logging Aggregation::
35+
===== Logging Aggregation
3636

3737
Component activity within the platform is logged in a way that makes it difficult to find, persist and consolidate this information.
3838
Log configuration is also a challenge.
@@ -47,19 +47,19 @@ In this release this has been added to the following components:
4747

4848
Support for other products will be added in future releases.
4949

50-
New Versions::
50+
===== New Versions
5151

5252
The following new product version is now supported:
5353

5454
* https://github.com/stackabletech/trino-operator/pull/358[Trino 403]
5555

56-
Deprecated Versions::
56+
===== Deprecated Versions
5757

5858
The following product version is no longer supported:
5959

6060
* https://github.com/stackabletech/druid-operator/pull/339[Druid 0.22.1]
6161

62-
Product features::
62+
===== Product features
6363

6464
Additionally, there are some individual product features that are noteworthy
6565

modules/ROOT/partials/release-notes/release-23.11.adoc

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,45 +11,45 @@ Released 2023-11-30.
1111

1212
The following new major platform features were added:
1313

14-
PodDisruptionBudgets::
14+
===== PodDisruptionBudgets
1515
Kubernetes has mechanisms to ensure minimal planned downtime.
1616
Our product operators deploy so-called PodDisruptionBudget (PDB) resources alongside the products.
1717
For every role that you specify (e.g. HDFS namenodes or Trino workers) a PDB is created.
1818
This will determine the extent to which roles for a given application may be inactive at any given time.
1919
See xref:concepts:operations/pod_disruptions.adoc[the documentation] for more details.
2020

21-
Graceful shutdown::
21+
===== Graceful shutdown
2222
Graceful shutdown refers to the managed, controlled shutdown of service instances in the manner intended by the software authors.
2323
Typically, an instance will receive a signal indicating the intent for the server to shut down, and it will initiate a controlled shutdown.
2424
Our operators configure a sensible amount of time Pods are granted to properly shut down without disrupting the availability of the product.
2525
See xref:concepts:operations/graceful_shutdown.adoc[the documentation] for more details.
2626

27-
Signed SDP product images::
27+
===== Signed SDP product images
2828
As of this release all Stackable product images are signed (the signing of operator images was delivered in SDP 23.7).
2929
Please see this xref:guides:enabling-verification-of-image-signatures.adoc[tutorial] for more information.
3030

31-
Airflow KubernetesExecutor::
31+
===== Airflow KubernetesExecutor
3232
Airflow clusters can now be configured to use Kubernetes executors, whereby pods are spun up for job tasks and terminated when complete, thus offering an alternative way to use resources without the need for job queuing.
3333

34-
Overridable Java security settings::
34+
===== Overridable Java security settings
3535
For JVM-based products (i.e. Druid, HBase, HDFS, Hive, Kafka, NiFi, Spark, Trino and ZooKeeper) it is now possible to provide custom security settings that override the default values.
3636
This allows the user to control things such as DNS lookup caches.
3737

38-
Stackable Cockpit::
38+
===== Stackable Cockpit
3939
This release includes a very early preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status.
4040

41-
stackablectl::
41+
===== stackablectl
4242
Our command line tool has been re-worked to use the same backbone as Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation].
4343

44-
Listener operator::
44+
===== Listener operator
4545
The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4.
4646
In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments.
4747
This is discussed in more detail in the xref:listener-operator:listenerclass.adoc[documentation].
4848

49-
Openshift certification::
49+
===== Openshift certification
5050
All Stackable operators in the 23.11 release have been certified for Openshift versions 4.11-4.13 and can be installed directly from the OperatorHub UI: image:openshift_operatorhub.png[OperatorHub in Openshift portal]
5151

52-
Product features::
52+
===== Product features
5353

5454
Additionally, there are some other individual product features that are noteworthy:
5555

@@ -692,7 +692,6 @@ kubectl replace -f https://raw.githubusercontent.com/stackabletech/zookeeper-ope
692692
[source,console]
693693
----
694694
customresourcedefinition.apiextensions.k8s.io "airflowclusters.airflow.stackable.tech" replaced
695-
customresourcedefinition.apiextensions.k8s.io "airflowdbs.airflow.stackable.tech" replaced
696695
customresourcedefinition.apiextensions.k8s.io "authenticationclasses.authentication.stackable.tech" replaced
697696
customresourcedefinition.apiextensions.k8s.io "s3connections.s3.stackable.tech" replaced
698697
...

modules/ROOT/partials/release-notes/release-23.4.adoc

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -29,19 +29,19 @@ It is recommended to install <<Release 23.4.1>> instead, as it contains relevant
2929

3030
The following new major platform features were added:
3131

32-
Cluster Operation::
32+
===== Cluster Operation
3333

3434
The first part of xref:concepts:operations/cluster_operations.adoc[Cluster operations] was rolled out in every applicable Stackable Operator.
3535
This supports pausing the cluster reconciliation and stopping the cluster completely.
3636
Pausing reconciliation will not apply any changes to the Kubernetes resources (e.g. when changing the custom resource).
3737
Stopping the cluster will set all replicas of StatefulSets, Deployments or DaemonSets to zero and therefore result in the deletion of all Pods belonging to that cluster (not the PVCs).
3838

39-
Status Field::
39+
===== Status Field
4040

4141
Operators of the Stackable Data Platform create, manage and delete Kubernetes resources: in order to easily query the health state of the products - and react accordingly - Stackable Operators use several predefined condition types to capture different aspects of a product's availability.
4242
See this xref:contributor:adr/ADR027-status[ADR] for more information.
4343

44-
Default / Custom Affinities::
44+
===== Default / Custom Affinities
4545

4646
In Kubernetes there are different ways to influence how Pods are assigned to Nodes.
4747
In some cases it makes sense to co-locate certain services that communicate a lot with each other, such as HBase regionservers with HDFS datanodes.
@@ -50,29 +50,29 @@ There may also be additional requirements e.g. placing important services - such
5050
This release implements default affinities that should suffice for many scenarios out-of-the box, while also allowing for custom affinity rules at a role and/or role-group level.
5151
See this xref:contributor:adr/ADR026-affinities.adoc[ADR] for more information.
5252

53-
Log Aggregation::
53+
===== Log Aggregation
5454

5555
The logging framework (added to the platform in Release 23.1) offers a consistent custom resource configuration and a separate, persisted sink (defaulting to OpenSearch).
5656
This has now been rolled out across all products.
5757
See this xref:contributor:adr/adr025-logging_architecture[ADR] and this xref:concepts:logging.adoc[concepts page] for more information.
5858

59-
Service Type::
59+
===== Service Type
6060

6161
The Service type can now be specified in all products.
6262
This currently differentiates between the internal ClusterIP and the external NodePort and is forward compatible with the xref:listener-operator:listenerclass.adoc[ListenerClass] for the automatic exposure of Services via the Listener Operator.
6363
This change is not backwards compatible with older platform releases.
6464
For security reasons, the default is set to the cluster-internal (ClusterIP) ListenerClass.
6565
A cluster can be exposed outside of Kubernetes by setting clusterConfig.listenerClass to external-unstable (NodePort) or external-stable (LoadBalancer).
6666

67-
New Versions::
67+
===== New Versions
6868

6969
No new product versions are supported in this platform release.
7070

71-
Deprecated Versions::
71+
===== Deprecated Versions
7272

7373
No product versions have been deprecated in this platform release.
7474

75-
Product features::
75+
===== Product features
7676

7777
Additionally, there are some individual product features that are noteworthy:
7878

0 commit comments

Comments
 (0)