Skip to content

Commit 0a61948

Browse files
fhennigrazvan
andauthored
docs: various small improvements (#763)
* various small docs fixes * Update docs/modules/kafka/pages/getting_started/first_steps.adoc * Update docs/modules/kafka/pages/getting_started/first_steps.adoc * Update docs/modules/kafka/pages/getting_started/installation.adoc * Update docs/modules/kafka/pages/usage-guide/operations/graceful-shutdown.adoc * Update docs/modules/kafka/pages/usage-guide/operations/pod-disruptions.adoc --------- Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
1 parent ffb6f3a commit 0a61948

14 files changed

+73
-58
lines changed

docs/modules/kafka/pages/getting_started/first_steps.adoc

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
= First steps
22
:description: Deploy and verify a Kafka cluster on Kubernetes with Stackable Operators, including ZooKeeper setup and data testing using kcat.
3+
:kcat-install: https://github.com/edenhill/kcat#install
34

4-
After going through the xref:getting_started/installation.adoc[] section and having installed all the operators, you will now deploy a Kafka cluster and the required dependencies. Afterwards you can <<_verify_that_it_works, verify that it works>> by producing test data into a topic and consuming it.
5+
After going through the xref:getting_started/installation.adoc[] section and having installed all the operators, you now deploy a Kafka cluster and the required dependencies.
6+
Afterward you can <<_verify_that_it_works, verify that it works>> by producing test data into a topic and consuming it.
57

68
== Setup
79

@@ -10,7 +12,8 @@ Two things need to be installed to create a Kafka cluster:
1012
* A ZooKeeper instance for internal use by Kafka
1113
* The Kafka cluster itself
1214

13-
We will create them in this order, each one is created by applying a manifest file. The operators you just installed will then create the resources according to the manifest.
15+
Create them in this order by applying the corresponding manifest files.
16+
The operators you just installed then create the resources according to the manifest.
1417

1518
=== ZooKeeper
1619

@@ -58,11 +61,12 @@ and apply it:
5861
include::example$getting_started/getting_started.sh[tag=install-kafka]
5962
----
6063

61-
This will create the actual Kafka instance.
64+
This creates the actual Kafka instance.
6265

6366
== Verify that it works
6467

65-
Next you will produce data into a topic and read it via https://github.com/edenhill/kcat#install[kcat]. Depending on your platform you may need to replace `kafkacat` in the commands below with `kcat`.
68+
Next you produce data into a topic and read it via {kcat-install}[kcat].
69+
Depending on your platform you may need to replace `kafkacat` in the commands below with `kcat`.
6670

6771
First, make sure that all the Pods in the StatefulSets are ready:
6872

docs/modules/kafka/pages/getting_started/index.adoc

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
= Getting started
22
:description: Start with Apache Kafka using Stackable Operator: Install, set up Kafka, and manage topics in a Kubernetes cluster.
33

4-
This guide will get you started with Apache Kafka using the Stackable Operator.
5-
It will guide you through the installation of the Operator and its dependencies, setting up your first Kafka instance and create, write to and read from a topic.
4+
This guide gets you started with Apache Kafka using the Stackable Operator.
5+
It guides you through the installation of the Operator and its dependencies, setting up your first Kafka instance and create, write to and read from a topic.
66

77
== Prerequisites
88

9-
You will need:
9+
You need:
1010

1111
* a Kubernetes cluster
1212
* kubectl
1313
* optional: Helm
1414
* https://github.com/edenhill/kcat#install[kcat] for testing
1515

16-
Resource sizing depends on cluster type(s), usage and scope, but as a starting point we recommend a minimum of the following resources for this operator:
16+
Resource sizing depends on cluster type(s), usage and scope, but as a starting point a minimum of the following resources is recommended for this operator:
1717

1818
* 0.2 cores (e.g. i5 or similar)
1919
* 256MB RAM

docs/modules/kafka/pages/getting_started/installation.adoc

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,20 @@
11
= Installation
22
:description: Install Stackable Operator for Apache Kafka using stackablectl or Helm, including dependencies like ZooKeeper and required operators for Kubernetes.
33

4-
On this page you will install the Stackable Operator for Apache Kafka and operators for its dependencies - ZooKeeper -
4+
Install the Stackable Operator for Apache Kafka and operators for its dependencies -- ZooKeeper --
55
as well as the commons, secret and listener operator which are required by all Stackable Operators.
66

7-
== Stackable Operators
7+
There are multiple ways to install the Stackable Operator for Apache Kafka.
8+
xref:management:stackablectl:index.adoc[] is the preferred way, but Helm is also supported.
9+
OpenShift users may prefer installing the operator from the RedHat Certified Operator catalog using the OpenShift web console.
810

9-
There are 2 ways to install Stackable Operators:
10-
11-
. Using xref:management:stackablectl:index.adoc[stackablectl]
12-
. Using Helm
13-
14-
=== stackablectl
15-
16-
The `stackablectl` command line tool is the recommended way to interact with operators and dependencies. Follow the
17-
xref:management:stackablectl:installation.adoc[installation steps] for your platform if you choose to work with
18-
`stackablectl`.
11+
[tabs]
12+
====
13+
stackablectl::
14+
+
15+
--
16+
The `stackablectl` command line tool is the recommended way to interact with operators and dependencies.
17+
Follow the xref:management:stackablectl:installation.adoc[installation steps] for your platform if you choose to work with `stackablectl`.
1918
2019
After you have installed `stackablectl`, run the following command to install all operators necessary for Kafka:
2120
@@ -24,16 +23,18 @@ After you have installed `stackablectl`, run the following command to install al
2423
include::example$getting_started/getting_started.sh[tag=stackablectl-install-operators]
2524
----
2625
27-
The tool will show
26+
The tool prints
2827
2928
[source]
3029
include::example$getting_started/install_output.txt[]
3130
3231
TIP: Consult the xref:management:stackablectl:quickstart.adoc[] to learn more about how to use `stackablectl`.
32+
--
3333
34-
=== Helm
35-
36-
You can also use Helm to install the operators. Add the Stackable Helm repository:
34+
Helm::
35+
+
36+
--
37+
Add the Stackable Helm repository:
3738
3839
[source,bash]
3940
----
@@ -47,8 +48,9 @@ Then install the Stackable Operators:
4748
include::example$getting_started/getting_started.sh[tag=helm-install-operators]
4849
----
4950
50-
Helm will deploy the operators in a Kubernetes Deployment and apply the CRDs for the Apache Kafka service (as well as
51-
the CRDs for the required operators). You are now ready to deploy Apache Kafka in Kubernetes.
51+
Helm deploys the operators in a Kubernetes Deployment and apply the CRDs for the Apache Kafka service (as well as the CRDs for the required operators).
52+
--
53+
====
5254

5355
== What's next
5456

docs/modules/kafka/pages/index.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ It is commonly used for real-time data processing, data ingestion, event streami
2121

2222
== Getting started
2323

24-
Follow the xref:kafka:getting_started/index.adoc[] which will guide you through installing The Stackable Kafka and ZooKeeper operators, setting up ZooKeeper and Kafka and testing your Kafka using `kcat`.
24+
Follow the xref:kafka:getting_started/index.adoc[] which guides you through installing The Stackable Kafka and ZooKeeper operators, setting up ZooKeeper and Kafka and testing your Kafka using `kcat`.
2525

2626
== Resources
2727

@@ -45,7 +45,7 @@ Kafka requires xref:zookeeper:index.adoc[Apache ZooKeeper] for coordination purp
4545
== Connections to other products
4646

4747
Since Kafka often takes on a bridging role, many other products connect to it.
48-
In the <<demos, demos>> below you will find example data pipelines that use xref:nifi:index.adoc[Apache NiFi with the Stackable operator] to write to Kafka and xref:nifi:index.adoc[Apache Druid with the Stackable operator] to read from Kafka.
48+
In the <<demos, demos>> below you find example data pipelines that use xref:nifi:index.adoc[Apache NiFi with the Stackable operator] to write to Kafka and xref:nifi:index.adoc[Apache Druid with the Stackable operator] to read from Kafka.
4949
But you can also connect using xref:spark-k8s:index.adoc[Apache Spark] or with a custom Job written in various languages.
5050

5151
== [[demos]]Demos

docs/modules/kafka/pages/reference/commandline-parameters.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ stackable-kafka-operator run --product-config /foo/bar/properties.yaml
2323

2424
*Multiple values:* false
2525

26-
The operator will **only** watch for resources in the provided namespace `test`:
26+
The operator **only** watches for resources in the provided namespace `test`:
2727

2828
[source]
2929
----

docs/modules/kafka/pages/reference/environment-variables.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ docker run \
3636

3737
*Multiple values:* false
3838

39-
The operator will **only** watch for resources in the provided namespace `test`:
39+
The operator **only** watches for resources in the provided namespace `test`:
4040

4141
[source]
4242
----

docs/modules/kafka/pages/usage-guide/configuration-environment-overrides.adoc

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,16 @@
22

33
The cluster definition also supports overriding configuration properties and environment variables, either per role or per role group, where the more specific override (role group) has precedence over the less specific one (role).
44

5-
IMPORTANT: Overriding certain properties which are set by operator (such as the ports) can interfere with the operator and can lead to problems.
5+
IMPORTANT: Overriding operator-set properties (such as the ports) can interfere with the operator and can lead to problems.
66

77
== Configuration Properties
88

99
For a role or role group, at the same level of `config`, you can specify: `configOverrides` for the following files:
1010

11-
- `server.properties`
12-
- `security.properties`
11+
* `server.properties`
12+
* `security.properties`
1313

14-
For example, if you want to set the `auto.create.topics.enable` to disable automatic topic creation, it can be configured in the `KafkaCluster` resource like so:
14+
For example, if you want to set the `auto.create.topics.enable` to disable automatic topic creation, it can be configured in the KafkaCluster resource like so:
1515

1616
[source,yaml]
1717
----
@@ -43,9 +43,13 @@ For a full list of configuration options we refer to the Apache Kafka https://ka
4343

4444
=== The security.properties file
4545

46-
The `security.properties` file is used to configure JVM security properties. It is very seldom that users need to tweak any of these, but there is one use-case that stands out, and that users need to be aware of: the JVM DNS cache.
46+
The `security.properties` file is used to configure JVM security properties.
47+
It is very seldom that users need to tweak any of these, but there is one use-case that stands out, and that users need to be aware of: the JVM DNS cache.
4748

48-
The JVM manages it's own cache of successfully resolved host names as well as a cache of host names that cannot be resolved. Some products of the Stackable platform are very sensible to the contents of these caches and their performance is heavily affected by them. As of version 3.4.0 Apache Kafka performs poorly if the positive cache is disabled. To cache resolved host names, you can configure the TTL of entries in the positive cache like this:
49+
The JVM manages it's own cache of successfully resolved host names as well as a cache of host names that cannot be resolved.
50+
Some products of the Stackable platform are very sensible to the contents of these caches and their performance is heavily affected by them.
51+
As of version 3.4.0 Apache Kafka performs poorly if the positive cache is disabled.
52+
To cache resolved host names, you can configure the TTL of entries in the positive cache like this:
4953

5054
[source,yaml]
5155
----

docs/modules/kafka/pages/usage-guide/operations/graceful-shutdown.adoc

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ You can configure the graceful shutdown as described in xref:concepts:operations
66

77
As a default, Kafka brokers have `30 minutes` to shut down gracefully.
88

9-
The Kafka broker process will receive a `SIGTERM` signal when Kubernetes wants to terminate the Pod.
10-
After the graceful shutdown timeout runs out, and the process still didn't exit, Kubernetes will issue a `SIGKILL` signal.
9+
The Kafka broker process receives a `SIGTERM` signal when Kubernetes wants to terminate the Pod.
10+
After the graceful shutdown timeout runs out, and the process is still running, Kubernetes issues a `SIGKILL` signal.
1111

1212
This is equivalent to executing the `bin/kafka-server-stop.sh` command, which internally executes `kill <kafka-pid>` (https://github.com/apache/kafka/blob/2c6fb6c54472e90ae17439e62540ef3cb0426fe3/bin/kafka-server-stop.sh#L34[code]).
1313

@@ -25,15 +25,15 @@ The broker logs the received signal as shown in the log below:
2525

2626
The https://kafka.apache.org/35/documentation/#basic_ops_restarting[Kafka documentation] does a very good job at explaining what happens during a graceful shutdown:
2727

28-
The Kafka cluster will automatically detect any broker shutdown or failure and elect new leaders for the partitions on that machine.
29-
This will occur if either a server fails or it is brought down intentionally for maintenance or configuration changes.
28+
The Kafka cluster automatically detects any broker shutdown or failure and elect new leaders for the partitions on that machine.
29+
This occurs if either a server fails or it is brought down intentionally for maintenance or configuration changes.
3030
For the latter cases Kafka supports a more graceful mechanism for stopping a server than just killing it.
31-
When a server is stopped gracefully it has two optimizations it will take advantage of:
31+
When a server is stopped gracefully it has two optimizations it takes advantage of:
3232

33-
1. It will sync all its logs to disk to avoid the need for any log recovery when it restarts (i.e. validating the checksum for all messages in the tail of the log). Log recovery takes time, so this speeds up intentional restarts.
34-
2. It will migrate any partitions the broker is the leader of, to other replicas prior to shutting down. This will make the leadership transfer faster and minimize the time each partition is unavailable to a few milliseconds.
33+
1. It syncs all its logs to disk to avoid the need for any log recovery when it restarts (i.e. validating the checksum for all messages in the tail of the log). Log recovery takes time, so this speeds up intentional restarts.
34+
2. It migrates any partitions the broker is the leader of, to other replicas prior to shutting down. This makes the leadership transfer faster and minimize the time each partition is unavailable to a few milliseconds.
3535

36-
Note that controlled shutdown will only succeed if all the partitions hosted on the broker have replicas (i.e. the replication factor is greater than 1 and at least one of these replicas is alive).
36+
Note that controlled shutdown only succeeds if all the partitions hosted on the broker have replicas (i.e. the replication factor is greater than 1 and at least one of these replicas is alive).
3737
This is generally what you want since shutting down the last replica would make that topic partition unavailable.
3838

3939
This operator takes care of that by only allowing a certain number of brokers to be offline as described in xref:usage-guide/operations/pod-disruptions.adoc[].

docs/modules/kafka/pages/usage-guide/operations/index.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22

33
This section of the documentation is intended for the operations teams that maintain a Stackable Data Platform installation.
44

5-
Please read the xref:concepts:operations/index.adoc[Concepts page on Operations] that contains the necessary details to operate the platform in a production environment.
5+
Read the xref:concepts:operations/index.adoc[Concepts page on Operations] that contains the necessary details to operate the platform in a production environment.

docs/modules/kafka/pages/usage-guide/operations/pod-disruptions.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33

44
You can configure the permitted Pod disruptions for Kafka nodes as described in xref:concepts:operations/pod_disruptions.adoc[].
55

6-
Unless you configure something else or disable our PodDisruptionBudgets (PDBs), we write the following PDBs:
6+
Unless you configure something else or disable the default PodDisruptionBudgets (PDBs), the operator writes the following PDBs:
77

88
== Brokers
9-
We only allow a single Broker to be offline at any given time, regardless of the number of replicas or `roleGroups`.
9+
Allow only a single Broker to be offline at any given time, regardless of the number of replicas or `roleGroups`.
1010
This is because we can not make any assumptions about topic replication factors.
Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
= Cluster ID
22

3-
Kafka has an internal check to ensure that a broker cannot join a different cluster to the one in which it was previously registered (this is important to avoid various kinds of metadata inconsistencies in the cluster). The clusterId is stored locally after initial registration and is verified upon cluster startup that it still matches what is in ZooKeeper.
3+
Kafka has an internal check to ensure that a broker cannot join a different cluster to the one in which it was previously registered (this is important to avoid various kinds of metadata inconsistencies in the cluster).
4+
The clusterId is stored locally after initial registration and is verified upon cluster startup that it still matches what is in ZooKeeper.
45

5-
This clusterId is stored in the `meta.properties` file in the folder specified by the `log.dirs` setting: this is persisted on a PVC created by Kafka. This PVC is not removed when the Kafka ZNode is deleted, which means that there are circumstances where this internal check will fail with the following error:
6+
This clusterId is stored in the `meta.properties` file in the folder specified by the `log.dirs` setting: this is persisted on a PVC created by Kafka. This PVC is not removed when the Kafka ZNode is deleted, which means that there are circumstances where this internal check fails with the following error:
67

78
[source,bash]
89
----
@@ -13,12 +14,15 @@ The Cluster ID <new Cluster ID> doesn't match stored clusterId <old Cluster ID>
1314

1415
=== Restarting a Kafka cluster
1516

16-
When re-starting a Kafka cluster, ensure that the Kafka ZNode is not removed: upon restart the cluster will attempt to register with the ZooKeeper cluster referenced in the ZNode and will check that the cluster IDs match. As the `meta.properties` file has not been changed this should not cause any problems.
17+
When re-starting a Kafka cluster, ensure that the Kafka ZNode is not removed: upon restart the cluster attempts to register with the ZooKeeper cluster referenced in the ZNode and checks that the cluster IDs match. As the `meta.properties` file has not been changed this should not cause any problems.
1718

1819
=== Replacing an existing ZNode
1920

20-
If the ZNode has been removed, then the Kafka PVC prefixed with `log-dirs-` will also have to be removed. This will result in the loss of topic metadata but is unavoidable since Kafka will need to re-register with ZooKeeper anyway. For instance, this will apply when breaking changes have been made to the ZooKeeper operator.
21+
If the ZNode has been removed, then the Kafka PVC prefixed with `log-dirs-` also has to be removed.
22+
This results in the loss of topic metadata but is unavoidable since Kafka needs to re-register with ZooKeeper anyway.
23+
For instance, this applies when breaking changes have been made to the ZooKeeper operator.
2124

2225
=== Updating the SDP release
2326

24-
Depending on the scope of any breaking changes, it may be possible to upgrade SDP and re-create clusters without having to touch the Kafka PVCs. In cases where deleting the aforementioned PVC is unavoidable this will also result in the loss of topic offset metadata.
27+
Depending on the scope of any breaking changes, it may be possible to upgrade SDP and re-create clusters without having to touch the Kafka PVCs.
28+
In cases where deleting the aforementioned PVC is unavoidable this also results in the loss of topic offset metadata.

docs/modules/kafka/pages/usage-guide/security.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ spec:
105105
== [[authorization]]Authorization
106106

107107
If you wish to include integration with xref:opa:index.adoc[Open Policy Agent] and already have an OPA cluster, then you can include an `opa` field pointing to the OPA cluster discovery `ConfigMap` and the required package.
108-
The package is optional and will default to the `metadata.name` field:
108+
The package is optional and defaults to the `metadata.name` field:
109109

110110
[source,yaml]
111111
----

docs/modules/kafka/pages/usage-guide/storage-resources.adoc

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ brokers:
1717
capacity: 2Gi
1818
----
1919

20-
In the above example, all Kafka brokers in the default group will store data (the location of the property `log.dirs`) on a `2Gi` volume.
20+
In the above example, all Kafka brokers in the default group store data (the location of the property `log.dirs`) on a `2Gi` volume.
2121

22-
If nothing is configured in the custom resource for a certain role group, then by default each Pod will have a `1Gi` large local volume mount for the data location.
22+
If nothing is configured in the custom resource for a certain role group, then by default each Pod has a `1Gi` large local volume mount for the data location.
2323

2424
== Resource Requests
2525

@@ -32,7 +32,8 @@ A minimal HA setup consisting of 2 Brokers has the following https://kubernetes.
3232
* `2560Mi` memory request and limit
3333
* `4Gi` persistent storage
3434

35-
Of course, additional services, require additional resources. For Stackable components, see the corresponding documentation on further resource requirements.
35+
Of course, additional services, require additional resources.
36+
For Stackable components, see the corresponding documentation on further resource requirements.
3637

3738
Corresponding to the values above, the operator uses the following resource defaults:
3839

0 commit comments

Comments
 (0)