From bc7d8befe9055d4ff61fe61ab355bc40d80d7338 Mon Sep 17 00:00:00 2001 From: Techassi Date: Mon, 25 Sep 2023 15:43:46 +0200 Subject: [PATCH 1/4] Update quickstart page --- modules/ROOT/pages/getting_started.adoc | 15 +++++- modules/ROOT/pages/quickstart.adoc | 66 +++++++++++-------------- 2 files changed, 43 insertions(+), 38 deletions(-) diff --git a/modules/ROOT/pages/getting_started.adoc b/modules/ROOT/pages/getting_started.adoc index 91d9e83e9..5d058be47 100644 --- a/modules/ROOT/pages/getting_started.adoc +++ b/modules/ROOT/pages/getting_started.adoc @@ -4,20 +4,23 @@ One of the best ways of getting started with a new platform is to try it out. An == About this guide -Firstly, let’s cover whether this *Getting Started* guide is right for you. This is intended as a learning tool to discover more about Stackable, its deployment and architecture. +Firstly, let#s cover whether this *Getting Started* guide is right for you. This is intended as a learning tool to discover more about Stackable, its deployment and architecture. * If you want to build a production cluster then this is not for you. This tutorial is to familiarize you with the Stackable architecture and is not a guide for building robust clusters. * This is intended for use in a private network or lab; it doesn't enable many security features such as authentication or encryption and should not be directly connected to the Internet. Be careful if you're deploying in the cloud as your instances may default to using public IPs. == Overview + Stackable is based on Kubernetes and uses this as the control plane to manage clusters. In this guide we will build a simple cluster with 3 services; Apache ZooKeeper, Apache Kafka and Apache NiFi. == Installing Kubernetes and kubectl + Stackable’s control plane is built around Kubernetes. Follow the xref:kubernetes.adoc#local-installation[instructions] on how to set up a local Kubernetes instance if you do not have access to a cluster and install kubectl. If you already have kubectl installed, and have access to a Kubernetes cluster, you can skip this step. == Installing Stackable + === Install stackablectl Install the Stackable command line utility xref:management:stackablectl:index.adoc[stackablectl] by following the installation steps for your platform on the xref:management:stackablectl:installation.adoc[installation] page. @@ -68,9 +71,11 @@ zookeeper 23.7.0 default deployed ---- == Deploying Stackable Services -At this point you’ve successfully deployed Kubernetes and the Stackable operators we need and are ready to deploy services to the cluster. To do this we provide service descriptions to Kubernetes for each of the services we wish to deploy. + +At this point you've successfully deployed Kubernetes and the Stackable operators we need and are ready to deploy services to the cluster. To do this we provide service descriptions to Kubernetes for each of the services we wish to deploy. === Apache ZooKeeper + We will deploy an Apache ZooKeeper instance to our cluster. [source,bash] @@ -105,6 +110,7 @@ EOF ---- === Apache Kafka + We will deploy an Apache Kafka broker that depends on the ZooKeeper service we just deployed. The zookeeperReference property below points to the namespace and name we gave to the ZooKeeper service deployed previously. [source,bash] @@ -139,6 +145,7 @@ EOF ---- === Apache NiFi + We will next deploy an Apache NiFi server. [source,bash] @@ -203,9 +210,11 @@ zookeeper-operator-deployment-64fcccc797-pckhf 1/1 Running 0 Since this is the first time that each of these services has been deployed to these nodes, it will take some time to download the software from the Stackable repository and deploy the services. Once all the pods are in the running state your cluster is ready to use. == Testing your cluster + If all has gone well then you will have successfully deployed a Stackable cluster and used it to start three services that should now be ready for you. === Apache ZooKeeper + We can test ZooKeeper by running the ZooKeeper CLI shell. The easiest way to do this is to run the CLI shell on the pod that is running ZooKeeper. [source,bash] @@ -221,6 +230,7 @@ The shell should connect automatically to the ZooKeeper server running on the po ---- === Apache Kafka + To test Kafka we'll create a topic, and verify that it was created. First create the topic with the following command: @@ -239,6 +249,7 @@ kubectl exec -i -t simple-kafka-broker-brokers-0 -c kafka -- \ ---- === Apache NiFi + Apache NiFi provides a web interface and the easiest way to test it is to view this in a web browser. To access the web interface we first need to get the ip address and port Nifi is listening on. To get the IP address we need to connect to (in this case `172.18.0.2`), run: diff --git a/modules/ROOT/pages/quickstart.adoc b/modules/ROOT/pages/quickstart.adoc index 1f39d2ce0..c8627dbf6 100644 --- a/modules/ROOT/pages/quickstart.adoc +++ b/modules/ROOT/pages/quickstart.adoc @@ -1,5 +1,6 @@ = Quickstart +:latest-release: https://github.com/stackabletech/stackable-cockpit/releases/tag/stackablectl-1.0.0-rc1 :cockpit-releases: https://github.com/stackabletech/stackable-cockpit/releases This is the super-short getting started guide that should enable you to get something up and running in less than three @@ -9,37 +10,30 @@ minutes (excluding download times). Install `stackablectl`, the Stackable CLI utility. -On Linux (Windows and MacOS instructions below): +=== Installation on Linux -[source,console] ----- -$ curl -L -o stackablectl https://github.com/stackabletech/stackable-cockpit/releases/latest/download/stackablectl-x86_64-unknown-linux-gnu ----- - -and mark it as executable: +Download the `stackablectl-x86_64-unknown-linux-gnu` binary file from the link:{latest-release}[latest release], then +rename the file to `stackablectl`. You can also use the following command: [source,console] ---- -$ chmod +x stackablectl +wget -O stackablectl https://github.com/stackabletech/stackable-cockpit/releases/download/stackablectl-1.0.0-rc1/stackablectl-x86_64-unknown-linux-gnu +# or +curl -L -o stackablectl https://github.com/stackabletech/stackable-cockpit/releases/download/stackablectl-1.0.0-rc1/stackablectl-x86_64-unknown-linux-gnu ---- -.Instruction for Windows and MacOS -[%collapsible] -==== -Download `stackablectl-x86_64-pc-windows-gnu.exe` (for Windows) or `stackablectl-x86_64-apple-darwin` (Mac OS Intel) or -`stackablectl-aarch64-apple-darwin` (MacOS ARM) from the {cockpit-releases}[latest release] and save it as -`stackablectl`. For Windows, you can simply execute it. For MacOS mark it as executable: +Mark the binary as executable: [source,console] ---- -$ chmod +x stackablectl +chmod +x stackablectl ---- -If you want to execute it from anywhere in your system, you need to add it to the system `PATH`. +Then, make sure it is present in your `$PATH`, like `/usr/local/bin`. + +=== Installation on macOS and Windows -NOTE: If macOS denies the execution of stackablectl go to `Settings` -> `Security & Privacy` -> `General`. Here you will -see a pop up asking if you want to allow access for `stackablectl`. You must allow access. -==== +See the xref:management:stackablectl:installation.adoc[guide] for detailed information about the installation process on macOS and Windows. == Install the Taxi data demo @@ -60,29 +54,29 @@ To list the installed installed Stackable services run the following command: [source,console] ---- -./stackablectl services list --all-namespaces +./stackablectl stacklet list ---- It will output something like: [source] ---- - PRODUCT NAME NAMESPACE ENDPOINTS EXTRA INFOS - - hive hive default hive 172.18.0.5:30298 - metrics 172.18.0.5:31633 - - opa opa default http http://172.18.0.5:30316 - - superset superset default external-superset http://172.18.0.4:32295 Admin user: admin, password: adminadmin - - trino trino default coordinator-http http://172.18.0.3:30167 - coordinator-metrics 172.18.0.3:31818 - coordinator-https https://172.18.0.3:30141 - - minio minio-trino default http http://172.18.0.3:31062 Third party service - console-http http://172.18.0.3:30503 Admin user: admin, password: adminadmin + PRODUCT NAME NAMESPACE ENDPOINTS + + hive hive default hive 172.18.0.5:30298 + metrics 172.18.0.5:31633 + + opa opa default http http://172.18.0.5:30316 + + superset superset default external-superset http://172.18.0.4:32295 + + trino trino default coordinator-http http://172.18.0.3:30167 + coordinator-metrics 172.18.0.3:31818 + coordinator-https https://172.18.0.3:3014 + + minio minio-trino default http http://172.18.0.3:31062 + console-http http://172.18.0.3:30503 ---- To connect to Superset, open the listed endpoint in your browser and log in with the username `admin` and the password -`adminadmin`. +`adminadmin`. Use the `stackablectl stacklet credentials` command to retrieve credentials for various stacklets. From 21ac3ea912b92a963e0af0fc0a6cdd5578dd1f98 Mon Sep 17 00:00:00 2001 From: Techassi Date: Tue, 26 Sep 2023 15:32:07 +0200 Subject: [PATCH 2/4] Add release note section to start page --- modules/ROOT/pages/index.adoc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/modules/ROOT/pages/index.adoc b/modules/ROOT/pages/index.adoc index c093a7320..fa931e719 100644 --- a/modules/ROOT/pages/index.adoc +++ b/modules/ROOT/pages/index.adoc @@ -66,6 +66,12 @@ which often do not play nicely together out-of-the-box. ++++ +== Release Notes + +The Stackable platform consists of multiple operators that work together. Periodically a platform release is made, +including all components of the platform at a specific version. See the latest release notes for 23.7 +xref:release_notes.adoc[here]. + == Components We are using Kubernetes as our deployment platform. And we're building {k8s-operators}[Operators] for each of the From 2f589dc01a0697ee2b4e80dc8d9bcc8c66665c47 Mon Sep 17 00:00:00 2001 From: Techassi Date: Tue, 26 Sep 2023 15:32:16 +0200 Subject: [PATCH 3/4] Various small adjustments --- modules/ROOT/pages/getting_started.adoc | 29 +++++++++------ modules/ROOT/pages/product-information.adoc | 4 +-- modules/ROOT/pages/quickstart.adoc | 35 ++++++++++--------- ...1-stackablectl_stacks_initial_version.adoc | 2 +- 4 files changed, 40 insertions(+), 30 deletions(-) diff --git a/modules/ROOT/pages/getting_started.adoc b/modules/ROOT/pages/getting_started.adoc index 5d058be47..e5e7614a4 100644 --- a/modules/ROOT/pages/getting_started.adoc +++ b/modules/ROOT/pages/getting_started.adoc @@ -4,7 +4,7 @@ One of the best ways of getting started with a new platform is to try it out. An == About this guide -Firstly, let#s cover whether this *Getting Started* guide is right for you. This is intended as a learning tool to discover more about Stackable, its deployment and architecture. +Firstly, let's cover whether this *Getting Started* guide is right for you. This is intended as a learning tool to discover more about Stackable, its deployment and architecture. * If you want to build a production cluster then this is not for you. This tutorial is to familiarize you with the Stackable architecture and is not a guide for building robust clusters. * This is intended for use in a private network or lab; it doesn't enable many security features such as authentication or encryption and should not be directly connected to the Internet. Be careful if you're deploying in the cloud as your instances may default to using public IPs. @@ -15,7 +15,7 @@ Stackable is based on Kubernetes and uses this as the control plane to manage cl == Installing Kubernetes and kubectl -Stackable’s control plane is built around Kubernetes. Follow the xref:kubernetes.adoc#local-installation[instructions] on how to set up a local Kubernetes instance if you do not have access to a cluster and install kubectl. +Stackable's control plane is built around Kubernetes. Follow the xref:kubernetes.adoc#local-installation[instructions] on how to set up a local Kubernetes instance if you do not have access to a cluster and install kubectl. If you already have kubectl installed, and have access to a Kubernetes cluster, you can skip this step. @@ -26,9 +26,10 @@ If you already have kubectl installed, and have access to a Kubernetes cluster, Install the Stackable command line utility xref:management:stackablectl:index.adoc[stackablectl] by following the installation steps for your platform on the xref:management:stackablectl:installation.adoc[installation] page. === Installing Stackable Operators + The Stackable operators are components that translate the service definitions deployed via Kubernetes into deploy services on the worker nodes. These can be installed on any node that has access to the Kubernetes control plane. In this example we will install them on the controller node. -Stackable operators can be installed using stackablectl. Run the following commands to install ZooKeeper, Kafka and NiFi from the Stackable 23.7 release. +Stackable operators can be installed using `stackablectl`. Run the following commands to install ZooKeeper, Kafka and NiFi from the Stackable 23.7 release. [source,bash] ---- @@ -61,13 +62,21 @@ helm install nifi-operator stackable-stable/nifi-operator --version=23.7 You can check which operators are installed using `stackablectl operator installed`: ----- -OPERATOR VERSION NAMESPACE STATUS LAST UPDATED -commons 23.7.0 default deployed 2023-07-27 09:41:05.769685041 +0200 CEST -kafka 23.7.0 default deployed 2023-07-27 09:41:27.685845379 +0200 CEST -nifi 23.7.0 default deployed 2023-07-27 09:41:40.928558978 +0200 CEST -secret 23.7.0 default deployed 2023-07-27 09:41:51.820834174 +0200 CEST -zookeeper 23.7.0 default deployed 2023-07-27 09:41:54.972145417 +0200 CEST +[source,console] +---- +┌────────────────────┬─────────┬─────────────────────┬──────────┬──────────────────────────────────────────┐ +│ OPERATOR ┆ VERSION ┆ NAMESPACE ┆ STATUS ┆ LAST UPDATED │ +╞════════════════════╪═════════╪═════════════════════╪══════════╪══════════════════════════════════════════╡ +│ commons-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-09-26 14:59:10.447836367 +0200 CEST │ +├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ kafka-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-09-26 14:59:25.162058457 +0200 CEST │ +├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ nifi-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-09-26 14:59:35.881227443 +0200 CEST │ +├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ secret-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-09-26 14:59:44.51273442 +0200 CEST │ +├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ zookeeper-operator ┆ 23.7.0 ┆ stackable-operators ┆ deployed ┆ 2023-09-26 14:59:58.196949027 +0200 CEST │ +└────────────────────┴─────────┴─────────────────────┴──────────┴──────────────────────────────────────────┘ ---- == Deploying Stackable Services diff --git a/modules/ROOT/pages/product-information.adoc b/modules/ROOT/pages/product-information.adoc index 0eb638e8a..5fea81db8 100644 --- a/modules/ROOT/pages/product-information.adoc +++ b/modules/ROOT/pages/product-information.adoc @@ -64,9 +64,9 @@ requires Helm version 3 or above. [#stackablectl-installation] -=== stackablectl +==== stackablectl -stackablectl is available for download, pre-built binaries are available on GitHub. The download link and installation +`stackablectl` is available for download, pre-built binaries are available on GitHub. The download link and installation steps are provided in the xref:management:stackablectl:installation.adoc[installation]. == System requirements diff --git a/modules/ROOT/pages/quickstart.adoc b/modules/ROOT/pages/quickstart.adoc index c8627dbf6..7bd52654e 100644 --- a/modules/ROOT/pages/quickstart.adoc +++ b/modules/ROOT/pages/quickstart.adoc @@ -43,7 +43,7 @@ Superset: [source,console] ---- -./stackablectl demo install trino-taxi-data +stackablectl demo install trino-taxi-data ---- TIP: Learn more about this demo in the demo docs: xref:demos:trino-taxi-data.adoc[] @@ -54,28 +54,29 @@ To list the installed installed Stackable services run the following command: [source,console] ---- -./stackablectl stacklet list +stackablectl stacklet list ---- It will output something like: [source] ---- - PRODUCT NAME NAMESPACE ENDPOINTS - - hive hive default hive 172.18.0.5:30298 - metrics 172.18.0.5:31633 - - opa opa default http http://172.18.0.5:30316 - - superset superset default external-superset http://172.18.0.4:32295 - - trino trino default coordinator-http http://172.18.0.3:30167 - coordinator-metrics 172.18.0.3:31818 - coordinator-https https://172.18.0.3:3014 - - minio minio-trino default http http://172.18.0.3:31062 - console-http http://172.18.0.3:30503 +┌──────────┬───────────────┬───────────┬──────────────────────────────────────────────┬─────────────────────────────────┐ +│ PRODUCT ┆ NAME ┆ NAMESPACE ┆ ENDPOINTS ┆ CONDITIONS │ +╞══════════╪═══════════════╪═══════════╪══════════════════════════════════════════════╪═════════════════════════════════╡ +│ hive ┆ hive ┆ default ┆ ┆ Available, Reconciling, Running │ +├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ opa ┆ opa ┆ default ┆ ┆ Available, Reconciling, Running │ +├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ superset ┆ superset ┆ default ┆ external-superset http://172.18.0.3:31974 ┆ Available, Reconciling, Running │ +├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ trino ┆ trino ┆ default ┆ coordinator-metrics 172.18.0.3:30788 ┆ Available, Reconciling, Running │ +│ ┆ ┆ ┆ coordinator-https https://172.18.0.3:31010 ┆ │ +├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ +│ minio ┆ minio-console ┆ default ┆ http http://172.18.0.3:32650 ┆ │ +└──────────┴───────────────┴───────────┴──────────────────────────────────────────────┴─────────────────────────────────┘ + +Use "stackablectl stacklet credentials [OPTIONS] " to display credentials for deployed stacklets. ---- To connect to Superset, open the listed endpoint in your browser and log in with the username `admin` and the password diff --git a/modules/contributor/pages/adr/ADR021-stackablectl_stacks_initial_version.adoc b/modules/contributor/pages/adr/ADR021-stackablectl_stacks_initial_version.adoc index 9bc600b67..467ff443b 100644 --- a/modules/contributor/pages/adr/ADR021-stackablectl_stacks_initial_version.adoc +++ b/modules/contributor/pages/adr/ADR021-stackablectl_stacks_initial_version.adoc @@ -58,7 +58,7 @@ By defining our own, thin, abstraction layer we can isolate our users from the c We could do nothing right now and instead use the _create_test_cluster.py_ script as our entry point for the initial release. * Good, this gives us time to design a fully thought out solution before implementing something -* Bad, _create_test_cluster.py_ doesn't feel very polished and we do want to introducet _stackablectl_ to the world as our tool of choice +* Bad, _create_test_cluster.py_ doesn't feel very polished and we do want to introduce _stackablectl_ to the world as our tool of choice === Implement basic definition of stacks From 2aabbe02ead7449f1af46155d6f4b9f216cb164d Mon Sep 17 00:00:00 2001 From: Techassi Date: Wed, 27 Sep 2023 17:01:59 +0200 Subject: [PATCH 4/4] Add links to VSC extensions --- modules/contributor/pages/index.adoc | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/modules/contributor/pages/index.adoc b/modules/contributor/pages/index.adoc index 656b2a8ec..0a7162f16 100644 --- a/modules/contributor/pages/index.adoc +++ b/modules/contributor/pages/index.adoc @@ -80,11 +80,11 @@ Of course you are free to use whatever works for you best. No editor is perfect For `VisualStudio Code` we also recommend the following extensions: -* Better Toml -* CodeLLDB (for debugging) -* Error Lens (inline error messages) -* AsciiDoc -* GitHub Pull requests and Issues -* GitLens -* Python -* Docker +* https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml[Even Better TOML] +* https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb[CodeLLDB] (for debugging) +* https://marketplace.visualstudio.com/items?itemName=usernamehw.errorlens[Error Lens] (inline error messages) +* https://marketplace.visualstudio.com/items?itemName=asciidoctor.asciidoctor-vscode[AsciiDoc] +* https://marketplace.visualstudio.com/items?itemName=GitHub.vscode-pull-request-github[GitHub Pull requests and Issues] +* https://marketplace.visualstudio.com/items?itemName=eamodio.gitlens[GitLens] +* https://marketplace.visualstudio.com/items?itemName=ms-python.python[Python] +* https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker[Docker]