-
-
Notifications
You must be signed in to change notification settings - Fork 12
Added license and overview page #439
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
b9272e7
Added license and overview page
fhennig b0872ff
Added page about kubernetes
fhennig 5a90c25
refactored Kubernetes info
fhennig ee78c00
fixed formatting
fhennig 097fa60
temporary changed stackablectl branch ref
fhennig 4849b72
temporary change reverted
fhennig bbb99d2
...
fhennig 0e0ba3c
reverted
fhennig 3ab26ab
restructured info into a Product Information page
fhennig 9158fb8
~
fhennig eea1a35
~
fhennig 1f13b9c
added required external components
fhennig fe69bea
~
fhennig f82ecb5
reverted branch references back to main
fhennig 232a3de
reverted branch references back to main
fhennig 147e079
fixed typo
fhennig 194cdde
temporary branch ref
fhennig 24121ec
temporary branch ref removed
fhennig 46f9e64
updates
fhennig aa4fafb
Added links
fhennig 52a8d57
Added another section
fhennig 61bb1a8
Update modules/ROOT/pages/product-information.adoc
fhennig 7c288c4
Update modules/ROOT/pages/product-information.adoc
fhennig 069e249
Update modules/ROOT/pages/product-information.adoc
fhennig 4ae594e
Update modules/ROOT/pages/product-information.adoc
fhennig 6920a8a
some changes
fhennig a8584a7
moved OPA
fhennig bebac36
another change
fhennig 7da0ddb
Added links to k8s distros
fhennig e6fb62a
restructuring
fhennig File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,3 @@ | ||
* xref:quickstart.adoc[] | ||
* xref:kubernetes.adoc[] | ||
* xref:getting_started.adoc[] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
// This second nav file is for another menu entry at the end of the navigation menu | ||
// see: https://docs.antora.org/antora/3.0/navigation/organize-files/#multiple-files-per-module | ||
* xref:release_notes.adoc[Release Notes] | ||
* xref:product-information.adoc[] | ||
* xref:licenses.adoc[Licenses] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
= Kubernetes | ||
|
||
The Stackable Data Platform runs on Kubernetes, a Kubernetes cluster is a prerequisite to running the platform. On this page you will find information on the supported Kubernetes distributions for production as well as how to set up a local test installation to try out parts of the platform right away. | ||
|
||
[#supported-production-distributions] | ||
== Supported production distributions | ||
|
||
The Stackable Data Platform requires a Kubernetes cluster to be present, where you can install things into. How to set up Kubernetes as well as a cluster depends on the distribution you chose. | ||
|
||
The following distributions are supported for a production setup of the Stackable Data Platform: | ||
|
||
include::partial$supported-kubernetes-distributions.adoc[] | ||
|
||
In this version of the SDP, the following Kubernetes versions are supported: | ||
|
||
include::partial$supported-kubernetes-versions.adoc[] | ||
|
||
Consult the xref:release_notes.adoc[release notes] to find out which specific versions are supported for the Stackable Data Platform you are using. | ||
|
||
[#local-installation] | ||
== Installing a testinging/development Kubernetes instance locally | ||
Stackable's control plane is built around Kubernetes, and we'll give some brief examples of how to install Kubernetes on your machine. | ||
|
||
=== Installing kubectl | ||
|
||
Stackable operators and their services are managed by applying manifest files to the Kubernetes cluster. For this purpose, you need to have the `kubectl` tool installed. Follow the instructions https://kubernetes.io/docs/tasks/tools/#kubectl[here] for your platform. | ||
|
||
=== Installing Kubernetes using Kind | ||
Kind offers a very quick and easy way to bootstrap your Kubernetes infrastructure in Docker. The big advantage of this is that you can simply remove the Docker containers when you're finished and clean up easily, making it great for testing and development. | ||
|
||
If you don't already have Docker then visit https://docs.docker.com/get-docker/[Docker Website] to find out how to install Docker. Kind is a single executable that performs the tasks of installing and configuring Kubernetes for you within Docker containers. The https://kind.sigs.k8s.io/docs/user/quick-start/[Kind Website] has instructions for installing Kind on your system. | ||
|
||
Once you have both of these installed then you can build a Kubernetes cluster in Docker. We're going to create a simple, single node cluster to test out Stackable, with the one node hosting both the Kubernetes control plane and the Stackable services. | ||
|
||
[source, bash] | ||
---- | ||
kind create cluster --name quickstart | ||
---- | ||
|
||
=== Installing Kubernetes using K3s | ||
K3s provides a quick way of installing Kubernetes. On your control node run the following command to install K3s: | ||
|
||
[source,bash] | ||
---- | ||
curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644 | ||
---- | ||
|
||
So long as you have an Internet connection K3s will download and automatically configure a simple Kubernetes environment. | ||
|
||
Create a symlink to the Kubernetes configuration from your home directory to allow tools like Helm to find the correct configuration. | ||
|
||
[source,bash] | ||
---- | ||
mkdir ~/.kube | ||
ln -s /etc/rancher/k3s/k3s.yaml ~/.kube/config | ||
---- | ||
|
||
|
||
=== Testing your Kubernetes installation | ||
|
||
To check if everything worked as expected you can use `kubectl cluster-info` to retrieve the cluster information. The output should look similar to: | ||
|
||
---- | ||
Kubernetes control plane is running at https://127.0.0.1:6443 | ||
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy | ||
---- | ||
|
||
If you set up your cluster using K3s you will additionally see the metrics server: | ||
|
||
---- | ||
Kubernetes control plane is running at https://127.0.0.1:6443 | ||
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy | ||
Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy | ||
---- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
= Licenses for the Stackable Data Platform | ||
|
||
The Stackable Data Platform is open source, and the source code of all the components can be found on GitHub. Licenses are also provided alongside the source code, in a file called `LICENSE`. | ||
|
||
== Operators | ||
|
||
Product Operators | ||
|
||
* https://github.com/stackabletech/airflow-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache Airflow. | ||
* https://github.com/stackabletech/druid-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache Druid. | ||
* https://github.com/stackabletech/hbase-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache HBase. | ||
* https://github.com/stackabletech/hdfs-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache Hadoop HDFS. | ||
* https://github.com/stackabletech/hive-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache Hive. | ||
* https://github.com/stackabletech/kafka-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache Kafka. | ||
* https://github.com/stackabletech/nifi-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache NiFi. | ||
* https://github.com/stackabletech/spark-k8s-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache Spark. | ||
* https://github.com/stackabletech/trino-operator/blob/main/LICENSE[License] for the Stackable Operator for Trino. | ||
* https://github.com/stackabletech/zookeeper-operator/blob/main/LICENSE[License] for the Stackable Operator for Apache ZooKeeper. | ||
* https://github.com/stackabletech/opa-operator/blob/main/LICENSE[License] for the Stackable Operator for OpenPolicyAgent. | ||
|
||
Additional Stackable Operators | ||
|
||
* https://github.com/stackabletech/commons-operator/blob/main/LICENSE[License] for the Stackable Commons Operator. | ||
* https://github.com/stackabletech/secret-operator/blob/main/LICENSE[License] for the Stackable Secret Operator. | ||
* https://github.com/stackabletech/listener-operator/blob/main/LICENSE[License] for the Stackable Listener Operator. | ||
|
||
== stackablectl | ||
|
||
https://github.com/stackabletech/stackablectl/blob/main/LICENSE[License] for stackablectl. | ||
|
||
== Product images | ||
|
||
https://github.com/stackabletech/docker-images/blob/main/LICENSE[License] for the product Docker images. | ||
|
||
The Docker images are built on the https://catalog.redhat.com/software/containers/ubi8/ubi-minimal/5c359a62bed8bd75a2c3fba8?container-tabs=overview[Red Hat ubi8-minimal base image]. It is https://www.redhat.com/licenses/EULA_Red_Hat_Universal_Base_Image_English_20190422.pdf[licensed seperately]. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
= Product information | ||
|
||
This page contains concrete specifications about the Stackable Data Platform (SDP) as a product, which components are included, how they are supplied and which external dependencies exist that you as a customer need to take care of. | ||
|
||
[#platform-components] | ||
== Platform components | ||
|
||
The Stackable Platform is made up of multiple components. Operators for data products, Operators for additional functionality to facilitate easy integration between the different products, as well as the data products themselves. Auxiliary software to interact with the platform, as well as software to ease the deployment of platform components. | ||
|
||
=== Data products and Kubernetes Operators | ||
|
||
The main components of the SDP are a set of open source data products. The SDP control plane consists of corresponding Kubernetes Operators. Each Operator is supplied in a Docker <<containers, container>> image. For every Operator there is also a Helm Chart to facilitate installation via Helm. Operators for Products supported by the Platform deploy these products as <<containers, containers>> in Kubernetes. | ||
|
||
Supported products: | ||
|
||
* xref:airflow:index.adoc[Apache Airflow] | ||
* xref:druid:index.adoc[Apache Druid] | ||
* xref:hbase:index.adoc[Apache HBase] (including xref:hbase:usage-guide/phoenix.adoc[Apache Phoenix]) | ||
* xref:hive:index.adoc[Apache Hive Metastore] | ||
* xref:hdfs:index.adoc[Apache Hadoop HDFS] | ||
* xref:kafka:index.adoc[Apache Kafka] | ||
* xref:nifi:index.adoc[Apache Nifi] | ||
* xref:spark-k8s:index.adoc[Apache Spark] (including xref:spark-k8s:usage-guide/history-server.adoc[Spark History Server]) | ||
* xref:superset:index.adoc[Apache Superset] | ||
* xref:trino:index.adoc[Trino] | ||
* xref:zookeeper:index.adoc[Apache Zookeeper] | ||
|
||
The product also includes the xref:opa:index.adoc[Open Policy Agent], and operators that provide additional functionality for managing and control SDP: xref:commons-operator:index.adoc[Commons], xref:secret-operator:index.adoc[Secret] and xref:listener-operator:index.adoc[Listener] Operator. | ||
|
||
The pages linked above also detail the use cases and features supported by each component. You can find additional information in the xref:release_notes.adoc[release notes]. Refer to the xref:operators:supported_versions.adoc[list of supported product versions] to find out which product versions are supported. | ||
|
||
|
||
[#stackablectl] | ||
=== stackablectl | ||
|
||
xref:stackablectl::index.adoc[stackablectl] is a commandline utility that makes it easier to install and interact with stackable components. | ||
|
||
== Supported installation methods | ||
|
||
[#containers] | ||
=== Operators and products | ||
|
||
All operators are supplied in container images. The products are also deployed in container images. | ||
The docker images are available for download here: https://repo.stackable.tech/#browse/browse:docker | ||
|
||
Stackable supports installing the Operators via https://helm.sh/[Helm] or with <<stackablectl>>. | ||
Every Operator includes installation instructions in the Getting started guide. | ||
|
||
==== Helm Charts | ||
|
||
The Helm Charts can be found here: https://repo.stackable.tech/#browse/browse:helm-stable Using the Helm Charts requires Helm version 3 or above. | ||
|
||
|
||
[#stackablectl-installation] | ||
=== stackablectl | ||
|
||
stackablectl is available for download, pre-built binaries are available on GitHub. The download link and installation steps are provided in the xref:stackablectl::installation.adoc[installation documentation]. | ||
|
||
== System requirements | ||
|
||
=== Operators | ||
|
||
Every Operator needs: | ||
|
||
* 0.2 cores (i.e. i5 or similar) | ||
* 256MB RAM | ||
|
||
=== stackablectl | ||
|
||
Hardware requirements and supported operating systems can be found in the xref:stackablectl::installation.adoc#system-requirements[system requirements section] of the stackablectl documentation. | ||
|
||
=== Stackable Data Platform Open Source Products | ||
|
||
The system requirements of each open source product depend on your specific use case. In the xref:stackablectl::demos/index.adoc[demos] you can find example use cases; every demo also has a _System requirements_ section which can provide a guideline for system sizing. Further example sizings can be found at https://ci.stackable.tech/job/<product>-operator-it-summary/ (where `<product>` is i.e. _druid_, _hbase_, etc.) in the `cluster-info.txt` file. These are the cluster sizes used for integration tests, and can be regarded as working cluster configurations. | ||
|
||
== Prerequisites and required external components | ||
|
||
Required external components are components that are required for the platform or parts of it to operate, but are not part of the Stackable Data Platform. Note that Stackable does neither distribute nor offer support for the external components. | ||
|
||
=== Kubernetes | ||
|
||
A Kubernetes cluster is required to install the Stackable Data Platform. The supported Kubernetes versions are: | ||
|
||
include::partial$supported-kubernetes-versions.adoc[] | ||
|
||
There are various Kubernetes distributions. Stackable supports: | ||
|
||
include::partial$supported-kubernetes-distributions.adoc[] | ||
|
||
=== Product specific dependencies | ||
|
||
The following products have required external components to run: | ||
|
||
* xref:airflow:required-external-components.adoc[Apache Airflow] | ||
* xref:druid:required-external-components.adoc[Apache Druid] | ||
* xref:hive:required-external-components.adoc[Apache Hive] | ||
* xref:superset:required-external-components.adoc[Apache Superset] | ||
|
||
== Optional and technology preview components | ||
|
||
=== Optional components | ||
|
||
Stackable software can be used with xref:stackablectl::demos/index.adoc[sample configurations] and third-party components, as outlined in the relevant documentation. These external components are not part of of the stackable products and can be used by the customer at their own risk. Stackable does not distribute these components and does not offer support for them. (See <<platform-components,Platform components>> above for a list of the supported components that are part of the platform) | ||
|
||
=== Technology preview components | ||
|
||
Some functionality of the platform might be labelled as _technology preview_. Stackable does not offer support for technology preview components, and therefore discourages their use in a production setting. Using these features/components is done so at your own risk. Customers are encouraged to provide feedback and suggestions for improvements on preview components. Neither Stackable nor third parties or licensees are obligated to distribute technology preview components or include them into the product. Technology preview components might be discontinued at any time. |
7 changes: 7 additions & 0 deletions
7
modules/ROOT/partials/supported-kubernetes-distributions.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
* https://aws.amazon.com/eks/[Amazon Elastic Kubernetes Service] | ||
* https://azure.microsoft.com/en-gb/products/kubernetes-service[Microsoft Azure Kubernetes Service] | ||
* https://cloud.google.com/kubernetes-engine[Google Kubernetes Engine] | ||
* https://cloud.ionos.com/managed/kubernetes[IONOS Managed Kubernetes] | ||
* https://www.rancher.com/products/rancher[SUSE Rancher] | ||
* https://www.suse.com/products/k3s/[SUSE K3S] | ||
* https://www.redhat.com/en/technologies/cloud-computing/openshift[Red Hat OpenShift] 4.10, 4.11 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.