Add Overview to Doc Content (#955)

* Add Overview to Doc Content

Adding a new Overview file to the documentation. This includes some content that is redundant to the designoverview and getting started sections. Assuming this looks like a good addition, I can create new PRs to rationalize Overview with the other existing content.

* Create storage-overview.md

Author: prlaurence

hugo/content/overview/_index.md

@@ -0,0 +1,7 @@
+---
+title: "Design"
+date:
+draft: false
+weight: 4
+---
+

hugo/content/overview/backup-restore-overview.md

@@ -0,0 +1,49 @@
+---
+title: "PostgreSQL Operator Backup and Restore Capability"
+date:
+draft: false
+weight: 5
+---
+
+## PostgreSQL Operator Backup and Restore Capability
+
+The PostgreSQL Operator provides users with the ability to manage PostgreSQL cluster backups through both native PostgreSQL backup functionality, as well as using pgbackrest, an open source backup and restore solution designed to scale up to the largest databases. By default, beginning with verison 4.0, the PostgreSQL Operator backup command performs a PostgreSQL pgbackrest backup.
+
+The three backup types that can be configured through the PostgreSQL Operator CLI are:
+
+* pgbackrest a simple, reliable backup and restore solution that can seamlessly scale up to the largest databases and workloads.  It provides full, incremental, differential
+backups, and point-in-time recovery.
+
+* pg_basebackup is used to take base backups of a running PostgreSQL database cluster. These are taken without affecting other clients to the database, and can be used both for
+point-in-time recovery and as the starting point for a log shipping or streaming replication standby servers. 
+
+* pg_dump is a utility for backing up a single PostgreSQL database. It makes consistent backups even if the database is being used concurrently. pg_dump does not block other users
+accessing the database (readers or writers).pg_dump 
+
+### pgBackRest Integration
+
+The PostgreSQL Operator integrates various features of the [pgbackrest backup and restore project](https://pgbackrest.org) to support backup and restore capability.
+
+The *pgo-backrest-repo* container acts as a pgBackRest remote repository for the PostgreSQL cluster to use for storing archive files and backups.
+
+The following diagrams depicts some of the integration features:
+
+![alt text](/operator-backrest-integration.png "Operator Backrest Integration")
+
+In this diagram, starting from left to right we see the following:
+
+ * a user when they enter *pgo backup mycluster --backup-type=pgbackrest* will cause a pgo-backrest container to be run as a Job, that container will execute a   *pgbackrest backup* command in the pgBackRest repository container to perform the backup function.
+
+ * a user when they enter *pgo show backup mycluster --backup-type=pgbackrest* will cause a *pgbackrest info* command to be executed on the pgBackRest repository container, the *info* output is sent directly back to the user to view
+
+ * the PostgreSQL container itself will use an archive command, *pgbackrest archive-push* to send archives to the pgBackRest repository container
+
+ * the user entering *pgo create cluster mycluster --pgbackrest* will cause a pgBackRest repository container deployment to be created, that repository is exclusively used for this Postgres cluster
+
+ * lastly, a user entering *pgo restore mycluster* will cause a *pgo-backrest-restore* container to be created as a Job, that container executes the *pgbackrest restore* command
+
+### Support for pgBackRest Use of S3 Buckets
+
+The PostgreSQL Operator supports the use AWS S3 storage buckets for the pgbackrest repository in any pgbackrest-enabled cluster.  When S3 support is enabled for a cluster, all archives will automatically be pushed to a pre-configured S3 storage bucket, and that same bucket can then be utilized for the creation of any backups as well as when performing restores.  Please note that once a storage type has been selected for a cluster during cluster creation (specifically `local`, `s3`, or _both_, as described in detail below), it cannot be changed.    
+
+The PostgreSQL Operator allows for the configuration of a single storage bucket, which can then be utilized across multiple clusters.  Once S3 support has been enabled for a cluster, pgbackrest will create a `backrestrepo` directory in the root of the configured S3 storage bucket (if it does not already exist), and subdirectories will then be created under the `backrestrepo` directory for each cluster created with S3 storage enabled.

hugo/content/overview/custom-resource-definitions.md

@@ -0,0 +1,39 @@
+---
+title: "Custom Resource Definitions Overview"
+date:
+draft: false
+weight: 5
+---
+
+## PostgreSQL Operator Custom Resource Definitions  
+
+The PostgreSQL Operator defines the following series of Kubernetes [Custom Resource Definitions (CRDs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#custom-resources):
+
+![Reference](/operator-crd-architecture.png)
+
+Each of these CRDs are used in the design of the PostgreSQL Operator to perform PostgreSQL related operations in order to enable with on-demand, PostgreSQL-as-a-Service workflows.  
+
+### Cluster (pgclusters)
+
+The Cluster or pgcluster CRD is used by the PostgreSQL Operator to define the PostgreSQL cluster definition and make new PostgreSQL cluster requests. 
+
+### Backup (pgbackups)
+
+The Backup or pgbackup CRD is used by the PostgreSQL Operator to perform a pgbasebackup and to hold the workflow and status of the last backup job.  Crunchy Data plans to deprecate this CRD in a future release in favor of a more general pgtask resource
+
+### Tasks (pgtask)
+
+The Tasks or pgtask CRD is used by the PostgreSQL Operator to perform workflow and other related administration tasks.  The pgtasks CRD captures workflows and administrative tasks for a given pgcluster. 
+
+### Replica (pgreplica)
+
+The Replica or pgreplica CRD is used by teh PostgreSQL Operator to create a PostgreSQL replica.  When a user creates a PostgreSQL replica, a pgreplica CRD is created to define that replica.
+
+Metadata about each PostgreSQL cluster deployed by the PostgreSQL Operator are stored within these CRD resources which act as the source of truth for the
+Operator.  The PostgreSQL Operator makes use of CRDs to maintain state and resource definitions as offered by the PostgreSQL Operator. 
+
+
+
+
+
+

hugo/content/overview/failover-overview.md

@@ -0,0 +1,39 @@
+---
+title: "Failover in the PostgreSQL Operator Overview"
+date:
+draft: false
+weight:
+---
+
+## Failover in the PostgreSQL Operator
+
+There are a number of potential events that could cause a primary database instance or cluster to become unavailable during the course of normal operations, including: 
+
+* A database storage (disk) failure or any other hardware failure 
+* The network on which the database resides becomes unreachable
+* The host operating system becomes unstable and crashes
+* A key database file becomes corrupted
+* Total loss of data center
+
+There may also be downtime events that are due to the normal case of operations, such as performing a minor upgrade, security patching of operating system, hardware upgrade, or other maintenance.
+
+To enable rapid recovery from the unavailability of the primary PostgreSQL instance within a PostgreSQL cluster, the PostgreSQL Operator supports both Manual and Automated failover within a single Kubernetes cluster. 
+
+### PostgreSQL Cluster Architecture 
+
+The failover from a primary PostgreSQL instances to a replica PostgreSQL instance within a PostgreSQL cluster. 
+
+### Manual Failover
+
+Manual failover is performed by PostgreSQL Operator API actions involving a *query* and then a *target* being specified to pick the fail-over replica target.
+
+### Automatic Failover 
+
+Automatic failover is performed by the PostgreSQL Operator by evaluating the readiness of a primary.  Automated failover can be globally specified for all clusters or specific clusters. If desired, users can configure the PostgreSQL Operator to replace a failed primary PostgreSQL instance with a new PostgreSQL replica.
+
+The PostgreSQL Operator automatic failover logic includes:
+
+ * deletion of the failed primary Deployment
+ * pick the best replica to become the new primary
+ * label change of the targeted Replica to match the primary Service
+ * execute the PostgreSQL promote command on the targeted replica

hugo/content/overview/namespace-overview.md

@@ -0,0 +1,35 @@
+---
+title: "PostgreSQL Operator Namespace Considerations Overview"
+date:
+draft: false
+weight: 6
+---
+
+## Kubernetes Namespaces and the PostgreSQL Operator
+
+In Kubernetes, namespaces provide the user the ability to divide cluster resources between multiple users (via resource quota).  
+
+The PostgreSQL Operator makes use of the Kubernetes Namespace support in order to define the Namespace to which the PostgreSQL Operator will deploy PostgreSQL clusters, enabling users to more easily allocate Kubernetes resources to specific areas within their business (users, projects, departments).  
+
+#### Namespaces Applied to Organizational Requirements
+
+Prior to version PostgreSQL Operator 4.0, the PostgreSQL Operator could only be deployed with a Namespace deployment pattern where both the PostgreSQL Operator and the PostgreSQL Clusters it deployed existed within a single Kubernetes namespace. 
+
+With the PostgreSQL Operator 4.0 release, the operator now supports a variety of Namespace deployment patterns, including:
+
+* **OwnNamespace** Operator and PostgreSQL clusters deployed to the same Kubernetes Namespace
+
+* **SingleNamespace and MultiNamespace** Operator and PostgreSQL clusters deployed to a predefined set of Kubernetes Namespaces
+
+* **AllNamespaces** Operator deployed into a single Kubernetes Namespace but watching all Namespaces on a Kubernetes cluster
+
+#### Configuration of the Namespace to which PostgreSQL Operator is Deployed
+
+In order to configure the Kubernetes Namespace within which the PostgreSQL Operator will run, it is necessary to configure the PGO_OPERATOR_NAMESPACE environment variable.  Both the Ansible and Bash installation method enable you to modify this PGO_OPERATOR_NAMESPACE environment variable in connection with the PostgreSQL Operator installation. 
+
+#### Configuration of the Namespaces to which PostgreSQL Operator will Deploy PostgreSQL Clusters 
+
+At startup time, the PostgreSQL Operator determines the Kubernetes Namespaces to which it will be able to deploy and administer PostgreSQL databases and clusters. The Kubernetes Namespace that the PostgreSQL Operator will be able to service is determined at startup time by the NAMESPACE environment variable.  The NAMESPACE variable is set as part of the PostgreSQL Operator installation process.  The format of the NAMESPACE value in the PostgreSQL Operator is modeled after the Operator Lifecycle Manager project. 
+
+
+

hugo/content/overview/node-affinitiy-overview.md

@@ -0,0 +1,32 @@
+---
+title: "Node Affinity in PostgreSQL Operator"
+date:
+draft: false
+weight: 3
+---
+
+## Node Affinity in PostgreSQL Operator 
+
+Kubernetes node affinity allows you to constrain which nodes your pod is eligible to be scheduled on, based on labels on the node.
+
+The PostgreSQL Operator provides users with the ability to add a node affinity section to a new Cluster Deployment.  By adding a node affinity section to the Cluster Deployment, users can direct Kubernetes to attempt to schedule a primary PostgreSQL instance within a cluster on a specific Kubernetes node.
+
+As an example, you can see the nodes on your Kubernetes cluster by running the following:
+```
+kubectl get nodes
+```
+
+You can then specify one of those Kubernetes node names (e.g. kubeadm-node2) when creating a PostgreSQL cluster;
+```
+pgo create cluster thatcluster --node-label=kubeadm-node2
+```
+
+The node affinity rule inserted in the Deployment uses a *preferred* strategy so that if the node were down or not available, Kubernetes will go ahead and schedule the Pod on another node.
+
+When you scale up a PostgreSQL cluster by adding a PostgreSQL replica instance, the scaling will take into account the use of `--node-label`.  If it sees that a PostgreSQL cluster was created with a specific node name, then the PostgreSQL replica Deployment will add an affinity rule to attempt to schedule the Pod. 
+
+
+
+
+
+

hugo/content/overview/postgres-operator-containers-overview.md

@@ -0,0 +1,49 @@
+## PostgreSQL Operator Containers Overview
+
+The PostgreSQL Operator orchestrates a series of PostgreSQL and PostgreSQL related containers containers that enable rapid deployment of PostgreSQL, including administration and monitoring tools in a Kubernetes environment. The PostgreSQL Operator supports PostgreSQL 9.5+ with multiple PostgreSQL cluster deployment strategies and a variety of PostgreSQL related extensions and tools enabling enterprise grade PostgreSQL-as-a-Service.   A full list of the containers supported by the PostgreSQL Operator is provided below.   
+
+### PostgreSQL Server and Extensions
+
+* **PostgreSQL** (crunchy-postgres).  PostgreSQL database server.  The crunchy-postgres container image is unmodified, open source PostgreSQL packaged and maintained by Crunchy Data. 
+
+* **PostGIS** (crunchy-postgres-gis).  PostgreSQL database server including the PostGIS extension. The crunchy-postgres-gis container image is unmodified, open source PostgreSQL packaged and maintained by Crunchy Data. This image is identical to the crunchy-postgres image except it includes the open source geospatial extension PostGIS for PostgreSQL in addition to the language extension PL/R which allows for writing functions in the R statistical computing language.
+
+### Backup and Restore
+
+* **pgBackRest** (crunchy-backrest-restore). pgBackRest is a high performance backup and restore utility for PostgreSQL.  The crunchy-backrest-restore container executes the pgBackRest utility, allowing FULL and DELTA restore capability.
+
+* **pg_basebackup** (crunchy-backup). pg_basebackup is used to take base backups of a running PostgreSQL database cluster. The crunchy-backup container executes a full backup against another database container using the standard pg_basebackup utility that is included with PostgreSQL.
+
+* **pgdump** (crunchy-pgdump). The crunchy-pgdump container executes either a pg_dump or pg_dumpall database backup against another PostgreSQL database.
+
+* **crunchy-pgrestore** (restore). The restore image provides a means of performing a restore of a dump from pg_dump or pg_dumpall via psql or pg_restore to a PostgreSQL container database.
+
+
+### Administration Tools
+
+* **pgAdmin4** (crunchy-pgadmin4). PGAdmin4 is a graphical user interface administration tool for PostgreSQL.  The crunchy-pgadmin4 container executes the pgAdmin4 web application.
+
+* **pgbadger** (crunchy-pgbadger).  pgbadger is a PostgreSQL log analyzer with fully detailed reports and graphs.  The crunchy-pgbadger container executes the pgBadger utility, which generates a PostgreSQL log analysis report using a small HTTP server running on the container.
+
+* **pg_upgrade**  (crunchy-upgrade). The crunchy-upgrade container contains 9.5, 9.6, 10, and 11 PostgreSQL packages in order to perform a pg_upgrade from 9.5 to 9.6, 9.6 to 10, and 10 to 11 versions.
+
+* **scheduler** (crunchy-scheduler).  The crunchy-scheduler container provides a cron like microservice for automating pgBaseBackup and pgBackRest backups within a single namespace. 
+
+### Metrics and Monitoring
+
+* **Metrics Collection** (crunchy-collect). The crunchy-collect container provides real time metrics about the PostgreSQL database via an API. These metrics are scraped and stored by a Prometheus time-series database and are then graphed and visualized through the open source data visualizer Grafana.  
+
+* **Grafana** (crunchy-grafana).  Grafana is an open source Visual dashboards are created from the collected and stored data that crunchy-collect and crunchy-prometheus provide for the crunchy-grafana container, which hosts an open source web-based graphing dashboard called Grafana.
+
+* **Prometheus** (crunchy-prometheus).  Prometheus is a multi-dimensional time series data model with an elastic query language. It is used in collaboration with Crunchy Collect and Grafana to provide metrics.
+
+### Connection Pooling and Load Balancing
+
+* **pgbouncer** (crunchy-pgbouncer).  pgbouncer is a lightweight connection pooler for PostgreSQL. The crunchy-pgbouncer container provides a pgbouncer image.
+
+* **pgpool** (crunchy-pgpool).  pgPool II is a middleware that works between PostgreSQL servers and a PostgreSQL database client.  The crunchy-pgpool container executes the utility. pgPool can be used to provide a smart PostgreSQL-aware proxy to a PostgreSQL cluster, both primary and replica, so that applications only have to work with a single database connection.
+
+
+
+
+

hugo/content/overview/postgres-operator-overview.md

@@ -0,0 +1,39 @@
+---
+title: "PostgreSQL Operator Overview"
+date:
+draft: false
+weight: 5
+---
+
+## PostgreSQL Operator Overview
+
+The PostgreSQL Operator extends Kubernetes to provide a higher- level abstraction enabling the rapid creation and management of PostgreSQL databases and clusters.  
+
+The PostgreSQL Operator include the following components:
+
+* PostgreSQL Operator
+* PostgreSQL Operator Containers
+* PostgreSQL Operator PGO Client
+* PostgreSQL Operator REST API Server
+* PostgreSQL PGO Schedule
+
+#### PostgreSQL Operator
+
+The PostgreSQL Operator makes use of Kubernetes “Custom Resource Definitions” or “CRDs” to extend Kubernetes with custom, PostgreSQL specific, Kubernetes objects such as “Database” and “Cluster”.  The PostgreSQL Operator users these CRDs to enable users to deploy, configure and administer PostgreSQL databases and clusters as Kubernetes-natvie, open source PostgreSQL-as-a-Service infrastructure. 
+
+#### PostgreSQL Operator Containers
+
+The PostgreSQL Operator orchestrates a series of PostgreSQL and PostgreSQL related containers containers that enable rapid deployment of PostgreSQL, including administration and monitoring tools in a Kubernetes environment. 
+
+#### PostgreSQL Operator PGO Client 
+
+The PostgreSQL Operator provides a command line interface (CLI), pgo. This CLI tool may be used an end-user to create databases or clusters, or make changes to existing databases.  The CLI interacts with the REST API deployed within the postgres-operator deployment.
+
+#### PostgreSQL Operator REST API Server
+
+A feature of the PostgreSQL Operator is to provide a REST API upon which users or custom applications can inspect and cause actions within the Operator such as provisioning resources or viewing status of resources.  This API is secured by a RBAC (role based access control) security model whereby each API call has a permission assigned to it. API roles are defined to provide granular authorization to Operator services.
+
+#### PostgreSQL Operator PGO Scheduler
+
+The PostgreSQL Operator includes a cron like scheduler application called pgo-scheduler. The purpose of pgo-scheduler is to run automated tasks such as PostgreSQL backups or SQL policies against PostgreSQL clusters created by the PostgreSQL Operator.  PGO Scheduler watches Kubernetes for configmaps with the label crunchy-scheduler=true in the same namespace where the PostgreSQL Operator is deployed. 
+