NSFS deployment with export of accounts (UID, GID configuration)

This is a WIP feature.

Step 1 - Deploy Latest NooBaa

Download the operator binary:

curl https://noobaa-operator-cli.s3.amazonaws.com/noobaa-operator-master-20210419 > noobaa

Use the CLI to install to the noobaa namespace:

noobaa install -n noobaa --operator-image='noobaa/noobaa-operator:master-20210419' --noobaa-image='noobaa/noobaa-core:master-20210419'

I also suggest updating the current namespace to noobaa so you don’t need to add “-n noobaa” to all kubectl / noobaa commands:

kubectl config set-context --current --namespace noobaa

Step 2 - Setup Filesystem PVC

For NSFS to work it requires a PVC for the filesystem, with a ReadWriteMany accessMode to allow any node in the cluster to use it.

In this example we are showing how to create a simple Local PV (similar to hostpath) for testing purposes:

Assuming the filesystem to expose is mounted in /nsfs in the node. We will create a local PV that represents the mounted file system on the node at /nsfs. Download and create the yamls attached below -

kubectl create -f nsfs-local-class.yaml
kubectl create -f nsfs-local-pv.yaml
kubectl create -f nsfs-local-pvc.yaml

nsfs-local-class.yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nsfs-local
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

nsfs-local-pv.yaml:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nsfs-vol
spec:
  storageClassName: nsfs-local
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /nsfs/
  capacity:
    storage: 1Ti
  accessModes:
    - ReadWriteMany
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/os
              operator: Exists

nsfs-local-pvc.yaml:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nsfs-vol
spec:
  storageClassName: nsfs-local
  resources:
    requests:
      storage: 1Ti
  accessModes:
    - ReadWriteMany

Step 3 - Patch endpoints to mount the PVC instead of the Operator (TEMPORARY STEP)

This step should be automated by the operator eventually.

Update the noobaa endpoints deployment to mount the volume -

kubectl patch deployment noobaa-endpoint --patch '{
  "spec": { "template": { "spec": {
    "volumes": [{
      "name": "nsfs",
      "persistentVolumeClaim": {"claimName": "nsfs-vol"}
    }],
    "containers": [{
      "name": "endpoint",
      "volumeMounts": [{ "name": "nsfs", "mountPath": "/nsfs" }]
    }]
  }}}
}'

Step 4 - Create Namespace Resource instead of the Operator (TEMPORARY STEP)

Create a namespace resource:

noobaa api pool_api create_namespace_resource '{
  "name": "fs1", 
  "nsfs_config": {
      "fs_backend": "GPFS",   
      "fs_root_path": "/nsfs/fs1"           
  }
}'

Supported backends: CEPH_FS, GPFS, NFSv4 The backend configuration allows optimization of flows for the underlying FS

Step 5 - Setup the filesystem ACL/permissions

Set up the ACLs/permissions of the mounted FS path to the needed UIDs, GIDs that would be used to access it

Here is an example: Locally on node giving a full access in order to support any UID, GID

mkdir -p /nsfs/fs1
chmod -R 777 /nsfs

Step 6 - Create Bucket

Create namespace bucket - These are the only required parameters for it to work:

noobaa api bucket_api create_bucket '{
  "name": "fs1-jenia-bucket",
  "namespace":{
    "write_resource": { "resource": "fs1", "path": "jenia/" },
    "read_resources": [ { "resource": "fs1", "path": "jenia/" }]
  }
}'

Step 7 - Setup S3 Account for NSFS

Create an account with NSFS configuration:

• Map the account to a UID/GID
• Set up the directory for new buckets created from S3 for this account (TBD)
• Note that allowed_buckets should be set to full_permission because the filesystem permissions of the UID will be used to resolve the allowed buckets for this account.

noobaa api account_api create_account '{
  "email": "jenia@noobaa.io",
  "name" : "jenia",
  "has_login": false,
  "s3_access": true,
  "allowed_buckets": { "full_permission": true },
  "nsfs_account_config": {
    "uid": *INSERT_UID*,
    "gid": *INSERT_GID*,
    "new_buckets_path": "TBD",
  }
}'

This should give out a response with the credentials to use

INFO[0001] ✅ RPC: account.create_account() Response OK: took 205.7ms 
access_keys:
- access_key: *NOOBAA_ACCOUNT_ACCESS_KEY*
  secret_key: *NOOBAA_ACCOUNT_SECRET_KEY*

You can also perform a list accounts command in order to see the configured NSFS accounts (besides all other accounts of the system)

noobaa api account_api list_accounts

If you are interested in a particular account you can read it directly

noobaa api account_api read_account '{
  "email": "jenia@noobaa.io"
}'

Step 8 - Connect S3 Client

Configure the S3 client application and access the FS via S3 from the endpoint

Application S3 config:

AWS_ACCESS_KEY_ID=*NOOBAA_ACCOUNT_ACCESS_KEY*
AWS_SECRET_ACCESS_KEY=*NOOBAA_ACCOUNT_SECRET_KEY*
S3_ENDPOINT=s3.noobaa.svc (or nodePort address from noobaa status)
BUCKET_NAME=fs1-jenia-bucket