Service account and tokens not deleted #4710

[jaydeepva]

Bug Report

When operator is installed by OLM it creates service account for the operator. Service account token secrets are subsequently created for the service account. When operator is uninstalled, is service account for operator is expected to be deleted? OR Is the service account expected to be deleted manually?

What did you do?

1. Create a service account for operator
2. Include service account yaml in config/rbac folder.

kind: ServiceAccount
apiVersion: v1
metadata:
  name: my-operator-service-account
  labels:
    app.kubernetes.io/instance: myapp
    app.kubernetes.io/managed-by: my-operator
    app.kubernetes.io/name: my-operator
    release: 1.0.0
imagePullSecrets:
  - name: my-key

3. Add service account yaml name to config/rbac/kustomization.yaml

resources:
- role.yaml
- role_binding.yaml
- leader_election_role.yaml
- leader_election_role_binding.yaml
- service_account.yaml

4. Run make bundle. The service account yaml is included in bundle folder

$ pwd
/operators/my-operator/bundle/manifests
$ ls -l
-rwxrwxrwx 1 elm elm   311 Feb 27 10:40 my-operator-service-account_v1_serviceaccount.yaml
-rwxrwxrwx 1 elm elm 47659 Feb 27 10:40 my-operator.clusterserviceversion.yaml

Verify that service account name is present in csv file at

install.spec.deployments[0].spec.template.spec.serviceAccount, install.spec.deployments[0].spec.template.spec.serviceAccountName and install.spec.permissions.rules.serviceAccountName

5. Deploy the operator through OLM. Ensure that service account is created. Also verify that token secrets for service account are created
6. Uninstall the operator

What did you expect to see?

1. The service account should get deleted
2. The service account tokens should be deleted

What did you see instead? Under which circumstances?

Service account and service account tokens are not deleted

$ oc get sa
NAME                               SECRETS   AGE
builder                            2         119d
default                            2         119d
deployer                           2         119d
my-operator-service-account   2         70d

$ oc get secrets
NAME                                               TYPE                                  DATA   AGE
builder-dockercfg-276cc                            kubernetes.io/dockercfg               1      119d
builder-token-f8znw                                kubernetes.io/service-account-token   4      119d
builder-token-mmwpw                                kubernetes.io/service-account-token   4      119d
default-dockercfg-llg5c                            kubernetes.io/dockercfg               1      119d
default-token-rhz7d                                kubernetes.io/service-account-token   4      119d
default-token-x2gxj                                kubernetes.io/service-account-token   4      119d
deployer-dockercfg-xw8qt                           kubernetes.io/dockercfg               1      119d
deployer-token-6hssq                               kubernetes.io/service-account-token   4      119d
deployer-token-9hzs7                               kubernetes.io/service-account-token   4      119d
my-operator-service-account-dockercfg-jqjcg   kubernetes.io/dockercfg               1      70d
my-operator-service-account-token-k8hzw       kubernetes.io/service-account-token   4      70d
my-operator-service-account-token-s8dd8       kubernetes.io/service-account-token   4      70d
my-operator-service-account-token-z9fzd       kubernetes.io/service-account-token   4      70d

Environment

• operator-lifecycle-manager version:

Image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fdc4106ae1d4219bbc1247ee66a1780266b1ee2ac10e5f11b690d2d0be6ac2e8

operator-lifecycle-manager version: 0.15.1

• Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.3+65bd32d", GitCommit:"65bd32d", GitTreeState:"clean", BuildDate:"2021-01-27T04:24:26Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes cluster type:

OpenShift

Additional context

See operator-framework/operator-sdk#4710

[estroz]

OLM intentionally does not delete service accounts that exist prior to operator installation., even those created from a bundle. I am not 100% clear on the reasons why not though OLM will ownerref a service account contained in a bundle to the bundle's CSV, so when the CSV is deleted the service account should be as well. @jaydeepva how are you uninstalling your operator?

/ping @kevinrizza

[jaydeepva]

@estroz the operator is uninstalled from OpenShift console

[exdx]

Is it possible that this SA has two owners, and another operator uses it as well? Or potentially the operands are still using the SA?

Could you provide us the ownerrefs for the SA in question -- that would help us debug this further.

[jaydeepva]

Surprisingly there is no ownerRef in SA. I can see that CSV status.requirementStatus.dependents shows SA

    - dependents:
        - group: rbac.authorization.k8s.io
          kind: PolicyRule
          message: >-
            namespaced
            rule:{"verbs":["get","list","watch","create","update","patch","delete"],"apiGroups":[""],"resources":["configmaps"]}
          status: Satisfied
          version: v1
        - group: rbac.authorization.k8s.io
          kind: PolicyRule
          message: >-
            namespaced
            rule:{"verbs":["get","update","patch"],"apiGroups":[""],"resources":["configmaps/status"]}
          status: Satisfied
          version: v1
...
...
      group: ''
      kind: ServiceAccount
      message: ''
      name: my-operator-service-account
      status: Present
      version: v1