Pod CrashLoopBackOff error with automountServiceAccountToken set to false

user_3148997 0
Viewed 187

Hi Kubernetes Experts,

I was using the following ServiceAccount creation config:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-scheduler
  namespace: kube-system

and the following Deployment config:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    component: scheduler
    tier: control-plane
  name: my-scheduler
  namespace: kube-system
spec:
  ...
  template:
    metadata:
      labels:
        component: scheduler
        tier: control-plane
        version: second
        k8s-custom-scheduler: my-scheduler
    spec:
      serviceAccountName: my-scheduler

Things were working fine, And now I want to make my pod more secure setting automountServiceAccountToken to false.

I changed my ServiceAccount creation and deployment config:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-scheduler
  namespace: kube-system
automountServiceAccountToken: false

Deployment config:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    component: scheduler
    tier: control-plane
  name: my-scheduler
  namespace: kube-system
spec:
  ...
  template:
    metadata:
      labels:
        component: scheduler
        tier: control-plane
        version: second
        k8s-custom-scheduler: my-scheduler
    spec:
      serviceAccountName: my-scheduler
      automountServiceAccountToken: false

After setting this my scheduler pod is not coming up and it says CrashLoopBackOff

Error:

I0325 17:37:50.304810       1 flags.go:33] FLAG: --write-config-to=""
I0325 17:37:50.891504       1 serving.go:319] Generated self-signed cert in-memory
W0325 17:37:51.168023       1 authentication.go:387] failed to read in-cluster kubeconfig for delegated authentication: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
W0325 17:37:51.168064       1 authentication.go:249] No authentication-kubeconfig provided in order to lookup client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication won't work.
W0325 17:37:51.168072       1 authentication.go:252] No authentication-kubeconfig provided in order to lookup requestheader-client-ca-file in configmap/extension-apiserver-authentication in kube-system, so request-header client certificate authentication won't work.
W0325 17:37:51.168089       1 authorization.go:177] failed to read in-cluster kubeconfig for delegated authorization: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
W0325 17:37:51.168102       1 authorization.go:146] No authorization-kubeconfig provided, so SubjectAccessReview of authorization tokens won't work.
W0325 17:37:51.168111       1 options.go:298] Neither --kubeconfig nor --master was specified. Using default API client. This might not work.
invalid configuration: no configuration has been provided

I believe we need to configure something more along with automountServiceAccountToken: false. Can someone point me to the additional configurations needed to use automountServiceAccountToken: false?

kubernetes-pod kubernetes security
1 Answers

Configure Service Accounts for Pods

You can access the API from inside a pod using automatically mounted service account credentials. In version 1.6+, you can opt out of automounting API credentials for a service account by setting automountServiceAccountToken: false on the service account or for a particular pod.

So, when you are creating a ServiceAccount and a Deployment like in your example yaml files, credentials for accessing the Kubernetes API are not automatically mounted to the Pod. But your k8s Deployment 'my-scheduler' requires them to access the API.

You can test your ServiceAccount with some dummy Deployment of nginx, for example. And it will work without mounting credentials.

Also, if you create a ServiceAccount like in your example:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-scheduler
  namespace: kube-system
automountServiceAccountToken: false

You can manually mount the API credentials like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-scheduler
  namespace: kube-system
...
spec:
  ...
  template:
    metadata:
      labels:
        app: my-scheduler
    spec:
      containers:
      - image: <YOUR_IMAGE>
        imagePullPolicy: Always
        name: my-scheduler
        volumeMounts:
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: kube-api-access
          readOnly: true
      serviceAccountName: my-scheduler
      volumes:
      - name: kube-api-access
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              expirationSeconds: 3607
              path: token
          - configMap:
              items:
              - key: ca.crt
                path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
              - fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
                path: namespace

See the Managing Service Accounts link for more information.

user_177394400
Related