MountVolume.SetUp failed for volume "kube-api-access-fcz9j" : object "default"/"kube-root-ca.crt" not registered

user_6923022 0
Viewed 13868

I have an on-premise kubernetes cluster v1.22.1 (1 master & 2 worker nodes) and wanted to run jenkins slave agents on this kubernetes cluster using kubernetes plugin on jenkins. Jenkins is currently hosted outside of K8s cluster, running 2.289.3. For Kubernetes credentials in Jenkins Cloud, I have created new service account with cluster role cluster-admin and provided token secret text to Jenkins. The connection between Jenkins and Kubernetes has been established successfully however when I am running a jenkins job to create pods in kubernetes, pods are showing error and not coming online.

Below are the Kubernetes Logs. enter image description here

enter image description here

Jenkins logs enter image description here Has any experienced such issue when connecting from Jenkins master, installed outside of kubernetes cluster?

jenkins-plugins kubernetes jenkins
3 Answers

Under your pod spec you can add automountServiceAccountToken: false. As described here

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  serviceAccountName: build-robot
  automountServiceAccountToken: false
  ...

This will not work if you need to access the account credentials.

user_10564600

I understand that rootCAConfigMap publishes the kube-root-ca.crt in each namespace for default service account. Starting from kubernetes version 1.22, RootCAConfigMap is set to true by default and hence when creating pods, this certificate for default account is being used. Please find more info on bound service account token based projected volumes - https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/

To stop creating automated volume using default service account or specific service account that has been used for creating pods, simply set "automountServiceAccountToken" to "false" under serviceaccount config which should then allow jenkins to create slave pods on Kubernetes cluster. I have tested this successfully in my on premise cluster.

user_69230220

I struggled with this problem for quite a while. For me the problem was more simple. First, I needed the correct permissions. The role for that serviceaccount needed to be in my case

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: build-autoscaler
  name: autoscaler-pod-read-writer
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods", "pods/exec", "pods/logs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""] # "" indicates the core API group
  resources: ["configmaps", "secrets"]
  verbs: ["get"]

(in particular, pay attention to pods/exec)

In addition to that, bear in mind that the serviceaccount in the cloud configuration, and the service account for the pod needs to be the same (I was using default in the pod and the SA that I had created for the cloud configuration)

user_542560
Related