What kubernetes permissions does GitLab runner kubernetes executor need?

user_90580 0
Viewed 3679

I've installed GitLab runner on a kubernetes cluster under a namespace gitlab-runner. Like so

# cat <<EOF | kubectl create -f -
{
  "apiVersion": "v1",
  "kind": "Namespace",
  "metadata": {
    "name": "gitlab-runner",
    "labels": {
      "name": "gitlab-runner"
    }
  }
}

# helm repo add gitlab https://charts.gitlab.io
# cat <<EOF|helm install --namespace gitlab-runner gitlab-runner -f - gitlab/gitlab-runner
gitlabUrl: https://gitlab.mycompany.com
runnerRegistrationToken: "c................Z"

The GitLab runner properly registers with the GitLab project but all jobs fail.

A quick look into the GitLab runner logs tells me that the service account used by the GitLab runner lack the proper permissions:

# kubectl logs --namespace gitlabrunner gitlab-runner-gitlab-runner-xxxxxxxxx
ERROR: Job failed (system failure): pods is forbidden: User "system:serviceaccount:gitlabrunner:default" cannot create resource "pods" in API group "" in the namespace "gitlab-runner"  duration=42.095493ms job=37482 project=yyy runner=xxxxxxx

What permission does the gitlab runner kubernetes executor need?

gitlab-ci-runner gitlab-ci kubernetes
3 Answers

I couldn't find in the GitLab runner documentation a list of permissions but I try adding permissions one by one and I compiled a list of the permission required for basic functioning.

The gitlab runner will use the service account system:serviceaccount:gitlab-runner:default so we need to create a role and assign that role to that service account.

# cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: gitlab-runner
  namespace: gitlab-runner
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["list", "get", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]

# kubectl create rolebinding --namespace=gitlab-runner gitlab-runner-binding --role=gitlab-runne r --serviceaccount=gitlab-runner:default

With that role assigned to the service account, GitLab runner will be able to create, execute and delete the pod and also access the logs.

user_905800

Unfortunately I couldn't find this in the official docs either just like @RubenLaguna stated. However, the default values.yaml of the kubernetes gitlab runner helm chart lets you define these RBAC rules nicely and does list some examples which I started with.

In my case I had to add a few and went with the following:

rbac:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "secrets", "configmaps"]
      verbs: ["get", "list", "watch", "create", "patch", "delete", "update"]
    - apiGroups: [""]
      resources: ["pods/exec", "pods/attach"]
      verbs: ["create", "patch", "delete"]
    - apiGroups: [""]
      resources: ["pods/log"]
      verbs: ["get"]
user_15213470

If you installed gitlab runners using helm chart then gitlab runners does not use the service account created by the helm chart instead it uses the default service account. There is a bug related to this. https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/353 To work around this problem we created a ClusterRoleBinding as below.

kind: ClusterRoleBinding
metadata:
  name: gitlab-runner-role-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: <helm-created-cluster-role-here>
subjects:
- kind: ServiceAccount
  name: default
  namespace: gitlab-runner
user_35710880
Related