inconsistent result on kubernetes permissions

user_5041045 0
Viewed 1564

I create a service account, a role, and a role binding that attaches the role to the SA.

I am trying to get the service account to list nodes but I can't, although the auth call tells me I can (for deployments it's even worst, the role do allow it, but I can't do anything):

#pods
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test auth can-i get pods
yes
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test get pods
NAME      READY   STATUS                       RESTARTS   AGE
no-st-1   0/1     CreateContainerConfigError   0          74m <<--ignore the ConfigError, this is expected.

# nodes 
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test auth can-i get nodes
Warning: resource 'nodes' is not namespace scoped
yes       <<<<------ ?? i can get nodes, but then i can't??
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test get nodes
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:psp-ns:sa-test" cannot list resource "nodes" in API group "" at the cluster scope

#deployments (at least this one is consistent, although the role should allow this)
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test auth can-i get deployments
no
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test get deployments
Error from server (Forbidden): deployments.apps is forbidden: User "system:serviceaccount:psp-ns:sa-test" cannot list resource "deployments" in API group "apps" in the namespace "psp-ns"

The role I am using should allow all 3 operations above:

Simons-MBP:psp simon$ kubectl describe role test-sa-role
Name:         test-sa-role
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","metadata":{"annotations":{},"name":"test-sa-role","namespace":"psp-ns"},"rules...
PolicyRule:
  Resources    Non-Resource URLs  Resource Names  Verbs
  ---------    -----------------  --------------  -----
  deployments  []                 []              [list get]
  nodes        []                 []              [list get]
  pods         []                 []              [list get]

And finally the role binding showing that the role is attached to this service account:

Simons-MBP:psp simon$ kubectl describe rolebinding rb-test
Name:         rb-test
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"RoleBinding","metadata":{"annotations":{},"name":"rb-test","namespace":"psp-ns"},"rol...
Role:
  Kind:  Role
  Name:  test-sa-role
Subjects:
  Kind            Name     Namespace
  ----            ----     ---------
  ServiceAccount  sa-test  psp-ns

Why is this happening? I was expecting to be able to get all nodes/pods/deployments, and that auth can-i returns yes for all calls I tried.

kubernetes
2 Answers
Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test auth can-i get nodes

Above command just checks what is the ClusterRole, Role and RoleBinding defined and since you have a Role which gives access to node it returns yes. It does not really consider the fact that node is a cluster coped resource and hence there should be a ClusterRole instead of Role.

Coming to why you are not able to access deployment it's because you might have apiGroups: [""] in the role but deployments are in apps API group. Also use the namespace psp-ns explicitly in the command

Simons-MBP:psp simon$ kubectl  --as=system:serviceaccount:psp-ns:sa-test auth can-i get deployments -n psp-ns

To be able to access node you need a ClusterRole instead of Role because node is a cluster coped resource.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" omitted since ClusterRoles are not namespaced
  name: node-clusterrole
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: psp-ns
  name: deployment-role
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "watch", "list"]

And then

  1. Create a RoleBinding or a ClusterRoleBinding to bind above ClusterRole node-clusterrole to the service account.
  2. Create a RoleBinding to bind above Role deployment-role to the service account
user_18394820

Service accounts are namespaced. From the kubernetes documentation https://kubernetes.io/docs/reference/access-authn-authz/authentication/ which says

In contrast, service accounts are users managed by the Kubernetes API. They are bound to specific namespaces, and created automatically by the API server or manually through API calls.

Namespaced API-Resources

kubectl api-resources --namespaced=true
NAME                        SHORTNAMES   APIGROUP                    NAMESPACED   KIND
bindings                                                             true         Binding
configmaps                  cm                                       true         ConfigMap
endpoints                   ep                                       true         Endpoints
events                      ev                                       true         Event
limitranges                 limits                                   true         LimitRange
persistentvolumeclaims      pvc                                      true         PersistentVolumeClaim
pods                        po                                       true         Pod
podtemplates                                                         true         PodTemplate
replicationcontrollers      rc                                       true         ReplicationController
resourcequotas              quota                                    true         ResourceQuota
secrets                                                              true         Secret
serviceaccounts             sa                                       true         ServiceAccount
services                    svc                                      true         Service
controllerrevisions                      apps                        true         ControllerRevision
daemonsets                  ds           apps                        true         DaemonSet
deployments                 deploy       apps                        true         Deployment
replicasets                 rs           apps                        true         ReplicaSet
statefulsets                sts          apps                        true         StatefulSet
localsubjectaccessreviews                authorization.k8s.io        true         LocalSubjectAccessReview
horizontalpodautoscalers    hpa          autoscaling                 true         HorizontalPodAutoscaler
cronjobs                    cj           batch                       true         CronJob
jobs                                     batch                       true         Job
leases                                   coordination.k8s.io         true         Lease
endpointslices                           discovery.k8s.io            true         EndpointSlice
events                      ev           events.k8s.io               true         Event
ingresses                   ing          extensions                  true         Ingress
ingresses                   ing          networking.k8s.io           true         Ingress
networkpolicies             netpol       networking.k8s.io           true         NetworkPolicy
poddisruptionbudgets        pdb          policy                      true         PodDisruptionBudget
rolebindings                             rbac.authorization.k8s.io   true         RoleBinding
roles                                    rbac.authorization.k8s.io   true         Role

Non-namespaced API-Resources

kubectl api-resources --namespaced=false
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
componentstatuses                 cs                                          false        ComponentStatus
namespaces                        ns                                          false        Namespace
nodes                             no                                          false        Node
persistentvolumes                 pv                                          false        PersistentVolume
mutatingwebhookconfigurations                  admissionregistration.k8s.io   false        MutatingWebhookConfiguration
validatingwebhookconfigurations                admissionregistration.k8s.io   false        ValidatingWebhookConfiguration
customresourcedefinitions         crd,crds     apiextensions.k8s.io           false        CustomResourceDefinition
apiservices                                    apiregistration.k8s.io         false        APIService
tokenreviews                                   authentication.k8s.io          false        TokenReview
selfsubjectaccessreviews                       authorization.k8s.io           false        SelfSubjectAccessReview
selfsubjectrulesreviews                        authorization.k8s.io           false        SelfSubjectRulesReview
subjectaccessreviews                           authorization.k8s.io           false        SubjectAccessReview
certificatesigningrequests        csr          certificates.k8s.io            false        CertificateSigningRequest
ingressclasses                                 networking.k8s.io              false        IngressClass
runtimeclasses                                 node.k8s.io                    false        RuntimeClass
podsecuritypolicies               psp          policy                         false        PodSecurityPolicy
clusterrolebindings                            rbac.authorization.k8s.io      false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io      false        ClusterRole
priorityclasses                   pc           scheduling.k8s.io              false        PriorityClass
csidrivers                                     storage.k8s.io                 false        CSIDriver
csinodes                                       storage.k8s.io                 false        CSINode
storageclasses                    sc           storage.k8s.io                 false        StorageClass
volumeattachments                              storage.k8s.io                 false        VolumeAttachment

And NODES are cluster resources and not namespaced. You can try clusterrole and clusterrolebinding with service account.

user_90104670
Related