GKE Autopilot Not Connecting to Cloud Filestore

user_5458157 0
Viewed 470

I have been trying to attach a PersistentVolume, backed by Cloud Filestore, to my Deployment in GKE Autopilot. The Filestore Instance is on the same network as the Autopilot cluster, though I am consistently receiving the following errors:

"Unable to attach or mount volumes: unmounted volumes=[prd-fileserver-tmp-dir-volume], unattached volumes=[kube-api-access-fk6zj prd-fileserver-tmp-dir-volume localssd]: timed out waiting for the condition"

And when I run kubectl describe pod [POD_NAME] on the cluster, I see this error in the output:

Output: mount.nfs: access denied by server while mounting 10.209.161.2:/prd-fileserver-tmp-dir-volume

Below are my YAML files:

Filestore-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: prd-fileserver
spec:
  capacity:
    storage: 20Gi
  accessModes:
  - ReadWriteMany
  nfs:
    path: /prd_fs_share
    server: [IP_ADDRESS]

Filestore-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: prd-fileserver-claim
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: ""
  volumeName: prd-fileserver
  resources:
    requests:
      storage: 20Gi

deployment.yaml

# This is how I am mounting the PVC
volumes:
      - name: prd-fileserver-tmp-dir-volume
        persistentVolumeClaim:
          claimName: prd-fileserver-claim
          readOnly: false
# Attaching to container
- mountPath: /mnt/disks/fileserver/tmp
          name: prd-fileserver-tmp-dir-volume

When I run this same configuration in GKE standard environment, everything works fine. My question is: can you use Cloud Filestore in GKE Autopilot? I am not seeing anythingthat suggests you can't.

Accessing file shares from Google Kubernetes Engine clusters from the public docs doesn't mention that you need to run a Privileged Pod, but I see various articles online that mention you have to run a Privileged Pod in GKE to access Filestore. That would be a limitation for GKE Autopilot given you can't run Privileged Pods.

google-kubernetes-engine google-cloud-firestore google-cloud-platform
2 Answers

According to Google Cloud Support, mounting a Filestore instance in GKE Autopilot is not possible, as of August 2021. Response below:

GKE autopilot can not run privileged pods as mentioned in this documentation. This is the tradeoff of an enforced security policies for the autopilot mode. As the autopilot exclude the nodes management, this is not possible to interact with the hosts, why privileged pods are not allowed. The recommended solution for you is running your workload on GKE standard for being able to mount filestore as expected.

user_54581570

I want add some comments to prevent people to misunderstood about this topic.

GKE autopilot can not run privileged pods

True. Autopilot mode does not allow privileged pods, to prevent the underlying nodes to be modified by those pods.

to access file shares from Google Kubernetes Engine clusters, you need to run a Privileged Pod.

Not true if the "file shares" here means Persistent Volume backed by Cloud Filestore or some other share filesystems. For PV, the actual accessing the underlying shared filesystem (mount) will be done by the node and the pod just get a mapped path. This is abstracted way and the pod actually don't know if the path is even remote or local.

I guess the docs mentioning about privileged pods are talking about "manual access" of file shares from container level. If you really need to mount a remote filesystem from your container directly, without making it as PV, the mount command will be executed inside the container and the pod should be privileged mode. (This is not a common use case)

mounting a Filestore instance in GKE Autopilot is not possible, as of August 2021.

Not true. There is no different in between standard mode and autopilot mode in the perspective of PV access.

I guess the original issue of the OP may be an ACL issue or another.

user_11110020
Related