GCP credentials with skaffold for local development

Viewed 698

What is the standard approach to getting GCP credentials into k8s pods when using skaffold for local development?

When I previously used docker compose and aws it was easy to volume mount the ~/.aws folder to the container and everything just worked. Is there an equivalent solution for skaffold and gcp?

2 Answers

When I previously used docker compose and aws it was easy to volume mount the ~/.aws folder to the container and everything just worked. Is there an equivalent solution for skaffold and gcp?

You didn't mention what kind of kubernetes cluster you have deployed locally but if you use Minikube it can be actually achieved in a very similar way.

Supposed you have already initialized your Cloud SDK locally by running:

gcloud auth login
gcloud container clusters get-credentials <cluster name> --zone <zone> --project <project name>
gcloud config set project <project name>

so you can run your gcloud commands on the local machine on which Minikube is installed. You can easily delagate this access to your Pods created either by Skakffold or manually on Minikube.

You just need to start your Minikube as follows:

minikube start --mount=true --mount-string="$HOME/.config/gcloud/:/home/docker/.config/gcloud/"

To make things simple I'm mounting local Cloud SDK config directory into Minikube host using as a mount point /home/docker/.config/gcloud/.

Once it is available on Minikube host VM, it can be easily mounted into any Pod. We can use one of the Cloud SDK docker images available here or any other image that comes with Cloud SDK preinstalled.

Sample Pod to test this out may look like the one below:

apiVersion: v1
kind: Pod 
metadata:
  name: cloud-sdk-pod
spec:
  containers:
  - image: google/cloud-sdk:alpine
    command: ['sh', '-c', 'sleep 3600']
    name: cloud-sdk-container
    volumeMounts:
    - mountPath: /root/.config/gcloud
      name: gcloud-volume
  volumes:
  - name: gcloud-volume
    hostPath:
      # directory location on host
      path: /home/docker/.config/gcloud
      # this field is optional
      type: Directory

After connecting to the Pod by running:

kubectl exec -ti cloud-sdk-pod -- /bin/bash

we'll be able to execute any gcloud commands as we are able to execute them on our local machine.

If you're using Skaffold with a Minikube cluster then you can use their gcp-auth addon which does exactly this, described here in detail.

Related