How to inject secret from Google Secret Manager into Kubernetes Pod as environment variable with Spring Boot?

user_2591194 0
Viewed 7465

For the life of Bryan, how do I do this?

Terraform is used to create an SQL Server instance in GCP. Root password and user passwords are randomly generated, then put into the Google Secret Manager. The DB's IP is exposed via private DNS zone.

How can I now get the username and password to access the DB into my K8s cluster? Running a Spring Boot app here.

This was one option I thought of:

In my deployment I add an initContainer:

- name: secrets
  image: gcr.io/google.com/cloudsdktool/cloud-sdk
  args: 
  - echo "DB_PASSWORD=$(gcloud secrets versions access latest --secret=\"$NAME_OF_SECRET\")" >> super_secret.env

Okay, what now? How do I get it into my application container from here?

There are also options like bitnami/sealed-secrets, which I don't like since the setup is using Terraform already and saving the secrets in GCP. When using sealed-secrets I could skip using the secrets manager. Same with Vault IMO.

google-secret-manager spring-boot kubernetes
5 Answers

On top of the other answers and suggestion in the comments I would like to suggest two tools that you might find interesting.

First one is secret-init:

secrets-init is a minimalistic init system designed to run as PID 1 inside container environments and it`s integrated with multiple secrets manager services, e.x. Google Secret Manager

Second one is kube-secrets-init:

The kube-secrets-init is a Kubernetes mutating admission webhook, that mutates any K8s Pod that is using specially prefixed environment variables, directly or from Kubernetes as Secret or ConfigMap.

It`s also support integration with Google Secret Manager:

User can put Google secret name (prefixed with gcp:secretmanager:) as environment variable value. The secrets-init will resolve any environment value, using specified name, to referenced secret value.

Here`s a good article about how it works.

user_121865850

How do I get it into my application container from here?

You could use a volume to store the secret and mount the same volume in both init container and main container to share the secret with the main container from the init container.

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: my-app
    image: my-app:latest
    volumeMounts:
    - name: config-data
      mountPath: /data
  initContainers:
  - name: secrets
    image: gcr.io/google.com/cloudsdktool/cloud-sdk
    args: 
    - echo "DB_PASSWORD=$(gcloud secrets versions access latest --secret=\"$NAME_OF_SECRET\")" >> super_secret.env
    volumeMounts:
    - name: config-data
      mountPath: /data
  volumes:
  - name: config-data
    emptyDir: {}
user_18394820

Using volumes of emptyDir with medium: Memory to guarantee that the secret will not be persisted.

...
volumes:
      - name: scratch
        emptyDir:
          medium: Memory
          sizeLimit: "1Gi"
...
user_38495550
user_30359470

If one has control over the image, it's possible to change the entry point and use berglas.

Dockerfile:

FROM adoptopenjdk/openjdk8:jdk8u242-b08-ubuntu  # or whatever you need

# Install berglas, see https://github.com/GoogleCloudPlatform/berglas
RUN mkdir -p /usr/local/bin/
ADD https://storage.googleapis.com/berglas/main/linux_amd64/berglas /usr/local/bin/berglas
RUN chmod +x /usr/local/bin/berglas

ENTRYPOINT ["/usr/local/bin/berglas", "exec", "--"]

Now we build the container and test it:

docker build -t image-with-berglas-and-your-app .
docker run \
    -v /host/path/to/credentials_dir:/root/credentials \
    --env GOOGLE_APPLICATION_CREDENTIALS=/root/credentials/your-service-account-that-can-access-the-secret.json \
    --env SECRET_TO_RESOLVE=sm://your-google-project/your-secret  \
    -ti image-with-berglas-and-your-app env

This should print the environment variables with the sm:// substituted by the actual secret value.

In K8s we run it with Workload Identity, so the K8s service account on behalf of which the pod is scheduled needs to be bound to a Google service account that has the right to access the secret.

In the end your pod description would be something like this:

apiVersion: v1
kind: Pod
metadata:
  name: your-app
spec:
  containers:
  - name: your-app
    image: image-with-berglas-and-your-app
    command: [start-sql-server]
    env:
      - name: AXIOMA_PASSWORD
        value: sm://your-google-project/your-secret
user_187750
Related