kustomize edit set image doesn't work with kustomize multibases and common base

user_4242801 0
Viewed 6253

I am using this example:

├── base
│   ├── kustomization.yaml
│   └── pod.yaml
├── dev
│   └── kustomization.yaml
├── kustomization.yaml
├── production
│   └── kustomization.yaml
└── staging
    └── kustomization.yaml

and in kustomization.yaml file in root:

resources:
- ./dev
- ./staging
- ./production

I also have the image transformer code in dev, staging, production kustomization.yaml:

images:
- name: my-app
  newName: gcr.io/my-platform/my-app

To build a single deployment manifest, I use:

(cd dev && kustomize edit set image my-app=gcr.io/my-platform/my-app:0.0.2 && kustomize build .)

which simply works!

to build deployment manifest for all overlays (dev, staging, production), I use:

(kustomize edit set image my-app=gcr.io/my-platform/my-app:0.0.2 && kustomize build .)

which uses the kustomization.yaml in root which contains all resources(dev, staging, production).

It does work and the final build is printed on console but without the image tag.

It seems like the kusotmize edit set image only updates the kustomizaion.yaml of the current dir.

Is there anything which can be done to handle this scenario in an easy and efficient way so the final output contains image tag as well for all deployments?

To test please use this repo

kustomize kubernetes
1 Answers

It took some time to realise what happens here. I'll explain step by step what happens and how it should work.

What happens

Firstly I re-created the same structure:

$ tree
.
├── base
│   ├── kustomization.yaml
│   └── pod.yaml
├── dev
│   └── kustomization.yaml
├── kustomization.yaml
└── staging
    └── kustomization.yaml

When you run this command for single deployment:

(cd dev && kustomize edit set image my-app=gcr.io/my-platform/my-app:0.0.2 && kustomize build .)

you change working directory to dev, manually override image from gcr.io/my-platform/my-app and adding tag 0.0.2 and then render the deployment.

The thing is previously added transformer code gets overridden by the command above. You can remove transformer code, run the command above and get the same result. And after running the command you will find out that your dev/kustomization.yaml will look like:

resources:
- ./../base
namePrefix: dev-
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
images:
- name: my-app
  newName: gcr.io/my-platform/my-app
  newTag: 0.0.2

Then what happens when you run this command from main directory:

(kustomize edit set image my-app=gcr.io/my-platform/my-app:0.0.2 && kustomize build .)

kustomize firstly goes to overlays and do transformation code which is located in overlays/kustomization.yaml. When this part is finished, image name is not my-app, but gcr.io/my-platform/my-app.

At this point kustomize edit command tries to find image with name my-app and can't do so and therefore does NOT apply the tag.

What to do

You need to use transformed image name if you run kustomize edit in main working directory:

$ kustomize edit set image gcr.io/my-platform/my-app=*:0.0.4 && kustomize build .

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: my-app
  name: dev-myapp-pod
spec:
  containers:
  - image: gcr.io/my-platform/my-app:0.0.4
    name: my-app
---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: my-app
  name: stag-myapp-pod
spec:
  containers:
  - image: gcr.io/my-platform/my-app:0.0.4
    name: my-app
user_155372010
Related