Deploy nginx with helm in non-default namespace

Billy Billy 0
Viewed 101

I have seen so many issues with Helm and default namespace here on SO that it is embarrassing to post yet another one. I believe I reviewed the ones I saw and did not find a solution to my problem.

Helm version is 3.9.3 Kubernetes: OpenShift 4.9

I create an nginx chart by simply doing:

helm create disco

When I deploy this application in default project/namespace, it works fine. However, when I deploy it in non-default workspace, it fails.

My command for deploying in non-default workspace:

oc new-project helm-sandbox
helm install disco . --namespace helm-sandbox

The CLI appears to indicate that the deployment succeeded. But the pod keeps having CrashLoopBackoff.

The pod logs show following error message:

nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2022/09/08 02:00:19 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

Any idea why this is failing on me?

openshift kubernetes helm3 nginx
1 Answers

By default, every Pod use default service account on OpenShift and the service account is attached restricted SCC.

Additionally, UID in Pod is randomly chose from the range which is defined at openshift.io/sa.scc.uid-range annotation in the namespace which the Pod is defined.

So, the random UID may have no permission to write the /var/cache/nginx/ directory.

However, Change UID with anyuid or nonroot SCCs to the service account used by Pod is not good idea unless the Pod needs strong permission to work.

If you want the cache directory is an ephemeral directory, mounting emptyDir or ephemeral volume are better to solve it.

If you want persist it, mount a Persistent Volume as /var/cache/nginx.

If you need to change UID, give anyuid or nonroot SCC to a service account. But, giving a SCC to pre-defined SCC is not good. So if you need to use a SCC other than restricted, you should create a service account and use it to start your Pod.

Hopefully, those are helpful to solve the issue.

hiroyukik0
Related