To verify my understanding is correct about "volume" defined in Dockerfile & specified in StatefulSet manifest of k8s

user_842225 0
Viewed 162

I am using kubernetes to deploy a postgresql pod.

My k8s manifest for that works nicely. I only paste the relevant code below:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgresql
spec:
  ...
  template:
  ...
    spec:
      containers:
      - name: postgresql
        image: postgres:14
      ...
      volumeMounts:
        - name: my-postgres
          mountPath: /var/lib/postgresql/data
        ...

  volumeClaimTemplates:
  - metadata:
      name: my-postgres
    spec:

As you can see above, under container spec, I declared the volumeMounts with a mount path. And a volumeClaimTemplates to provision the persistent in cloud.

Like said, it works fine.

when I open the Dockerfile of official postgresql here. On line 186, there is VOLUME /var/lib/postgresql/data.

I would like to verify if I understand the whole thing correctly:

  1. Is it so that the line 186 VOLUME ... in postgres Dockerfile is only meant to tell its official mount point? I mean if I use the Dockerfile to run a container, it DOESN'T create a volume but I need to specific a volume & mount to that mount point. Am I right here?

  2. In my k8s manifest above, under volumeMounts: the mountPath: value has to be exactly the same as the line 186 VOLUME ... in postgres Dockerfile, because that is the mount point they told. Am I right here?

kubernetes docker postgresql
4 Answers
  • I created simple Dockerfile to examine "VOLUME" instruction in Dockerfile as below :
[root@machine stack]# cat Dockerfile
FROM busybox
VOLUME /insidecontiner
RUN echo "writing From Container" >> /insidecontiner/file
CMD sleep 1000

[root@machine stack]# docker build -t test .
Sending build context to Docker daemon   7.68kB
Step 1/4 : FROM busybox
 ---> 69593048aa3a
Step 2/4 : VOLUME /insidecontiner
 ---> Running in 71188c3529d8
Removing intermediate container 71188c3529d8
 ---> 6a83730242a2
Step 3/4 : RUN echo "writing From Container" >> /insidecontiner/file
 ---> Running in d73b82b60333
Removing intermediate container d73b82b60333
 ---> c2cfb57a967d
Step 4/4 : CMD sleep 1000
 ---> Running in aff9f1d4a0b3
Removing intermediate container aff9f1d4a0b3
 ---> 834cc7a98e91
Successfully built 834cc7a98e91
  • There were no docker volumes in the system before running this contaainer
[root@machine stack]# docker volume ls
DRIVER              VOLUME NAME
  • Once i created a container using this image , It automatically created a docker volume
[root@machine ~]# docker run -d --name boo test
e5994bb0975635354f60a238c19db472700949946c15b6a916e0c2e4ffed33e3

[root@machine ~]# docker volume ls
DRIVER              VOLUME NAME
local               2d7ec8ea6f61cd4c8758d2da666b11b28970b9ff630053bd4109a13f4516c504

[root@machine ~]# docker volume inspect 2d7ec8ea6f61cd4c8758d2da666b11b28970b9ff630053bd4109a13f4516c504
[
    {
        "CreatedAt": "2022-06-11T13:03:05-04:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/2d7ec8ea6f61cd4c8758d2da666b11b28970b9ff630053bd4109a13f4516c504/_data",
        "Name": "2d7ec8ea6f61cd4c8758d2da666b11b28970b9ff630053bd4109a13f4516c504",
        "Options": null,
        "Scope": "local"
    }
]
  • That volume is mounted inside the container at /insidecontiner :
[root@machine ~]# docker exec -it boo /bin/sh
/ # df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  50.0G     18.7G     31.3G  37% /
tmpfs                    64.0M         0     64.0M   0% /dev
tmpfs                    62.8G         0     62.8G   0% /sys/fs/cgroup
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/mapper/rhel-root

                         50.0G     18.7G     31.3G  37% /insidecontiner

Coming to your question "Is it so that the line 186 VOLUME ... in postgres Dockerfile is only meant to tell its official mount point? I mean if I use the Dockerfile to run a container, it DOESN'T create a volume but I need to specific a volume & mount to that mount point. Am I right here?"

  • /var/lib/postgresql/data is the place where postgres keeps in datbases
  • It does create a volume automatically
  • It mounts inside the container to the location mention in the Dockerfile

Now to Kubernetes behavior

  • Creating a pod without mentioning any volumes in pod yaml run the pod successfully.ContainerRuntime on the worker node(docker in this example) will a local docker volume for that container inside the pod
[root@machine stack]# kubectl run boo --image prasasai/test
pod/boo created

[root@machine stack]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE     IP               NODE                                     NOMINATED NODE   READINESS GATES
boo    1/1     Running   0          3m51s   192.168.187.80   machine1  <none>           <none>

[root@machine1 ~]# docker volume inspect 4dbf6bc943a55c89a45d01c4aa3e860141bb7efe01ce046959c5a208dfab7e3e
[
    {
        "CreatedAt": "2022-06-11T17:26:11-04:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/4dbf6bc943a55c89a45d01c4aa3e860141bb7efe01ce046959c5a208dfab7e3e/_data",
        "Name": "4dbf6bc943a55c89a45d01c4aa3e860141bb7efe01ce046959c5a208dfab7e3e",
        "Options": null,
        "Scope": "local"
    }
]
  • If i mention volumeMounts section matching the VOLUME in Dockerfile then kubernetes is taking care of the volume mapping and ContainerRuntime is not creating the volume :
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: boo
  name: boo
spec:
  containers:
  - image: prasasai/test
    name: boo
    volumeMounts:
    - name: mount1
      mountPath: /insidecontiner
  volumes: 
  - name: mount1 
    hostPath:
      path: /here

[root@machine stack]# kubectl get pods -o wide
NAME   READY   STATUS    RESTARTS   AGE   IP               NODE                                     NOMINATED NODE   READINESS GATES
boo    1/1     Running   0          13s   192.168.187.85   machine1   <none>           <none>

[root@machine1 ~]# docker volume ls
DRIVER    VOLUME NAME

" In my k8s manifest above, under volumeMounts: the mountPath: value has to be exactly the same as the line 186 VOLUME ... in postgres Dockerfile, because that is the mount point they told. Am I right here?"

  • Yes , Thats absolutely correct.
user_36380810

In simple way,

VOLUME in Dockerfile creates and random unnamed volumes when you run it without mounting it to any point by default.

NO it is not equalent to the one you have in k8s manifest. Volume in Dockerfile is more of a metadata to the image and the manifest k8 file is definition of what pod needs to mount with.

You don't have to copy mountpath in dockerfile. Just mention it when you are deploying it.

  • Example:

docker run -v myvolume:/my/mountpath myimage

user_112185050

if I understood you correctly, your question is "why is a volume specified in the dockerfile, if there is no volume created during the deployment of the container?"

postgresql needs a directory to write its data to. if the default configuration for postgresql is set to /var/lib/postgresql/data, the application expects that it can write its data into that directory.

when deploying postgresql in a classic, non containerized manner, it would start to write the data into this directory path which would be available on the operating system of the host it was installed on.

when deploying postgresql as a container, the operating system is virtualized by your container engine and therefore the directory "/var/lib/postgresql/data" is the directory inside of the container. that means if you restart it, all data would be lost, because it is not persisted on the host it was deployed to.

if you want to keep your data, even if the container restarts, you have to mount a volume inside the container during runtime which keeps on living after the container is stopped.

TL;DR:

  1. yes, you are correct. a volume to persist the data has to be created outside of the container and has to be mounted during the deployment onto "/var/lib/postgresql/data" if you don't alter the default configuration.

  2. yes, you are correct here as well, because it is basically the same mechanism. for the container, it doesn't matter how you provide the volume and the mechanism differ in kubernetes, docker and the other container engines. when deploying on kubernetes, volumes and volumeMounts are the correct way of configuring this behaviour.

user_58512130

Is it so that the line 186 VOLUME ... in postgres Dockerfile is only meant to tell its official mount point? I mean if I use the Dockerfile to run a container, it DOESN'T create a volume but I need to specific a volume & mount to that mount point. Am I right here?

Yes and no. You need to specify a volume and mount it to achieve persistence, that much is correct. What you do NOT need to do is to mount it to that specific directory (though I would advice to do so).

In my k8s manifest above, under volumeMounts: the mountPath: value has to be exactly the same as the line 186 VOLUME ... in postgres Dockerfile, because that is the mount point they told. Am I right here?

No. You could well mount your volume anywhere into the path of /var/lib/postgresql/data, say /var/lib/postgresql/. Technically. However, that comes with its own caveats.

However, all in all, you are really reinventing the wheel. You might want to look into helm charts, specifically bitnami's PostgreSQL helm chart.

user_12967070
Related