Docker images and containers: size on-disk, virtual size, running size confusion

user_1144952 0
Viewed 379

Could someone explain or link an article that explain about Docker 'sizes' ?

For example - I pulled some Windows based containers:

docker system df --verbose
Images space usage:

REPOSITORY                                                                                 TAG                                    IMAGE ID       CREATED        SIZE      SHARED SIZE   UNIQUE SIZE   CONTAINERS
mcr.microsoft.com/windows/servercore                                                       1909-amd64                             8116ea20e860   3 weeks ago    5.509GB   5.509GB       0B            0
mcr.microsoft.com/dotnet/framework/aspnet                                                  4.8-windowsservercore-1909             eff771fe260f   3 weeks ago    7.523GB   7.523GB       0B            0
mcr.microsoft.com/windows/servercore                                                       ltsc2019                               31902e4b25a6   4 weeks ago    5.212GB   0B            5.212GB       0
mcr.microsoft.com/powershell                                                               lts-windowsservercore-2004             7c3eedbd9958   6 weeks ago    4.84GB    0B            4.84GB        0
mcr.microsoft.com/dotnet/framework/sdk                                                     4.8                                    27c31113cb5e   3 months ago   8.667GB   0B            8.667GB       0
microsoft/mssql-server-windows-developer                                                   2017-GA                                454b8faa6c43   3 years ago    11.55GB   0B            11.55GB       0

Containers space usage:

CONTAINER ID   IMAGE                          COMMAND   LOCAL VOLUMES   SIZE      CREATED         STATUS                    NAMES
db7868c7480c   mcr.microsoft.com/powershell   "cmd"     0               0B        4 seconds ago   Exited (0) 1 second ago   happy_agnesi

Local Volumes space usage:

VOLUME NAME                                                        LINKS     SIZE
721e8ed501b31c35f3d847b09189ce21a1c33c32b0daff48fa2a17e57c92afda   0         0B
73f678eec23d645575b96df63d48c9b3c7b3c379bd11f01d12d6fc0a7180fa2d   0         0B
e1e165e9776dda6afb73ba8af79e337f6ef19268935fdf485aead0942bb5934d   0         0B
3210cffd7ef3a592487b69695338448cb9aa31917c2dc2c25e628e866a71ba6a   0         0B

Build cache usage: 0B

CACHE ID   CACHE TYPE   SIZE      CREATED   LAST USED   USAGE     SHARED

So as I understand from data above for Docker image mcr.microsoft.com/windows/servercore for example:

  1. I need around 5.509GB on my physical hard-disk ?
  2. Once I start the image docker run -d ..... does this size stacked up ? So 2 running containers of above image will take around 5.509 Gb + 5.509 Gb = 11.018 Gb on my physical hard-disk ?
  3. Most of the layers of mcr.microsoft.com/dotnet/framework/aspnet based on layers of mcr.microsoft.com/windows/servercore image - so does it have any impact on docker push command (to private repo) ? Any impact on docker pull command ?
  4. Inside the container I write a lot of logs (Gigabytes of logs). It is not something virtual - it should also take space on my physical hard-disk ? (I known it will be deleted once container stopped but the question about running state) - is there any place/command where I could see this size ? Could I limit this size on Windows images ?
  5. Any gotchas about physical disk space using that I forgot ? Stopped containers usage/holding space?
  6. A little bit explanation about memory of running container: it not shared between containers (?) so let say that running 100 memory hungry containers will effectively eat all physical memory on my computer and then what ? Running new container will be denied ? Everything gonna freeze ? Any tools to prevent this ?
docker
1 Answers
  1. If the image is 5GB you need 5GB. If you already have a few layers you only need space for the layers you don't have. For example:
# You already have an Image that consists of these layers
3333
2222
1111

# You pull an image that consists of these layers:
AAAAA   <-- You only need to pull (and need additional space) for this layer
22222
11111

If you remove the first image only the layer 3333 will be removed, but the other 2 layers are still used by a different image and can't be removed.

  1. You only need the space for the layers once no matter the amount of containers you start. A running container doesn't take up the space of the image. It creates a DIFF layer on top of the image and only needs space for the changes compared to the image. For example if you start a debian container and create a file /opt/helloworld.txt the container doesn't need the space for the debian image because all layers are shared space. You only need the space for the file /opt/helloworld.txt
  2. The same applies to your private repo only the layers that are not there will be pushed. If you base all your images on mcr.microsoft.com/windows/servercore You only need space for your own additional layers.
  3. You should NOT write in the container. Either write them to stdout or to volume. If you write them to stdout you can use logrotate and max-size rules and it will be located at /var/lib/docker/containers/<container>/<container>-json.log (your docker path might vary). And yes it takes the space on the harddrive. Your running container has an entry in /var/lib/docker/containers/overlay2/ with all the changed on the filesystem of the container compared to your image (e.g. if you write a log that didn't exist before it will be located there). If the directory only contains the logfile(s) you can create a volume for the directory and use logrotate from the host. Otherwise you're stuck with it and the space it eats up.
  4. Containers don't use up any significant space on your disk (only a few kb + stdout + filesystem changes) unless you write a lot to stdout and don't rotate the logfiles (see 4.) and the space is freed only when they are removed. Volumes are not automatically removed so they will take up space after you removed a container. They are supposed to be persistent between container restarts though.
  5. If you are out of memory the host will slowly freeze and be unresponsive or your host will start killing random stuff. Nothing special about docker afaik. You can make memory constraints if you use tools like docker-swarm or Kubernetes and apparently also with vanilla docker nowaday, but I didn't read it.
user_152190640
Related