Docker images disappearing over time

Viewed 4969

I loaded some docker images running

docker load --input <file>

I can then see these images when executing

docker image ls

After a while images start disappearing. Every few minutes there are less and less images listed. I did not run any of images yet. What could be the cause of this issue?

EDIT: This issue arises with docker inside minikube VM.

2 Answers

Since you've mentioned that Docker daemon runs inside minikube VM, I assume that you might hit K8s Garbage collection mechanism, which keeps system utilization on appropriate level and reduce amount of unused containers(built from images) by adjusting the specific thresholds.

These eviction thresholds are fully managed by Kubelet k8s node agent, cleaning uncertain images and containers according to the parameters(flags) propagated in kubelet configuration file.

Therefore, I guess you can investigate K8s eviction behavior looking at the certain thresholds, adjusted in kubelet config file which is generated by minikube bootstrapper in the following path /var/lib/kubelet/config.yaml.

As mention in @mk_sta answer to fix issue you need:

Create or edit /var/lib/kubelet/config.yaml with

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
    imagefs.available: "5%"

Default value is 15%

minikube stop
minikube start --extra-config=kubelet.config=/var/lib/kubelet/config.yaml

Or free more space on docker partition.

https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/#create-the-config-file

https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#hard-eviction-thresholds

Related