Can I remove a single layer from from docker to prevent caching?

user_1892584 0
Viewed 585

I have a long-running docker build process, so I would prefer not to disable caching for the entire build (with --no-cache). However, I would like to invalidate caching for a particular step.

I had a bright idea: remove the cached layer and rebuild so this has to rebuild.

I used:

docker build --progress=plain

to get hold of the sha of the cached layer:

#16 [stage-9  3/15] RUN pip install -r /tmp/requirements.lock
#16 sha256:e4ac79a1eac5702cd296ccf33a1cfa2e0c3890c77d42737dc62a3b26ac3e798e
#16 CACHED

But then I got this error

> docker rmi e4ac79a1eac5702cd296ccf33a1cfa2e0c3890c77d42737dc62a3b26ac3e798
Error: No such image: e4ac79a1eac5702cd296ccf33a1cfa2e0c3890c77d42737dc62a3b26ac3e798

Is there an (easy) way of deleting this layer?

Note: For most use cases (and maybe even this one) you might like to use the --no-cache option for docker build

docker caching build
6 Answers

Not directly possible. But can be done with some changes to your docker file.

To forcibly break the cache, you can use "build-time arguments" (ARG); changing the value of that argument breaks the cache, and every step after it, for example:

FROM something
RUN apt-get update && apt-get install foo bar baz ......

ARG CACHE_DATE=2016-01-01

# steps below will always be executed if `CACHE_DATE` is changed to a unique value
RUN blablabla

And set a new date for CACHE_DATE during build:

docker build --build-arg CACHE_DATE="$(date)" ....

details taken directly from this github issue.

user_4330730

maybe you can remove by this command:

docker rm e4ac79a1eac5702cd296ccf33a1cfa2e0c3890c77d42737dc62a3b26ac3e798

Because you are still using the same image and only different containers.

user_181399530

Theres a hacky way to do this if you'd like, and it might not work for your use case... but during the tons of QA I've done over the years for my team at cycle.io - I found that if I needed to control a caching layer I could move it towards the end of the Dockerfile and put a "copy" step for any arbitrary file (say a .txt file) right before it. If you make a change to the txt file it invalidates the caching of all following steps.

user_61266010

Unfortunately, with docker, you only have the option of using --no-cache to completely turn off using the cache.

If you're happy editing the dockerfile before building the image, then the easiest way to go about this would be to add another step to the image.

While I've never tried doing this, if you only want to use the cache up to a certain point, it may be worth seeing if creating your own base/parent image is something that would work in your case. The downside to this is you would have two dockerfiles and two images but you could on your first dockerfile (the base/parent image) use the cache but then on your second dockerfile use the --no-cache option.

user_198991590

To invalidate cache for a particular step use ARGS before that step

ARGS anchor cached
RUN .... # step to

then pass different value each time

docker run --build-arg ANCHOR="`date`" .

user_41190550

Docker creates container images using layers. Each command that is found in a Dockerfile creates a new layer. Each layer contains the filesystem changes to the image for the state before the execution of the command and the state after the execution of the command.

Docker uses a layer cache to optimize and speed up the process of building Docker images.

Docker Layer Caching mainly works on the RUN, COPY and ADD commands, which will be explained in more detail next.

user_159090960
Related