How to determine the last successful layer in failing docker builds

Viewed 472

When a build failed in a previous version of Docker, I could count on output like this:

 ---> 8f9941c285e2
Removing intermediate container 9a543227b6b8
Step 6/21 : RUN adduser -s /bin/sh -h /home/user -D user  && mkdir -p /whl  && chown -R user:user /srv /whl
 ---> Running in 93a90935664d
 ---> 7f700d063b68
Removing intermediate container 93a90935664d

If the step failed, the intermediate image 7f700d063b68 would still exist. I could docker run it and try to figure out what happened.

But now, using docker client 20.10.2 I only get this output:

 => [dev 1/3] COPY --from=build /whl /whl
    .1s
 => [dev 2/3] COPY --chown=user:user . .
    .9s
 => ERROR [dev 3/3] RUN python3 -m pip install --find-links /whl -e ".[bpython,test]" && command -v bpython
    .3s

I don't see anything in the docker build --help output that would seem to provide more debuggability here. docker build doesn't accept --debug or --verbose flags.

How can I discover what the last successful image/layer was, so I can enter it and debug with context?

2 Answers

i think you are using BuildKit, if the variable DOCKER_VARIABLE is DOCKER_BUILDKIT=1? then you are using the Buildkit.

To use what was used previously to view the output of a build, then set,

DOCKER_BUILDKIT=0 docker build -t sometag .

The BuildKit offers many improvements compared to the previous builder.

More information on the docker documentation,

https://docs.docker.com/develop/develop-images/build_enhancements/

For those like me using Buildkit, it apparently doesn't write out every intermediate image anymore.

To debug a failing step, the recommended method is to:

  1. Comment out all the steps after the last successful one in the Dockerfile

  2. Build an image (it will reuse the cache and just write out the finished image)

  3. Run a container on the image you just generated and debug as needed

Related