Is it possible to docker build a multi-staged image in parallel?

user_5345715 0
Viewed 7083

I have a rather large Dockerfile that has multiple layers. Some of these layers need quite some time. I noticed that many things do not depend on each other.

So hence the obvious question: Can I docker build in parallel?

The docker build only seems to have options that limit the build speed, instead of speeding it up (e.g. --memory).

Example: Imagine you have a Dockerfile that looks like this. I now want to call a docker build --some-flag that builds all stages in parallel unless they have to work for each other. 

FROM someImage AS stage1
# do some task

FROM otherImage AS stage2
# do another taks

FROM yetAnotherImg As stage3
# more work
COPY --from=stage2 ... # stage3 has to wait for stage2 to finish

Do you know if --some-flag exists? Do you know a different way how do achieve the goal?

EDIT:

The only thing I can think about is splitting the Dockerfile up in many more stages and thus making modifications less painful, but this is not really an ideal solution

docker-build docker parallel-processing
1 Answers

Thanks at m303945 for this answer!

This answer gives you an example how to build in parallel:

Dockerfile:

FROM alpine as s1
RUN sleep 10 && echo "s1 done"

FROM alpine as s2
RUN sleep 10 && echo "s2 done"

FROM alpine as s3
RUN sleep 10 && echo "s3 done"

Sequential: docker build . will take around 30 seconds.

Parallel: DOCKER_BUILDKIT=1 docker build . will take around 10 seconds.

Sources:

docker documentation

amazing blog post

user_53457150
Related