I'm building Docker containers in a GitHub Actions workflow, and I have a custom caching solution that uses Docker buildx. Essentially, I use the docker buildx build --cache-from and --cache-to arguments to dump the cache from the last build, store it externally, and load it for the next built in order to save time.
For the most part, this is working beautifully. Most of the build time is spent on pip install since we're using Alpine and there are no wheels available, so it has to build the packages, so skipping that step with a cache is a huge savings.
There are two relevant lines in the Dockerfile:
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
I'm encountering this odd issue, though, where requirements.txt doesn't change, and the first line (the COPY) correctly uses the cache, but the second line (the RUN) does not use the cache, and proceeds to re-build all of the packages.
I cannot figure out what could be causing this. If the COPY uses the cache properly, then it must be recognizing that requirements.txt hasn't changed, and if requirements.txt hasn't changed, I don't understand why it wouldn't use the cache for the RUN command.
And the worst part is, it's intermittent. Some builds it uses the cache, some builds it re-builds the packages, when in neither case has requirements.txt (or any preceeding Dockerfile line) changed.
Am I missing something?