What I'm doing:
I'm dealing with a setup where the slowest step of my build (>=90% of the run time) is pushing a single image layer to a remote repository. However the file names and contents, the only updates I care about, of that layer only ever changes every few weeks or months and I'm trying to get Docker to reuse some layer that has already been uploaded.
This should in principle be possible because this layer is the the first and only layer on top of a stable image. My Dockerfile look like approximately:
FROM some_stable_base:latest
COPY ["/data/", "/data/"]
Everything works fine if I can use the local build cache, but that assumption doesn't generally apply: in practice many builds are run without the benefit of a populated cache, so the only resource common across all builds will be the remote repository the layers are uploaded to.
Also, I'm having to contend with the issue that the automated build environment has to re-get the files in /data/ each build (this is not too slow) and as a result the file's metadata is different for each build which, as best I can tell, results in each build seeing any COPY/ADD of them into the image as a new, never before seen layer that needs to be uploaded afresh.
Question:
What I'd like to do is make docker ignore (or zero out, or whatever) all the meta data (other than in my case: path and content, maybe group/owner and permissions) so that the layer matches and get that coveted "Layer already exists" when I go to push it.
Is that possible?
In principle it should be, every relevant part of the layer is the same between most builds. But in practice it's looking like docker lacks the ability to be told to ignore the irreverent parts (like file timestamps) that do change.
If it's not possible, I have some other workarounds, but they impose constraints that make "other things" more complicated.
Justifying wanting this:
This would seem to be a generally desirable feature; may builds will only care about file paths and content and be able to produce bit identical content even without a primed cache. But sharing that primed cache across build and build environments is complex and expensive. Being able to reproduce layers would remove that requirement. If fact it seems that it's not uncommon to assume things work the way I'm wanting them to.