We have a complex pipeline within a monorepo, which creates multiple child-pipelines. We would like to cache some folders for each (parallel run) sub-pipeline. To do so, we define a variable, which targets the relative path of each sub-project / pipeline, like this:
cache:
key:
files:
- ${PROJECT_DIR}/package-lock.json
- ${PROJECT_DIR}/package.json
paths:
- ${PROJECT_DIR}/node_modules
- ${PROJECT_DIR}/.npm
- ${PROJECT_DIR}/cypress-cache
policy: pull
We put this is on a common base-yaml for all jobs, therefore expecting to have a cache, if the package-lock.json and package.json don't change.
Each of these child-pipelines has a job, which is responsible to get the dependencies and cache it:
prepare:
artifacts:
paths:
- ${WORKING_DIR}/package.json
- ${WORKING_DIR}/package-lock.json
- ${BUILD_ENV_FILE}
reports:
dotenv: ${BUILD_ENV_FILE}
when: always
cache:
policy: pull-push
extends: .build-base
script:
- echo HELLO PROJECT ${PROJECT_DIR:-$CI_PROJECT_DIR}
..get dependencies code etc.
The echo is a test of mine to be sure I got the correct path.
Now the strange part: the first prepare-job in a subpipeline doesn't find the cache, restores the dependencies, and pushes them, so far so good. But the second prepare-job of a different pipeline takes the exact same cache, although having different dependencies. This obviously breaks a lot of stuff, as now dependencies are missing etc.
What I don't get: I would expect the cache to check the content hash keys, which would be different between the different child prepare jobs, but this doesn't seem to be the case. The other question is about scoping: Can it generally work to have different cache paths? From the Cache URL (/project//default-6-non_protected) I see that it just caches on project level with a generic name.
I would generally expect GitLab to take the full path-path as well as the content into consideration, but that doesn't seem to be the case?