Gitlab CI : how to cache node_modules from a prebuilt image?

user_3430600 0
Viewed 5460

The situation is this: I'm running Cypress tests in a Gitlab CI (launched by vue-cli). To speed up the execution, I built a Docker image that contains the necessary dependencies. How can I cache node_modules from the prebuilt image to use it in the test job ? Currently I'm using an awful (but working) solution:

testsE2e:
  image: path/to/prebuiltImg
  stage: tests
  script:
    - ln -s /node_modules/ /builds/path/to/prebuiltImg/node_modules
    - yarn test:e2e
    - yarn test:e2e:report

But I think there must be a cleaner way using the Gitlab CI cache.

I've been testing:

cacheE2eDeps:
  image: path/to/prebuiltImg
  stage: dependencies
  cache:
    key: e2eDeps
    paths:
      - node_modules/
  script:
    - find / -name node_modules # check that node_modules files are there
    - echo "Caching e2e test dependencies"

testsE2e:
  image: path/to/prebuiltImg
  stage: tests
  cache:
    key: e2eDeps
  script:
    - yarn test:e2e
    - yarn test:e2e:report

But the job cacheE2eDeps displays a "WARNING: node_modules/: no matching files" error. How can I do this successfully? The Gitlab documentation doesn't really talk about caching from a prebuilt image...

The Dockerfile used to build the image :

FROM cypress/browsers:node13.8.0-chrome81-ff75

COPY . .
RUN yarn install
node-modules cypress gitlab-ci
1 Answers

There is not documentation for caching data from prebuilt images, because it’s simply not done. The dependencies are already available in the image so why cache them in the first place? It would only lead to an unnecessary data duplication.

Also, you seem to operate under the impression that cache should be used to share data between jobs, but it’s primary use case is sharing data between different runs of the same job. Sharing data between jobs should be done using artifacts.

In your case you can use cache instead of prebuilt image, like so:

variables:
  CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

testsE2e:
  image: cypress/browsers:node13.8.0-chrome81-ff75
  stage: tests
  cache:
    key: "e2eDeps"
    paths:
      - node_modules/
      - cache/Cypress/
  script:
    - yarn install
    - yarn test:e2e
    - yarn test:e2e:report

The first time the above job is run, it’ll install dependencies from scratch, but the next time it’ll fetch them from the runner cache. The caveat is that unless all runners that run this job share cache, each time you run it on a new runner it’ll install the dependencies from scratch.

Here’s the documentation about using yarn with GitLab CI.

Edit:

To elaborate on using cache vs artifacts - artifacts are meant for both storing job output (eg. to manually download it later) and for passing results of one job to another one from a subsequent stage, while cache is meant to speed up job execution by preserving files that the job needs to download from the internet. See GitLab documentation for details.

Contents of node_modules directory obviously fit into the second category.

user_96244520
Related