I am trying to write a GitHub Actions workflow to cache my Conda environment, such that in subsequent runs, the CI/CD can skip the protracted Conda dependency resolving step and just restore what it needs from cache. (Note: I only care about caching the environment; I'm not interested in caching the Conda or pip package caches.)
From reading the documentation for actions/cache and conda-incubator/setup-miniconda, this is what I have come up with (running on GitHub-hosted Ubuntu):
- name: Setup Conda
uses: conda-incubator/setup-miniconda@v2
with:
activate-environment: MY_ENV
auto-activate-base: false
- name: Establish Conda environment directory
id: conda_env
run: |
conda info --json | jq -r '"::set-output name=dir::\(.envs_dirs[0])"'
- name: Setup Conda environment caching
id: cache
uses: actions/cache@v3
env:
# Increase this to manually invalidate the cache
CACHE_NUMBER: 0
with:
path: ${{ steps.conda_env.outputs.dir }}
key: ${{ runner.os }}-conda-${{ hashFiles('environment.yml', 'requirements.txt') }}-${{ env.CACHE_NUMBER }}
- name: Update Conda environment
run: conda env update -n MY_ENV -f environment.yml
if: steps.cache.outputs.cache-hit != 'true'
In the first run through, this appears to work just fine: It sets up the cache directory correctly to where conda-incubator/setup-miniconda is keeping its environments (by default this seems to be /usr/share/miniconda/envs); Conda does the dependency resolution and installation of packages; then, at the end of the pipeline, actions/cache archives and uploads about 2GiB of data.
In subsequent runs, the cache key is correctly identified and it starts downloading. However, the unpacking fails with the following permission errors:
/usr/bin/tar: ../../../../../usr/share/miniconda/envs: Cannot utime: Operation not permitted
/usr/bin/tar: ../../../../../usr/share/miniconda/envs: Cannot change mode to rwxr-xr-x: Operation not permitted
/usr/bin/tar: Exiting with failure status due to previous errors
Warning: Tar failed with error: The process '/usr/bin/tar' failed with exit code 2
/usr/share/miniconda/envs is definitely writeable; that's where those 2GiB of data were written to in the first run. However, only root can change things like the atime and file permissions. I presume I cannot get actions/cache to run its untar step as root. How, therefore, does one get around this?