How to cache pip packages within Azure Pipelines

user_11685534 0
Viewed 1718

Although this source provides a lot of information on caching within Azure pipelines, it is not clear how to cache Python pip packages for a Python project.

How to proceed if one is willing to cache Pip packages on an Azure pipelines build?

According to this, it may be so that pip cache will be enabled by default in the future. As far as I know it is not yet the case.

azure-pipelines pip caching python
2 Answers

I used the pre-commit documentation as inspiration:

and configured the following Python pipeline with Anaconda:

pool:
  vmImage: 'ubuntu-latest'

variables:
  CONDA_ENV: foobar-env
  CONDA_HOME: /usr/share/miniconda/envs/$(CONDA_ENV)/

steps:
- script: echo "##vso[task.prependpath]$CONDA/bin"
  displayName: Add conda to PATH

- task: Cache@2
  displayName: Use cached Anaconda environment
  inputs:
    key: conda | environment.yml
    path: $(CONDA_HOME)
    cacheHitVar: CONDA_CACHE_RESTORED

- script: conda env create --file environment.yml
  displayName: Create Anaconda environment (if not restored from cache)
  condition: eq(variables.CONDA_CACHE_RESTORED, 'false')

- script: |
    source activate $(CONDA_ENV)
    pytest
  displayName: Run unit tests
user_950

To cache a standard pip install use this:

variables:
  # variables are automatically exported as environment variables
  # so this will override pip's default cache dir
  - name: pip_cache_dir
    value: $(Pipeline.Workspace)/.pip

steps:
  - task: Cache@2
    inputs:
      key: 'pip | "$(Agent.OS)" | requirements.txt'
      restoreKeys: |
        pip | "$(Agent.OS)"
      path: $(pip_cache_dir)
    displayName: Cache pip

  - script: |
      pip install -r requirements.txt
     displayName: "pip install"
Ted_Elliott_165010
Related