Bitbucket Pipelines - share pip libraries across multiple steps

user_9858045 0
Viewed 390

I am trying to store pip libraries installed in the initial step as artifacts, so that it can be reused in the parallel steps later on. Unfortunately, when "Test part 1" and "Test part 2" are executed, mentioned libraries does not exist.

Moreover, mentioned artifacts are not visible even in the Artifacts tab in Bitbucket window

image: python:3.8

options:
  max-time: 20

definitions:
  steps:
    - step: &fetch-and-build
        name: Update image
        caches:
          - pip
          - docker
        services:
          - docker
        script:
          - pip install -r requirements.txt -U
          - pip list -v
        artifacts:
          - /usr/local/lib/python3.8/**
  services:
    postgres:
      image: postgres
      memory: 512
      variables:
        POSTGRES_HOST_AUTH_METHOD: 'trust'
    redis:
      image: redis
      memory: 256
    docker:
      memory: 2048

  default:
    - step: *fetch-and-build
    - parallel:
      - step:
          name: "Test part 1"
          caches:
            - pip
            - docker
          script:
            - pip list -v
            - export COVERAGE_PROCESS_START=./.coveragerc
            - coverage run --parallel-mode --concurrency=multiprocessing --rcfile=./.coveragerc manage.py test -v 3 --parallel=5 payments
          services:
            - redis
            - postgres
            - docker
          artifacts:
            - htmlcov/**
            - htmlcov/index.html
            - coverage/.coverage
            - /usr/local/lib/python3.8/**
      - step:
            name: "Test part 2"
            caches:
              - pip
              - docker
            script:
              - export COVERAGE_PROCESS_START=./.coveragerc
              - coverage run --parallel-mode --concurrency=multiprocessing --rcfile=./.coveragerc manage.py test -v 3 --parallel=5 feed jobs
            services:
              - redis
              - postgres
              - docker
            artifacts:
              - htmlcov/**
              - htmlcov/index.html
              - coverage/.coverage
              - /usr/local/lib/python3.8/**
bitbucket-pipelines bitbucket
1 Answers

For the "artifacts are not visible even in the Artifacts tab in Bitbucket window"

Currently Artifacts Path in bitbucket pipeline is limited to the files in the build directory. To work around this, first copy the necessary files into the build dir, then in artifacts, mention the the path

For Example:

  - step:
      name: "Test part 1"
      caches:
        - pip
        - docker
      script:
        - pip list -v
        - mkdir artifact #to create single dir for artifacts, and create subfolders if you want to
        - cp /usr/local/lib/python3.8/** ./artifact/python #just like this you can copy multiple files which are not in your current build dir.
        - export COVERAGE_PROCESS_START=./.coveragerc
        - coverage run --parallel-mode --concurrency=multiprocessing --rcfile=./.coveragerc manage.py test -v 3 --parallel=5 payments
      services:
        - redis
        - postgres
        - docker
      artifacts:
        - ./artifact #you can mention the artifacts dir.

In the next step you can copy the files to your destinations from artifact.

user_165603570
Related