How to download build artifact from other versions (runs) published build artifact?

user_5880017 0
Viewed 266

My pipeline publishes two different build artifacts when all its tests have passed - stage: publish_pipeline_as_build.

One of my tests needs to use the build that was made in the current run, of the current version. But additionally, I need to get the build artifact of the previous version, in order to run some compatibility tests.

How do I download the build artifact from that other pipeline run?

I know the build artifact name (from runtime script), but how would I find that?

I tried playing around with azure-cli az pipelines runs artifact list. It requies a --run-id and actually my script won’t have that. So far I kind of managed, assuming the response of az pipelines runs list retuns the latest match to the query first:

az pipelines runs list --project PROJNAME --query "[?sourceBranch=='refs/heads/releases/R21.3.0.2']" | jq '.[0]'

I currently seem to run out of Ideas. Perhaps just some confused/frustrated questions that pop up:

  • How can I find that specific build artifact name's latest version and download it?
  • How are pipeline tasks fed with runtime generated values?
  • Is this so ridiculously difficult when doing it in Azure DevOps, or am I just going the wrong way?

The job I'm trying to get there with:

jobs:
- job: test_session_integration
  dependsOn: easysales_Build
  steps:
    - template: ./utils/cache_yarn_and_install.yml
    - template: ./utils/update_webdriver.yml
    - template: ./utils/download_artifact.yml
      parameters:
        artifact: easysales_$(Build.BuildId)_build
        path: $(System.DefaultWorkingDirectory)/dist
    # current release name as output
    - template: ./utils/get_release_name.yml 
    # previous release name, branch and build name output
    - template: ./utils/get_prev_release.yml
    # clone prev version manually - can't use output variables as task input
    # (BTW: why? that is super inconvenient, is there really no way?)
    - bash: |
        git clone --depth 1 -b $(get_prev_release.BRANCH_NAME) \
          "https://${REPO_USERNAME}:${REPO_TOKEN}@dev.azure.com/organisation/PROJECTNAME/_git/frontend-app" \
          ./reference
      workingDirectory: $(System.DefaultWorkingDirectory)
      env:
        REPO_TOKEN: $(GIT_AUTH_TOKEN)
        REPO_USERNAME: $(GIT_AUTH_USERNAME)
      name: clone_reference_branch

Any clues?

I'd be glad for any rubber ducking hints or clues on how I would be able to achieve what I need.
I'm new to Azure DevOps and currently struggle to find orientation in the vast but also quite in many places bits and pieces documentation Microsoft offers to me. It's fine, but frankly I struggle quite a bit with it. Is it just me having this problem?

All stages and full YAML on pastebin

The main template with the stages (Expanded templates made with "download full YAML"):

stages:
  - stage: install_prepare
    displayName: install & prepare
    jobs:
      - template: az_templates/install_hls_lib_build_job.yml

  - stage: test_and_build
    displayName: test and build projects
    dependsOn: install_prepare
    jobs:
      - template: az_templates/build_projects_jobs.yml
      - template: az_templates/test_session_integration_job.yml

  - stage: publish_pipeline_as_build
    displayName: Publish finished project artifacts as builds
    dependsOn: test_and_build
    jobs:
      - template: az_templates/build_artifact_publish_jobs.yml
azure-pipelines azure-devops
1 Answers

I by now found a solution. Perhaps not a definitive one, but should sort of work:

  1. In my library variable group I added a azure devops personal access token with read access to the necessary groups as ADO_PAT_TOKEN

  2. I Sign in with azure-cli with that token

  3. I get the latest run id with azure cli

      - bash: |
          az devops configure --defaults organization=$(System.CollectionUri)
          az devops configure --defaults project=$(System.TeamProject)
          echo "$AZ_DO_TOKEN" | az devops login
          AZ_QUERY="[?sourceBranch=='refs/heads/$(prev_release.BRANCH_NAME)'] | [0].id"
          ID=$(az pipelines runs list --query-order FinishTimeDesc --query "$AZ_QUERY")
          echo "##vso[task.setvariable variable=ID;isOutput=true]$ID"
        env:
          AZ_DO_TOKEN: $(ADO_PAT_TOKEN)
        name: prev_build_run
  1. I then download the artifact with azure-cli and the queried run id

      - bash: |
          az pipelines runs artifact download \
            --artifact-name 'easysales_$(prev_release.PREV_RELEASE_VERSION)' \
            --run-id $(prev_build_run.ID) \
            --path '$(System.DefaultWorkingDirectory)/reference/dist/easySales'
        workingDirectory: $(System.DefaultWorkingDirectory)
        name: download_prev_release_build_artifact

This roughly seems to work for me now... finally

Missing

The personal access token I added to the secrets may work, but AFAIS these tokens can not be created with no expiry date further than one year in the future. That is not ideal, since I don't want my pipeline to stop working and perhaps no one around knows how to fix this.

Perhaps someone knows how I can use azure CLI within the current pipeline without authentication?
Given it accesses only the current organization and project?

Or does anyone see a more elegant solution to my admittedly clumsy solution.

user_58800170
Related