Azure Pipelines - Handling builds for Dependent downstream pipelines

Viewed 57

We have more number of common upstream pipelines - pipleline-a, pipleline-b, pipeline-c, pipeline-d … each in its own repository - repository-a, repository-b, repository-c, repository-d… My target pipeline, say pipeline-y in repository-y, has a dependency on these upstream pipelines artifacts and the target pipeline needs to build when there is a change to any of the upstream libraries and the corresponding upstream pipeline builds successfully. In other words, target pipeline-y needs to be triggered if any of the upstream pipelines completed successfully due to changes in them (CI triggers for upstream libraries work fine in their own pipelines).

We currently achieved this, using the resources pipelines trigger in the target pipeline-y, as below:

Upstream Pipeline - pipeline-a.yml

trigger:

- repository-a*

steps

- task: Maven@3
  inputs:
  mavenPomFile: 'pom.xml'
  publishJUnitResults: false
  javaHomeOption: 'JDKVersion'
  mavenVersionOption: 'Default'
  mavenAuthenticateFeed: true
  effectivePomSkip: false
  sonarQubeRunAnalysis: false
  goals: 'package deploy'
  

Target pipeline-y.yml resources section

resources:
pipelines:

- pipeline: pipeline-a
  source: pipeline-a
  trigger:
  branches:
  - 'pipeline-a-v1*'
- pipeline: pipeline-b
  source: pipeline-b
  trigger:
  branches:
  - 'pipeline-b-v1*'
- pipeline: pipeline-c
  source: pipeline-c
  trigger:
  branches:
  - 'pipeline-c-v1*'
- pipeline: pipeline-d
  source: pipeline-d
  trigger:
  branches:
  - 'pipeline-d-v1*'
- pipeline: pipeline-e
  source: pipeline-e
  trigger:
  branches:
  - 'pipeline-e-v1*'

This works fine.

My question is, as we add more upstream common libraries, we have to update the resources section in the target downstream. When there are new versions of upstream libraries, we have to modify the version in resources-pipelines-pipiline-trigger - branches from “pipeline-a-v1” to “pipeline-a-v2”.

Is there a better way to do this? Can a variable be used in the resources-pipelines-pipeline-trigger - branches - example pipeline-a-$(version) . Can version be derived using Build system variables as below:

I tried

variables: version: $[replace(variables['Build.SourceBranchName'], variables['Build.Repository.Name'], '')]

It did not seem to work.

1 Answers

It's not possible to dynamically specify resources in YAML.

A suggestion could be to use REST API hooks when new pipelines are added. Then trigger a program that generates new YAML for pipeline-y.yml.

Related