How to update only the pipeline in Azure Devops when using YAML pipelines and gitflow?

Question

We're currently using the Classic Azure Pipelines (both build and release pipelines). They work well because if we need to make a change to a pipeline, it's easy just to modify the pipeline and schedule a build from the UI.

Now Microsoft is pushing the YAML pipelines and infrastructure-as-code so we're planning on using the YAML and multi-stage pipelines on our next project.

I haven't found a way to update only the pipeline now when the configuration is within source control. For example, say a pipeline for test environment breaks for some reason, and I need to fix it by making changes only to the YAML configuration. Would I make a change to a Release branch only and then merge it back to develop branch?

We used to have dedicated Build Pipeline and dedicated Release Pipeline in the Classic mode, so that a Build pipeline would create a package regardless of the branch it was triggered on. Then the Release Pipeline would be triggered and it would check the branch where the build artifact is coming from and deploy the package using the corresponding configuration.

This gave us a lot of flexibility so that we could deploy versions automatically and manually whenever and where ever we wanted. I'm not seeing any benefit gained from a YAML pipeline.

For example we could make a production build (master branch) and use that build in a test environment to try to reproduce errors.

How would this work with YAML pipelines when I cannot just modify only the pipeline but have to take all changes that are coming from a development branch? Is it possible (or feasible) to use only YAML build pipelines and use the classic release pipeline?

user_4013975 · Answer

We utilize deployment rings for our application, we have a singular build pipeline that utilizes conditional logic to either include or exclude specific tasks depending on the source branch. If you absolutely have to have different pipelines I would recommend utilizing template files for the common steps (very similar to task groups in classic pipelines). With YAML you can make stage, job, or step templates, so these go beyond the capabilities of the task groups. The solution that I've implemented looks something like this:

build-repo
- templates
  - stage
    - parallel-unit-test.yml
    - build-artifacts.yml
    - publish-artifacts.yml
    - deploy.yml
  - step
    - pre-build.yml
    - build.yml
    - post-build.yml
    - unit-test.yml
    - download-artifacts.yml
    - publish-artifacts.yml
    - deploy.yml
app-repo
- pipelines
  - master-release.yml
  - ring-deploy.yml
- [Project Files]

All of the templates in the build repository are coded to be as service agnostic as possible. Everything that can change/will change has been parameterized. We even use YAML logic expressions ${[ if eq(parameters.shouldRun, true) }}: to turn on or off steps that are only applicable to specific environments/rings. I would recommend reading these docs:

Templates

YAML Expressions

user_2347999 · Answer

First of all if you don't see any specific reason (like a feature which is available in YAML based pipeline) don't do this. They are not 1 to 1 and there are some features on classic release which are not available in YAML. There is a chance that gap will be closed in the future, but this is not done yet.

If you want to use yaml pipeline for builds and keep classic approach for releases it is perfectly doable. You consume artifact in the same way as you do with artifact published in classic build. And if you decide to make that change this is really easy as you may preview YAML for steps in classic pipeline.

But if you want to multi-stage approach and integrate you release as deployments there you should be familiar with two key things:

With YAML templates you will put all your 5 tasks from release stage into one unit which later you will use as a separate deployment stage. You will make parameters there instead of hard-coded values so at mail level you will define where exactly you deploy an artifact.

And with condition you will control which stage will be triggered. As a simple example you may check this YAML from documentation (you have a link above):

variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/master')]

stages:
- stage: A
  jobs:
  - job: A1
    steps:
      - script: echo Hello Stage A!

- stage: B
  condition: and(succeeded(), eq(variables.isMain, true))
  jobs:
  - job: B1
    steps:
      - script: echo Hello Stage B!
      - script: echo $(isMain)

But please be aware that with YAML you cannot trigger just specific job. So it means that you cannot run for instance deployment to DEV env. If you want to do this, you have to run whole pipeline (it may change in the future as Azure DevOps team is working on this if I recall).

user_1281548 · Answer

For the uninitiated in Classic Release Pipelines we're discussing a feature similar to an an artifact filter. Click the Pre-deployment conditions for the stage and then click Add under Artifact filters. Select your artifact and specify which branches the stage should trigger on. Here we can specify that the Stage should only trigger if the artifact was built from a certain branch.

There isn't a similarly named feature for YAML CD in Azure Pipelines. We can achieve a similar effect using a stage condition as described here. From those linked docs we can see an example;

variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/master')]

stages:
- stage: A
  jobs:
  - job: A1
    steps:
      - script: echo Hello Stage A!

- stage: B
  condition: and(succeeded(), eq(variables.isMain, true))
  jobs:
  - job: B1
    steps:
      - script: echo Hello Stage B!
      - script: echo $(isMain)

In the example stage A will always run but stage B will only run if the SourceBranch is master.

How to update only the pipeline in Azure Devops when using YAML pipelines and gitflow?

3 Answers