How to have multiple foreach iteration azurepipeline yaml?

Viewed 1491

in azuredevops yaml pipeline, can we have nested forloop ? or multiple for loop.

My requirements here is to iterate the pipeline through the mapped variable values and this should be done across environments.

Eg: In my ADO repo i have 10 folders for each apps and in each apps i have 4 different variabletemplate files for each environment ,namely "vars-development.yaml, vars-staging.yaml, vars-prod.yaml, vars-dr.yaml"

What i need to have is in each of this environment specific variable template, i need to have differet alert rule inputs (there will be multiple alerts per environment ). variables.development.yaml

variables:
  alert1:
     environment: development
     subscription: 'mysubscription'
     resourceGroup: myrg
     AlertRuleName: myalertrule
     query:myquery
     timeWindowInMinutes: 1    
     severity: 1
     operator: GreaterThanOrEqual
     threshold: 20
     
  alert2:
     environment: development
     subscription: 'mysubscription'
     resourceGroup: myrg
     AlertRuleName: myalertrule
     query:myquery
     timeWindowInMinutes: 1    
     severity: 1
     operator: GreaterThanOrEqual
     threshold: 20
     autoMitigate: false
     
etc.............

  alertn: 

azure-pipeline.yaml

name: $(Build.SourceBranchName)-$(Build.BuildId)
trigger: none

parameters:
- name: appname
  type: string
  values:
    - app1
    - app2
    - app3
    - app4
    - app5
    - app5
stages:
  - template: loganalytic-alertrulet.yaml
    parameters:
      rootModuleFolder: ${{ parameters.appname }}
      envList:
        - development
        - staging
        - dr
        - production

loganalytic-alertrulet.yaml

parameters:
- name: appname
  type: string
  default: ""
- name: envList
  type: object
- name: rootModuleFolder
  type: string

stages:
- stage: BuilD_ARM_LogAnalytic_Artifact
  displayName: 'Build and validate Query Based Alert for aggregator Count'
  jobs:
    - ${{ each environment in parameters.envList }}:
      - job: BuilD_ARM_LogAnalytic_Artifact_${{ environment }}
        variables:
          - group: ${{ environment }}-alert-config
          - template: ../${{ parameters.rootModuleFolder }}/vars-${{ environment }}.yaml      
        workspace:
          clean: all
        pool:
          ${{ if eq(environment, 'development')}}:
            name: devpool
          ${{ if eq(environment, 'staging')}}:
            name: stagepool
          ${{ if eq(environment, 'production')}}:
            name: prodpool
          ${{ if eq(environment, 'dr')}}:
            name: drpool               
        steps:
          - ${{ each alert in variable.alertList }}:        
            - task: qetza.replacetokens.replacetokens-task.replacetokens@3
              inputs:
                rootDirectory: '$(System.DefaultWorkingDirectory)/'
                targetFiles: '$(System.DefaultWorkingDirectory)/logsearch-numberofresult/*.json'
                encoding: 'auto'
                writeBOM: true
                actionOnMissing: 'warn'
                keepToken: false
                tokenPrefix: '#{'
                tokenSuffix: '}#'
                useLegacyPattern: false
                enableTelemetry: true
            - task: AzureCLI@2
              displayName: "validate the templates"
              inputs:
                azureSubscription: ${{ variables.subscription }}
                scriptType: 'bash'
                scriptLocation: 'inlineScript'
                inlineScript: 'az group deployment validate --resource-group ${{ variables.resourceGroup }} --template-file $(System.DefaultWorkingDirectory)/logsearch-numberofresult/numofresult_template.json --parameters $(System.DefaultWorkingDirectory)/ogsearch-numberofresult/numofresulparam.json'
            - task: AzureCLI@2
              displayName: "verify the change result"
              inputs:
                azureSubscription: ${{ variables.subscription }}
                scriptType: 'bash'
                scriptLocation: 'inlineScript'
                inlineScript: 'az deployment group what-if --resource-group ${{ variables.resourceGroup }} --template-file $(System.DefaultWorkingDirectory)/logsearch-numberofresult/numofresult_template.json --parameters $(System.DefaultWorkingDirectory)/logsearch-numberofresult/numofresulparam.json'          
            - task: PublishBuildArtifacts@1
              inputs:
                PathtoPublish: '$(System.DefaultWorkingDirectory)/05-logsearch-numberofresult/'
                ArtifactName: 'arm-source-${{ variables.environment }}'
                publishLocation: 'Container'
              name: "Publish_arm_code"
              displayName: "Publish arm code as build artifact"
    
- ${{ each environment in parameters.envList }}:         
    - stage: Create_LogSearch__Alert_${{ environment }}
      displayName: 'Create LogSearch Count Aggregation alert Alert ${{ environment }}'
      condition: succeeded()
      variables:
        - group: ${{ environment }}-alert-config
        - template: ../${{ parameters.rootModuleFolder }}/vars-${{ environment }}.yaml
      jobs:     
      - deployment: deploy_Alert_${{ environment }}
        ${{ if eq(variables.subscription, 'NonProd') }}:
          environment: DEV        
        ${{ if eq(variables.subscription, 'Test') }}:    
          environment: DEV
        ${{ if eq(variables.subscription, 'PROD') }}:
          environment: PROD
        ${{ if eq(variables.subscription, 'DR') }}:    
          environment: DR              
        displayName: "Creating ALert in ${{ environment }}"
        pool:
          ${{ if eq(variables.subscription, 'NonProd') }}:          
            name: devpool
          ${{ if eq(variables.subscription, 'Test') }}:
            name: stagepool
          ${{ if eq(variables.subscription, 'PROD') }}:          
            name: prodpool
          ${{ if eq(variables.subscription, 'DR') }}:
            name: drpool                
        strategy:
          runOnce:
            deploy:
              steps:
              - task: DownloadBuildArtifacts@0
                inputs:
                  artifactName: 'arm-source-${{ environment }}'
                  downloadPath: $(System.ArtifactsDirectory)              
              - task: CopyFiles@2
                inputs:
                  sourceFolder: $(System.ArtifactsDirectory)/arm-source-${{ environment }}
                  contents: '**'
                  targetFolder: $(System.DefaultWorkingDirectory)/arm-source-${{ environment }}
                  cleanTargetFolder: true                   
              - task: AzureResourceGroupDeployment@2
                ${{ if eq(environment, 'development') }}:              
                  inputs:
                    azureSubscription: 'NonProd'
                    action: 'Create Or Update Resource Group'
                    resourceGroupName: ${{ variables.resourcegroup }}
                    location: 'US'
                    templateLocation: 'Linked artifact'
                    csmFile: '$(System.DefaultWorkingDirectory)/arm-source-development/numofresult_template.json '
                    csmParametersFile: '$(System.DefaultWorkingDirectory)/arm-source-development/numofresulparam.json'
                ${{ if eq(environment, 'staging') }}:   
                  inputs:
                    azureSubscription: 'NonProd'
                    action: 'Create Or Update Resource Group'
                    resourceGroupName: ${{ variables.resourcegroup }}
                    location: 'US'
                    templateLocation: 'Linked artifact'
                    csmFile: '$(System.DefaultWorkingDirectory)/arm-source-staging/numofresult_template.json '
                    csmParametersFile: '$(System.DefaultWorkingDirectory)/arm-source-staging/numofresulparam.json'
                ${{ if eq(environment, 'production') }}:   
                  inputs:
                    azureSubscription: 'PROD'
                    action: 'Create Or Update Resource Group'
                    resourceGroupName: ${{ variables.resourcegroup }}
                    location: 'West Europe'
                    templateLocation: 'Linked artifact'
                    csmFile: '$(System.DefaultWorkingDirectory)/arm-source-production/numofresult_template.json '
                    csmParametersFile: '$(System.DefaultWorkingDirectory)/arm-source-production/numofresulparam.json'
                ${{ if eq(environment, 'dr') }}:   
                  inputs:
                    azureSubscription: 'DR'
                    action: 'Create Or Update Resource Group'
                    resourceGroupName: ${{ variables.resourcegroup }}
                    location: 'US'
                    templateLocation: 'Linked artifact'
                    csmFile: '$(System.DefaultWorkingDirectory)/arm-source-dr/numofresult_template.json '
                    csmParametersFile: '$(System.DefaultWorkingDirectory)/arm-source-dr/numofresulparam.json'                                           

so here I am using arm template to create the alert and need to generate these alert rules across the 4 environments. in Azurepipeline yaml, How i can use the 2 level foreach iteration for alert rules as well as environment widse.

1 Answers

See the example to understand how you can handle complex objects with nested loops in Azure pipelines

parameters:
- name: environmentObjects
  type: object
  default: 
  - environmentName: 'dev'
    regionAbrvs: ['westeurope']
  - environmentName: 'uat'
    regionAbrvs: ['france', 'uk']


steps:
- ${{ each environmentObject  in parameters.environmentObjects }}: 
  - ${{ each regionAbrv  in environmentObject.regionAbrvs }}:
    - script: echo ${{ regionAbrv  }}

Results:

enter image description here enter image description here enter image description here

For additional scenarios;

https://github.com/microsoft/azure-pipelines-yaml/blob/master/design/each-expression.md#scenarios

Related