Azure DevOps Pipelines - Combining variables with wrapped job variables in an each loop

Question

In this scenario I want to combine job variables from jobs passed into a template with a list of common variables defined in the template (specifically output variables from a fixed job within the template).

To demonstrate this, in this example I've extended a Microsoft sample by attempting to combine a variable within the template to the passed in job. Assume for the sake of the example that bar.foo does exist and is outputted by steps in CredScan:

parameters:
- name: jobs
  type: jobList
  default: []

jobs:
- job: CredScan                       # Cred scan first
  pool: MyCredScanPool
  steps:
  - task: MyCredScanTask@1

- ${{ each job in parameters.jobs }}: # Then each job
  - ${{ each pair in job }}:          # Insert all properties other than "dependsOn" and "variables"
      ${{ if ne(pair.key, 'dependsOn') }}:
        ${{ if ne(pair.key, 'variables') }}:
          ${{ pair.key }}: ${{ pair.value }}
    dependsOn:                        # Inject dependency
    - CredScan
    - ${{if job.dependsOn}}:            
      - ${{ job.dependsOn }}
    variables:
      bar.foo: $[ dependencies.CredScan.outputs['bar.foo'] ]
      ${{ if job.variables }}:
        ${{ job.variables }}

This gives the error "Expected a mapping". Moving the indentation around does little to change this. Added a hyphen changes this to "While parsing a block mapping, did not find expected key".

I'm guessing that this is because of the various structures you can use for variables, e.g. key pair with a colon or explicitly defining name/value. Somehow combining the variable objects isn't working. Tried looping the contents of variables and then inserting the key and values by that also failed.

Also tried the variable insertion example

${{ insert }}: ${{ job.variables }}

What does at least compile is adding the job variables by themselves:

    ${{ if job.variables }}:
      variables: ${{ job.variables }}

But then adding additional variables to this causes errors again, I assume because of the data type structures.

    ${{ if job.variables }}:
      variables: ${{ job.variables }}
      bar.foo: $[ dependencies.CredScan.outputs['bar.foo'] ]

Or

    ${{ if job.variables }}:
      variables:
      ${{ job.variables }}
      bar.foo: $[ dependencies.CredScan.outputs['bar.foo'] ]

Note: I actually wouldn't want the if job.variables check above the whole variables section, because I'd want to add my common variables regardless of if the job already has some of it's own. It's unclear if this can be removed entirely or live within the variables section.

Help greatly appreciated!

user_2885807 · Answer

Cracked it finally.

Two issues were blocking previous attempts. Firstly, I needed to again iterate over the variables within job.variables rather then just trying to dump job.variables in, but when I'd tried this previously I could never get it to compile. This was due to using hyphens and them not being needed at all in this context.

Second issue was the name of the "key" in the variable pair. In the context of variables it's "name" and not "key", which is of course obvious when you realise as they are defined that way when setting them explicitly.

The working result is as follows, hope this helps someone else:

variables:
  bar.foo: $[ dependencies.CredScan.outputs['bar.foo'] ]
  ${{ if job.variables }}:
    ${{ each var in job.variables }}:
      ${{var.name}}: ${{ var.value }}

Azure DevOps Pipelines - Combining variables with wrapped job variables in an each loop

1 Answers