How can I get a list of directory names to pass in as an array to an Azure Pipeline template?

Viewed 1038

I'm trying to get a list of directories and then pass that array to an azure pipeline template. Here's the code for getting the array names:

- script: |
    cd $(Build.StagingDirectory)
    directoryList=( $(ls -d */ | sed 's#/##') ) # sed gets rid of the trailing slash
    echo $directoryList
    echo "##vso[task.setvariable variable=directoryList]$directoryList"
    
# Archive all directories in staging directory created by sam build
- template: zip-template.yml
  parameters:
    directories:
      - $(directoryList)

One thing I noticed is that echo $(directoryList) only outputs the first entry. When I try to pass this to the template it only performs 1 iteration which makes me think I'm not creating the array correctly. Here's the azure pipeline template that should receive the list of directory names:

parameters:
  - name: "directories"
    type: object
    default: {}

steps:
  - ${{ each dir in parameters.directories }}:
      - task: ArchiveFiles@2
        displayName: "Zip ${{ dir }}"
        inputs:
          rootFolderOrFile: '$(Build.StagingDirectory)/${{dir}}'
          includeRootFolder: false
          archiveType: 'zip'
          archiveFile: '$(Build.StagingDirectory)/${{dir}}/package.zip'
          replaceExistingArchive: true 

Another odd outcome I observed is that the displayName outputs "Zip ($directoryList)" when I run the pipeline. What can I do to get the list of directories an pass that to the template to iterate through and run the zip task multiple times?

2 Answers

There are a number of things going on here, and ultimately I don't believe this is possible the way you're doing it.

The immediate cause of your problem with directoryList is that the syntax to access all elements of a Bash array is not $arrayName, but ${arrayName[@]}. $arrayName is equivalent to accessing element 0 (${arrayName[0]}).

BTW, parsing ls is a bad practice. A better way here would be:

IFS=$'\n' directoryList=($(find ./* -maxdepth 1 -type d -printf '%f\n'))

This prevents all manner of issues that could occur when your directories have weird names (if any contain newlines, this could still get messed up, but in this context, if that happens you probably have bigger problems).

I do not know if a VSO log command (the last line of your script) will split up a variable containing spaces or newlines into an array that you can loop over -- I'm not too familiar with the for-each template syntax. You might want to check this, though, because the log command is just writing a string of text onto standard output, it's not somehow passing the array itself back to Azure DevOps.

Another odd outcome I observed is that the displayName outputs "Zip ($directoryList)" when I run the pipeline. What can I do to get the list of directories an pass that to the template to iterate through and run the zip task multiple times?

${{ }} expressions are evaluated at template compile time, prior to when your script is running to populate the directoryList variable on the pipeline using the VSO log command. You have to use $[ ] expressions if you want to evaluate them at runtime. Unfortunately, I do not believe there is a way to use $[ ] expressions to loop over a step:

Compile time expressions can be used anywhere; runtime expressions can be used in variables and conditions.

I'm thinking your best bet is to do the archive-files step within a Bash script, perhaps looping over that array you created earlier. Or just hard-code the list of directories into your pipeline or put them in a variable in the pipeline YAML that you have to update when the list of directories changes. The best approach may depend on how often that list of directories changes.

I am afraid that it's not supported defining an array as a variable, the variable syntax is variables: { string: string }.

Here is a Ticket about the Variable, you could refer to it.

The parameters support passing the Array.

From your requirement, you need to use command to get the directory list. But the Yaml parameters couldn't be set in the command.

In summary, we cannot pass the array to the template via variable to run the archive file task multiple times.

Workaround:

You could use Powershell script to get the directory list and archive them as ZIP.

Here is the Pipeline example(The tool is 7-zip):

steps:
- powershell: |
         $arr = Get-ChildItem '$(Build.SourcesDirectory)' | 
          Where-Object {$_.PSIsContainer} | 
          Foreach-Object {$_.Name}
   
         echo "##vso[task.setvariable variable=arr;]$arr"
  displayName: 'PowerShell Script'


- powershell: |
   $string = "$(arr)"
   
   $Data=$string.split(" ")
   
   foreach($item in $Data){
   
     function create-7zip([String] $aDirectory, [String] $aZipfile){
       [string]$pathToZipExe = "$($Env:ProgramFiles)\7-Zip\7z.exe";
       [Array]$arguments = "a", "-tzip", "$aZipfile", "$aDirectory", "-r";
       & $pathToZipExe $arguments;
   }
   
   
   create-7zip "$(build.sourcesdirectory)" "$(build.sourcesdirectory)/$item/package.zip"
     
   }
  displayName: 'PowerShell Script'

Explanation:

The first Powershell task is used to get the folder list in the target folder(source directory.) Then it will be send to the variable value, the type is string.

The second powershell task could get the variable and split it.

Finally, loop the result and compress it into zip through the script.

Result:

enter image description here

Related