in Azure DevOps, where are the stages of a pipeline actually run?

Question

I'm coming to Azure DevOps from my experience with Jenkins, where it was clear that the code in the pipelines being run is actually run on the Jenkins server (whatever machine is hosting that). So I assumed that Azure DevOps pipelines were running somehow somewhere on https://dev.azure.com/. Maybe this is true, maybe it isn't. Maybe it depends on how the "environment" is set up and what resources are in that environment. My multi-stage pipeline looks like this:

# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4

trigger:
- master

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'



stages: 
- stage: build
  displayName: Build
  jobs:
  - job: Build


    steps:

    - checkout: self


    #without this first one, bad things happen!!
    - task: NuGetCommand@2
      inputs:
        command: 'restore'
        restoreSolution: '**/*.sln'
        feedsToUse: 'config'
        nugetConfigPath: 'NuGet.config'
        
    - task: NuGetToolInstaller@1


    - task: VSBuild@1
      inputs:
        solution: '$(solution)'
        msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'

    - task: VSTest@2
      inputs:
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'

    - task: PublishBuildArtifacts@1
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)'
        ArtifactName: 'drop'
        publishLocation: 'Container'


- stage: Release
  displayName: Release
  condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
  jobs:

  - deployment:
    displayName: Release
    environment: 
        name: QA
        resourceType: VirtualMachine
    strategy:
      runOnce:
        deploy:
          steps:
          - task: CopyFiles@2
            inputs:  
              SourceFolder: '$(Agent.WorkFolder)\1\drop'
              Contents: '**\*.zip'
              OverWrite: true
              TargetFolder: 'C:\QA\XXX\YYY'

I know that 'C:\QA\XXX\YYY' is on the server I am deploying to (a resource in the QA Azure DevOps environment). But where are these locations?

$(Build.ArtifactStagingDirectory)

$(Agent.WorkFolder)

Some of these have "D:" in their path, and yet there is no D: on the server I am deploying to. Microsoft's documentation never really seems to address this issue, and yet understanding it would seem to be essential to anyone trying to get up to speed with Azure DevOps.

user_174843 · Answer

The pipeline runs on the agent, which can be located in one of two places:

Microsoft-Hosted Agents each run on a single-purpose VM which is freshly created by Microsoft, and destroyed after a single pipeline job is finished.

Self-Hosted Agents are run by you, on your infrastructure (either locally or in the cloud). It's normal for them to persist, and run multiple jobs. You can run them:

on a local physical server
on an Azure VM
on an Azure VMSS

Whatever kind of agent you use, those pre-defined variables ($(Build.ArtifactStagingDirectory) and $(Agent.WorkFolder)) correspond to local file locations on the server where the agent is running.

Your choice of the "windows-latest" pool means that your pipeline runs on a Microsoft-hosted agent, on a Windows VM.

user_781754 · Answer

pool:
   vmImage: 'windows-latest'

That's the pool you're specifying where your jobs should run.

Your deployment job is using the runOnce strategy, which is telling it to run that deployment job a single time on the windows-latest agent pool. If you want to run a job across machines within an environment, you need to use an appropriate deployment strategy, such as rolling or canary.

in Azure DevOps, where are the stages of a pipeline actually run?

2 Answers