Overview
I'm using artifacts to persist some files and builds across stages. In one case it is working, and in another, it is not despite being able to see the artifact is published.
What Works
For example, this uploads my k8s manifests to an artifact so they can be access in the deployment stage to AKS and it works perfectly:
# publishStage.yaml
stages:
- stage: Publish
displayName: Publish artifacts
dependsOn:
- SeleniumTests
- Changed
condition: succeeded()
variables:
anyServicesChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'] ]
anyConfigsChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.anyConfigsChanged'] ]
jobs:
- job: Publish
condition: or(eq(variables.anyServicesChanged, true), eq(variables.anyConfigsChanged, true), eq(variables['Build.Reason'], 'Manual'))
displayName: Publishing artifacts...
steps:
- upload: k8s
artifact: k8s
# deployStage.yaml
parameters:
- name: tag
default: ''
- name: tagVersion
default: ''
stages:
- stage: Deploy
displayName: Deployment stage...
dependsOn:
- Publish
- Changed
condition: succeeded()
variables:
anyServicesChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'] ]
anyConfigsChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.anyConfigsChanged'] ]
servicesChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.servicesChanged'] ]
jobs:
- deployment: Deploy
condition: or(eq(variables.anyServicesChanged, true), eq(variables.anyConfigsChanged, true), eq(variables['Build.Reason'], 'Manual'))
displayName: Deploying services...
environment: 'App Production AKS'
strategy:
runOnce:
deploy:
steps:
- task: KubernetesManifest@0
displayName: Create imagePullSecret
inputs:
action: createSecret
secretName: $(imagePullSecret)
kubernetesServiceConnection: 'App Production AKS'
dockerRegistryEndpoint: $(dockerRegistryServiceConnection)
- template: deployStep.yaml
parameters:
tag: ${{ parameters.tag }}
tagVersion: ${{ parameters.tagVersion }}
serviceName: api-v1
pathName: api
# deployStep.yaml
parameters:
- name: tag
default: ''
- name: tagVersion
default: ''
- name: serviceName
default: ''
- name: pathName
default: ''
steps:
- task: KubernetesManifest@0
condition: contains(variables['servicesChanged'], '${{ parameters.serviceName }}')
displayName: Deploy to ${{ parameters.pathName }} Kubernetes cluster...
inputs:
action: deploy
kubernetesServiceConnection: 'App Production AKS'
manifests: |
$(Pipeline.Workspace)/k8s/aks/${{ parameters.pathName }}.yaml
imagePullSecrets: |
$(imagePullSecret)
containers: |
$(containerRegistry)/$(imageRepository)-${{ parameters.pathName }}:${{ parameters.tag }}-${{ parameters.tagVersion }}
What Doesn't Work
However, now what I'm trying to do is build a service in the application, run unit tests, and take the exact same build that was tested and use it for building the Docker image.
I'm trying to do this with the following:
# unitTestsStage.yaml
stages:
- stage: UnitTests
displayName: Run unit tests for services...
dependsOn: Changed
condition: succeeded()
jobs:
- template: ../secretsJob.yaml
- template: pythonJob.yaml
parameters:
serviceName: api-v1
pathName: api
# pythonJob.yaml
parameters:
- name: serviceName
type: string
default: ''
- name: pathName
type: string
default: ''
jobs:
- job: UnitTests
displayName: Running unit tests for ${{ parameters.serviceName }}...
variables:
servicesChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.servicesChanged'] ]
condition: or(contains(variables['servicesChanged'], '${{ parameters.serviceName }}'), eq(variables['Build.Reason'], 'Manual'))
dependsOn: Secrets
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.8'
- script: |
cd ${{ parameters.pathName }} &&
python -m pip install --upgrade pip &&
pip install -r requirements.txt
displayName: Install requirements for ${{ parameters.pathName }}...
- script: cd ${{ parameters.pathName }} && coverage run --omit='manage.py,config/*,.venv*,*/*__init__.py,*/tests.py,*/admin.py' manage.py test && coverage report
displayName: Run unit tests and coverage for ${{ parameters.pathName }}...
env:
DJANGO_SECRET_KEY: $(PROD-DJANGOSECRETKEY)
DJANGO_DEBUG: $(PROD-DJANGODEBUG)
DOMAIN: $(PROD-DOMAIN)
PGDATABASE: $(PROD-PGDATABASE)
PGDATABASEV2: $(PROD-PGDATABASEV2)
PGUSER: $(PROD-PGUSER)
PGPASSWORD: $(PROD-PGPASSWORD)
PGHOST: $(PROD-PGHOST)
PGPORT: $(PROD-PGPORT)
- upload: ${{ parameters.pathName }}
artifact: ${{ parameters.pathName }}
condition: succeeded()
# buildStage.yaml
parameters:
- name: tag
default: ''
- name: tagVersion
default: ''
#
stages:
- stage: BuildAndPush
displayName: Build and Push Docker images of services...
dependsOn:
- UnitTests
- Changed
condition: succeeded()
variables:
anyServicesChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'] ]
servicesChanged: $[ stageDependencies.Changed.Changes.outputs['detectChanges.servicesChanged'] ]
jobs:
- job: BuildAndPush
condition: or(eq(variables.anyServicesChanged, true), eq(variables['Build.Reason'], 'Manual'))
displayName: Building and Push Docker images of services...
steps:
- template: buildStep.yaml
parameters:
tag: ${{ parameters.tag }}
tagVersion: ${{ parameters.tagVersion }}
serviceName: api-v1
pathName: api
# buildStep.yaml
parameters:
- name: tag
default: ''
- name: tagVersion
default: ''
- name: serviceName
default: ''
- name: pathName
default: ''
steps:
- task: Docker@2
condition: contains(variables['servicesChanged'], '${{ parameters.serviceName }}')
displayName: Build and Push ${{ parameters.pathName }} Docker image
inputs:
command: buildAndPush
repository: $(imageRepository)-${{ parameters.pathName }}
dockerfile: $(Pipeline.Workspace)/${{ parameters.pathName }}/Dockerfile
buildContext: $(Pipeline.Workspace)/${{ parameters.pathName }}
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
${{ parameters.tag }}-${{ parameters.tagVersion }}
I can see the api artifact was published:
But then when it comes actually pulling it for building the Docker image I just get:
Starting: Build and Push api Docker image
==============================================================================
Task : Docker
Description : Build or push Docker images, login or logout, start or stop containers, or run a Docker command
Version : 2.187.0
Author : Microsoft Corporation
Help : https://aka.ms/azpipes-docker-tsg
==============================================================================
##[error]Unhandled: No Dockerfile matching /home/vsts/work/1/api/Dockerfile was found.
Finishing: Build and Push api Docker image
If I had a script task to ls -la $(Pipeline.Workspace) I get:
drwxr-xr-x 6 vsts docker 4096 Jul 9 16:19 .
drwxr-xr-x 7 vsts root 4096 Jul 9 16:19 ..
drwxr-xr-x 2 vsts docker 4096 Jul 9 16:19 TestResults
drwxr-xr-x 2 vsts docker 4096 Jul 9 16:19 a
drwxr-xr-x 2 vsts docker 4096 Jul 9 16:19 b
drwxr-xr-x 12 vsts docker 4096 Jul 9 16:19 s
Question
So what am I doing wrong here that referencing the artifact is working in one case and not the other?
