The deployment environment 'Staging' in your bitbucket-pipelines.yml file occurs multiple times in the pipeline

Viewed 6646

I'm trying to get Bitbucket Pipelines to work with multiple steps that define the deployment area. When I do, I get the error

Configuration error The deployment environment 'Staging' in your bitbucket-pipelines.yml file occurs multiple times in the pipeline. Please refer to our documentation for valid environments and their ordering.

From what I read, the deployment variable has to happen on a step by step basis.

How would I set up this example pipelines file to not hit that error?

image: ubuntu:18.04

definitions:
    steps:
        - step: &build
            name: npm-build
            condition:
                changesets:
                    includePaths:
                        # Only run npm if anything in the build directory was touched
                        - "build/**"
            image: node:14.17.5
            script:
              - echo 'build initiated'
              - cd build
              - npm install
              - npm run dev
              - echo 'build complete'
            artifacts:
              - themes/factor/css/**
              - themes/factor/js/**
        - step: &deploychanges
            name: Deploy_Changes
            deployment: Staging
            script:
              - echo 'Installing server dependencies'
              - apt-get update -q
              - apt-get install -qy software-properties-common
              - add-apt-repository -y ppa:git-ftp/ppa
              - apt-get update -q
              - apt-get install -qy git-ftp
              - echo 'All dependencies installed'
              - echo 'Transferring changes'
              - git ftp init --user $FTP_USER --passwd $FTP_PASSWORD $FTP_ADDRESS push --force --changed-only -vv
              - echo 'File transfer complete'
        
        - step: &deploycompiled
            name: Deploy_Compiled
            deployment: Staging
            condition:
                changesets:
                    includePaths:
                        # Only run npm if anything in the build directory was touched
                        - "build/**"
            script:
              - echo 'Installing server dependencies'
              - apt-get update -q
              - apt-get install -qy software-properties-common
              - add-apt-repository -y ppa:git-ftp/ppa
              - apt-get update -q
              - apt-get install -qy git-ftp
              - echo 'All dependencies installed'
              - echo 'Transferring compiled assets'
              - git ftp init --user $FTP_USER --passwd $FTP_PASSWORD $FTP_ADDRESS push --all --syncroot themes/factor/css/ -vv
              - git ftp init --user $FTP_USER --passwd $FTP_PASSWORD $FTP_ADDRESS push --all --syncroot themes/factor/js/ -vv
              - echo 'File transfer complete'

pipelines:
    branches:
        master:
            - step: *build
                <<: *deploychanges
                deployment: Production
            - step:            
                <<: *deploycompiled
                deployment: Production

        dev:
            - step: *build
            - step: *deploychanges
            - step: *deploycompiled
6 Answers

The workaround for the issue with reusing Environment Variables without using the deployment clause for more than one steps in a pipeline I have found is to dump ENV VARS to a file and save it as an artifact that will be sourced in the following steps.

The code snippet for it would look like:

  steps:

    - step: &set-environment-variables
        name: 'Set environment variables'
        script:
          - printenv | xargs echo export > .envs
        artifacts:
          - .envs


    - step: &next-step
        name: "Next step in the pipeline"
        script:
          - source .envs
          - next_actions


pipelines:

  pull-requests:
    '**':
      - step:
          <<: *set-environment-variables
          deployment: my-deployment
      - step:
          <<: *next-step
          name: "Name of the next step being executed"

  branches:
    staging:
      - step:
          <<: *set-environment-variables
          deployment: my-deployment
      - step:
          <<: *next-step
          name: "Name of the next step being executed"

So far this solution works for me.

Normally what happens is that you deploy to an environment. So there is one step which deploys. So particularly u should put your "deployment" group to it specifically. This is how Bitbucket manages that if the deployment of the code happened or not. So its like you can have multiple steps where in one you are testing unit cases, integration cases, another one for building the binaries and the last one as an artifact deploys to the env marking deployment group. see the below example.

definitions: 
  steps:
    - step: &test-vizdom-services
        name: "Vizdom services unit Tests"
        image: mcr.microsoft.com/dotnet/core/sdk:3.1
        script:     
            - cd ./vizdom/vizdom.services.Tests
            - dotnet test vizdom.services.Tests.csproj    


pipelines:
  custom:
    DEV-AWS-api-deploy:  
      - step: *test-vizdom-services        
      - step:
          name: "Vizdom Webapi unit Tests"
          image: mcr.microsoft.com/dotnet/core/sdk:3.1
          script:
              - export ASPNETCORE_ENVIRONMENT=Dev       
              - cd ./vizdom/vizdom.webapi.tests
              - dotnet test vizdom.webapi.tests.csproj    
      - step:
          deployment: DEV-API
          name: "API: Build > Zip > Upload > Deploy"
          image: mcr.microsoft.com/dotnet/core/sdk:3.1
          script:
              - apt-get update
              - apt-get install zip -y
              - mkdir -p ~/deployment/release_dll
              - cd ./vizdom/vizdom.webapi
              - cp -r ../shared_settings ~/deployment              
              - dotnet publish vizdom.webapi.csproj -c Release -o ~/deployment/release_dll
              - cp Dockerfile ~/deployment/
              - cp -r deployment_scripts ~/deployment
              - cp deployment_scripts/appspec_dev.yml ~/deployment/appspec.yml
              - cd ~/deployment
              - zip -r $BITBUCKET_CLONE_DIR/dev-webapi-$BITBUCKET_BUILD_NUMBER.zip .
              - cd $BITBUCKET_CLONE_DIR
              - pipe: atlassian/aws-code-deploy:0.5.3
                variables:
                  AWS_DEFAULT_REGION: 'us-east-1'
                  AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                  AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                  COMMAND: 'upload'
                  APPLICATION_NAME: 'ss-webapi'
                  ZIP_FILE: 'dev-webapi-$BITBUCKET_BUILD_NUMBER.zip'
                  S3_BUCKET: 'ss-codedeploy-repo'
                  VERSION_LABEL: 'dev-webapi-$BITBUCKET_BUILD_NUMBER.zip'
              - pipe: atlassian/aws-code-deploy:0.5.3
                variables:
                  AWS_DEFAULT_REGION: 'us-east-1'
                  AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                  AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                  COMMAND: 'deploy'
                  APPLICATION_NAME: 'ss-webapi'
                  DEPLOYMENT_GROUP: 'dev-single-instance'
                  WAIT: 'false'
                  S3_BUCKET: 'ss-codedeploy-repo'
                  VERSION_LABEL: 'dev-webapi-$BITBUCKET_BUILD_NUMBER.zip'

So as you can see I have multiple steps for running test cases but I would finally build the binaries and deploy the code in final step. I could have broken it into separate steps but I dont want to waste the minutes of having to use another step because cloning and copying the artifact takes some time. Right now there are three steps it could have been broken into 4. where the 4th one would have been the deployment step. I hope this brings some clarity.

Also you can modify the names of the deployment groups as per your needs and can have up to 50 deployment groups :)

Little did I know, it's intentional that deploy happens in one step and you can only define on one step the deployment environment. The following setup is what worked for us (plus the appropriate separate git-ftp files):

image: ubuntu:18.04

definitions:
    steps:
        - step: &build
            name: Build
            condition:
                changesets:
                    includePaths:
                    # Only run npm if anything in the build directory was touched
                        - "build/**"
            image: node:15.0.1
            script:
              - echo 'build initiated'
              - cd build
              - npm install
              - npm run prod
              - echo 'build complete'
            artifacts:
              - themes/factor/css/**
              - themes/factor/js/**
        - step: &deploy
            name: Deploy
            deployment: Staging
            script:
              - echo 'Installing server dependencies'
              - apt-get update -q
              - apt-get install -qy software-properties-common
              - add-apt-repository -y ppa:git-ftp/ppa
              - apt-get update -q
              - apt-get install -qy git-ftp
              - echo 'All dependencies installed'
              - echo 'Transferring changes'
              - git ftp init --user $FTP_USER --passwd $FTP_PASSWORD $FTP_ADDRESS push --force --changed-only -vv
              - echo 'File transfer complete'

pipelines:
    branches:
        master:
            - step: *build
            - step:            
                <<: *deploy
                deployment: Production

        dev:
            - step: *build
            - step: *deploy

I guess we cannot use combine the Deployment with either Artifact or Cache. If I use standalone Deployment, so I can use the same deployment for multiple steps (as my screenshot). In case I add cache/artifact, will get same error as yours. enter image description here enter image description here

With your cases, to solve the problems, you either solve the errors as following options:

  • Combine all steps into one big step
  • Or create different deployment variable group Staging DeployChanges and Staging DeployComplied, this way may lead to duplicate variable

ex:

        - step: &deploychanges
            name: Deploy_Changes
            deployment: Staging DeployChanges
            script:
              - ....
        
        - step: &deploycompiled
            name: Deploy_Compiled
            deployment: Staging DeployComplied
            ....

Related