Gitlab CI/CD: variables in anchor/global variables not available to Runner

user_9878332 0
Viewed 3630

I want to make variables available to all my Node deployment jobs to DRY out my Gitlab pipeline, but I also want to provide other variables that are dependent on each individual job. This will help when i am dealing with multiple Node services that need unique env vars.

Reading the docs, it seems there are two ways you could do this in the .gitlab-ci.yml file:

Method #1

Include the global variables in the YAML anchor definition, hoping it will merge with the job-specific vars:

# anchor: global Node image deployment job
.deploy_node_image: &deploy_node_image
  image: docker:latest
  stage: deploy
  services:
    - docker:dind
  variables:  # global variables
    REGION: "region"
    ACCOUNT_ID: "id"
    CLUSTER_NAME: "cluster"
  script:
    - apk add py-pip
    - pip install awscli
    - echo $CLUSTER_NAME $REGION $IMAGE_NAME $ACCOUNT_ID $SERVICE_NAME $SERVICE_DIR
    - aws ecr get-login-password --region $REGION | docker login --username AWS
      --password-stdin https://$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com
    - cd $CI_PROJECT_DIR/ingenio/new-backend/$SERVICE_DIR
    - docker build
      -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_NAME:latest .
    - docker push
      $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_NAME:latest
    - aws ecs update-service
      --region $REGION
      --cluster $CLUSTER_NAME
      --service $SERVICE_NAME

# specific job
deploy:users-service:
  <<: *deploy_node_image
  variables:
    IMAGE_NAME: "users-api"
    SERVICE_NAME: "serviceName"
    SERVICE_DIR: "/endpoint"

Method #2

Define a set of global variables and merge it with the job-specific variables:

node_variables: &node_globals
  variables:
    REGION: "region"
    ACCOUNT_ID: "id"
    CLUSTER_NAME: "cluster"

# anchor: global Node image deployment job
.deploy_node_image: &deploy_node_image
  image: docker:latest
  stage: deploy
  services:
    - docker:dind
  script:
    - apk add py-pip
    - pip install awscli
    - echo $CLUSTER_NAME $REGION $IMAGE_NAME $ACCOUNT_ID $SERVICE_NAME $SERVICE_DIR
    - aws ecr get-login-password --region $REGION | docker login --username AWS
      --password-stdin https://$ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com
    - cd $CI_PROJECT_DIR/ingenio/new-backend/$SERVICE_DIR
    - docker build
      -t $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_NAME:latest .
    - docker push
      $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$IMAGE_NAME:latest
    - aws ecs update-service
      --region $REGION
      --cluster $CLUSTER_NAME
      --service $SERVICE_NAME

# specific job
deploy:users-service:
  <<: *deploy_node_image
  variables:
    <<: *node_globals
    IMAGE_NAME: "users-api"
    SERVICE_NAME: "serviceName"
    SERVICE_DIR: "/endpoint"

Neither work, and the first AWS ECR command fails because $REGION is null, as well as all the variables that were included in the Node anchor from Method 1 and the global variables from Method 2 from the log statement in the Node anchor. Only the vars included in the actual job are seen in the logs, which makes me think it just overrides.

So how can I merge them?

gitlab gitlab-ci yaml
1 Answers

Problem

When you merge two jobs, either using anchors or inheritance (using extends), the "child" job will override any sections of the parent. The sections themselves do not get merged. So the parent variables section is basically overridden and ignored by the child variables section. The extension mechanism is a part of the YAML standard and has nothing to do with GitLab.

Solution

The variables section just creates environment variables, so you can do the same thing in a before_script section, as follows:

# Note: these are global variables. You don't need to add them to an anchor.
variables:
  REGION: "region"
  ACCOUNT_ID: "id"
  CLUSTER_NAME: "cluster"

.deploy_node_image:
  image: docker:latest
  stage: deploy
  # You can define the "global" variables section here INSTEAD if you want.
  script:
    - echo $REGION
    - echo $IMAGE_NAME

deploy:users-service:
  <<: *deploy_node_image
  before_script:
    - export IMAGE_NAME="users-api"
    - export SERVICE_NAME="serviceName"
    - export SERVICE_DIR="/endpoint"

See GitLab environment variables for more info.

user_57527300
Related