Docker tagging strategy in CI systems (GitLab)

Viewed 621

We are using GitLab and thus there might be some answer specific to that platform, but I think the question is equally relevant on other CI system as well.

In our CI system we have a step to build a Docker image. Actually we have 2 separate jobs:

The first job will build a base image that includes all the external dependencies. This job will only run if the Dockerfile has changed. Which is rare.

Then we have some test jobs that use the most recent version of the base image.

If that also passes we have a second job that is using the most recent base image to create an image that already includes our source code as well and the compiled versions of the code that need compilation. Till now we taged everything with “latest”, but it is not a good solution. I wonder if there are recommended ways to tag the images. Especially given that we also start to use branches and Merge-Requests (Pull Request in GitHub speak) so we have to decide how to tag the images that might have been created in a branch and what to use during the merge request?

Is there a "best practice" for this or are there several "best practices"?

1 Answers

Based on your question, I try to briefly explain how we used docker tagging but also referencing the correct image from within the pipeline. We had quite some problems with this, as we used multiple registries (including GitLab's own). And each environment had to create custom tags and push the images to different registries. To solve all issues, we did the following (I hope this info will help you!):

  • Use a GitLab variable file to distinguish between variables for different environments and registries
  • Use Git commit SHA to tag Docker images
  • Use variable strings in deployment files
  • interpolate variables via a bash or node script

Using GitLab variable files

We always use 3 files for a pipeline:

  • .gitlab-ci-vars.yml
  • .gitlab-ci-jobs.yml
  • .gitlab-ci.yml

We use this because we can have different docker registries per environment. So now we can easily target our dev, acc and prod registry if we like. That makes a nice seperation between dev, acc and prod images.

Examples .gitlab-ci-vars.yml:

# .gitlab-ci-vars.yml

variables:
  FORCE_COLOR: 1
  NODE_ENV: dev
  CI_DOCKER_NAME: ${CI_REGISTRY_IMAGE}

.vars-dev:
  variables:
    NODE_ENV: dev
    REGISTRY: my.dev.registry
    IMAGE_NAME: ${NODE_ENV}-api:${CI_COMMIT_SHA}

.vars-acc:
  variables:
    NODE_ENV: acc
    REGISTRY: my.acc.registry
    IMAGE_NAME: ${NODE_ENV}-api:${CI_COMMIT_SHA}

.vars-prod:
  variables:
    NODE_ENV: prod
    REGISTRY: my.prod.registry
    IMAGE_NAME: ${NODE_ENV}-api:${CI_COMMIT_SHA}

Example of a reusable job in .gitlab-jobs.yml:

# .gitlab-jobs.yml

.publish_gitlab:
  stage: publish
    image: docker
    services:
      - name: docker:dind
        alias: docker
    before_script:
      - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY
    script:
      - docker build -t ${REGISTRY}/${IMAGE_NAME} .
      - docker push ${REGISTRY}/${IMAGE_NAME}

We can now invoke it from the .gitlab-ci.yml as follow:

include:
  - ".gitlab-ci-vars.yml"
  - ".gitlab-ci-jobs.yml"

publish:image:dev:
  extends: .publish_gitlab
  # target dev env vars
  variables: !reference [.vars-dev, variables]
  resource_group: dev
  environment:
    name: dev
  only:
    - develop

publish:image:acc:
  extends: .publish_gitlab
  variables: !reference [.vars-acc, variables]
  resource_group: acc
  environment:
    name: acc
  only:
    - /^release.*$/

publish:image:prod:
  extends: .publish_gitlab
  variables: !reference [.vars-prod, variables]
  resource_group: prod
  environment:
    name: prod
  only:
    - master

Use variable strings in deployment files

As the tag is random (a Git SHA) you do not know the exact name. And thus we use in K8s or any other file were we need to target our new docker image with tag, strings which we interpolate using a nodejs script. In this case we always target the correct image and will never have conflicts with other tags.

Example k8s file:

apiVersion: apps/v1
...
    spec:
      containers:
        - name: my-image-name
          # target the name we defined in .gitlab-ci-vars.yml
          image: __REPO__/__IMAGE_NAME__
          ports:
            - containerPort: __CONTAINER_PORT__
          env:
            - name: NODE_ENV
              value: __NODE_ENV__

We now provision al those variables by overriding them just before applying a deployment (or run a test, or what ever).

const fs = require('fs');
const path = require('path');

const {
  NODE_ENV,
  REGISTRY,
  IMAGE_NAME,
} = process.env

const DEFAULT_PROVISION_MAP = {
  __REPO__: REGISTRY,
  __IMAGE_NAME__: IMAGE_NAME,
  __CONTAINER_PORT__: 3000,
  __NODE_ENV__: NODE_ENV,
}

async function provisionK8sFiles() {
  const K8S_TEMPLATE_DIR = 'k8s';
  const DIR = path.join('gitlab', K8S_TEMPLATE_DIR);
  const K8S_FILES = await fs.promises.readdir(DIR);

  const MAPPING = {
    ...DEFAULT_PROVISION_MAP,
  }

  for (const file of K8S_FILES) {
    const filePath = path.join(DIR, file);

    let content = await fs.promises.readFile(filePath, {
      encoding: 'utf-8'
    });

    content = Object.keys(MAPPING).reduce((acc, curr) => {
      return acc.replace(new RegExp(curr, 'g'), MAPPING[curr])
    }, content)

    fs.writeFileSync(filePath, content, 'utf8')
  }
}

provisionK8sFiles()

And to update all the variables from within the pipeline we use a before_script to target the correct image.

  before_script:
    - node ./bin/provision-k8s.js

Conslusion

Try to make everything dynamic, and interpolate the correct image and sha via bash or node script.

Related