Masked variables when overriding service commands are empty

Question

I want to use Testcontainers for my JUNIT tests and so I created this:

image: gitlab.registry.example:5005/my-custom-maven-image

variables:
  MAVEN_CLI_OPTS: "--batch-mode -s $CI_PROJECT_DIR/.m2/settings.xml"

stages:
  - test

test:
  stage: test
  script:
    - mvn $MAVEN_CLI_OPTS clean test
  services:
    - name: docker:dind
      alias: docker
      command:
        - /bin/sh
        - -c
        - "DOCKER_AUTH_CONFIG=`echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}"` && mkdir -p "/root/.docker" && echo "${DOCKER_AUTH_CONFIG}" > "/root/.docker/config.json" && cat /root/.docker/config.json && update-ca-certificates && dockerd-entrypoint.sh || exit"
  variables:
    # Instruct Testcontainers to use the daemon of DinD.
    DOCKER_HOST: "tcp://docker:2375"
    # Instruct Docker not to start over TLS.
    DOCKER_TLS_CERTDIR: ""
    DOCKER_TLS_VERIFY: 0
    # Improve performance with overlayfs.
    DOCKER_DRIVER: overlay2

This gives me the following output when the runner tries to spawn the dind container:

{"auths":{"gitlab.registry.example:5005":{"username":"gitlab-ci-token","password":""}}}

As you can see the password is empty. Printing the CI_REGISTRY_PASSWORD variable in a before_script shows me [masked] as I would expect.

I am about to create an issue in the gitlab-runner project but I wanted to make sure what I did is not wrong beforehand.

Update: Created an issue in the gitlab-runner project

user_5465973 · Answer

It looks like a bug indeed: CI_REGISTRY_PASSWORD variable is not present at all in the container running DinD service, where it's properly set in job container.

I reproduced your issue by re-using your example in a simplified way:

test:
  stage: test
  script:
    - echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD"
    # - sleep 9999
  services:
    - name: docker:dind
      alias: docker
      command:
        - /bin/sh
        - -c
        - echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD" && dockerd-entrypoint.sh || exit

This shows in Gitlab UI:

# Services logs (not always shown)
Registry registry.novadiscovery.net - User gitlab-ci-token - Password 

# Script logs
$ echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD"
Registry registry.mycompany.com - User gitlab-ci-token - Password [MASKED]

At first I thought the variable was somehow hidden from Gitlab log UI, but instead of being shown as [masked] it was simply not shown at all. However, when inspecting underlying containers running jobs, we can see variable is indeed absent from DinD service:

# Running docker inspect command on machine running Gitlab Runner

# Inspect DinD service container
# CI_REGISTRY_PASSWORD does not exists
docker inspect runner-zz-qri9h-project-663-concurrent-0-8f92ad27e7b78f1c-docker-0 | jq .[0].Config.Env | grep CI_REGISTRY
  "CI_REGISTRY_USER=gitlab-ci-token",
  "CI_REGISTRY=registry.mycompany.com",
  "CI_REGISTRY_IMAGE=registry.mycompany.com/pierre.beucher/sandbox",

# Inspect job container
# CI_REGISTRY_PASSWORD is set
docker inspect runner-zz-qri9h-project-663-concurrent-0-8f92ad27e7b78f1c-build-2 | jq .[0].Config.Env | grep CI_REGISTRY
  "CI_REGISTRY_USER=gitlab-ci-token",
  "CI_REGISTRY_PASSWORD=xxx",
  "CI_REGISTRY=registry.mycompany.com",
  "CI_REGISTRY_IMAGE=registry.mycompany.com/pierre.beucher/sandbox",

By comparing variables between job container and service container, it seems all secret or sensible pre-defined CI variables are missing from the services containers. From the above comparison, the following variables were missing (there may be others):

CI_JOB_TOKEN
CI_BUILD_TOKEN
CI_REGISTRY_PASSWORD
CI_REPOSITORY_URL
CI_DEPENDENCY_PROXY_PASSWORD
CI_JOB_JWT

Tested on Gitlab 13.11.3 and Gitlab Runner 13.2.1

Masked variables when overriding service commands are empty

1 Answers