Gitlab CI CD: keeping docker env files

Question

So I need to store somewhere .env files for docker-compose. One approach is to store their contents in masked variables in gitlab CI/CD but it seems not secure for me as hacking quite a lot of apps would only take someone to crack a gitlab account.

I would like to store .env files in a directory on server and copy them to new pulled repository path in the first job of CI/CD. I tried artifacts for that, but they are uploaded to gitlab and can be viewed there and I didn't manage to find them in the later jobs (ls in after_script didn't show them).

How could I copy .env files into all jobs and not upload them on gitlab?

.gitlab-ci.yml

before_script:
  - docker info
  - docker compose --version

copy_env_files:
  script:
    - cp /home/myuser/myapp/env.* .
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  artifacts:
    paths:
      - env.*

build_image:
  script:
    - docker-compose -f docker-compose.yml up -d --build
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  artifacts:
    paths:
      - env.*

collect_static_files:
  script:
    - docker-compose exec web python manage.py collectstatic --no-input
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  artifacts:
    paths:
      - env.*

migrate_database:
  script:
    - docker-compose exec web python manage.py migrate --no-input
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
  artifacts:
    paths:
      - env.*

after_script:
  - docker container ls
  - pwd
  - ls

VonC_6309 · Answer

How could I copy .env files into all jobs and not upload them on gitlab?

By integrating your gitlab-ci job to an external vault, where sensitive data would securely reside.

For instance: "Authenticating and reading secrets with HashiCorp Vault", but it is for GitLab premium only.

You still can use external secrets in CI

Configure your vault and secrets.

Generate your JWT and provide it to your CI job.

Runner contacts HashiCorp Vault and authenticates using the JWT.

HashiCorp Vault verifies the JWT.

HashiCorp Vault checks the bounded claims and attaches policies.

HashiCorp Vault returns the token.

Runner reads secrets from the HashiCorp Vault.

user_9016861 · Answer

I should have added "cp /home/myuser/myapp/env.* ." into before_script, not separate it as a job.

I also fixed my errors with django --no-input (by adding -T to docker exec) which occured after docker was successfully builded.

before_script:
  - docker info
  - docker compose --version
  - cp /home/myuser/myproject/env.* .

build_image:
  script:
    - docker-compose -f docker-compose.yml up -d --build
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"

collect_static_files:
  script:
    - docker-compose exec -T web python manage.py collectstatic --no-input
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"

migrate_database:
  script:
    - docker-compose exec -T web python manage.py migrate --no-input
  rules:
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"

after_script:
  - docker container ls

Gitlab CI CD: keeping docker env files

2 Answers