How can I disable concurrent jobs on the same gitlab runner with docker executor?

Viewed 1401

I am using gitlab community edition 14.4.1 along with one single gitlab runner with version 14.4.0. Its configuration reads as follows:

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "My runner"
  limit = 1
  url = "https://my-gitlab-instance.com"
  token = "my-gitlab-token"
  executor = "docker"
  [runners.custom_build_dir]
    enabled = true
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "gitlab/dind:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/builds:/builds:rw", "/cache"]
    shm_size = 1000000000

Note in particular the two options

concurrent = 1

and

[[runners]]
  limit = 1

Now, I have a pipeline where some stages have multiple jobs. From the above runner configuration, I expect that each job would run one after the other, sequentially. However, the jobs are run concurrently on the same runner, causing most of them to fail because of git locks.

Why is that so? How can I really disable concurrency within a runner? It seems like disabling concurrency is my only option. Indeed, I tried to investigate how to make it work in a concurrent way by e.g. defining in my .gitlab-ci.yml the GIT_CLONE_PATH like this

variables:
  GIT_CLONE_PATH: ${CI_BUILDS_DIR}/${CI_CONCURRENT_ID}/${CI_PROJECT_NAME}

but that doesn't work because the $CI_CONCURRENT_ID is not filled correctly by gitlab (it's always 0, whatever happens).

Increasing the number of runners is not a solution, because I can still observe the very same effect. Sometimes, multiple jobs would still run on the same runner at about the same time. It cannot be that there is no work-around, right? How can I solve this issue?

In the end, I want to use multiple runners, but of course with a distribution of max one job / runner otherwise my pipelines are not reliable (i.e. they can fail because of those annoying git locks errors). For that to work, I need concurrency within my runners to be disabled.

1 Answers

If u are looking to run jobs in a stage one at a time u can use resource groups option.

job1:
 resource_group: team
 script:
   - echo "Hello"

job2:
 resource_group: team
 script:
   - echo "hai"

In this case job1 will run and only after it is completed the other job will run.

Related