Run specific job in Gitlab CI base on a condition

Question

I have a repo QA/tests which I want to run all the jobs when there is a push to this repo.

I used a script to generate the jobs dynamically:

job-generator:
  stage: generate
  tags:
    - kuber
  script:
    - scripts/generate-job.sh > generated-job.yml
  artifacts:
    paths:
      - generated-job.yml

main:
  trigger:
    include:
      - artifact: generated-job.yml
        job: job-generator
    strategy: depend

At the next step, I have another repo products/first which I want to run a specific job in QA/tests at every push in the products/first so I tried:

stages:
  - test

tests:
  stage: test
  variables:
    TARGET: first
  trigger:
    project: QA/tests
    branch: master
    strategy: depend

Then I tried to define a global TARGET: all variable in my main gitlab-ci.yml and override it with the TARGET: first in the above YAML.

generate-job.sh:

#!/bin/bash

PRODUCTS=("first" "second" "third")

for P in "${PRODUCTS[@]}"; do
cat << EOF
$P:
  stage: test
  tags:
    - kuber
  script:
    - echo -e "Hello from $P"
  rules:
    - if: '"$TARGET" == "all"'
      when: always
    - if: '"$TARGET" == $P'
      when: always

EOF
done

But no results. the downstream pipeline doesn't have any job at all!

Any idea?

user_3708208 · Answer

I am not sure if this is now helpful, but this looks like an over complicated approach from the outside. I have to say i have limited knowledge and my answer is based on assumption:

the QA/tests repository contains certain test cases for all repositories
QA/tests has the sole purpose of containing the tests, not an overview over the projects etc.

My Suggestion

As QA/tests is only containing tests which should be executed against each project, i would create a docker image out of it which is contains all the tests and can actually execute them. (lets calls it qa-tests:latest)

Within my projects i would add a step which uses this images, with my source code of the project and executes the tests:

qa-test:
  image: qa-tests:latest
  script:
    - echo "command to execute scripts"
  # add rules here accordingly

this would solve the issue with each push into the repositories. For an easier usage, i could create a QA-Tests.gitlab-ci.yml file which can be included by the sub-projects with

include:
  - project: QA/tests
    file: QA-Tests.gitlab-ci.yml

this way you do not need to do updates with in the repositories if the ci snippet changes.

Finally to trigger the execution on each push, you only need to trigger the pipelines of the subprojects from the QA/tests.

Disclaimer

As i said, i have only a limited few, as the goal is described but not the motivation. With this approach you remove some of the directive calls - mainly the ones triggering from sub projects to QA/tests. And it generates a clear structure, but it might not fit your needs.

user_4565015 · Answer

I solved it with:

gitlab-ci.yml:

variables:
  TARGET: all

job-generator:
  stage: generate
  tags:
    - kuber
  script:
    - scripts/generate-job.sh > generated-job.yml
  artifacts:
    paths:
      - generated-job.yml

main:
  variables:
    CHILD_TARGET: $TARGET
  trigger:
    include:
      - artifact: generated-job.yml
        job: job-generator
    strategy: depend

and use CHILD_TARGET in my generate-job.sh:

#!/bin/bash

PRODUCTS=("first" "second" "third")

for P in "${PRODUCTS[@]}"; do
cat << EOF
$P:
  stage: test
  tags:
    - kuber
  script:
    - echo -e "Hello from $P"
  rules:
    - if: '\$CHILD_TARGET == "all"'
      when: always
    - if: '\$CHILD_TARGET == "$P"'
      when: always

EOF
done

So I could call it from other projects like this:

stages:
  - test

e2e-tests:
  stage: test
  variables:
    TARGET: first
  trigger:
    project: QA/tests
    branch: master
    strategy: depend

Run specific job in Gitlab CI base on a condition

2 Answers