How to setup a dynamic gitlab-ci file

Thanks to Ivan I found the perfect solution, what i've done is create 1 gitlab-ci that includes all others and within each specific file I used the only keyword like Ivan suggested.

Now my files look like this:

The gitlab-ci that is present on project level ( in each project ):

include:
  - project: 'tools/gitlab-ci'
    file: '/selector.yml'

Then my selector.yml file looks like this:

include:
  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-global.yml'

  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-nobuild.yml'

  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-legacy.yml'

  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-multitheme.yml'

  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-default.yml'

Since all files are being read, I had to move my before_script tasks to each individual stage otherwise gitlab only read the last before_script declaration, so I created a template/anchor for this:

.before_script_template: &before-script-legacy
  before_script:
    - apt-get update -y && apt-get install -yqqf openssh-client git unzip sshpass rsync --fix-missing

    # Is probably already present but just to be sure.
    - npm install -g gulp
    - npm install -g bower

    - echo "You've set the GITLABFILE var to legacy and therefore get this file"

compile_legacy:
  stage: build
  <<: *before-script-legacy
  script:
    - #Some stuff here....
  only:
    variables:
      - $GITLABFILE == "legacy" && $SKIP_COMPILE == null

Also note that each stage has to have a unique identifier/job name!

Why don't you put conditional logics inside these individual gitlab-ci files, and then just include them all.

Something like this - in your main file:

variables:
  GITLABFILE: legacy

include:
  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci.yml'

  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-legacy.yml'

  - project: 'tools/gitlab-ci'
    file: '/gitlab-ci-legacy-nobuild.yml'

And then in each sub-files - e.g. in gitlab-ci-legacy.yml:

job:
  script:
    - echo Do something
  only:
    variables:
      - $GITLABFILE == "legacy"

See more: https://docs.gitlab.com/ee/ci/yaml/#onlyvariablesexceptvariables

A possible solution: see GitLab 14.3 (September 2021)

Include GitLab CI/CD configuration based on conditions

include is one of the most popular keywords to use when writing a full CI/CD pipeline.
If you are building larger pipelines, you are probably using the include keyword to bring external YAML configuration into your pipeline.

In this release, we are expanding the power of the keyword so you can use include with rules conditions.

Now, you can decide when external CI/CD configuration should or shouldn’t be included. This will help you write a standardized pipeline with the ability to dynamically modify itself based on the conditions you choose.

See Documentation and Issue.

Example:

You can only use if rules in include, and only with certain variables.
rules keywords such as changes and exists are not supported.

include:
  - local: builds.yml
    rules:
      - if: '$INCLUDE_BUILDS == "true"'

test:
  stage: test
  script: exit 0

See GitLab 14.5 (November 2021)

Conditional includes with exists keyword

The include keyword is one of the most popular ones to use when writing a full CI/CD pipeline. If you are building larger pipelines, you are probably using the include keyword to bring external YAML configuration into your pipeline.

In this release, we are expanding the power of the keyword so you can use include with exists in the rules condition. Now, you can decide when external CI/CD configuration should or shouldn’t be included based on when certain files exist in the repository.

See Documentation and Issue.