Gitlab-CI: Specify that Job C should run after Job B if Job A fails

user_3012814 0
Viewed 27416

Imagine you have the following Pipeline:

Job A (deploy) -> Job B (test) -> Job C (remove test deployment)

The pipeline should deploy a test image and test it after a successful deployment. After the test I want to run a cleanup script regardless of the test output, but only if the test image (Job A) was deployed.

To summarize this: I want Gitlab to execute Job C only if Job A succeeds, but after Job B.

Things that won't work:

  • when: on-failure (Job A or Job B could failed, but only Job A is important)
  • when: always (maybe Job A failed which causes Job C to fail)
  • when: on-success (requires all jobs to succeed)

I know that GitLab has a feature called DAG Pipelines which allow you to specify multiple dependencies on other jobs with the needs keyword, but sadly the when keyword is always scoped to all prior jobs. So you are not able to say something like:

when:
    on-success: job-a
    always: job-b

Do I miss something or is there no way to achieve such a behaviour?

gitlab gitlab-ci
3 Answers

The needs DAG field can be used to conditionally execute the cleanup (Job C), if Job B fails or succeeds, but NOT when it is skipped because Job A failed.

Create 2 cleanup jobs that match the following boolean conditions:

  • (Job A succeeds and Job B succeeds): If all previous tasks succeed (Job A and Job B), we can run the cleanup with when: on_success. However, this will not trigger if Job A succeeds and Job B fails.
  • (Job A succeeds and Job B fails): To circumvent the previous scenario with an untriggered cleanup (Job C), we make use of the fact that if Job B fails, this implies that Job A succeeded in the pipeline. By creating a duplicate cleanup task and specifying a needs tag on Job B and when: on_failure, the cleanup task will only run if Job A succeeds and Job B fails.

To reiterate: a cleanup job will run if (Job A succeeds and Job B succeeds) or (Job A succeeds and Job B fails), which by boolean expression reduction is equivalent to (Job A succeeds).

An obvious caveat here is that there are now 2 cleanup jobs that are displayed in the pipeline; however, they are mutually exclusive and only one could ever be executed.

Here is a sample configuration:

stages:
  - deploy
  - test
  - cleanup

deploy_job:
  stage: deploy
  script:
    - echo Deployed
    - "true"
  when: always

test_job:
  stage: test
  script:
    - echo Executing tests
    - "true"
  when: on_success

# a YAML anchor reduces repetition
.cleanup_job: &cleanup_job
  stage: cleanup
  script:
    - echo Cleaned up deployment

cleanup_deployment_success:
  when: on_success
  <<: *cleanup_job

cleanup_deployment_failure:
  needs: ["test_job"]
  when: on_failure
  <<: *cleanup_job

With various intentional fail conditions, the following pipeline states are produced:

  • failed pipeline: Job A succeeds and Job B fails
  • failed pipeline: Job A fails and Job B is skipped
  • passed pipeline: Job A succeeds and Job B succeeds

Logically, this indicates that regardless of whether Job B succeeded or failed, Job C runs if Job A succeeded. Furthermore, the failure state is preserved in the overall pipeline.

user_143521610

The needs DAG field can be used to conditionally execute the cleanup (Job C), if Job B fails or succeeds, but NOT when it is skipped because Job A failed.

That might have changed with GitLab 13.11 (April 2021)

Optional DAG ('needs:') jobs in CI/CD pipelines

The directed acyclic graph (DAG) in GitLab CI/CD lets you use the needs syntax to configure a job to start earlier than its stage (as soon as dependent jobs complete). > We also have the rules, only, or except keywords, which determine if a job is added to a pipeline at all.

Unfortunately, if you combine needs with these other keywords, it’s possible that your pipeline could fail when a dependent job does not get added to a pipeline.

In this release, we are adding the optional keyword to the needs syntax for DAG jobs.

  • If a dependent job is marked as optional but not present in the pipeline, the needs job ignores it.
  • If the job is optional and present in the pipeline, the needs job waits for it to finish before starting.

This makes it much easier to safely combine rules, only, and except with the growing popularity of DAG.

https://about.gitlab.com/images/13_11/optional.png -- Optional DAG ('needs:') jobs in CI/CD pipelines

See Documentation and Issue.

VonC_63090

I would like to add another alternative. I am currently looking to implement this exact use case for e2e-tests and my preference will probably be to delay the reporting of test results by using artifacts. The setup would be something like this:

Job A (deploy) -> Job B (run test, collect results into artifact) -> Job C (undeploy) -> Job D (publish test results from artifact)

Job B would be configured to always be successful, and only save its findings as an artifact (e.g. JUnit xml). Job D would then publish the results and the pipeline would fail here if tests were unsuccessful. This should preserve pipeline failure state, even if Job C and Job D run unconditionally with the previously mentioned technique of duplicating the Job with multiple when: conditions.

user_9162230
Related