Airflow - Skip future task instance without making changes to dag file

user_11902443 0
Viewed 821

  • I have a DAG 'abc' scheduled to run every day at 7 AM CST and there is task 'xyz' in that DAG.

  • For some reason, I do not want to run one of the tasks 'xyz' for tomorrow's instance.

How can I skip that particular task instance?

  • I do not want to make any changes to code as I do not have access to Prod code and the task is in Prod environment now.
  • Is there any way to do that using command line ?

Appreciate any help on this.

airflow
1 Answers

You can mark the unwanted tasks as succeeded using the run command. The tasks marked as succeeded will not be run anymore.

Assume, there is a DAG with ID a_dag and three tasks with IDs dummy1, dummy2, dummy3. We want to skipp the dummy3 task from the next DAG run.

First, we get the next execution date:

$ airflow next_execution a_dag
2020-06-12T21:00:00+00:00

Then we mark dummy3 as succeeded for this execution date:

$ airflow run -fAIim a_dag dummy3 '2020-06-12T21:00:00+00:00'

To be sure, we can check the task state. For the skipped task it will be success:

$ airflow task_state a_dag dummy3 '2020-06-12T21:00:00+00:00'
...
success

For the rest of the tasks the state will be None:

$ airflow task_state a_dag dummy1 '2020-06-12T21:00:00+00:00'
...
None
user_69421340
Related