How to re-run all failed tasks in Apache Airflow?

Question

I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.

I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS. This SO post suggests using the GUI to "clear" failed task:

How to restart a failed task on Airflow

This approach works if you have a handful number of failed tasks.

I am wondering if we can bypass the GUI and do it problematically, through command line something like:

airflow_clear_failed_tasks dag_id execution_data

user_5691525 · Answer

Use the following command to clear only failed tasks:

airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id

Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear

user_12796375 · Answer

The command to clear only failed tasks was updated. It is now (Airflow 2.0 as of March 2021):

airflow tasks clear [-s START_DATE] [-e END_DATE] --only-failed dag_id

How to re-run all failed tasks in Apache Airflow?

2 Answers