Branch airflow tasks based on schedule

user_9079461 0
Viewed 62

Is there any way we can branch airflow schedules.

Eg.

If the date is between 1-7, branch X should execute.

If the date is between 8-end_of_month branch Y should execute.

ie.

  Task_1
    |
  Task_2
   /\
X_1  Y_1
 |    |
X_2  Y_2

This will help to avoid new replica version of DAG and save maintenance cost.

This scenario can further be extended to regular days vs monthend, weekday vs weekends etc..

airflow
1 Answers

If the tasks take place in one DAG, you can download execution_date in the Python function and select which task to perform based on it. E.g:

def _branch_operator_func(**kwargs):
    execution_date = kwargs['execution_date']
    if execution_date.day >= 1 and execution_date.day <= 7:
        return 'X_1'
    elif (...)
user_128795930
Related