Is the Airflow BranchDateTimeOperator > or >= for the upper and lower bounds?

user_1256041 0
Viewed 66

Suppose I execute this BranchDateTimeOperator on the following schedules (ds values). Which tasks will be executed and which will be skipped?

branch_task = BranchDateTimeOperator(
    task_id='create_records_attributed',
    depends_on_past=True,
    use_task_execution_date=True,
    follow_task_ids_if_true=['task_a'],
    follow_task_ids_if_false=['task_b'],
    target_upper=datetime(2022, 5, 20),
    target_lower=None,
    dag=dag,
)
  • datetime(2022, 5, 19)
  • datetime(2022, 5, 20)
  • datetime(2022, 5, 21)
airflow
1 Answers

In your example task_a will be followed if target_lower < now < target_upper otherwise task_b will follow.

BranchDateTimeOperator makes more sense if used with use_task_execution_date=True in this scenario it compares target_lower < DAG logical_date < target_upper so it also relevant when you backfill a task or clear historical runs because the date which will be compared is the DAG running date rather than current date.

Alternatively, note that the target parameters accept also datetime.time so for scheduled runs you might want to use datetime.time rather than datetime.datetime for example to check if the time of running is between 08:00 to 10:00

user_146244090
Related