Apache airflow scheduler not triggering once in a month task as expected

user_9176914 0
Viewed 121

Here is the dag, which I want to execute on fixed date of every month, as of now I kept it on 18th of every month. But the task is getting triggered daily by the scheduler. catchup_by_default = False is set to false in airflow.cfg file

default_args = {
'owner': 'muddassir',
'depends_on_past': True,
'start_date': datetime(2021, 3, 18),
'retries': 1,
'schedule_interval': '0 0 18 * *',
}

Image 1

enter image description here

Image 2

enter image description here

Image 3

enter image description here

Image 4

enter image description here

airflow-scheduler airflow
1 Answers

you have mentioned schedule_interval in your default_args, which is not the way to schedule the DAG. default_args are actually applied to the tasks, as they are passed to Operators and not the DAG itself.

you can modify your code as follows, just by removing the schedule_interval from default_args and passing it in the DAG instance as follows and you can set catchup flag as False to avoid any backfills :

# assuming this is how you initialize your DAG
dag = DAG('your DAG UI name', default_args=default_args, schedule_interval = '0 0 18 * *', catchup=False)
user_66291480
Related