Airflow execution_timeout resetting every retry

user_7151534 0
Viewed 2611

I want that the task fails after 12 hours it started, I have set execution_timeout to 12 hours:

Operator is SQLSensor, since it sometimes lost connection and fails, I set the retries to 3.

However, because of the retries, task starts over after every timeout.

1: task started at 10:00

2: timeout at 22:00

3: then task started again at 22:01

I don't want this, how can I set total timeout for all retries?

log_sensor = SqlSensor(
    task_id='log_sensor',
    conn_id='mssql_etl',
    sql=my_query,
    retries=3,
    execution_timeout=timedelta(hours=12),
    dag=dag)
airflow
1 Answers

I think you want to use the sla kwarg, as described in the documentation here https://airflow.apache.org/docs/1.10.2/code.html?highlight=task%20timeout

sla (datetime.timedelta) – time by which the job is expected to succeed. Note that this represents the timedelta after the period is closed. For example if you set an SLA of 1 hour, the scheduler would send an email soon after 1:00AM on the 2016-01-02 if the 2016-01-01 instance has not succeeded yet. The scheduler pays special attention for jobs with an SLA and sends alert emails for sla misses. SLA misses are also recorded in the database for future reference. All tasks that share the same SLA time get bundled in a single email, sent soon after that time. SLA notification are sent once and only once for each task instance.

user_40248400
Related