Timedeltasensor delaying from schedule interval

user_5259592 0
Viewed 1060

I have a job which runs at 13:30. Of which first task takes almost 1 hour to complete after that we need to wait 15 mins. So, I am using Timedeltasensor like below.

waitfor15min = TimeDeltaSensor(
        task_id='waitfor15min',
        delta=timedelta(minutes=15),
        dag=dag)

However in logs, It is showing schedule_interval + 15 min like below

[2020-11-05 20:36:27,013] {time_delta_sensor.py:45} INFO - Checking if the time (2020-11-05T13:45:00+00:00) has come
[2020-11-05 20:36:27,013] {base_sensor_operator.py:79} INFO - Success criteria met. Exiting.
[2020-11-05 20:36:30,655] {logging_mixin.py:95} INFO - [2020-11-05 20:36:30,655] {jobs.py:2612} INFO - Task exited with return code 0

How can I create delay between job??

airflow-scheduler airflow
1 Answers

You could use PythonOperator and write a function that simply waits 15 minutes. There is an example on how a wait task could look like:

def my_sleeping_function(random_base, **kwargs)):
    """This is a function that will run within the DAG execution"""
    time.sleep(random_base)


# Generate 5 sleeping tasks, sleeping from 0.0 to 0.4 seconds respectively
for i in range(5):
    task = PythonOperator(
        task_id='sleep_for_' + str(i),
        python_callable=my_sleeping_function,
        op_kwargs={'random_base': float(i) / 10},
        provide context=true,
        dag=dag,
    )

    run_this >> task
user_28388670
Related