I am having issues scheduling my DAG runs properly, especially in terms of schedule_interval & start_date.
Airflow's default behavior doesn't suit my use case, wherein if I set an arbitrarily early date for start_date, airflow will always trigger the initial run outside the "intuitive" first schedule (e.g. if I set my schedule_interval to run at 12AM every Wednesday & Friday, and I created this DAG on a Monday with an arbitrarily early date many weeks back, there is a cron run at the exact instance the DAG is created in the Monday, which I do not want).
Understand from reference that the scheduled run will happen after 1 scheduled_interval cycle. While I have taken reference to How to work correctly airflow schedule_interval & https://airflow.apache.org/docs/1.10.6/scheduler.html#backfill-and-catchup, they generally show examples related to regular intervals (e.g. hourly, daily, monthly, etc.).
However, in my scenario, I have a schedule_interval of 0 16 * * 1,2,0 & 0 16 * * 1,3,5. As the intervals between runs are irregular, how do I set my start_date such that:
- The first run happens exactly on the next datetime when it is "intuitive" to run? e.g.
- if I create these 2 DAGs on a Saturday, I would want my DAGs to make their first run at 1600h on Sunday & Monday respectively
- if I create these 2 DAGs on Monday 1700h, I would want my DAGs to make their first run at 1600h on Tuesday & Wednesday respectively