i have configured apache-airflow with postgreSSQL database and in my airflow i have running 1 dag, now its running successfully but if scheduler have any issue means how i get that and what is the way to check that, kindly give some idea and solution.
1 Answers
- Airflow exposes
/healthendpoint for this purpose - Also do check
REST APIreference, it has many useful endpoints for several common day-to-day stuff like triggering a DAG or returning the latest runs of DAGs
UPDATE-1
Apparently just because scheduler is running, doesn't necessarily mean that it will actually trigger a DAG; for e.g. this
- you can think of it like there could be internal bugs / interesting corrupt internal states of Airflow that may cause it to not trigger DAGs
- Thus people have gone a step ahead and they schedule a
canary DAG(a dummy DAG which does nothing but runs every few minutes). Then by monitoring metrics (thinkprometheus) of canary DAG, they can reliably affirm if Airflow scheduler is working as expected or not.
Related