I have to create a Kubernetes cron job that will try to get the current status and duration of a particular DAG. So that I should be able to notify respective stakeholders that this DAG is running for this much time
I have to create a Kubernetes cron job that will try to get the current status and duration of a particular DAG. So that I should be able to notify respective stakeholders that this DAG is running for this much time
You can get the dag_run_id information with a python request. You will be able to calculate the running time afterwards.
response = requests.get(url=f"{base_url}/dags/{dag_id}/dagRuns/{dag_run_id}", headers={"Authorization": _basic_auth_str("username", "password")})