I have a use case when the user make a request from the client app (Angular) to my backend app (Node) that manualy run a task on Airflow. Airflow is not waiting for the complete run to respond to the backend and confirm that the task is running.
The thing I can do is to make an HTTP request to the backend to tell that the task is completed.
So, is there a way for the client to wait for a response from my backend server that will return when the task is completed with the Airflow ping?
Instead of implementing a complicated push event from airflow to your backend, you can pull the event every X seconds from your backend. For example, the backend submit query airflow API to run a dag, then checks if the run is finished every 1 second, once it is finished, it respond to the client request.
But if the event you want to wait takes much time, it's better to only register the query, then send the result by email/message.
For example if the client wants to prepare some data, and to do this, the backend should run an Airflow dag, so instead of blocking the client request, you can send 200 with message the query is successfully registered, and add an extra task in the Airflow dag to send the result to the client when the data is ready.
