I have submitted spark job through airflow sometimes job works and sometimes it don't give output at all .
Even after 2-3 hrs of waiting job is not giving any detail apart from
Waiting for job output...
I am using dataproc-1-4-deb10
Its simple job like pulling data from jdbc using pysaprk. Also it works without error sometimes and sometimes doesnt at all.