I have a Spark job. When i submit it to server, it works fine. But i use Apache Airflow to submit job, it fail (i use command ssh to submit job).
My data has about 1-2 million lines. And I repartion 200 in that data.
Error in airflow console: airflow
Error in executor: executor
Note: i fiexd it by convert to cluster mode submit