I am scheduling a oozie job with the following structure in azure hdinsight spark2 cluster. I scheduled the job using the following these following commands,
oozie job -config /job.properties -run
oozie job -config /coordinator.properties -run
But I am getting the following Error as
Status: ERROR
Error Code: JA018
Error Message: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code
My workflow.xml file:
<workflow-app name="sparkshellwf" xmlns="uri:oozie:workflow:0.3">
<start to="sparkshellwf"/>
<action name="sparkshellwf">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>$SPARK_HOME/bin/spark-submit</exec>
<!--Adding all arguments needed/optional for Spark-submit here-->
<argument>--class</argument>
<argument>${Spark_Driver}</argument>
<argument>--master</argument>
<argument>${Spark_Master}</argument>
<argument>--deploy-mode</argument>
<argument>${Spark_Mode}</argument>
<argument>--num-executors</argument>
<argument>${numExecutors}</argument>
<argument>--driver-memory</argument>
<argument>${driverMemory}</argument>
<argument>--executor-memory</argument>
<argument>${executorMemory}</argument>
<argument>--executor-cores</argument>
<argument>${executorCores}</argument>
<argument>${workflowRoot}/lib/${sparkJar}</argument>
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Job failed, error message[${wf:errorMessage(wf:lastErrorNode())}] </message>
</kill>
<end name="end"/>
</workflow-app>
But the Spark job is running correctly,without any error
I am getting the oozie response is as like above.
