Error Code: JA018 whie runnnig oozie workflow in HDInsight spark2 cluster

Viewed 5387

I am scheduling a oozie job with the following structure in azure hdinsight spark2 cluster. I scheduled the job using the following these following commands,

oozie job -config /job.properties -run
oozie job -config /coordinator.properties -run

But I am getting the following Error as

Status: ERROR
Error Code: JA018
Error Message: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code

enter image description here

My workflow.xml file:

<workflow-app name="sparkshellwf" xmlns="uri:oozie:workflow:0.3">
  <start to="sparkshellwf"/>
  <action name="sparkshellwf">
    <shell xmlns="uri:oozie:shell-action:0.1">
      <job-tracker>${jobTracker}</job-tracker>
      <name-node>${nameNode}</name-node>
      <configuration>
        <property>
          <name>mapred.job.queue.name</name>
          <value>${queueName}</value>
        </property>
      </configuration>
      <exec>$SPARK_HOME/bin/spark-submit</exec>
      <!--Adding all arguments needed/optional for Spark-submit here-->
      <argument>--class</argument>
      <argument>${Spark_Driver}</argument>
      <argument>--master</argument>
      <argument>${Spark_Master}</argument>
      <argument>--deploy-mode</argument>
      <argument>${Spark_Mode}</argument>
      <argument>--num-executors</argument>
      <argument>${numExecutors}</argument>
      <argument>--driver-memory</argument>
      <argument>${driverMemory}</argument>
      <argument>--executor-memory</argument>
      <argument>${executorMemory}</argument>
      <argument>--executor-cores</argument>
      <argument>${executorCores}</argument>
      <argument>${workflowRoot}/lib/${sparkJar}</argument>
    </shell>
    <ok to="end"/>
    <error to="fail"/>
  </action>
  <kill name="fail">
    <message>Job failed, error message[${wf:errorMessage(wf:lastErrorNode())}] </message>
  </kill>
  <end name="end"/>
</workflow-app>

But the Spark job is running correctly,without any error

I am getting the oozie response is as like above.

1 Answers
Related