I installed cdh 5.11.0, along with spark 2 (according to this doc: https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html). However, when I tried to launch a spark job using spark-submit2, the error below occurred:
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:111)
at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:102)
at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:746)
at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
at org.apache.spark.deploy.yarn.ApplicationMaster$.log(ApplicationMaster.scala:746)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:761)
at org.apache.spark.deploy.yarn.ExecutorLauncher$.main(ApplicationMaster.scala:795)
at org.apache.spark.deploy.yarn.ExecutorLauncher.main(ApplicationMaster.scala)
Caused by: java.lang.ClassNotFoundException: org.slf4j.impl.StaticLoggerBinder
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 8 more
I realized there was something wrong with the classpath, but can I fix it with the Cloudera Manager? Or should I go and classpath manually on each nodes?