Spark Launcher using redirectToLog doesn't write to log

Viewed 414

I am using SparkLauncher in order to submit spark applications. I would like to redirect log information to my log file defined by logback.xml.I am using com.typesafe.scalalogging.LazyLogging.

This how I tried to do it and it's not working:

val sparkLauncher: SparkLauncher = new SparkLauncher(envVariable)
      .setAppName("spark")
      .redirectToLog(this.getClass.getName)
      .setSparkHome(sparkInput.sparkHomePath)
      .setMaster(sparkInput.master)
      .setMainClass(sparkInput.classPath)
      .setAppResource(sparkInput.jarPath)
      .setVerbose(true) 
val sparkLauncher: SparkLauncher = buildSparkLauncher(sparkInput)
val p = sparkLauncher.launch()
p.waitFor()
0 Answers
Related