How to make SparkSession and Spark SQL implicits globally available (in functions and objects)?

Viewed 1654

I have a project with many .scala files inside a package. I want to use Spark SQL as follows:

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions._

val spark: SparkSession = SparkSession.builder()
                                      .appName("My app")
                                      .config("spark.master", "local")
                                      .getOrCreate()

// For implicit conversions like converting RDDs to DataFrames
import spark.implicits._

Is it a good practice to wrap the above code inside a singleton object like:

object sparkSessX{
   import org.apache.spark.sql.SparkSession
   import org.apache.spark.sql.functions._

   val spark: SparkSession = SparkSession.builder()
                                      .appName("My App")
                                      .config("spark.master", "local")
                                      .getOrCreate()

   // For implicit conversions like converting RDDs to DataFrames
   import spark.implicits._
}

and every class to extend or import that object?

1 Answers
Related