How to fix "error: encountered unrecoverable cycle resolving import"?

user_127320 0
Viewed 1273

How to resolve the following compile error?

SOApp.scala:7: error: encountered unrecoverable cycle resolving import.
Note: this is often due in part to a class depending on a definition nested within its companion.
If applicable, you may wish to try moving some members into another object.
  import spark.implicits._

Code:

object SOApp extends App with Logging {
  // For implicit conversions like converting RDDs to DataFrames
  import spark.implicits._

  import org.apache.spark.sql.SparkSession  
  val spark = SparkSession
    .builder()
    .appName("Stackoverflow App")
    .master("local[*]")
    .getOrCreate()
}
apache-spark-sql apache-spark scala
1 Answers

tl;dr Move import spark.implicits._ after val spark = SparkSession...getOrCreate().

That name spark causes a lot of confusion since it could refer to org.apache.spark package as well as to spark value.

Unlike Java, Scala allows for import statements in many more places.

What you could consider a Spark SQL idiom is to create a spark value that gives access to implicits. In Scala, you can only bring implicits into scope from stable objects (like values) so the following is correct:

  // For implicit conversions like converting RDDs to DataFrames
  import spark.implicits._

And as you comment says, it's to bring implicit conversions of RDDs to DataFrames (among the things).

This is not to import org.apache.spark package, but for the implicit conversions.

user_13053440
Related