I new to both Scala and SBT, and in an attempt to learn something new, am trying to run through the book "Building a recommendation engine with Scala". The example libraries referenced in the book have now been replaced by later versions or in some cases seemingly superseded by different techniques (casbah to Mongo Scala driver). This has led to me producing some potentially incorrect SBT build files. With my initial build file, I had;
name := "BuildingScalaRecommendationEngine"
scalaVersion := "2.12.1"
version :="1.0"
libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-streaming_2.11" % "2.1.1"
libraryDependencies += "org.mongodb.scala" %% "mongo-scala-driver" % "2.1.0"
libraryDependencies += "org.apache.kafka" % "kafka_2.12" % "0.10.2.0"
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.5.1"
This was leading to a build error like;
[error] Modules were resolved with conflicting cross-version suffixes in {file:/C:/Dev/learning/scala/Tutorial/src/}src:
[error] org.scala-lang.modules:scala-xml _2.11, _2.12
java.lang.RuntimeException: Conflicting cross-version suffixes in: org.scala-lang.modules:scala-xml
I tried following some examples as suggested at this link which I found at some Gitter conversation here. The suggestions were all a little beyond my understanding at this point though.
I managed to work around the error through mostly trial and error via amending my build file to look like the following;
name := "BuildingScalaRecommendationEngine"
scalaVersion := "2.12.1"
version :="1.0"
libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "2.1.0" excludeAll(
ExclusionRule(organization = "org.scala-lang.modules")
)
libraryDependencies += "org.apache.spark" % "spark-streaming_2.11" % "2.1.1" excludeAll(
ExclusionRule(organization = "org.scala-lang.modules")
)
libraryDependencies += "org.mongodb.scala" %% "mongo-scala-driver" % "2.1.0" excludeAll(
ExclusionRule(organization = "org.scala-lang.modules")
)
libraryDependencies += "org.apache.kafka" % "kafka_2.12" % "0.10.2.0" excludeAll(
ExclusionRule(organization = "org.scala-lang.modules")
)
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.5.1" excludeAll(
ExclusionRule(organization = "org.scala-lang.modules")
)
Is there something I could have done with my original build file to get around this error?
Is there a combination of library dependency values that is causing this in the first place?
I aim to get a better understanding of Scala and SBT to overcome this, but it is somewhat frustrating in the meantime.