Missing Scala-Reflect Deps when Using SBT Subprojects

Question

I am encountering issues when trying to run a subproject of an sbt project w/ multiple subprojects. The project structure is as follows - one main project, multi_app, and two subprojects, app1 and app2.

val baseDependencies = Seq(
        "org.apache.spark" %% "spark-sql" % "2.4.4" % "provided",
        "org.scalameta" %% "mdoc" % "2.2.14" % "provided"
    )


lazy val commonSettings = Seq(
    organization := ...,
    version := "1.0",
    scalaVersion := "2.12.12",
    crossScalaVersions := Seq("2.11.12"),
    resolvers ++= Seq(...),
  libraryDependencies ++= baseDependencies
)

lazy val testSettings = Seq(
  fork in Test := true,
  javaOptions ++= Seq("-Xms512M", "-Xmx2048M", "-XX:MaxPermSize=2048M", "-XX:+CMSClassUnloadingEnabled"),
  testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-oD"),
)


lazy val multi_app = project
  .in(file("."))
  .settings(name := "multi_app")
  .settings(commonSettings: _*)
  .settings(testSettings: _*)
  .aggregate(app1)
  .aggregate(app2)


lazy val app1 = project
  .in(file("./app1"))
  .settings(commonSettings: _*)
  .settings(testSettings: _*)
  .settings(
    name := "app1",
    mainClass in Compile := Some("sparkapps.multi_app.App1"),
    mainClass in run := Some("sparkapps.multi_app.App1"),
    mainClass in assembly := Some("sparkapps.multi_app.App1")
  )
  .dependsOn(app2)

lazy val app2 = project
  .in(file("./app2"))
  .settings(commonSettings: _*)
  .settings(testSettings: _*)
  .settings(
    name := "app2",
  )

All app2 does is implement the below function, which is then imported in imported in app1's main class and called. The only purpose multi_app serves is to build an uber jar if I wished to add more main classes and run those from a single executable.

object MySharedObject {
    def printWord = println("word")
}

When running the below workflow, however, I am receiving the following exception:

sbt "project app1" compile
sbt "project app1" run

[error] (run-main-0) java.lang.NoClassDefFoundError: scala/reflect/api/TypeCreator
[error] java.lang.NoClassDefFoundError: scala/reflect/api/TypeCreator
[error]         at sparkapps.multi_app.App1.main(Main.scala)
[error]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]         at java.lang.reflect.Method.invoke(Method.java:498)
[error] Caused by: java.lang.ClassNotFoundException: scala.reflect.api.TypeCreator
[error]         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
[error] stack trace is suppressed; run last Compile / bgRun for the full output
[error] Nonzero exit code: 1
[error] (Compile / run) Nonzero exit code: 1
[error] Total time: 3 s, completed Sep 13, 2022 8:39:22 PM

What would cause the scala-reflect dependencies to be missing when running a subproject? It should be included via the commonSettings scalaVersion, no? If, rather than breaking this into multiple projects, I kept it all in one, the workflow executes fine.

Missing Scala-Reflect Deps when Using SBT Subprojects

0 Answers