Why can't I seem to exclude this transitive dependency from build.sbt? (Continues to show up in ivyReport)

Viewed 678

I'm working with a project that compiles but throws an exception while loading:

[error] (run-main-5) java.util.ServiceConfigurationError: com.fasterxml.jackson.databind.Module: com.spotify.ffwd.http.fasterxml.jackson.datatype.jdk8.Jdk8Module not a subtype
[error] java.util.ServiceConfigurationError: com.fasterxml.jackson.databind.Module: com.spotify.ffwd.http.fasterxml.jackson.datatype.jdk8.Jdk8Module not a subtype
[error]     at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:590)
[error]     at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1237)
[error]     at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator$1.run(ServiceLoader.java:1268)
[error]     at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator$1.run(ServiceLoader.java:1267)
[error]     at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
[error]     at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1270)
[error]     at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1300)
[error]     at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1385)
[error]     at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:994)
[error]     at org.apache.beam.sdk.options.PipelineOptionsFactory.<clinit>(PipelineOptionsFactory.java:468)
[error]     at com.spotify.scio.ScioContext$.defaultOptions(ScioContext.scala:293)
[error]     at com.spotify.scio.ScioContext$.apply(ScioContext.scala:216)
[error]     at com.mycompany.data.adaccountpipeline.endpoints.ExportUsersJob$.main(ExportUsersJob.scala:25)
[error]     at com.mycompany.data.adaccountpipeline.endpoints.ExportUsersJob.main(ExportUsersJob.scala)
[error]     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error]     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]     at java.base/java.lang.reflect.Method.invoke(Method.java:567)

The exact problem was reported at https://github.com/spotify/ffwd/issues/75 already:

Jar-file of ffwd-http-client contains incorrect records in META-INF/services. [...] This breaks java service discovery. For example, having ffwd-http-client on classpath and doing new ObjectMapper().findAndRegisterModules() results in this error:

Since the author has not released a fix, I'm trying to work around it.

  1. Does it seem like the right approach to try removing the transitive dependency ffwd-http-client altogether?

  2. If the transitive dependency has been successfully removed, should it stop appearing in ivyReport? Or how can I check?


Things I've tried...

In my build.sbt, I've tried to remove the dependency in several ways...

libraryDependencies ++= Seq(
  "org.slf4j" % "slf4j-simple" % slf4jVersion,
  "com.mycompany.ads" % "ad-account-service-client" % adAccountServiceClientVersion,
  "com.mycompany.ads" % "ad-account-service" % adAccountServiceClientVersion,
  "com.mycompany.ads" % "adserver-data-pipelines" % adServerPipelineVersion,
  "com.mycompany.ads" % "adstudio-common-server" % adStudioCommonVersion,
  "com.mycompany" % "padlock-service-api" % padlockApiVersion,
  "org.scala-lang.modules" % "scala-java8-compat_2.11" % scalaJavaCompatVersion,
  "com.spotify.ffwd" % "ffwd-http-client" % "0.4.0" intransitive()
)
  .map(_.exclude("com.spotify.ffwd", "ffwd-http-client"))
  .map(_.exclude("com.spotify.ffwd", "*")),

But if I do ivyReport and show ivyReport I still see

    <module organisation="com.spotify.ffwd" name="ffwd-http-client">
        <revision name="0.4.0" status="release" pubdate="20171219102239" resolver="sbt-chain" artresolver="sbt-chain" homepage="https://github.com/spotify/ffwd" downloaded="false" searched="false" default="false" conf="compile, runtime(*), master(compile), runtime, compile(*), master" position="106">
            <license name="The Apache Software License, Version 2.0" url="http://www.apache.org/licenses/LICENSE-2.0.txt"/>
            <metadata-artifact status="no" details="" size="3463" time="0" location="/Users/acheong/.ivy2/cache/com.spotify.ffwd/ffwd-http-client/ivy-0.4.0.xml" searched="false" origin-is-local="false" origin-location="https://artifactory.spotify.net/artifactory/repo/com/spotify/ffwd/ffwd-http-client/0.4.0/ffwd-http-client-0.4.0.pom"/>
            <caller organisation="com.spotify.metrics" name="semantic-metrics-ffwd-http-reporter" conf="compile, runtime" rev="0.4.0" rev-constraint-default="0.4.0" rev-constraint-dynamic="0.4.0" callerrev="1.0.2"/>
            <artifacts>
                <artifact name="ffwd-http-client" type="jar" ext="jar" status="no" details="" size="6270059" time="0" location="/Users/acheong/.ivy2/cache/com.spotify.ffwd/ffwd-http-client/jars/ffwd-http-client-0.4.0.jar">
                    <origin-location is-local="false" location="https://artifactory.spotify.net/artifactory/repo/com/spotify/ffwd/ffwd-http-client/0.4.0/ffwd-http-client-0.4.0.jar"/>
                </artifact>
            </artifacts>
        </revision>
    </module>

So am I not excluding the dependency? Or am I not checking dependencies correctly?

I've also tried https://github.com/jrudolph/sbt-dependency-graph but IntelliJ runs out of memory even after I increase memory for the SBT plugin.

1 Answers

I was making an amateur mistake. Not all my attempts were actually in effect, because I didn't know:

You must run the reload command to re-read build.sbt!

Or, restart sbt, which I did sometimes but not always, hence my attempts applying inconsistently.

Coming from C++/Makefiles and Java/Maven and only dipping my feet in Scala/SBT to make a minor change, I expected SBT to "know" if any files were touched with a more recent timestamp than the last build, and refresh accordingly. I didn't realize that the sbt interactive shell essentially uses build.sbt as its static configuration.

Related