Spark-Solr error java.lang.ClassNotFoundException: org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter

user_2329168 0
Viewed 355

Have anyone tried spark-solr https://github.com/lucidworks/spark-solr

When I run this

val options = Map("collection" -> "DS", "zkhost" -> "url:2181,url:2181,url:2181")
val df = spark.read.format("solr").options(options).load

Then I get this error

  at solr.DefaultSource.createRelation(DefaultSource.scala:14)
  at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:317)
  at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
  ... 49 elided
Caused by: java.lang.ClassNotFoundException: org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter
  at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  ... 54 more

My setup is Ubuntu 18, Solr 7.4, Spark 2.4.0, Scala 2.11.12

apache-spark solr scala
1 Answers

The solrj lib is missing in the project.

Add the solrj lib as nit should work for you.

user_36360710
Related