Spark + Kafka Integration error. NoClassDefFoundError: org/apache/spark/sql/internal/connector/SimpleTableProvider

user_3012044 0
Viewed 518

I am using Kafka 2.5.0 and Spark 3.0.0. I'm trying to import some data from kafka into spark. The following code snippet gives me an erorr:

spark.readStream.format("kafka").option("kafka.bootstrap.servers", "localhost:9092").option("subscribe", "topic1").load()

The error I get says

java.lang.NoClassDefFoundError: org/apache/spark/sql/internal/connector/SimpleTableProvider
apache-kafka apache-spark scala
1 Answers

This error is mainly due to spark-kafka dependency conflicts.

You can check the supporting scala version in maven repository if not yet.

If the error still occurring then you can share more details like the groupId, artifactId along with version.

user_139892900
Related