I am working on integrating Kafka with Google PubSub by using the CloudPubSubConnector provided here: https://github.com/GoogleCloudPlatform/pubsub/tree/master/kafka-connector#cloudpubsubconnector-configs with a locally hosted Kafka server running on my machine. When I run the connector, I get the following stack trace:
[2020-05-29 15:20:01,678] ERROR WorkerSinkTask{id=CPSSinkConnector-9} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:186)
java.lang.NoClassDefFoundError: Could not initialize class com.google.pubsub.v1.ProjectTopicName
at com.google.pubsub.kafka.sink.CloudPubSubSinkTask.createPublisher(CloudPubSubSinkTask.java:353)
at com.google.pubsub.kafka.sink.CloudPubSubSinkTask.start(CloudPubSubSinkTask.java:143)
at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:305)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:193)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
[2020-05-29 15:20:01,678] INFO Created connector CPSSinkConnector (org.apache.kafka.connect.cli.ConnectStandalone:112)
[2020-05-29 15:20:01,678] ERROR WorkerSinkTask{id=CPSSinkConnector-9} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:187)
[2020-05-29 15:20:01,678] INFO Stopping CloudPubSubSinkTask (com.google.pubsub.kafka.sink.CloudPubSubSinkTask:386)
I am trying to understand what the issue is. The connector works fine if I host Kafka on a GCP VM, and my maven dependencies seem to be set up correctly, but it can't find the ProjectTopicName class for some reason.
UPDATE: RESOLVED This issue does not persist when using Kafka version 2.4.1 instead of 2.5.0