Confluent Kafka Connector throws "No suitable driver found for jdbc:postgresql"

Viewed 446

I am working on Multinode Confluent kafka setup with 3 kafka brokers where there separate node for KSQL & schema-registry and running connector in separate node. All the mapping between those brokers was successful.

I am trying to load AVRO format data into postgresql table from kSQL stream topic "twitter-csv_avro" using confluent postgresql sink connector. Here is the postgres-sink.properties file

# key.converter=org.apache.kafka.connect.storage.StringConverter
# key.converter.schema.registry.url=http://host:8081
# value.converter=io.confluent.connect.avro.AvroConverter
# value.converter.schema.registry.url=http://host:8081
# schema.registry.url=http://host:8081
#Postgres sink connector id name which must be unique for each connector
name=twitter-streaming-postgres-sink
# Name of the connector class to be run
connector.class=io.confluent.connect.jdbc.JdbcSinkConnector
# Max number of tasks to spawn for this connector instance
tasks.max=1
# Kafka topic name which is input data to postgresql table
topics=TWITTER_CSV_AVRO
# Postgresql Database connection details
connection.url=jdbc:postgresql://host:5432/d2insights?user=postgres&password=*******
key.converter.schemas.enable=false
value.converter.schemas.enable=true
auto.create=false
auto.evolve=true
key.ignore=true
# Postgresql Table name
table.name.format=twitter

# Primary key configuration that
# pk.mode=none
#record_key=??

Here is the connect-avro-standalone.properties file

bootstrap.servers=XXX.XXX.XX.XX:9092,XXX.XXX.XX.XX:9092,XXX.XXX.XX.XX:9092
key.converter=org.apache.kafka.connect.storage.StringConverter
key.converter.schema.registry.url=http://XXX.XXX.XX.XX:8081
key.converter.enhanced.avro.schema.support=true
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://XXX.XXX.XX.XX:8081

value.converter.enhanced.avro.schema.support=true
offset.storage.file.filename=/tmp/connect.offsets
rest.port=9080
plugin.path=/usr/share/java/kafka-connect-jdbc

As i am using confluent-5.2.1 version, I am using default postgresql-42.2.10.jar which is under /usr/share/java/kafka-connect-jdbc and postgresql-10.12 version database.

Here is command that i am using to run the connector

connect-standalone /etc/schema-registry/connect-avro-standalone-postgres.properties /etc/kafka/connect-postgresql-sink1.properties

Although postgresql jar exists I am getting below exception

[2020-08-25 15:42:29,378] INFO Unable to connect to database on attempt 2/3. Will retry in 10000 ms. (io.confluent.connect.jdbc.util.CachedConnectionProvider:99)
java.sql.SQLException: No suitable driver found for jdbc:postgresql://XXX.XXX.XX.XX:5432/d2insights
        at java.sql.DriverManager.getConnection(DriverManager.java:689)
        at java.sql.DriverManager.getConnection(DriverManager.java:208)
        at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.getConnection(GenericDatabaseDialect.java:224)
        at io.confluent.connect.jdbc.util.CachedConnectionProvider.newConnection(CachedConnectionProvider.java:93)
        at io.confluent.connect.jdbc.util.CachedConnectionProvider.getConnection(CachedConnectionProvider.java:62)
        at io.confluent.connect.jdbc.sink.JdbcDbWriter.write(JdbcDbWriter.java:56)
        at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:74)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:539)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:322)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
        at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
        at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
        at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Can someone suggest or tell me what I am doing wrong over here.

0 Answers
Related