In Kafka Connect, how to connect with multiple kafka clusters?

user_15819351 0
Viewed 1594

I set the kafka connect cluster in distributed mode and I wanna get connections with multiple kafka CLUSTERS, not just multiple brokers. Target brokers can be set with bootstrap.servers in connect-distributed.properties. So, at first, I set broker1 from kafka-cluster-A like below:

bootstrap.servers=broker1:9092

Absolutely, it worked well.

And then, I added broker2 from kafka-cluster-B like below:

bootstrap.servers=broker1:9092,broker2:9092

So, these two brokers are in the different clusters. And this didn't work at all. Without any error, it was just stuck and there was no answer with the request like creating connector through the REST API.

How can I connect with multiple kafka clusters?

apache-kafka-connect apache-kafka
1 Answers

As far as I know, you can only connect a Kafka Connect worker to one Kafka cluster.

If you have data on different clusters that you want to handle with Kafka Connect then run multiple Kafka Connect worker processes.

user_3506130
Related