Kafka-how to send messages to specific partition based on a table's field value via Debezium configuration

Viewed 56

is it possible to send messages to specific partition based on a table field value? For example,i have a column called customer, which has 4 values ,say customer1,customer2,customer3,customer4. I want to send to their corresponding partition.

is it posiible to achive this in debezium configuration?

1 Answers

By default, Debezium will write Kafka records into partitions based on the record key, e.g. the database rows id. There's no guarantee "customer1" goes to "partition 1", or that 2 customers will end up in the same partition (e.g. you may have more customers than partitions)

To explicitly map the data to numbered partitions, you'll need to implement your own Partitioner Java interface and add it to the Connect worker classpath and set producer.override.partitioner.class in the Debezium config.

Or you can just let the producer partition based on the key of the records, as is expected.

Related