how to consume message from specific offset using Kafka HTTP Sink Connector

user_17424027 0
Viewed 79

I'm using Kafka Confluent HTTP Sink Connector to consume the message from Kafka topic, by default it read all the messages from beginning. I've use case where I want to stop the connector for some time and resume the HTTP sink connector later. When I resume the connector I want to read the message from specific offset.

I was looking offset related configuration here https://docs.confluent.io/kafka-connect-http/current/connector_config.html but I couldn't find any.

How can I consume the message from specific offset?

confluent-platform kafka-consumer-api apache-kafka-connect apache-kafka
1 Answers

All (at least most) sink connectors use Kafka Consumer groups.

You can use kafka-consumer-groups --reset-offsets command to first seek the group for the connector to specific offsets. And then, when it starts again, it should begin to read from that position.

by default it read all the messages from beginning

Not necessarily. By default, auto.offset.reset=latest

user_23086830
Related