Kafka: Fetch offset is out of range for partition. How to avoid data loss?

user_225986 0
Viewed 5497
Fetcher INFO : Fetch offset .... is out of range for partition ..., resetting offset

Trying to understand what happens when this happens. If offset we are trying to to read is missing (I assume because of kafka GC'ed the offet) and auto.offset.reset = latest do we skip over data to the latest offset?

What is the safe configuration to avoid data loss?

spring-kafka apache-kafka
1 Answers

For me, I Fixed it by adding the below line in my KafkaConsumer configurations

properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
user_24930200
Related