what is the main purpose of auto-offset-reset: latest and enable-auto-commit: false in Kafka consumers.
Is there a way in Kafka consumer I can check if I am able to receive the duplicate record or process the previous record during KafkaListner
What are all the necessary things to keep in mind while creating the Consumer side in spring boot and all the important things for logging, so that logging will have a meaning
purpose of
auto-offset-reset: latest
To start at the end of the topic rather than the beginning
... and
enable-auto-commit: false
So that your client (Spring) manages (or not) the offsets of the consumer rather than automatically every 500 messages, by default.
RE: duplicates - You will need to store all your records persistently somewhere else (in-memory or a database) in order to properly detect or process duplicates / previous offsets without rewinding the consumer via a seek method.
Adequate logging should be setup by Spring Boot by default.
