There are two parts to my question.
- In below Springboot KafkaListener implementation when does offset get committed for offset strategy
auto-offset-reset: latestandenable-auto-commit: true? Is it immediately after the message is received by the consumer or after the whole method implementing the KafkaListener has completed?
KafkaConsumer.java
@KafkaListener(topics = "${spring.kafka.consumer.topic}")
public ResponseEntity<String> consume(String message) {
log.info("Message recieved from Kafka topic {}", message); // offset committed HERE?
KafkaResponse kafkaResponse = new Gson().fromJson(message, KafkaResponse.class);
myBusinessService.processKafkaResponse(kafkaResponse);
return new ResponseEntity<>("Successfully Received", HttpStatus.OK);// OR offset committed HERE?
}
- In
application.ymlfor the below properties which statements are True/False/what's the correct answer?:
max-poll-records: 100 max-poll-interval-ms: 200000 enable-auto-commit: true auto-commit-interval: 3000 auto-offset-reset: latest isolation-level: READ_UNCOMMITTED fetch-max-bytes: 52428800
- Between
auto-offset-reset: latestandauto-commit-interval: 3000if the Kafka consumer breaks down before 3 seconds then no offset would be committed for any record processed during those 3 seconds? - Between
max-poll-interval-ms: 200000andauto-commit-interval: 3000Kafka consumer would surely be polling the broker after an interval of200000seconds even if the consumer hasn't finished committing the current batch of offsets in3000seconds? - Between the combination of
max-poll-records: 100andfetch-max-bytes: 52428800which would take a priority if 99th record has exceeded 52428800 bytes?
Thanks in advance!