how to find out if it is a retry message in spring kafka Listener

user_15285766 0
Viewed 543

We are trying to solve an edge case scenario, in spring boot kafka listener:

@KafkaListener(topics="topicName", groupId="consumerGoupId")
public void consumeNotification(@Payload Datarequest request,
                          @Header(KafkaHeaders.OFFSET) Long offset,
                          @Header(KafkaHeaders.RECEIVED_PARTITION_ID)Integer partitionId) 
{
    processRequest(request);
}

Now for some reason when processRequest is going on, my spring boot app gets Shutdown,the offset doesn't get committed and when it is up,the listener picks up the same message for processing again.

My use case is : Is there any way to know Listener has got this message as part of Retry?

Because since process request was stopped in between due to app Crash, if the same message has picked up by listener again, I have to write some logic in processRequest to start processing from where it stopped.

spring-kafka kafka-consumer-api apache-kafka spring-boot java
1 Answers

Right; the only guarantee is at least once delivery; there is no way to know if a message has been redelivered (after a restart - we can tell if it's a redelivery in the same application instance - see https://docs.spring.io/spring-kafka/docs/current/reference/html/#delivery-header).

Since you can't tell in all cases, you have to make your listener idempotent - e.g. by storing the topic/partition/offset along with the data, so you can check if it has already been processed.

user_12407630
Related