Is it possible to retrieve messages in Kafka console conumer for a particular timestamp range?
For example kafka messags between 08:00 to 09:00 yesterday.
2 Answers
You can use kcat for consuming messages between two timestamps:
kcat -b localhost:9092 -C -t mytopic -o s@1568276612443 -o e@1568276617901
where
s@denotes the starting timestamp in mse@denotes the ending timtestamp in ms (non-inclusive)
Yes, you can do it since Kafka version 0.10.1.
Use the function offsetsForTimes in KafkaConsumer:
Look up the offsets for the given partitions by timestamp. The returned offset for each partition is the earliest offset whose timestamp is greater than or equal to the given timestamp in the corresponding partition. This is a blocking call. The consumer does not have to be assigned the partitions.
Related