I am using Kafka Consumer to read from several topics and I need one of those to have higher priority. The processing takes a lot of time and there are always many messages in (low priority) topics, but I need the messages from other one to be processed as soon as possible.
It's similar question as Does Kafka support priority for topic or message? but this one is using old API.
In new API (0.10.1.1), there are methods
KafkaConsumer::pause(Collection)
KafkaConsumer::resume(Collection)
But it's not clear to me, how to effectively detect that there are new messages in high priority topic and it is necessary to pause consumption from the other topics.
Any ideas/examples?