Kafka conumer.poll() argument

user_8055025 0
Viewed 48

What does the argument in the .poll method mean?

ConsumerRecords<String, String> records = consumer.poll(100);

Does it mean that consumer will block the execution for 100ms when there is no data in the partition? Or does it mean that it will keep polling for 100ms and batch those data for 100ms and then return?

If I may ask it differently, does the argument hold meaning only in cases of no data in the partition or does it mean that data or no-data the consumer will block for 100ms?

kafka-consumer-api apache-kafka java
2 Answers

The poll call is a blocking call from the kafka consumer. The call will return in one of the two conditions:

  1. poll duration times out: The kafka cluster will return the consumer with all the new messages that have been published to the topic partition (that consumer has subscribed to) and have not been processed by the consumer.
  2. max.poll.records have been reached: As part of kafka consumer config, max number of records to be read in a poll is configured as well. So if we are polling for 1min with max poll records as 1000, the poll call will return if 1000 new messages are available even if 1min is NOT over yet.
user_139580410
user_130899750
Related