confluent kafka python Consumer.poll() always return None while official console-consumer works properly

Viewed 703

The Consumer instance from confluent-kafka python client always returns None when calling poll() with timeout set.

The topic does contains some message and the official console consumer works fine:

$ vim ~/client.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=XXXXXXXXXX password="XXXXXXXXXX";

$ ~/kafka_2.13-2.4.0/bin/kafka-console-consumer.sh --topic my_topic --bootstrap-server somehost.:30742 --from-beginning --consumer.config ~/client.properties --group somenewgroup
msg1
msg2
msg3

But Consumer.poll() methods always return None. even when I change the password or host to an invalid value it also returns None.

I set a logger for the consumer but nothing is logged.

the python code is as follows

consumer=Consumer({'sasl.mechanisms': "SCRAM-SHA-512",
                   'security.protocol': 'SASL_PLAINTEXT',
                   'sasl.username': 'XXXXXXXXXX',
                   'sasl.password': 'XXXXXXXXXX',
                   'bootstrap.servers': 'somehost.:30742',
                   "group.id":"somenewgroup",
                   'auto.offset.reset': 'beginning',
                   'logger':logger
                   },logger=logger)

consumer.subscribe(["my_topic"])

while True:
    msg = consumer.poll(timeout=1.0)
    print("poll success")
    if msg is None:print("msg is None!")
0 Answers
Related