How to list Kafka consumer group using python

user_5186059 0
Viewed 4714

I want to get Kafka consumer group list with python but I couldn't.

I use to zookeeper python client( kazoo) but consumer group list empty because this method for old consumer and we are not using old consumer.

How can I get consumer group list with python code?

./kafka-consumer-groups.sh  -bootstrap-server localhost:9092 -list
kafka-consumer-api apache-kafka python
3 Answers

You can easily list consumer groups with kafka-python.

Just send a ListGroupsRequest to any of the brokers in your cluster.

For example:

from kafka import BrokerConnection
from kafka.protocol.admin import *
import socket

bc = BrokerConnection('localhost', 9092, socket.AF_INET)
bc.connect_blocking()

list_groups_request = ListGroupsRequest_v1()

future = bc.send(list_groups_request)
while not future.is_done:
    for resp, f in bc.recv():
        f.success(resp)

for group in future.value.groups:
    print(group)
user_17651890

KIP-222 was recently introduced to enable Java clients to retrieve all consumer groups. Maybe it needs more time to have librdkafka or python client support this. See details in this issue.

user_70286970

You may do this just by using kafka-python module.

from kafka import KafkaAdminClient
client = KafkaAdminClient(bootstrap_servers="localhost:9092")
for group in client.list_consumer_groups():
    print(group[0])
user_15512460
Related