Whenever a new Kafka consumer group is created, I would like to be able to specify some key-value pairs metadata, and later will reuse it in other systems. Also, it will be useful to have the possibility to update these metadata, when a new client will join consumer group. Does Kafka consumer group support such possibility?
In our case, consumer groups created by Kafka clients like kafka-streams or spring-kafka. One of example, why it might be useful, is in a monitoring system. Right now we automatically create Kafka lag alert with specific constant threshold on any newly created consumer group. By having metadata, we could propagate custom Kafka lag threshold from service configs where consumer is running, and fetch these metadata in another monitoring service from Kafka broker (architecture where we have multiple micro-services and single monitoring service for them).
As a dirty workaround, we could put these key-value pairs into consumer group id itself by joining into a string with some character (e.g. group.id=my_custom_consumer_group_key1_value1_key2_value2), but definitely, it's not a good idea, as it's error-prone, using group id in a way as it not designed to be used, absent contract for supported metadata keys and restricted with group id limitations like max allowed length or supporting characters.