What happens to a Kafka consumer group on the Kafka server when the underlying Kafka topic is deleted?

Viewed 104

Let's say I have the Kafka topic cars.

I also have a consumer group cars-consumers that is subscribed to the cars topic. The cars-consumers consumer group is currently at the offset 89.

When I now delete the cars topic, what happens to the cars-consumers consumer group? Will it also be deleted?

If not, what happens when I re-create a cars topic? Will the cars-consumers consumer group still be at offset 89?

1 Answers

It will not be immediately deleted.

The __consumer_offsets topic maintains the group name as a record key, with topic-partition offsets stored in the values. It will eventually get compacted if there are no commits to the same group.

In other words, if the group is subscribed to cars topic and others, then the committed offsets for the cars topic will remain as long as commits continue for the other topics (since they are all grouped by the group key). So, if you recreate the topic for a group that has more than just that topic, you might then see old offsets. However, if the offsets are at 89, as you say, and this is not immediately available on the next consumer poll, then auto.offset.reset will take affect, either going to the beginning or end of the topic

Related