Is there anyway to use different auto.offset.reset strategy for different input topics in kafka streams app?

user_1828260 0
Viewed 396

The use case is: I have a kafka streams app that consumer from an input topic, and output to a intermediate topic, then in the same streams another topology consume from this intermediate topic.

Whenever the application id is updated, both topic start to consumer from earliest. I want to change the auto.offset.reset for the intermediate topic to latest while keep that to earliest for the input topic.

apache-kafka-streams apache-kafka
1 Answers

Yes. You can set the reset strategy for each topic via:

// Processor API
topology.addSource(AutoOffsetReset offsetReset, String name, String... topics); 

// DSL
builder.stream(String topic, Consumed.with(AutoOffsetReset offsetReset));
builder.table(String topic, Consumed.with(AutoOffsetReset offsetReset));

All those methods have some overloads that allow to set it.

user_49530790
Related