How to properly order multiple kafka with same event type topics by timestamp with streams DSL?

Viewed 103

I have multiple input topics from different systems with the same event type, e.g.:

topic1:
{ id: 1, timestamp: t1, content: "..." }
{ id: 2, timestamp: t3, content: "..." }
{ id: 3, timestamp: t5, content: "..." }

topic2:
{ id: 4, timestamp: t2, content: "..." }
{ id: 5, timestamp: t4, content: "..." }
{ id: 6, timestamp: t6, content: "..." }

and I want them in an output topic as:

topic out:
{ id: 1, timestamp: t1, content: "..." }
{ id: 4, timestamp: t2, content: "..." }
{ id: 2, timestamp: t3, content: "..." }
{ id: 5, timestamp: t4, content: "..." }
{ id: 3, timestamp: t5, content: "..." }
{ id: 6, timestamp: t6, content: "..." }

How can I achieve this? KStream#merge does not guarantee the order. The difference of the timestamps in the topics can be huge (±years).

Is the only way to use the Processor API? And if so, is there an efficient way to not iterate through all events?

0 Answers
Related