How to apply an offset to Tumbling Window, in order to delay the starting of Windows<TimeWindow> in Kafka Streams

Viewed 27

I'm calculating a simple mean on a dataset with values for May 2022, using different windows sizes. Using 1 hour windows there are no problems, while using 1 week and 1 month windows, records are not evaluated correctly.

As discussed here the problem is due to the fact that time is divided since the Unix epoch (01-01-1970) into equal-sized chunks (windows) of the specified duration, and then incoming events are assigned into those chunks (windows).

So this means that using 31 days windows, in Kafka Streams time is divided like that:

 01-01-1970 : 31-01-1970
 01-02-1970 : 03-02-1970
 ...
[14-04-2022 : 15-05-2022] <-- Our Window
 16-05-2022 : 15-06-2022
 ...

so not having the 01-05-2022 : 31-05-2022 window as desired.

In that discussion (about Flink), the solution was to apply an offset of 17 days to the Tumbling Window, in order to shift the window start from 14-04 to 01-05:

var monthResult = keyed
        .window(TumblingEventTimeWindows.of(Time.days(31),Time.days(17)))
        .aggregate(new AvgQ1(Config.MONTH))
        .name("Monthly Window Mean AggregateFunction");

But using Kafka Stream, i don't found an offset function, or something that let me to achieve the same result.

This is how i'm actually defining my window:

var grouped = keyed
    .groupByKey(Grouped.with(Serdes.Long(), EventSerde.Event()))
    .windowedBy(TimeWindows.ofSizeWithNoGrace(Duration.ofMinutes(10)))
    .reduce((o, v1) -> o);
0 Answers
Related