Why does Redis TimeSeries not capture the last element in an aggregation?

user_11271048 0
Viewed 277

I am trying to wrap my head around how Redis' timeseries rule creation works, and I was confused why Redis ignores the last item in an aggregation, and wondering if that is expected behaviour.

I have created a sample code in redis-cli to illustrate:

127.0.0.1:6379> FLUSHALL
OK
127.0.0.1:6379> TS.CREATE source
OK
127.0.0.1:6379> TS.CREATE sumRule
OK
127.0.0.1:6379> TS.CREATERULE source sumRule AGGREGATION sum 1
OK
127.0.0.1:6379> TS.ADD source * 1
(integer) 1592638841406
127.0.0.1:6379> TS.ADD source * 2
(integer) 1592638843446
127.0.0.1:6379> TS.ADD source * 3
(integer) 1592638846536
127.0.0.1:6379> TS.ADD source * 4
(integer) 1592638849085
127.0.0.1:6379> TS.GET sumRule
1) (integer) 1592638846536
2) 3

I expected here the last item to be 4 not 3. Is this expected behaviour, because it needs to look backwards to make aggregations? Or does it require more than one value to make the aggregation?

redis
1 Answers

Thanks for trying RedisTimeSeries.

Let's breakdown whats actually happening and then I'll explain why. When you create a rule you need to specify 2 parameters: aggregation type and bucket size. (an important note here would be that bucket size is in milliseconds and not seconds)

Once you have a rule, every sample that is being added is used to calculate the current aggregated value, once the bucket context is over we flush the value to the downsampled key. How do we know a bucket context is over? if you have a bucket of 5 seconds, we will keep the context open until we get a sample that is out of the 5 seconds range.

So why you see only the previous bucket? There are 2 reasons for that:

  1. This is an optimisation to avoid opening the downsampled key and writing a sample (overriding a sample is a cpu intensive operation since we have compressed data structure to keep the data in memory)
  2. API - we preferred not to return a partial bucket to avoid showing "in flight" data.

I hope this will answer your questions.

user_1865340
Related