Do we have a option to get data in KSQL streams from specific time-period/Timestamp

user_2240572 0
Viewed 2221

I know , in KSQL we can set offset to earliest or latest But can we get data from specific time period i.e I need to get data inserted to a topic from 06-May-2020 ?

ksqldb confluent-platform apache-kafka
1 Answers

In ksqlDB you can query from the beginning (SET 'auto.offset.reset' = 'earliest';) or end of a topic (SET 'auto.offset.reset' = 'latest';).

You cannot currently (0.8.1 / CP 5.5) seek to an arbitrary offset.

What you can do is start from the earliest offset and then use ROWTIME in your predicate to identify messages that match your requirement. 

SELECT * 
  FROM MY_SOURCE_STREAM 
WHERE  ROWTIME>=1588772149620

Note that this scans through sequentially so depending on how much data you have in your topic may not be particularly fast.

user_3506130
Related