What's the difference between temporal table function and versioned table

user_4443784 0
Viewed 177

In Flink 1.12, Flink introduced a new concept which is called versioned table, It is very similar with temporal table function,but I am kind of confused between these two concepts.

temporal table function only supports append-only table(Please correct me if I am wrong),such as:

(changelog kind) update_time   currency   rate
================ ============= =========  ====
+(INSERT)        09:00:00      Yen        102
+(INSERT)        09:00:00      Euro       114
+(INSERT)        09:00:00      USD        1
+(INSERT)        11:15:00      Euro       119
+(INSERT)        11:49:00      Pounds     108

Can we call the above append-only table also a versioned table, From the official documentation it looks the answer is NO? . But I think it can be called a versioned table since it has kept the change history, and I can get the version(snapshot) of the table for the given time point (this is the confusing part to me)

In the following link, it is converting the append-only table to a versioned table by defining a versioned-table-views, with this additional conversion, it looks that append-only table is not versioned table?

Could you please explain about what is versioned table and the difference between versioned table and temporal table function. The official documentation doesn't explain very clearly.

https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/streaming/versioned_tables.html#versioned-table-views,


-- Define a versioned view
CREATE VIEW versioned_rates AS              
SELECT currency, rate, update_time              -- (1) `update_time` keeps the event time
  FROM (
      SELECT *,
      ROW_NUMBER() OVER (PARTITION BY currency  -- (2) the inferred unique key `currency` can be a primary key
         ORDER BY update_time DESC) AS rownum 
      FROM currency_rates)
WHERE rownum = 1; 

-- the view `versioned_rates` will produce a changelog as the following.
(changelog kind) update_time currency   rate
================ ============= =========  ====
+(INSERT)        09:00:00      Yen        102
+(INSERT)        09:00:00      Euro       114
+(INSERT)        09:00:00      USD        1
+(UPDATE_AFTER)  10:45:00      Euro       116
+(UPDATE_AFTER)  11:15:00      Euro       119
+(INSERT)        11:49:00      Pounds     108
apache-flink
1 Answers

You can view Temporal Table Function as a previous version of some sorts of versioned tables (This is also stated in the legacy features part of Flink).

In your first example, you create an append only table which will hold multiple keys of the same currency (i.e. EURO), which makes it ineligible to be a primary key.

Can you call an append only table a versioned table? The definition on Versioned Tables in doc says:

Flink SQL can define versioned tables over any dynamic table with a PRIMARY KEY constraint and time attribute.

This implies that an append only table cannot serve as a versioned table since it can't, as we said, hold true the primary key constraint.

But, since the append only table in your example does hold the relevant information to become a versioned table, containing inserts as well as updates and deletes, we can turn it to one with a deduplication query, as you posted above.

user_18708030
Related