Databricks Delta Live Tables: Difference between STREAMING and INCREMENTAL

user_12886610 0
Viewed 1712

Is there a difference between CREATE STREAMING LIVE TABLE and CREATE INCREMENTAL LIVE TABLE? The documentation is mixed: For instance, STREAMING is used here, while INCREMENTAL is used here. I have tested both and so far I have not noticed any difference.

delta-live-tables databricks
1 Answers

There are two aspects here:

  1. Conceptual - incremental means that the minimal data changes are applied to a destination table, we don't recompute full data set when new data arrive. This is how is explained in the Getting Started book.

  2. Syntax - CREATE INCREMENTAL LIVE TABLE was the original syntax for pipelines that were processing streaming data. But it was deprecated in the favor of CREATE STREAMING LIVE TABLE, but the old syntax is still supported for compatibility reasons.

Alex_Ott_186270
Related