Spark Delta format on non-Databricks platforms

Viewed 134

To improve query speed, Delta Lake on Databricks supports the ability to optimize the layout of data stored in cloud storage. Delta Lake on Databricks supports two layout algorithms: bin-packing and Z-Ordering.

If you run on-prem (not in the cloud) and use the delta format with Spark, thus not on Databricks, can the Z-Ordering be used? Or is it only available on Databricks run-time?

My assumption is yes, but just want to be really clear as I do not have RHEL cluster at hand.

1 Answers

The z-ordering is supported only in the Delta Lake on Databricks Runtime.

Related