Databricks Streaming and Cluster updates?

Viewed 43

Does anyone know if there's any documentation about how to handle Structured Streaming in Databricks and cluster maintenance updates (runtime, OS, etc)?

I would like to know more about how it works, and if we (user) need to handle that manually or if there's some sort of mechanism that handles that internally.

1 Answers

See recovery from failures under streaming.

For any infrastructure problem that could occur:
Hence, to make your queries fault tolerant, you must enable query checkpointing and configure Databricks jobs to restart your queries automatically after a failure.

Related