HA in databrick clusters in Azure and AWS

Viewed 190

It is clear the benefits of spark to deal with vms disruption within a zone, but what happens if a zone goes down. Are the vms in the running cluster failing over to a second zone in the same region or is this something to be done manually?

br, Mike

1 Answers

databricks is being deployed in following mode :

Active deployment: Users can connect to an active deployment of a Databricks workspace and run workloads. Jobs are scheduled periodically using Databricks scheduler or other mechanism. Data streams can be executed on this deployment as well. Some documents might refer to an active deployment as a hot deployment.

Passive deployment: Processes do not run on a passive deployment. IT teams can setup automated procedures to deploy code, configuration, and other Databricks objects to the passive deployment. A deployment becomes active only if a current active deployment is down. Some documents might refer to a passive deployment as a cold deployment.

For more details , you can ref link

Related