We need to create a shared storage account for Synapse and Databricks, however we can only use existing storage accounts in Synapse while Databricks creates separate resource groups on its own and there is no option to use existing one also why there the managed resource group created by databricks have locks in it?
1 Answers
There are two things regarding storage accounts & Databricks:
Databricks automatically creates a storage account for each workspace to hold so-called DBFS Root. This storage account is meant to be used to keep only temporary data, libraries, cluster logs, models, etc. It's not designed to keep the production data as this storage account isn't accessible outside of the Databricks workspace.
Databricks can work with storage accounts created outside of the workspace (documentation) - just create a dedicated storage account to keep your data, and access it using the
abfssprotocol as described in the documentation, or mount it into workspace (although it's not recommended anymore). And then you can access that storage account from Synapse & other tools as well.
Related