Check if azure databricks mount point exists from .NET

user_7074969 0
Viewed 233

I work on an app which does some kind of data engineering and we use Azure ADLS for data storage and Databricks for data manipulation. There are two approaches in order to retrieve the data, the first one uses the Storage Account and Storage account secret key and the other approach uses mount point. When I go with the first approach, I can successfully check, from .NET, whether the Storage account and it's corresponsive Secret key correspond to each other and return a message whether the credentials are right or not. However, I need to do the same thing with the mount point i.e. determine whether the mount point exists in dbutils.fs.mounts() or anywhere in the storage (I don't know how mount point exactly works and if it stores data in blob).

The flow for Storage account and Secret key is the following:

  1. Try to connect using the BlobServiceClient API from Microsoft;
  2. If it fails, return a message to the user that the credentials are invalid;
  3. If it doesn't fail, proceed further.

I'm not that familiar with /mnt/ and stuff because I mostly do .NET but is there a way to check from .NET whether a mount point exists or not?

azure-blob-storage azure-databricks databricks .net-core
1 Answers

Mount point is just a kind of reference to the underlying cloud storage. dbutils.fs.mounts() command needs to be executed on some cluster - it's doable, but it's not fast & cumbersome.

The simplest way to check that is to use List command of DBFS REST API, passing the mount point name /mnt/<something> as path parameter. If it doesn't exist, you'll get error message RESOURCE_DOES_NOT_EXIST:

{
  "error_code": "RESOURCE_DOES_NOT_EXIST",
  "message": "No file or directory exists on path /mnt/test22/."
}
Alex_Ott_186270
Related