Creating DataBricks Azure Key Vault Secret Scope Backend using Rest Api and DataBricks CLI

Syamala Kumar Rama Koti Reddy 0
Viewed 59

I am trying to create the secrete scope which is backend by azure key vault so when I tried with secret api, I got an error has

Input:

response = requests.post(
    'https://%s/api/2.0/secrets/scopes/create' % (DOMAIN),
    headers={'Authorization': 'Bearer %s' % TOKEN,},
    json={"scope": "my-simple-azure-keyvault-scope",
    "scope_backend_type": "AZURE_KEYVAULT",
  "backend_azure_keyvault":
  {
    "resource_id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azure-rg/providers/Microsoft.KeyVault/vaults/my-azure-kv",
    "dns_name": "https://my-azure-kv.vault.azure.net/"
  },
  "initial_manage_principal": "users"
})

Output:

{
  "error_code": "INVALID_PARAMETER_VALUE",
  "message": "Scope with Azure KeyVault must have userAADToken defined!"
}

I tried with Different method which is Databricks Cli for that I faced a error, So I created a Secret scope using databricks UI and tried to access through cli

Input:

databricks configure --token
Databricks Host (should begin with https://): https://adb-...azuredatabricks.net/
Token: 

D:\Users\>databricks secrets list-scopes

Output:

Error: b'Bad Request'

In Cmd prompt

Is there a way to get AAD token, I have implemented Microsoft documents for getting token but didn't work

azure-keyvault azure-databricks azure python
1 Answers

This is well known limit described in the documentation linked in the comment - you must use user's AAD token to create a secret scope baked by the Azure KeyVault.

The simplest way to create a secret scope is not to use API, but instead use the Databricks CLI - there is a command to create a secret scope:

databricks secrets create-scope --scope <scope-name> \  
  --scope-backend-type AZURE_KEYVAULT --resource-id <azure-keyvault-resource-id> \
  --dns-name <azure-keyvault-dns-name>

You also don't even need to configure CLI - specify DATABRICKS_HOST and DATABRICKS_TOKEN environment variables, like this:

export DATABRICKS_TOKEN=$(az account get-access-token -o tsv --query accessToken --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d)
export DATABRICKS_HOST=https://adb-.....azuredatabricks.net

Note that here we're getting user's AAD token by using az account get-access-token command from Azure CLI (make sure you did az login before)

Alex Ott0
Related