Sync Azure AD users & groups to Azure Databricks workspace without using Azure Databricks SCIM Connector

user_9280820 0
Viewed 458

Do anyone have a good solution to sync azure ad users & groups to azure databricks workspace? The Azure Databricks SCIM Connector requires premium workspace and premium azure ad account.

If anyone have done this or have seen any good github resources that could be used as a starting point please share otherwise please comment on my suggested high level idea/logic below:

Using terraform/azure cli and databricks rest api:

Sync groups

  • config: get list of definied azure ad groups
  • databricks rest api: get list of databricks workspace groups (exclude admin and all users groups)
  • script: compare existing databricks workspace groups with list of defined azure ad groups in config.

if group exist in list but not databricks workspace:

  • databricks rest api: create groups in databricks workspace

if group exist in databricks workspace but not in list of defined azure ad groups in config:

  • databricks rest api: delete groups in databricks workspace

Sync users

Add users

  • databricks rest api: get list of all users in databricks workspace
  • azure cli: iterate through list of defined azure ad groups and get list of members

for each azure ad group

  • script: compare against list of members in azure ad group with list of all users in databricks workspace.

if members is not in databricks workspace.

  • databricks rest api: create users in databricks workspace
  • databricks rest api: add users to group in databricks workspace

Delete users

  • azure cli: iterate through list of defined azure ad groups and get list of unique members.
  • script: compare list of unique members in all azure ad groups with list of all users in databricks workspace and get all users that do not match.
  • databricks rest api: delete users in databricks workspace
azure-databricks databricks
1 Answers

From my point of view the simplest way to achieve this is to combine user & group resources of the Databricks Terraform Provider with Azure AD Terraform provider - with it you can easily pull groups & users from AAD, and apply these data to create users & groups in Databricks. And Terraform will take care for storing the state, finding the differences, delete users, etc. (just don't forget to store the Terraform state somewhere durable, like, ADLS, or something like)

The code for doing that is too long to be posted here as the answer, but you can find it here.

Alex_Ott_186270
Related