Databricks Identity Sync from Microsoft Entra ID

Identity management is essential for any application to ensure that the right people have the right level of access with the appropriate permissions. When using Azure as the cloud provider with Databricks, Microsoft provides built-in integrations that simplify identity and access management.

In Databricks, identities such as users, groups, and service principals can be created directly within the platform. Alternatively, identities that already exist in Microsoft Azure (Microsoft Entra ID) can be synchronized with Databricks, allowing organizations to centrally manage identities and access.

Let’s demonstrate how identities can be created and synchronized between Azure and Databricks.

Prerequisite

Have a Databricks workspace with Unity Catalog enabled and a metastore configured for centralized data governance.

Create user and group in Microsoft Entra ID

The below users and groups created in Microsoft Entra ID and the users added in the group.

Enable Automatic Identity management

Traditionally, this required configuring SCIM provisioning, which automatically syncs users and groups from Entra ID to Databricks.

When automatic identity management is enabled, all users, groups, and group memberships sync from Microsoft Entra ID to Azure Databricks so SCIM provisioning is not necessary.

Databricks recommends using automatic identity management.