Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This page explains how to authorize access to Azure Databricks resources using the Azure Databricks CLI and REST APIs. It describes different authorization methods, when to use them, and how to configure authentication for your use case.
To access Azure Databricks resources with the CLI or REST APIs, you must authenticate using a Azure Databricks account with appropriate permissions. Your Azure Databricks administrator or a user with administrator privileges configures the account.
Account and API types
There are two types of accounts that you can use for authentication:
- User account: For interactive CLI commands and API calls.
- Service principal: For automated CLI commands and API calls without human interaction.
You can use an MS Entra service principal to authorize access to your Azure Databricks account or workspace. See Authenticate with MS Entra service principals. However, Databricks recommends using a Databricks service principal with OAuth, as its access tokens are more robust for Databricks-only authorization.
Azure Databricks has two types of APIs that require different authentication approaches:
- Account-level APIs: Available to account owners and admins, hosted on the account console URL. See account-level APIs.
- Workspace-level APIs: Available to workspace users and admins, hosted at workspace URLs. See workspace-level APIs.
Authorization methods
Choose the authorization method that best fits your use case. Azure Databricks tools and SDKs support multiple authorization methods, so you can select the most appropriate one for your scenario.
| Method | Description | Use case |
|---|---|---|
| Databricks OAuth token federation (Recommended) | OAuth tokens from your identity provider for users or service principals. | Enables you to authenticate to Azure Databricks without managing Databricks secrets. |
| Databricks OAuth for service principals (OAuth M2M) | Short-lived OAuth tokens for service principals. | Unattended authentication scenarios, such as fully automated and CI/CD workflows. |
| OAuth for users (OAuth U2M) | Short-lived OAuth tokens for users. | Attended authentication scenario, where you use your web browser to authenticate with Azure Databricks in real time, when prompted. |
| Azure managed service identity authorization | Microsoft Entra ID tokens for Azure managed identities. | Use only with Azure resources that support managed identities, such as Azure virtual machines. |
| Microsoft Entra ID service principal authorization | Microsoft Entra ID tokens for Microsoft Entra ID service principals. | Use only with Azure resources that support Microsoft Entra ID tokens and do not support managed identities, such as Azure DevOps. |
| Azure CLI authorization | Microsoft Entra ID tokens for users or Microsoft Entra ID service principals. | Use to authorize access to Azure resources and Azure Databricks using the Azure CLI. |
| Microsoft Entra ID user authorization | Microsoft Entra ID tokens for users. | Use only with Azure resources that only support Microsoft Entra ID tokens. Databricks does not recommend that you create Microsoft Entra ID tokens for Azure Databricks users manually. |
Unified authentication
Azure Databricks unified authentication provides a consistent way to configure authentication across all supported tools and SDKs. This approach uses standard environment variables and configuration profiles to store credential values, so you can run CLI commands or call APIs without repeatedly configuring authentication.
Unified authentication handles account types differently:
- User authorization: OAuth automatically creates and manages access tokens for tools that support unified authentication. See Authorize user access to Azure Databricks with OAuth.
- Service principal authorization: OAuth requires client credentials (client ID and secret) that Azure Databricks provides after you assign the service principal to workspaces. See Authorize service principal access to Azure Databricks with OAuth.
To use OAuth access tokens, your Azure Databricks workspace or account administrator must grant your user account or service principal the CAN USE permission for the account and workspace features that your code will access.
Environment variables
To configure unified authentication, set the following environment variables. Some variables apply to both user and service principal authorization, while others are required only for service principals.
| Environment variable | Description |
|---|---|
DATABRICKS_HOST |
The URL of either your Azure Databricks account console (https://accounts.azuredatabricks.net) or your Azure Databricks workspace (https://{workspace-url-prefix}.azuredatabricks.net). Choose based on the type of operations your code performs. |
DATABRICKS_ACCOUNT_ID |
Used for Azure Databricks account operations. This is your Azure Databricks account ID. To get it, see Locate your account ID. |
DATABRICKS_CLIENT_ID |
(Service principal OAuth only) The client ID you were assigned when creating your service principal. |
DATABRICKS_CLIENT_SECRET |
(Service principal OAuth only) The client secret you generated when creating your service principal. |
Rather than setting environment variables manually, consider defining them in a Databricks configuration profile (.databrickscfg) on your client machine.
Configuration profiles
Azure Databricks configuration profiles contain settings and credentials that Azure Databricks tools and SDKs need to authorize access. These profiles are stored in local client files (typically named .databrickscfg) for your tools, SDKs, scripts, and apps to use.
For more information, see Azure Databricks configuration profiles.
Third-party integrations
When integrating with third-party services and tools, you must use the authentication mechanisms provided by those services. However, Azure Databricks provides support for common integrations:
- Databricks Terraform Provider: Access Azure Databricks APIs from Terraform using your Azure Databricks user account. See Provision a service principal by using Terraform.
- Git providers: GitHub, GitLab, and Bitbucket can access Azure Databricks APIs using a Databricks service principal. See Service principals for CI/CD.
- Jenkins: Access Azure Databricks APIs using a Databricks service principal. See CI/CD with Jenkins on Azure Databricks.
- Azure DevOps: Access Azure Databricks APIs using an MS Entra ID-based service principal. See Authenticate with Azure DevOps on Azure Databricks.