Authentication for the Databricks CLI

This article explains how to set up authentication between the Azure Databricks CLI and your Azure Databricks accounts and workspaces. It assumes you already installed the Azure Databricks CLI. See Install or update the Databricks CLI.

Before you run Azure Databricks CLI commands, you must configure authentication for the accounts or workspaces that you plan to use. The required setup depends on whether you want to run workspace-level commands, account-level commands, or both.

To view available CLI command groups, run databricks -h. For the list of corresponding REST API operations, see Databricks REST API.

For information about Microsoft Entra authentication to Databricks with Azure DevOps specifically, see Authenticate with Azure DevOps on Azure Databricks.

OAuth machine-to-machine (M2M) authentication

Machine-to-machine (M2M) authentication with OAuth allows services, scripts, or applications to access Databricks resources without interactive user sign-in. Instead of relying on personal access tokens (PATs) or user credentials, M2M authentication uses a service principal and an OAuth client credential flow to request and manage tokens.

To configure and use OAuth M2M authentication:

1. Complete the OAuth M2M authentication setup steps. See Authorize service principal access to Azure Databricks with OAuth.

2. Create a Azure Databricks configuration profile with the following fields in your .databrickscfg file.

   For account-level commands

   [<some-unique-configuration-profile-name>]
   host          = <account-console-url>
   account_id    = <account-id>
   client_id     = <service-principal-client-id>
   client_secret = <service-principal-oauth-secret>
   

   For workspace-level commands

   [<some-unique-configuration-profile-name>]
   host          = <workspace-url>
   client_id     = <service-principal-client-id>
   client_secret = <service-principal-oauth-secret>
   

To use the profile, pass it with the --profile or -p flag in CLI commands. For example:

databricks account groups list -p <profile-name>

Press Tab after --profile or -p to show a list of available profiles.

OAuth user-to-machine (U2M) authentication

With OAuth user-to-machine (U2M) authentication, you log in interactively and the CLI manages short-lived tokens on your behalf. OAuth tokens expire in under an hour, which reduces risk if a token is accidentally exposed. See Authorize user access to Azure Databricks with OAuth.

To log in:

For account-level commands

databricks auth login --host <account-console-url> --account-id <account-id>

For workspace-level commands

databricks auth login --host <workspace-url>

The CLI guides you through a browser-based login flow. When you finish, the CLI saves the credentials as a configuration profile. You can accept the suggested profile name or enter your own.

To use the profile, pass it with the --profile or -p flag in CLI commands. For example:

databricks clusters list -p <profile-name>

Press Tab after --profile or -p to show a list of available profiles.

Azure managed identities authentication

Azure managed identities authentication uses managed identities for Azure resources (formerly Managed Service Identities (MSI)) for authentication. See What are managed identities for Azure resources?. See also Authenticate with Azure managed identities.

To create an Azure user-assigned managed identity, do the following:

1. Create or identify an Azure VM and install the Databricks CLI on it, then assign your managed identity to your Azure VM and your target Azure Databricks accounts, workspaces, or both. See Use Azure managed identities with Azure Databricks.

2. On the Azure VM, create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values.

   For account-level commands, set the following values in your .databrickscfg file:

   [<some-unique-configuration-profile-name>]
   host            = <account-console-url>
   account_id      = <account-id>
   azure_client_id = <azure-managed-identity-application-id>
   azure_use_msi   = true
   

   For workspace-level commands, set the following values in your .databrickscfg file:

   [<some-unique-configuration-profile-name>]
   host            = <workspace-url>
   azure_client_id = <azure-managed-identity-application-id>
   azure_use_msi   = true
   

   For workspace-level commands, if the target identity has not already been added to the workspace, then specify azure_workspace_resource_id along with the Azure resource ID, instead of host along with the workspace URL. In this case, the target identity must have at least Contributor or Owner permissions on the Azure resource.

3. On the Azure VM, use the Databricks CLI's --profile or -p option followed by the name of your configuration profile to set the profile for Databricks to use, for example, databricks account groups list -p <configuration-profile-name> or databricks clusters list -p <configuration-profile-name>.

   Tip

   You can press Tab after --profile or -p to display a list of existing available configuration profiles to choose from, instead of entering the configuration profile name manually.

Microsoft Entra ID service principal authentication

Microsoft Entra ID service principal authentication uses the credentials of a Microsoft Entra ID service principal to authenticate. To create and manage service principals for Azure Databricks, see Service principals. See also Authenticate with MS Entra service principals.

To configure and use Microsoft Entra ID service principal authentication, you must have the Authenticate with the Azure CLI installed locally. You must also do the following:

1. Create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values.

   For account-level commands, set the following values in your .databrickscfg file:

   [<some-unique-configuration-profile-name>]
   host                = <account-console-url>
   account_id          = <account-id>
   azure_tenant_id     = <azure-service-principal-tenant-id>
   azure_client_id     = <azure-service-principal-application-id>
   azure_client_secret = <azure-service-principal-client-secret>
   

   For workspace-level commands, set the following values in your .databrickscfg file:

   [<some-unique-configuration-profile-name>]
   host                = <workspace-url>
   azure_tenant_id     = <azure-service-principal-tenant-id>
   azure_client_id     = <azure-service-principal-application-id>
   azure_client_secret = <azure-service-principal-client-secret>
   

   For workspace-level commands, if the target Microsoft Entra ID service principal has not already been added to the workspace, then specify azure_workspace_resource_id along with the Azure resource ID, instead of host along with the workspace URL. In this case, the target Microsoft Entra ID service principal must have at least Contributor or Owner permissions on the Azure resource.

2. Use the Databricks CLI's --profile or -p option followed by the name of your configuration profile, as part of the Databricks CLI command call, for example databricks account groups list -p <configuration-profile-name> or databricks clusters list -p <configuration-profile-name>.

   Tip

   You can press Tab after --profile or -p to display a list of existing available configuration profiles to choose from, instead of entering the configuration profile name manually.

Azure CLI authentication

Azure CLI authentication uses the Azure CLI to authenticate the signed-in entity. See also Authenticate with the Azure CLI.

To configure Azure CLI authentication, you must do the following:

1. Have the Azure CLI installed locally.

2. Use the Azure CLI to log in to Azure Databricks by running the az login command. See Sign in with the Azure CLI.

3. Create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values.

   For account-level commands, set the following values in your .databrickscfg file:

   [<some-unique-configuration-profile-name>]
   host       = <account-console-url>
   account_id = <account-id>
   

   For workspace-level commands, set the following values in your .databrickscfg file:

   [<some-unique-configuration-profile-name>]
   host = <workspace-url>
   

4. Use the Databricks CLI's --profile or -p option followed by the name of your configuration profile, as part of the Databricks CLI command call, for example databricks account groups list -p <configuration-profile-name> or databricks clusters list -p <configuration-profile-name>.

   Tip

   You can press Tab after --profile or -p to display a list of existing available configuration profiles to choose from, instead of entering the configuration profile name manually.

Authentication order of evaluation

Whenever the Databricks CLI authenticates to a Azure Databricks workspace or account, it looks for required settings in the following order:

1. Bundle settings files, for commands run from a bundle working directory. Bundle settings files can't contain credential values directly.
2. Environment variables, as listed in this article and in Environment variables and fields for unified authentication.
3. Configuration profiles in the .databrickscfg file.

As soon as the CLI finds the required setting, it stops searching other locations.

Examples:

• If a DATABRICKS_TOKEN environment variable is set, the CLI uses it, even if multiple tokens exist in .databrickscfg.
• If no DATABRICKS_TOKEN is set, and a bundle environment references a profile name such as dev → profile DEV, the CLI uses the credentials from that profile in .databrickscfg.
• If no DATABRICKS_TOKEN is set, and a bundle environment specifies a host value, the CLI searches for a profile in .databrickscfg with a matching host and uses its token.

Personal access token authentication (deprecated)

Azure Databricks personal access token authentication uses an Azure Databricks personal access token to authenticate the target Azure Databricks entity, such as an Azure Databricks user account. See Authenticate with Azure Databricks personal access tokens (legacy).

To create a personal access token, follow the steps in Create personal access tokens for workspace users.