Get Microsoft Entra ID tokens manually

This page describes how to manually generate Microsoft Entra ID access tokens for users and service principals to authenticate with Azure Databricks REST APIs. Manual token generation is an advanced technique.

When to use manual token generation

Use manual token generation only when:

• You need to integrate with systems that cannot use Azure CLI or unified authentication
• You require explicit control over token lifecycle and refresh
• You are debugging authentication issues

For all other use cases, use the recommended authentication methods that handle token management automatically.

Get tokens for users

Use the Azure CLI or the Microsoft Authentication Library (MSAL) to get Microsoft Entra ID access tokens for users.

Azure CLI method

1. Get the Azure subscription ID for your user account:

   • From the Azure Databricks workspace: Click your username > Azure Portal > Overview, and find the Subscription ID.

   • With the Azure CLI: Run the following command (replace with your workspace URL):

     az databricks workspace list --query "[?workspaceUrl==\`adb-0000000000000000.0.azuredatabricks.net\`].{id:id}" -o tsv
     
     # /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-ws
     

     The 00000000-0000-0000-0000-000000000000 after /subscriptions/ is your subscription ID.

     If you get a tenant error, sign in to the correct tenant:

     az login -t <tenant-id>
     

     See Get subscription and tenant IDs in the Azure portal.

2. Sign in to Azure:

   az login
   

3. Set the correct subscription:

   az account set -s <subscription-id>
   

4. Generate the access token:

   az account get-access-token \
   --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d \
   --query "accessToken" \
   -o tsv
   

   The resource ID 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d is the standard identifier for Azure Databricks across all Azure environments.

MSAL method

Use the Microsoft Authentication Library (MSAL) to get Microsoft Entra ID access tokens programmatically. MSAL supports two flows:

• Authorization code flow (interactive): Launches a browser for user sign-in. Use this when two-factor or federated authentication is enabled, or when admin consent is required.
• Username-password flow (programmatic): Authenticates with a username and password. Only use this when you have authority to sign in programmatically.

Before using MSAL, you must register an application in Microsoft Entra ID. See Register an app by using the Azure portal. When registering:

1. Set Supported account types to Accounts in this organizational directory only (Single tenant).
2. Set Redirect URI to Public client/native (mobile & desktop) with value http://localhost.
3. Note the Application (client) ID and Directory (tenant) ID from the app's Overview page.
4. Add the AzureDatabricks API permission:
   1. Go to API permissions > Add a permission.
   2. Search for and select AzureDatabricks.
   3. Enable user_impersonation and click Add permissions.
   4. Click Grant admin consent (requires admin privileges).

Authorization code flow

Install the MSAL Python SDK:

pip install msal

Save the following code as get-tokens.py:

from msal import PublicClientApplication
import sys

# Provide client ID and tenant ID as command-line arguments
client_id = sys.argv[1] if len(sys.argv) > 1 else '<client-id>'
tenant_id = sys.argv[2] if len(sys.argv) > 1 else '<tenant-id>'

scopes = ['2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default']

app = PublicClientApplication(
  client_id=client_id,
  authority=f"https://login.microsoftonline.com/{tenant_id}"
)

result = app.acquire_token_interactive(scopes=scopes)

if 'error' in result:
  print(f"Error: {result['error']}")
  print(f"Description: {result['error_description']}")
else:
  print(f"Access token:\n{result['access_token']}")
  print(f"\nRefresh token:\n{result['refresh_token']}")

Run the script:

python get-tokens.py <client-id> <tenant-id>

Username-password flow

Save the following code as get-tokens-user.py:

from msal import PublicClientApplication
import sys

client_id = sys.argv[1] if len(sys.argv) > 1 else '<client-id>'
tenant_id = sys.argv[2] if len(sys.argv) > 1 else '<tenant-id>'
username = sys.argv[3] if len(sys.argv) > 1 else '<username>'
password = sys.argv[4] if len(sys.argv) > 1 else '<password>'

scopes = ['2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default']

app = PublicClientApplication(
  client_id=client_id,
  authority=f"https://login.microsoftonline.com/{tenant_id}"
)

result = app.acquire_token_by_username_password(
  username=username,
  password=password,
  scopes=scopes
)

if 'error' in result:
  print(f"Error: {result['error']}")
  print(f"Description: {result['error_description']}")
else:
  print(f"Access token:\n{result['access_token']}")
  print(f"\nRefresh token:\n{result['refresh_token']}")

Run the script:

python get-tokens-user.py <client-id> <tenant-id> <username> <password>

Get tokens for service principals

Service principals use the OAuth 2.0 client credentials flow and can have different access control than regular users.

Create a service principal

If you don't have a service principal, create one using the Azure portal or Azure CLI:

Azure portal

1. Sign in to the Azure portal.
2. Switch to the correct tenant using Directories + subscriptions if needed.
3. Search for and select Microsoft Entra ID.
4. Click + Add > App registration.
5. Enter a Name and select Accounts in this organizational directory only (Single tenant).
6. Click Register.
7. Copy these values from the Overview page:
   • Application (client) ID
   • Directory (tenant) ID
8. Go to Certificates & secrets > Client secrets > New client secret.
9. Add a description, set an expiry period, and click Add.
10. Copy and securely store the client secret Value.

Azure CLI

See Create an Azure service principal with Azure CLI for complete instructions.

Generate a token

Generate a token using the REST API or Azure CLI. Gather the following information first:

• Tenant ID: The Directory (tenant) ID from Microsoft Entra ID
• Client ID: The Application (client) ID from Microsoft Entra ID
• Client secret: The client secret Value from Microsoft Entra ID

REST API method

curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \
https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token \
-d 'client_id=<client-id>' \
-d 'grant_type=client_credentials' \
-d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \
-d 'client_secret=<client-secret>'

The access token is in the access_token field of the response.

Azure CLI method

1. Get your Azure subscription ID. See Get tokens for users.

2. Sign in with your service principal:

   az login \
   --service-principal \
   -t <tenant-id> \
   -u <client-id> \
   -p <client-secret>
   

3. Set the correct subscription:

   az account set -s <subscription-id>
   

4. Generate the access token:

   az account get-access-token \
   --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d \
   --query "accessToken" \
   -o tsv
   

Use tokens with Databricks APIs

After generating a token, use it with the Databricks REST API.

Service principals in the workspace

If your service principal is already added to the workspace using the Service Principals API or Databricks CLI:

Databricks CLI

databricks clusters list -p <profile-name-with-token>

curl

curl -X GET \
-H 'Authorization: Bearer <access-token>' \
https://<databricks-instance>/api/2.0/clusters/list

Service principals with Azure role

If your service principal has the Contributor or Owner role on the workspace resource in Azure but isn't in the Azure Databricks workspace yet:

1. Get a management token for Azure Resource Manager:

   curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \
   https://login.microsoftonline.com/<tenant-id>/oauth2/token \
   -d 'client_id=<client-id>' \
   -d 'grant_type=client_credentials' \
   -d 'resource=https%3A%2F%2Fmanagement.core.windows.net%2F' \
   -d 'client_secret=<client-secret>'
   

2. Call the Databricks API with both tokens:

   curl -X GET \
   -H 'Authorization: Bearer <databricks-access-token>' \
   -H 'X-Databricks-Azure-SP-Management-Token: <management-access-token>' \
   -H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Databricks/workspaces/<workspace-name>' \
   https://<databricks-instance>/api/2.0/clusters/list
   

After first authentication, the service principal becomes a workspace admin.

Refresh tokens

If you obtained a refresh token with your access token, use it to get a new token. Microsoft Entra ID access tokens expire after 60-90 minutes by default.

Save the following code as refresh-tokens.py:

from msal import PublicClientApplication
import sys

client_id = sys.argv[1] if len(sys.argv) > 1 else '<client-id>'
tenant_id = sys.argv[2] if len(sys.argv) > 1 else '<tenant-id>'
refresh_token = sys.argv[3] if len(sys.argv) > 1 else '<refresh-token>'

scopes = ['2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default']

app = PublicClientApplication(
  client_id=client_id,
  authority=f"https://login.microsoftonline.com/{tenant_id}"
)

result = app.acquire_token_by_refresh_token(
  refresh_token=refresh_token,
  scopes=scopes
)

if 'error' in result:
  print(f"Error: {result['error']}")
  print(f"Description: {result['error_description']}")
else:
  print(f"\nNew access token:\n{result['access_token']}")
  print(f"\nNew refresh token:\n{result['refresh_token']}")

Run the script:

python refresh-tokens.py <client-id> <tenant-id> <refresh-token>

Troubleshoot token issues

This section describes common token errors and how to validate access tokens.

Failed to get token with username and password

The user or administrator has not consented to use the application with ID <client-id>.
Send an interactive authorization request for this user and resource.

Solution: Ensure the AzureDatabricks resource is added to your application. Use the authorization code flow (interactive method) to consent to permissions. After consenting, you can use the username-password flow. See Get tokens for users.

Redirect URIs do not match

The reply URL specified in the request does not match the reply URLs configured for the application: '<application-id>'

Solution: Verify the redirect URI in your request matches one configured in the application.

Validate an access token

Make sure that your Microsoft Entra ID access token includes the correct information. See validate tokens.

• aud: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d (Databricks resource ID)
• iss: https://sts.windows.net/<tenant-id>/
• tid: Workspace tenant ID
• nbf/exp: Current time falls between these values
• unique_name: User exists in the workspace

Validate the signature using public certs from the OIDC endpoints.

Decode without signature verification:

The following code shows the payload of the token. You must first install the PyJWT library using pip install pyjwt and the cryptography library using pip install cryptography.

import jwt

def decode_token(token):
  algorithm = jwt.get_unverified_header(token)['alg']
  decoded = jwt.decode(token, algorithms=[algorithm], options={"verify_signature": False})
  for key in decoded.keys():
    print(f"{key}: {str(decoded[key])}")

Decode with signature verification:

import jwt
import requests
from cryptography.x509 import load_pem_x509_certificate
from cryptography.hazmat.backends import default_backend

PEMSTART = '-----BEGIN CERTIFICATE-----\n'
PEMEND = '\n-----END CERTIFICATE-----\n'

def get_public_key_for_token(kid):
  response = requests.get('https://login.microsoftonline.com/common/.well-known/openid-configuration').json()
  pubkeys = requests.get(response['jwks_uri']).json()['keys']

  for key in pubkeys:
    if key['kid'] == kid:
      cert_str = PEMSTART + str(key['x5c'][0]) + PEMEND
      cert_obj = load_pem_x509_certificate(bytes(cert_str, 'ascii'), default_backend())
      return cert_obj.public_key()

def aad_access_token_decoder(access_token):
  header = jwt.get_unverified_header(access_token)
  public_key = get_public_key_for_token(header['kid'])
  decoded = jwt.decode(access_token, key=public_key, algorithms='RS256',
                      audience='2ff814a6-3304-4ab8-85cb-cd0e6f879c1d')
  for key in decoded.keys():
    print(f"{key}: {str(decoded[key])}")

Use online JWT decoders like jwt.ms or jwt.io for non-sensitive tokens.