Configure a federation policy

Databricks OAuth token federation enables you to securely access Databricks APIs using tokens from your identity provider (IdP). To enable OAuth token federation, you must configure a federation policy, either as Databricks account-wide or for workloads.

This page describes how to create and configure an OAuth token federation policy.

Workload identity federation

Workload identity federation allows your automated workloads running outside of Azure Databricks to access Azure Databricks APIs without the need for Azure Databricks secrets. Account admins can configure workload identity federation using a service principal federation policy.

A service principal federation policy is associated with a service principal in your Azure Databricks account, and specifies:

• The identity provider (or issuer) from which the service principal can authenticate.
• The workload identity (or subject) that is permitted to authenticate as the Azure Databricks service principal.

For example, given the following service principal federation policy for a Github Actions workload:

• Issuer: https://token.actions.githubusercontent.com
• Audiences: https://github.com/my-github-org
• Subject: repo:my-github-org/my-repo:environment:prod

You can use this JWT body to authenticate to Azure Databricks:

{
  "iss": "https://token.actions.githubusercontent.com",
  "aud": "https://github.com/my-github-org",
  "sub": "repo:my-github-org/my-repo:environment:prod"
}

Configure a service principal federation policy

Account admins can configure a service principal federation policy using the Databricks CLI or the Databricks API. You can create a maximum of 20 service principal federation policies per Azure Databricks service principal.

To configure a service principal federation policy, you must specify the following:

• Issuer URL: An HTTPS URL that identifies the workload identity provider, specified in the iss claim of workload identity tokens.

• Subject: The unique identifier for the workload in the workload runtime environment. If unspecified, the default is sub.

• Audiences: The intended recipient of the token, specified in the aud claim. The token is considered a match if its audience matches at least one audience in the policy. If unspecified, the default is your Azure Databricks account ID.

• Subject claim: (Optional) Specifies the token claim that contains the workload identity (also called the subject) of the token. If not set, Azure Databricks uses sub by default. Databricks recommends keeping the default sub claim for workload identity federation. Only choose a different claim if sub isn't a suitable or stable subject identifier, which is rare. For details, see Example service principal federation policies.

• Token signature validation: (Optional) The public keys, or their URL, in JSON Web Key Sets (JWKS) format used to validate token signatures. JWKS JSON supports up to 5 keys. If your identity provider publishes more, use a JWKS URI instead.

  If unspecified, Azure Databricks retrieves the keys from the issuer's well-known endpoint, which is the recommended approach. Your identity provider must serve OpenID Provider Metadata at <issuer-url>/.well-known/openid-configuration that includes a jwks_uri specifying the location of the public keys used to verify token signatures.

Databricks UI

1. As an account admin, sign in to the Azure Databricks account console at https://accounts.azuredatabricks.net.
2. Click User management.
3. Go to the Service principals tab.
4. Select the service principal to create the policy for.
5. Go to the Credentials & secrets tab.
6. Under Federation policies tab, click Create policy.
7. Select a federated credential provider and configure the corresponding fields.
8. Click Create policy.

Databricks CLI

You can't use the Azure Databricks CLI in the Azure Databricks workspace web terminal to create a federation policy.

1. Install or update to the newest version of the Azure Databricks CLI.

2. As an account admin, authenticate to your Azure Databricks account using the CLI. Specify the ACCOUNT_CONSOLE_URL and your Azure Databricks ACCOUNT_ID:

   databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}
   

3. Get the numeric ID of the service principal that will have the federation policy applied to it. (For example, 3659993829438643.)

   If you know the service principal application ID (typically a GUID value, such as bc3cfe6c-469e-4130-b425-5384c4aa30bb) in advance, you can then determine the service principal numeric ID using the Databricks CLI:

   databricks account service-principals list --filter 'applicationId eq "<service-principal-application-id>"'
   

4. Create the service principal federation policy. Here is an example of creating a federation policy for a GitHub Action:

   databricks account service-principal-federation-policy create ${SERVICE_PRINCIPAL_NUMERIC_ID} --json \
   '{
     "oidc_policy": {
       "issuer": "https://token.actions.githubusercontent.com",
       "audiences": [
         "https://github.com/my-github-org"
       ],
       "subject": "repo:my-github-org/my-repo:environment:prod"
     }
   }'
   

Databricks Account API

1. Get the service principal numeric ID (for example, 3659993829438643) from the account console or using the Service Principals API.

2. Create the service principal federation policy:

   curl --request POST \
     --header "Authorization: Bearer $TOKEN" \
     "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/servicePrincipals/${SERVICE_PRINCIPAL_NUMERIC_ID}/federationPolicies" \
     --data '{
       "oidc_policy": {
         "issuer": "https://token.actions.githubusercontent.com",
         "audiences": [
           "https://github.com/my-github-org"
         ],
         "subject": "repo:my-github-org/my-repo:environment:prod"
       }
     }'
   

   For complete API reference documentation, see the Account Federation Policy API.

Example Databricks service principal federation policies

The following table provides example service principal federation policies and the matching JWT body.

For complete configuration steps for enabling workload identity federation for some of these common identity providers, see Enable workload identity federation in CI/CD.

Account-wide token federation

Account admins can configure OAuth token federation in the Azure Databricks account using an account federation policy. An account federation policy enables all users and service principals in your Azure Databricks account to access Databricks APIs using tokens from your identity provider. An account federation policy specifies:

• The identity provider or issuer from which Azure Databricks will accept tokens.
• The criteria for mapping a token to the corresponding Azure Databricks user or service principal.

For example, given a federation policy with the following fields:

• Issuer: https://idp.mycompany.com/oidc
• Audiences: databricks
• Subject claim: sub

Use this JWT body to authenticate to Azure Databricks as username@mycompany.com:

{
  "iss": "https://idp.mycompany.com/oidc",
  "aud": "databricks",
  "sub": "username@mycompany.com"
}

Configure an account federation policy

Account admins can configure an account federation policy using the Azure Databricks UI, the Databricks CLI or the Databricks REST API. You can specify a maximum of 20 account federation policies in your Azure Databricks account.

To configure an account federation policy, you must specify the following:

• Issuer URL: An HTTPS URL that identifies your identity provider, specified in the iss claim of your tokens.

• Audiences: The intended recipient of the token, specified in the aud claim. The token is considered a match if its audience matches at least one audience in the policy. If unspecified, the default is your Azure Databricks account ID.

• Subject claim: The token claim that contains the Azure Databricks username of the user that the token was issued for. If unspecified, the default is sub.

• Token signature validation: (Optional) The public keys, or their URL, in JSON Web Key Sets (JWKS) format used to validate token signatures. JWKS JSON supports up to 5 keys. If your identity provider publishes more, use a JWKS URI instead.

  If unspecified, Azure Databricks retrieves the keys from the issuer's well-known endpoint, which is the recommended approach. Your identity provider must serve OpenID Provider Metadata at <issuer-url>/.well-known/openid-configuration that includes a jwks_uri specifying the location of the public keys used to verify token signatures.

Databricks UI

1. As an account admin, sign in to the Azure Databricks account console at https://accounts.azuredatabricks.net.
2. Click Settings and go to the Authentication tab.
3. Under Federation policies, click Create policy.
4. Enter the issuer URL, audiences, subject claim, and optional token signature validation.
5. Click Create policy.

Databricks CLI

You can't use the Azure Databricks CLI in the Azure Databricks workspace web terminal to create a federation policy.

1. Install or update to the newest version of the Databricks CLI.

2. As an account admin, authenticate to your Azure Databricks account using the CLI. Specify the ACCOUNT_CONSOLE_URL and your Azure Databricks ACCOUNT_ID.

   databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}
   

3. Create the account federation policy. For example:

   databricks account federation-policy create --json \
   '{
     "oidc_policy": {
       "issuer": "https://idp.mycompany.com/oidc",
       "audiences": [
         "databricks"
       ],
       "subject_claim": "sub"
     }
   }'
   

Databricks Account API

The following Azure Databricks REST API call creates an account federation policy:

curl --request POST \
  --header "Authorization: Bearer $TOKEN" \
  "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/federationPolicies" \
  --data '{
    "oidc_policy": {
      "issuer": "https://idp.mycompany.com/oidc",
      "audiences": [
        "databricks"
      ],
      "subject_claim": "sub"
    }
  }'

For complete API reference documentation, see the Account Federation Policy API.

Example account federation policies

The following table provides example account federation policies and the matching JWT body.

Next steps

After you configure a federation policy for your account:

• Configure your identity provider (IdP) to generate tokens that your users can exchange with Azure Databricks. Refer to your IdP’s documentation for setup details. For instructions to enable workload identity federation with common IdPs, see Enable workload identity federation in CI/CD.
• Use a JWT from your IdP to access the Azure Databricks API by first exchanging it for a Azure Databricks OAuth token. Include the Azure Databricks OAuth token in the Bearer: header of your API call to complete the request. The JWT must be valid and signed using the RS256 or ES256 algorithms. For implementation details, see Authenticate with an identity provider token.