Use cluster connect to securely connect to Azure Arc-enabled Kubernetes clusters

2025-01-24

With cluster connect, you can securely connect to Azure Arc-enabled Kubernetes clusters from anywhere without requiring any inbound port to be enabled on the firewall.

Access to the apiserver of the Azure Arc-enabled Kubernetes cluster enables the following scenarios:

Interactive debugging and troubleshooting.
Cluster access to Azure services for custom locations and other resources created on top of it.

Before you begin, review the conceptual overview of the cluster connect feature.

Prerequisites

An Azure account with an active subscription. Create an account for free.
An existing Azure Arc-enabled Kubernetes connected cluster.
- If you haven't connected a cluster yet, use our quickstart.
- Upgrade your agents to the latest version.
Enable the network requirements for Arc-enabled Kubernetes, including all endpoints listed as required for cluster connect.

Azure CLI
Azure PowerShell

Install or update Azure CLI to the latest version.
Install the latest version of the connectedk8s Azure CLI extension:
```
az extension add --name connectedk8s
```
If you've already installed the connectedk8s extension, update the extension to the latest version:
```
az extension update --name connectedk8s
```

Replace the placeholders and run the below command to set the environment variables used in this document:

CLUSTER_NAME=<cluster-name>
RESOURCE_GROUP=<resource-group-name>
ARM_ID_CLUSTER=$(az connectedk8s show -n $CLUSTER_NAME -g $RESOURCE_GROUP --query id -o tsv)

Install the latest version of Azure PowerShell.

Replace the placeholders and run the below command to set the environment variables used in this document:

$CLUSTER_NAME = <cluster-name>
$RESOURCE_GROUP = <resource-group-name>
$ARM_ID_CLUSTER = (Get-AzConnectedKubernetes -ResourceGroupName $RESOURCE_GROUP -Name $CLUSTER_NAME).Id

Set up authentication

On the existing Arc-enabled cluster, create the ClusterRoleBinding with either Microsoft Entra authentication or service account token.

Get the objectId associated with your Microsoft Entra entity. For single user accounts, get the user principal name (UPN) associated with your Microsoft Entra entity.
- For a Microsoft Entra group account:
```
AAD_ENTITY_ID=$(az ad signed-in-user show --query id -o tsv)
```
- For a Microsoft Entra single user account:
```
AAD_ENTITY_ID=$(az ad signed-in-user show --query userPrincipalName -o tsv)
```
- For a Microsoft Entra application:
```
AAD_ENTITY_ID=$(az ad sp show --id <id> --query id -o tsv)
```
Authorize the entity with appropriate permissions.
- If you use Kubernetes native ClusterRoleBinding or RoleBinding for authorization checks on the cluster, with the kubeconfig file pointing to the apiserver of your cluster for direct access, you can create one mapped to the Microsoft Entra entity that needs to access this cluster. For example:
```
kubectl create clusterrolebinding demo-user-binding --clusterrole cluster-admin --user=$AAD_ENTITY_ID
```
- If you use Azure RBAC for authorization checks on the cluster, you can create an applicable Azure role assignment mapped to the Microsoft Entra entity. For example:
```
az role assignment create --role "Azure Arc Kubernetes Viewer" --assignee $AAD_ENTITY_ID --scope $ARM_ID_CLUSTER
az role assignment create --role "Azure Arc Enabled Kubernetes Cluster User Role" --assignee $AAD_ENTITY_ID --scope $ARM_ID_CLUSTER
```

Get the objectId associated with your Microsoft Entra entity. For single user accounts, get the user principal name (UPN) associated with your Microsoft Entra entity.
- For a Microsoft Entra group account:
```
$AAD_ENTITY_ID = (az ad signed-in-user show --query id -o tsv)
```
- For a Microsoft Entra single user account:
```
$AAD_ENTITY_ID = (az ad signed-in-user show --query userPrincipalName -o tsv)
```
- For a Microsoft Entra application:
```
$AAD_ENTITY_ID = (az ad sp show --id <id> --query objectId -o tsv)
```
Authorize the entity with appropriate permissions.
- If you use native Kubernetes ClusterRoleBinding or RoleBinding for authorization checks on the cluster, with the kubeconfig file pointing to the apiserver of your cluster for direct access, you can create one mapped to the Microsoft Entra entity that needs to access this cluster. For example:
```
kubectl create clusterrolebinding demo-user-binding --clusterrole cluster-admin --user=$AAD_ENTITY_ID
```
- If you use Azure RBAC for authorization checks on the cluster, you can create an applicable Azure role assignment mapped to the Microsoft Entra entity. For example:
```
az role assignment create --role "Azure Arc Kubernetes Viewer" --assignee $AAD_ENTITY_ID --scope $ARM_ID_CLUSTER
az role assignment create --role "Azure Arc Enabled Kubernetes Cluster User Role" --assignee $AAD_ENTITY_ID --scope $ARM_ID_CLUSTER
```

Service account token authentication option

Azure CLI
Azure PowerShell

With the kubeconfig file pointing to the apiserver of your Kubernetes cluster, run this command to create a service account. This example creates the service account in the default namespace, but you can substitute any other namespace for default.
```
kubectl create serviceaccount demo-user -n default
```
Create a ClusterRoleBinding to grant this service account the appropriate permissions on the cluster. If you used a different namespace in the first command, substitute it here for default.
```
kubectl create clusterrolebinding demo-user-binding --clusterrole cluster-admin --serviceaccount default:demo-user
```

Create a service account token:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: demo-user-secret
  annotations:
    kubernetes.io/service-account.name: demo-user
type: kubernetes.io/service-account-token
EOF

TOKEN=$(kubectl get secret demo-user-secret -o jsonpath='{$.data.token}' | base64 -d | sed 's/$/\n/g')

Get the token to output to console
```
echo $TOKEN
```

With the kubeconfig file pointing to the apiserver of your Kubernetes cluster, run this command to create a service account. This example creates the service account in the default namespace, but you can substitute any other namespace for default.
```
kubectl create serviceaccount demo-user -n default
```
Create a ClusterRoleBinding or RoleBinding to grant this service account the appropriate permissions on the cluster. If you used a different namespace in the first command, substitute it here for default.
```
kubectl create clusterrolebinding demo-user-binding --clusterrole cluster-admin --serviceaccount default:demo-user
```

Create a service account token. Create a demo-user-secret.yaml file with the following content:

apiVersion: v1
kind: Secret
metadata:
  name: demo-user-secret
  annotations:
    kubernetes.io/service-account.name: demo-user
type: kubernetes.io/service-account-token

Then run these commands:

kubectl apply -f demo-user-secret.yaml

$TOKEN = ([System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String((kubectl get secret demo-user-secret -o jsonpath='{$.data.token}'))))

Get the token to output to console.
```
echo $TOKEN
```

Access your cluster from a client device

Now you can access the cluster from a different client. Run the following steps on another client device.

Sign in using either Microsoft Entra authentication or service account token authentication.
Get the cluster connect kubeconfig needed to communicate with the cluster from anywhere (even outside the firewall surrounding the cluster), based on the authentication option used:
- For Microsoft Entra authentication:
```
az connectedk8s proxy -n $CLUSTER_NAME -g $RESOURCE_GROUP
```
- For service account token authentication:
```
az connectedk8s proxy -n $CLUSTER_NAME -g $RESOURCE_GROUP --token $TOKEN
```
  Note
  
  This command opens the proxy and blocks the current shell.
In a different shell session, use kubectl to send requests to the cluster:
```
kubectl get pods -A
```

You should now see a response from the cluster containing the list of all pods under the default namespace.

Known limitations

When making requests to the Kubernetes cluster, if the Microsoft Entra service principal used is a part of more than 200 groups, you might see the following error:

Overage claim (users with more than 200 group membership) for SPN is currently not supported. For troubleshooting, please refer to aka.ms/overageclaimtroubleshoot

This is a known limitation. To get past this error:

Create a service principal, which is less likely to be a member of more than 200 groups.
Sign in to Azure CLI with the service principal before running the az connectedk8s proxy command.

Next steps

Set up Microsoft Entra RBAC on your clusters.
Deploy and manage cluster extensions.
Help to protect your cluster in other ways by following the guidance in the security book for Azure Arc-enabled Kubernetes.

Feedback

Was this page helpful?