Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
When you access a private AKS cluster, you need to connect to the cluster from the cluster virtual network, a peered network, or a configured private endpoint. These approaches require configuring a VPN, Express Route, deploying a jumpbox within the cluster virtual network, or creating a private endpoint inside of another virtual network.
With the Azure CLI, you can use command invoke to access private clusters without the need to configure a VPN or Express Route. command invoke allows you to remotely invoke commands, like kubectl and helm, on your private cluster through the Azure API without directly connecting to the cluster. The Microsoft.ContainerService/managedClusters/runcommand/action and Microsoft.ContainerService/managedclusters/commandResults/read actions control the permissions for using command invoke.
With the Azure portal, you can use the Run command feature to run commands on your private cluster. The Run command feature uses the same command invoke functionality to run commands on your cluster.
Tip
You can use Microsoft Copilot in Azure to run kubectl commands in the Azure portal. For more information, see Work with AKS clusters efficiently using Microsoft Copilot in Azure.
The pod created by the Run command provides kubectl and helm for operating your cluster. jq, xargs, grep, and awk are available for Bash support.
Before you begin
Before you begin, make sure you have the following resources and permissions:
- An existing private cluster. If you don't have one, see Create a private AKS cluster.
- The Azure CLI version 2.24.0 or later. Run
az --versionto find the version. If you need to install or upgrade, see Install Azure CLI. - Access to the
Microsoft.ContainerService/managedClusters/runcommand/actionandMicrosoft.ContainerService/managedclusters/commandResults/readroles on the cluster.
Limitations
This feature is designed to simplify cluster access and is not designed for programmatic access. You should leverage direct API access via Bastion, VPN, or Express Route for programmatic calls to your cluster.
If you have a program invoke Kubernetes using Run command, the following disadvantages apply:
- You only get exitCode and text output, and you lose API level details.
- One extra hop introduces extra failure points.
The pod created by the Run command has 200m CPU and 500Mi memory requests, and a 500m CPU and 1Gi memory limits. In cases where all your node is full or over committed, the pod might be unable to be scheduled within the ARM API timeout of 60 seconds. This means that the Run command invocation would fail.
command invoke runs the commands from your cluster, so any commands ran in this manner are subject to your configured networking restrictions and any other configured restrictions. Make sure there are enough nodes and resources in your cluster to schedule this command pod.
Note
The output for command invoke is limited to 512kB in size.
Run commands on your AKS cluster
Below are examples of how to use az aks command invoke to execute commands against a private AKS cluster. These examples assume you have an existing resource group and AKS cluster.
Use command invoke to run a single command
You can run a command on your cluster using the az aks command invoke --command command. The following example command runs the kubectl get pods -n kube-system command on the myPrivateCluster cluster in myResourceGroup.
First, set environment variables for your resource group and cluster name to use in subsequent commands.
export AKS_RESOURCE_GROUP="myResourceGroup"
export AKS_CLUSTER_NAME="myPrivateCluster"
The environment variables above will allow you to run AKS commands in the next sections without having to rewrite their names.
To run a single kubectl command on your AKS cluster:
az aks command invoke \
--resource-group $AKS_RESOURCE_GROUP \
--name $AKS_CLUSTER_NAME \
--command "kubectl get pods -n kube-system"
Use command invoke to run multiple commands
You can also run multiple commands. The following example executes three helm commands in sequence on the cluster.
az aks command invoke \
--resource-group $AKS_RESOURCE_GROUP \
--name $AKS_CLUSTER_NAME \
--command "helm repo add bitnami https://charts.bitnami.com/bitnami && helm repo update && helm install my-release bitnami/nginx"
Use command invoke to run commands with an attached file
When using the --file parameter with az aks command invoke, the file must exist and be accessible in your current working directory. Below, we create a minimal deployment file for demonstration.
To run a command with a file attached, first create a Kubernetes manifest file named deployment.yaml. The following example creates a small nginx deployment and applies it with command invoke:
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-demo
spec:
replicas: 1
selector:
matchLabels:
app: nginx-demo
template:
metadata:
labels:
app: nginx-demo
spec:
containers:
- name: nginx
image: nginx:1.21.6
ports:
- containerPort: 80
EOF
az aks command invoke \
--resource-group $AKS_RESOURCE_GROUP \
--name $AKS_CLUSTER_NAME \
--command "kubectl apply -f deployment.yaml -n default" \
--file deployment.yaml
Use command invoke to run commands with all files in the current directory attached
Use only small, necessary files to avoid exceeding system size limits. Below, two minimal YAML files are created before attaching them.
cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-demo
spec:
replicas: 1
selector:
matchLabels:
app: nginx-demo
template:
metadata:
labels:
app: nginx-demo
spec:
containers:
- name: nginx
image: nginx:1.21.6
ports:
- containerPort: 80
EOF
cat <<EOF > configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config
data:
welcome-message: "Hello from configmap"
EOF
az aks command invoke \
--resource-group $AKS_RESOURCE_GROUP \
--name $AKS_CLUSTER_NAME \
--command "kubectl apply -f deployment.yaml -f configmap.yaml -n default" \
--file deployment.yaml \
--file configmap.yaml
Disable Run command
You can disable the Run command feature by setting .properties.apiServerAccessProfile.disableRunCommand to true.
Troubleshooting
For information on the most common issues with az aks command invoke and how to fix them, see Resolve az aks command invoke failures.
Next steps
In this article, you learned how to access a private cluster and run commands on that cluster. For more information on AKS clusters, see the following articles:
Azure Kubernetes Service