This article explains how to add and configure a catalog for your Microsoft Dev Box dev center or project.
Catalogs help you provide a set of curated image definitions for your development teams to create ready-to-code dev boxes. You can attach your own source control repository from GitHub or Azure Repos as a catalog.
To further secure your templates, the catalog is encrypted; Dev Box supports encryption at rest with platform-managed encryption keys, which Microsoft for Azure Services manages.
Attaching catalogs at the project level enables platform engineers to provide image definitions that are specific to each development teams. Additionally, it empowers dev team leads assigned as Project Admins to manage the image definitions made available to their teams.
Platform engineers have full control over the use of catalogs at the project level. The use of project-level catalogs must be enabled at the dev center level before a catalog can be added to a project. Platform engineers can also configure which types of catalogs items, such as image definitions, can be consumed at the project level.
By default, use of catalogs at the project level is disabled and none of the catalog item types are enabled. Environment definitions from a project-level catalog are synced and usable under two conditions. First, you must enable project-based catalogs at the corresponding dev center level. Second, you must enable the use of image definitions for the project.
Add a catalog to a project
You must enable project-level catalogs at the dev center level before you can add a catalog to a project. You should also enable the use of image definitions at the project level.
To enable the use of project-level catalogs at the dev center level:
In the Azure portal, navigate to your dev center.
In the left menu, under Settings, select Dev center settings.
Under Project level catalogs, select Enable catalogs per project, and then select Apply.
To enable the use of image definitions in the project:
In the Azure portal, navigate to your project.
In the left menu, under Settings, select Catalogs.
On the Catalogs page, select Catalog item permissions.
In the Catalog item settings pane, select Azure deployment image definitions to enable the use of image definitions at the project level.
Now, you can add a catalog to the project.
For catalogs that use a managed identity or Personal Access Token (PAT) for authentication, you must assign a managed identity for the project. For catalogs that use a PAT, you must store the PAT in a key vault and grant the managed identity access to the key vault secret.
Before you can attach a catalog to a dev center or project, you must configure a managed identity, also called a Managed Service Identity (MSI). You can attach either a system-assigned managed identity (system-assigned MSI) or a user-assigned managed identity (user-assigned MSI). You then assign roles to the managed identity to allow the dev center or project to create environment types in your subscription and read the Azure Repos project that contains the catalog repo.
If your dev center or project doesn't have an MSI attached, follow the steps in Configure a managed identity to create one and to assign roles for the managed identity.
To learn more about managed identities, see What are managed identities for Azure resources?
Add a catalog
You can add a catalog from an Azure Repos repository or a GitHub repository. You can choose to authenticate by assigning permissions to an MSI or by using a PAT, which you store in a key vault.
Select the tab for the type of repository and authentication you want to use.
To add a catalog, complete the following tasks:
- Assign permissions in Azure Repos for the managed identity.
- Add your repository as a catalog.
Assign permissions in Azure Repos for the managed identity
You must give the managed identity permissions to the repository in Azure Repos.
Sign in to your Azure DevOps organization.
Note
Your Azure DevOps organization must be in the same directory as the Azure subscription that contains your dev center or project.
Select Organization settings.
On the Overview page, select Users.
On the Users page, select Add users.
Complete Add new users by entering or selecting the following information, and then select Add:
| Name |
Value |
| Users or Service Principals |
Enter the name of your dev center or project.
When you use a system-assigned MSI, specify the name of the dev center or project, not the object ID of the managed account. When you use a user-assigned MSI, use the name of the managed account. |
| Access level |
Select Basic. |
| Add to projects |
Select the project that contains your repository. |
| Azure DevOps Groups |
Select Project Readers. |
| Send email invites (to Users only) |
Clear the checkbox. |
Add your repository as a catalog
Dev Box supports attaching Azure Repos repositories and GitHub repositories. You can store a set of curated IaC templates in a repository. Attaching the repository to a dev center or project as a catalog gives your development teams access to the templates and enables them to quickly create consistent dev boxes.
The following steps let you attach an Azure Repos repository.
In the Azure portal, navigate to your dev center or project.
In the left menu under Environment configuration, select Catalogs, and then select Add.
In Add catalog, enter the following information, and then select Add:
| Field |
Value |
| Name |
Enter a name for the catalog. |
| Catalog location |
Select Azure DevOps. |
| Authentication type |
Select Managed Identity. |
| Organization |
Select your Azure DevOps organization. |
| Project |
From the list of projects, select the project that stores the repo. |
| Repo |
From the list of repos, select the repo you want to add. |
| Branch |
Select the branch. |
| Folder path |
Dev Box retrieves a list of folders in your branch. Select the folder that stores your IaC templates. |
In Catalogs for the dev center or project, verify that your catalog appears. When the connection is successful, the Status reads Sync successful. Connecting to a catalog can take a few minutes the first time.
To add a catalog, complete the following tasks:
- Get the clone URL for your Azure Repos repository.
- Create a personal access token (PAT).
- Store the PAT as a key vault secret in Azure Key Vault.
- Add your repository as a catalog.
Get the clone URL for your Azure Repos repository
Go to the home page of your team collection (for example, https://contoso-web-team.visualstudio.com), and then select your project.
Get the clone URL of your Azure Repos Git repo.
Copy and save the URL.
Create a personal access token in Azure Repos
Go to the home page of your team collection (for example, https://contoso-web-team.visualstudio.com) and select your project.
Create a PAT.
Copy and save the generated token to use later.
Create a Key Vault
You need an Azure Key Vault to store the PAT used to grant Azure access to your repository. Key vaults can control access with either access policies or role-based access control (RBAC). If you have an existing key vault, you can use it, but you should check whether it uses access policies or RBAC assignments to control access. For help with configuring an access policy for a key vault, see Assign a Key Vault access policy.
Use the following steps to create an RBAC key vault:
Sign in to the Azure portal.
In the Search box, enter Key Vault.
From the results list, select Key Vault.
On the Key Vault page, select Create.
On the Create key vault tab, provide the following information:
| Name |
Value |
| Name |
Enter a name for the key vault. |
| Subscription |
Select the subscription in which you want to create the key vault. |
| Resource group |
Either use an existing resource group or select Create new and enter a name for the resource group. |
| Location |
Select the location or region where you want to create the key vault. |
Leave the other options at their defaults.
On the Access policy tab, select Azure role-based access control, and then select Review + create.
On the Review + create tab, select Create.
If your organization's policies require you to keep your Key Vault private from the internet, you can create a firewall rule to disable or limit public access and set your Key Vault to allow trusted Microsoft services to bypass your rule. Key vaults with private endpoints or private link integration are not currently supported for this scenario.
To learn how to allow trusted Microsoft services to bypass the firewall, see Configure Azure Key Vault networking settings.
Store the personal access token in the key vault
In the Key Vault, on the left menu, select Secrets.
On the Secrets page, select Generate/Import.
On the Create a secret page:
- In the Name box, enter a descriptive name for your secret.
- In the Secret value box, paste the PAT that you copied earlier.
- Select Create.
Get the secret identifier
Get the path to the secret you created in the key vault.
In the Azure portal, navigate to your key vault.
On the key vault page, from the left menu, select Secrets.
On the Secrets page, select the secret you created earlier.
On the versions page, select the CURRENT VERSION.
On the current version page, for the Secret identifier, select copy.
Add your repository as a catalog
In the Azure portal, go to your dev center or project.
Ensure that the identity attached to the dev center or project has access to the key vault secret where your personal access token is stored.
In the left menu under Environment configuration, select Catalogs, and then select Add.
In Add catalog, enter the following information, and then select Add:
| Field |
Value |
| Name |
Enter a name for the catalog. |
| Catalog location |
Select Azure DevOps. |
| Authentication type |
Select Personal Access Token. |
| Organization |
Select the organization that hosts the catalog repo. |
| Project |
Select the project that stores the catalog repo. |
| Rep |
Select the repo that stores the catalog. |
| Folder path |
Select the folder that holds your IaC templates. |
| Secret identifier |
Enter the secret identifier that contains your PAT for the repository.
When you copy a secret identifier, the connection string includes a version identifier at the end, like in this example: https://contoso-kv.vault.azure.net/secrets/GitHub-repo-pat/9376b432b72441a1b9e795695708ea5a.
Removing the version identifier ensures that Dev Box fetches the latest version of the secret from the key vault. If your PAT expires, only the key vault needs to be updated.
Example secret identifier: https://contoso-kv.vault.azure.net/secrets/GitHub-repo-pat |
In Catalogs, verify that your catalog appears. If the connection is successful, the Status is Connected.
To add a catalog, complete the following tasks:
- Install and configure the Microsoft Dev Center app
- Assign permissions in GitHub for the repos.
- Add your repository as a catalog.
Install Microsoft Dev Center app
Sign in to the Azure portal.
Navigate to your dev center or project.
In the left menu under Environment configuration, select Catalogs, and then select Add.
In the Add catalog pane, enter, or select the following:
| Field |
Value |
| Name |
Enter a name for the catalog. |
| Catalog source |
Select GitHub. |
| Authentication type |
Select GitHub app. |
To install the Microsoft Dev Center app, select configure your repositories.
If you're prompted to authenticate to GitHub, authenticate.
On the Microsoft DevCenter page, select Configure.
Select the GitHub organization that contains the repository you want to add as a catalog. You must be an owner of the organization to install this app.
On the Install Microsoft DevCenter page, select Only select repositories, select the repository you want to add as a catalog, and then select Install.
You can select multiple repositories to add as catalogs. You must add each repository as a separate catalog, as described in Add your repository as a catalog.
On the Microsoft DevCenter by Microsoft would like permission to: page, review the permissions required, and then select Authorize Microsoft Dev Center.
Add your repository as a catalog
Switch back to the Azure portal.
In Add catalog, enter the following information, and then select Add:
| Field |
Value |
| Repo |
Select the repository that you want to add as a catalog. |
| Branch |
Select the branch. |
| Folder path |
Select the folder that contains subfolders that hold your image definitions. |
In Catalogs, verify that your catalog appears. When the connection is successful, the Status reads Sync successful.
To add a catalog, complete the following tasks:
- Get the clone URL of your GitHub repository.
- Create a personal access token (PAT) in GitHub.
- Store the PAT as a key vault secret in Azure Key Vault.
- Add your repository as a catalog.
Get the clone URL of your GitHub repository
Go to the home page of the GitHub repository that contains the template definitions.
Get the GitHub repository clone URL.
Copy and save the URL.
Create a personal access token in GitHub
Dev Box supports authenticating to GitHub repositories by using either classic tokens or fine-grained tokens. In this example, you create a fine-grained token.
Go to the home page of the GitHub repository that contains the template definitions.
In the upper-right corner of GitHub, select the profile image, and then select Settings.
On the left sidebar, select Developer settings > Personal access tokens > Fine-grained tokens.
Select Generate new token.
On the New fine-grained personal access token page, provide the following information:
| Name |
Value |
| Token name |
Enter a descriptive name for the token. |
| Expiration |
Select the token expiration period in days. |
| Description |
Enter a description for the token. |
| Resource owner |
Select the owner of the repository. |
| Repository access |
Select Only select repositories. |
| Select repositories |
Select the repository that contains the image definitions. |
| Repository permissions |
Expand Repository permissions, and for Contents, from the Access list, select Code read. |
Select Generate token.
Copy and save the generated token to use later.
Important
When working with a private repository stored within a GitHub organization, you must ensure that the GitHub PAT is configured to give access to the correct organization and the repositories within it.
- Classic tokens within the organization must be SSO authorized to the specific organization after they are created.
- Fine-grained tokens must have the owner of the token set as the organization itself to be authorized.
Incorrectly configured PATs can result in a Repository not found error.
Create a Key Vault
You need an Azure Key Vault to store the PAT that is used to grant Azure access to your repository. Key vaults can control access with either access policies or role-based access control (RBAC). If you have an existing key vault, you can use it, but you should check whether it uses access policies or RBAC assignments to control access. For help with configuring an access policy for a key vault, see Assign a key vault access policy.
Use the following steps to create an RBAC key vault:
Sign in to the Azure portal.
In the Search box, enter key vault.
From the results list, select Key Vault.
On the Key Vault page, select Create.
On the Create key vault tab, provide the following information:
| Name |
Value |
| Name |
Enter a name for the key vault. |
| Subscription |
Select the subscription in which you want to create the key vault. |
| Resource group |
Either use an existing resource group or select Create new and enter a name for the resource group. |
| Location |
Select the location or region where you want to create the key vault. |
Leave the other options at their defaults.
On the Access policy tab, select Azure role-based access control, and then select Review + create.
On the Review + create tab, select Create.
If your organization's policies require you to keep your Key Vault private from the internet, you can create a firewall rule to disable or limit public access and set your Key Vault to allow trusted Microsoft services to bypass your rule. Key vaults with private endpoints or private link integration are not currently supported for this scenario.
To learn how to allow trusted Microsoft services to bypass the firewall, see Configure Azure Key Vault networking settings.
Store the personal access token in the key vault
In the Key Vault, on the left menu, select Secrets.
On the Secrets page, select Generate/Import.
On the Create a secret page:
- In the Name box, enter a descriptive name for your secret.
- In the Secret value box, paste your PAT.
- Select Create.
Get the secret identifier
Get the path to the secret you created in the key vault.
In the Azure portal, navigate to your key vault.
On the key vault page, from the left menu, select Secrets.
On the Secrets page, select the secret you created earlier.
On the versions page, select the CURRENT VERSION.
On the current version page, for the Secret identifier, select copy.
Add your repository as a catalog
In the Azure portal, go to your dev center or project.
Ensure that the managed identity attached to the dev center or project has access to the key vault secret where your personal access token is stored.
In the left menu under Environment configuration, select Catalogs, and then select Add.
In Add catalog, enter the following information, and then select Add.
| Field |
Value |
| Name |
Enter a name for the catalog. |
| Catalog location |
Select GitHub. |
| Repo |
Enter or paste the clone URL for either your GitHub repository or your Azure Repos repository.
Sample catalog example: https://github.com/Azure/deployment-environments.git |
| Branch |
Enter the repository branch to connect to.
Sample catalog example: main |
| Folder path |
Enter the folder path relative to the clone URI that contains subfolders that hold your image definitions.
The folder path is for the folder with subfolders containing environment definition environment files, not for the folder with the environment definition environment file itself. The following image shows the sample catalog folder structure.
Sample catalog example: /Environments
The folder path can begin with or without a forward slash (/). |
| Secret identifier |
Enter the secret identifier that contains your PAT for the repository.
When you copy a secret identifier, the connection string includes a version identifier at the end, like in this example: https://contoso-kv.vault.azure.net/secrets/GitHub-repo-pat/9376b432b72441a1b9e795695708ea5a.
Removing the version identifier ensures that Dev Box fetch the latest version of the secret from the key vault. If your PAT expires, only the key vault needs to be updated.
Example secret identifier: https://contoso-kv.vault.azure.net/secrets/GitHub-repo-pat |
In Catalogs, verify that your catalog appears. When the connection is successful, the Status reads Sync successful.
View synced catalog items
Regardless of the type of repository you use, you can view the catalog items that are synced from the catalog.
On the left menu for your dev center or project, under Environment configuration, select Catalogs.
In the Catalogs pane, select the catalog name.
You see a list of successfully synced catalog items.
Update a catalog
If you update the definition or template contents in the attached repository, you can provide the latest set of image definitions to your development teams by syncing the catalog. You can sync a catalog manually or automatically.
Manually sync a catalog
When you manually sync a catalog, Dev Box scans through the repository and makes the latest list of image definitions available to all of the associated projects in the dev center.
On the left menu for your dev center, under Environment configuration, select Catalogs.
Select the specific catalog, and then from the command bar, select Sync.
Automatically sync a catalog
When you configure a catalog to sync automatically, Dev Box scans through the repository every 30 minutes and makes the latest list of image definitions available to all of the associated projects in the dev center.
On the left menu for your dev center or project, under Environment configuration, select Catalogs.
Select the specific catalog, and then select edit.
In the Edit catalog pane, select Automatically sync this catalog, and then select Save.
If an autosync fails, you should perform a manual sync. Dev Box doesn't make any further autosync attempts until a manual sync succeeds.
Delete a catalog
You can delete a catalog to remove it from the Dev Box dev center or project. Templates in a deleted catalog aren't available to development teams when they deploy new dev boxes. Update the environment definition reference for any existing dev boxes that were created by using the image definitions in the deleted catalog. If the reference isn't updated and the environment is redeployed, the deployment fails.
To delete a catalog:
On the left menu for your dev center or project, under Environment configuration, select Catalogs.
Select the specific catalog, and then select Delete.
In the Delete catalog dialog, select Continue to delete the catalog.
Troubleshoot catalog sync errors
When you add or sync a catalog, you might encounter a sync error or warning. A sync error indicates that a catalog failed to sync successfully, A sync warning indicates that some or all of the catalog items have errors. You can view the sync status and errors in the Azure portal, or use the Azure CLI and REST API to troubleshoot and resolve the errors.
View catalog sync status
In the Azure portal, you can get more information about the catalog sync status and any warnings or errors by selecting the status link. The status link opens a pane that shows the sync status, the number of image definitions that were added, and the number of image definitions that were ignored or failed.
View catalog sync failures
On the left menu for your dev center or project, under Environment configuration, select Catalogs.
In the Status column, select the status link for the catalog that failed to sync.
You see a details pane that shows the changes in the last sync, the number of sync errors, and the type of errors.
View catalog sync warnings
On the left menu for your dev center or project, under Environment configuration, select Catalogs.
In the Status column, select the status link for the catalog that synced but reports a warning.
You see a details pane that shows the changes in the last sync, the number of item errors, and the type and source of each error.
You can view items that synced successfully from a catalog that also reports sync errors. From the Catalogs pane, select the catalog name.
You see a list of successfully synced catalog items.
Troubleshoot catalog sync errors by using the Azure CLI
Use the Azure CLI or the REST API to GET the catalog. The GET response shows you the type of error:
- Ignored image definitions that were detected to be duplicates.
- Invalid image definitions that failed due to schema, reference, or validation errors.
Resolve ignored environment definition errors
An ignored environment definition error occurs if you add two or more image definitions that have the same name. You can resolve this issue by renaming image definitions so that each environment definition has a unique name within the catalog.
Resolve invalid environment definition errors
An invalid environment definition error might occur for various reasons:
Manifest schema errors. Ensure that your environment definition environment file matches the required schema.
Validation errors. Check the following items to resolve validation errors:
- Ensure that the environment file's engine type is correctly configured.
- Ensure that the environment definition name is between 3 and 63 characters.
- Ensure that the environment definition name includes only characters that are valid for a URL, which are alphanumeric characters and these symbols:
~ ! , . ' ; : = - _ + ( ) * & $ @
Reference errors. Ensure that the template path that the environment file references is a valid relative path to a file in the repository.
Related content
