Set up data source connection for data quality in Unified Catalog

2025-10-17

Data source connections set up the authentication needed to profile your data for statistical snapshot, or scan your data for data quality anomalies and scoring.

Setting up data source connections is the fourth step in the data quality life cycle for a data asset. Previous steps are:

Assign users data quality steward permissions in Unified Catalog to use all data quality features.
Register and scan a data source in your Microsoft Purview Data Map.
Add data assets to a data product.

Prerequisites

To create connections to data assets, your users must be in the data quality steward role.
You need at least read access to the data source for which you're setting up the connection.

Supported multicloud data sources

Browse the supported data source document to view the list of supported data sources, including file formats for data profiling and data quality scanning, with and without virtual network support.

Currently, data quality scans can only run by using Managed Identity as an authentication option. Data quality services run on Apache Spark 3.4 and Delta Lake 2.4.

Important

To access these sources, either you need to set your Microsoft Azure Storage sources to have an open firewall, to Allow Trusted Azure Services, or to use private endpoints follow the guideline documented in the data quality managed virtual network configuration guide.

Set up data source connection

Follow these steps to create a new connection for the data products and data assets in a governance domain.

In Unified Catalog, select Health management, then select Data quality.
Select a governance domain from the list.
From the Manage dropdown list, select Connections.
On the Connections page, select New.
On the Create connection flyout pane, enter a Display name and an optional Description.
Select a Source type.
Select one of the data sources: Azure subscription, Data Map, or enter a data source manually. Depending on which data source you choose, enter the required access details. The connection is then tested.
If the test connection is successful, select Submit to complete the connection setup.

Tip

You can also create a connection to your resources by using private endpoints and a Microsoft Purview Data Quality managed virtual network. Learn more about setting up managed virtual networks for data quality.
Connection setup steps vary for native connectors. Check the connection setup steps from native connectors articles to setup connection for Azure Databricks, Snowflake, Google BigQuery, and Azure Synapse connectors.
To set up Azure Dedicated SQL Pool (formerly SQL DW) connection, users need to select source type as Azure SQL database and add sqldatawarehouse.database.windows.net as endpoint name.
The virtual network region is auto populated from the selected source region. Find details on managing virtual network provisioning.

Grant Microsoft Purview permissions on the source

After you create the connection, you need to grant Microsoft Purview managed identity permissions on your data sources to scan them:

To scan Azure Data Lake Storage Gen2, assign the storage blob data reader role to Microsoft Purview Managed Identity. Follow the steps to assign managed identity permissions.
To scan an Azure SQL database, assign the db_datareader role to the Microsoft Purview Managed Identity. Follow the steps to assign managed identity permissions.

Next steps

Configure and run data profiling for an asset in your data source.
Set up data quality rules based on the profiling results, and apply them to your data asset.
Configure and run a data quality scan on a data product to assess the quality of all supported assets in the data product.
Review your scan results to evaluate your data product's current data quality.

Feedback

Was this page helpful?