Share via

Export dataverse data to in delta lake format not working

Alexander O'Reilly 0 Reputation points
2025-10-24T05:35:11.4133333+00:00

We have setup Azure Synapse Link for Dataverse and followed the documentation here: 

  1. https://free.blessedness.top/en-us/power-apps/maker/data-platform/azure-synapse-link-synapse
  2. Export Microsoft Dataverse data in Delta Lake format - Power Apps | Microsoft Learn

According to this we should see a near real-time table and a snapshot table in our Azure Synapse Analytics environment but the near real-time table is missing.

Documentation:

User's image

Our environment: 

User's image

Similarly the delta files have never been created in the appropriate /deltalake folder of our ADLS Gen2 storage location:

User's image

 The spark pool appears to be writing the csv's correctly to the target folders in our storage account ADLS Gen2 and the spark job states that it was successful in synapse workspace. 

User's image

However when I dig into the logs of the spark executions there's an error in the DeltaConversion job stating "invalid authority" when trying to access the default workspace storage location. It's referring to an abfss path that looks incorrect to me as it's in the format of:

  • abfss://[mystorageaccount]/default/default@[mystorageaccount].dfs.core.windows.net/synapse/workspaces/[mysynapseworkspace]/warehouse

 I am assuming it should be in the format of: 

  • abfss://default@[mystorageaccount].dfs.core.windows.net/synapse/workspaces/[mysynapseworkspace]/warehouse

User's image

There is also an error reading / writing to the metastore. 

User's image

The above issues could be a symptom of something else - any assistance with what is going on would be appreciated.

FYI we are using managed VNet and managed private endpoints. I understand this article https://free.blessedness.top/en-us/power-apps/maker/data-platform/azure-synapse-link-synapse states that, "Synapse workspaces featuring managed private endpoints, data exfiltration protection, or managed virtual networks aren't supported." however the architecture we have was advised to us by Microsoft resources in Australia we no longer have access to.

Spark version 3.4

Delta Lake 2.4

 

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
Answer accepted by question author
  1. Vinodh247 39,201 Reputation points MVP Volunteer Moderator
    2025-10-26T11:18:45.39+00:00
    1. The delta conversion is failing because the ABFSS URI is malformed, the container/account parts are swapped and an extra path segment appears: abfss://[mystorageaccount]/default/default@[mystorageaccount].dfs.core.windows.net/... instead of the correct form: abfss://<container>@<storageaccount>.dfs.core.windows.net/<path>
    2. That malformed URI produces an "invalid authority" error and prevents delta files / near real-time table from being created.
    3. Two common reasons for this in Synapse + Synapse Link:
      1. Synapse workspace default storage/container is configured incorrectly (container name and account were mixed or set to the storage account name instead of the container).
      2. Your network configuration (managedVNet / private endpoints / data exfiltration protection) is blocking the service or causing Synapse Link to attempt an incorrect internal path. Note: Synapse Link for Dataverse does not support managed private endpoints / managed VNet per the product doc.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.