How to resolve Azure Databricks private endpoint URLs from on-prem over ExpressRoute

Question

How to resolve Azure Databricks private endpoint URLs from on-prem over ExpressRoute

krutibasa majhi 25

Hello Friends,

We have set up Azure Databricks with two private endpoints — Databricks UI-API, and Browser Authentication. From an Azure VM within the virtual network, name resolution correctly resolves to the private IPs. However, when we try to access Databricks from our on-premises network (connected via ExpressRoute), the workspace URLs still resolve to public IPs.

To fix this, I configured a conditional forwarder on the on-prem DNS server, pointing the Databricks private DNS zone to the Azure DNS IP (168.63.129.16), but nslookup from on-prem returned a timeout. I also tested by manually creating a DNS zone in the on-prem DNS and mapping the Databricks workspace URLs to their private endpoint IPs. In that case, name resolution worked, but the Databricks Account Console (https://accounts.azuredatabricks.net/) became inaccessible. Once I removed those DNS records, the console started working again.

Question: How can I configure DNS so that on-premises users can access Azure Databricks via private endpoints over ExpressRoute, while still maintaining access to the Databricks Account Console?

Thank you

Krutibasa

Smaran Thoomu 31,910 Reputation points Microsoft External Staff Moderator

2025-10-16T12:00:59.0766667+00:00
Hi krutibasa majhi
Thank you for reaching out and providing the detailed background - that really helps. I understand you’ve configured Azure Databricks with private endpoints (UI/API and Browser Auth) and are trying to ensure on-premises users can resolve those workspace URLs to private IPs over ExpressRoute, without breaking access to the public Databricks Account Console.

Here’s some guidance to help you resolve this:

Confirm Private DNS Zone Link

Make sure your private DNS zone for Databricks (e.g., privatelink.azuredatabricks.net) is linked to the VNet that’s connected to ExpressRoute via your Azure Private DNS setup.

The VNet link ensures name resolution works for all connected networks, including on-prem, if DNS forwarding is correctly configured.

DNS Forwarding Setup

On your on-prem DNS server, configure a conditional forwarder for the Databricks private zone (privatelink.azuredatabricks.net) to your Azure DNS forwarder or custom DNS server inside Azure (not directly to 168.63.129.16 — that address doesn’t respond to external queries).

Typically, you deploy an Azure DNS Forwarder VM (using DNS Proxy like Azure Firewall DNS proxy or Windows DNS) in the connected VNet, and forward requests there.

Keep the Account Console Public

The Account Console (accounts.azuredatabricks.net) remains a public endpoint. Do not override or create DNS records for it in your private DNS or on-prem DNS; it must continue to resolve via public DNS.

Only the workspace URLs (e.g., <workspace>.privatelink.azuredatabricks.net) should resolve privately.

Recommended Topology

Azure VM/VNet → Private DNS zone linked to VNet

On-prem DNS → Conditional forwarder → Azure DNS forwarder (resolves private zones)

Public names (like Account Console) → Resolve through normal public DNS

For reference: Configure DNS for private endpoint

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
krutibasa majhi 25 Reputation points

2025-10-17T10:05:45.4+00:00

@Smaran Thoomu Thanks for the advice. Currently, we are not using any DNS server in Azure — it’s hosted on-premises. We have created entries in the forward lookup zone for the frontend URL, backend private link URL, and the private link browser authentication URL. DNS resolution is working, and we’re able to access the URLs using private IPs. However, I’m not sure if this is the right approach to achieve the connectivity.

However, I have a question: if the ExpressRoute link goes down, since the URLs are mapped to private IPs, will the DNS query automatically fall back to public DNS, or will the packets be dropped?

We’re trying this workaround as Azure DNS Resolver is quite costly. Please suggest if there’s a better alternative solution.

Regards,

Krutibasa
Smaran Thoomu 31,910 Reputation points Microsoft External Staff Moderator

2025-10-17T12:10:11.02+00:00
krutibasa majhi Thank you for sharing the additional details - it’s great to hear that your current DNS setup is successfully resolving the Databricks private endpoint URLs to private IPs.

Regarding your question about fallback behavior if the ExpressRoute link goes down:

When the workspace URLs are statically mapped to private IPs (via on-prem DNS records), the DNS resolution will not automatically fall back to public DNS.

Since those private IPs are only routable through ExpressRoute or VPN, if the link goes down, the packets will fail to reach Azure (i.e., they’ll be dropped) - there’s no automatic DNS failover mechanism from private to public resolution.

To handle this scenario:

Option 1 – Use Conditional Forwarding with Split Resolution: Instead of static A records, use a conditional forwarder for privatelink.azuredatabricks.net that forwards to Azure (via an Azure DNS forwarder VM or lightweight DNS proxy).

When ExpressRoute is up, resolution happens through the Azure path to private IPs.

If ExpressRoute is down, the forwarder becomes unreachable, and DNS queries fall back to your public resolver - allowing access through the public endpoint (if your firewall rules permit).

This avoids manual DNS record changes.

Option 2 – Use a Cost-Effective DNS Forwarder in Azure: Instead of Azure DNS Resolver (which can be costly for smaller environments), you can deploy a small Windows or Linux VM running DNS forwarding or use Azure Firewall DNS proxy - both are viable, low-cost alternatives.

This VM can forward only Databricks-related zones to Azure’s internal resolver, minimizing management overhead.

Your current manual DNS entry approach works technically, but it’s static and doesn’t support dynamic failover. Using conditional forwarding ensures more flexibility and resiliency.

Hope this helps. Do let us know if you any further queries.

1 answer

Your answer

krutibasa majhi 25 Reputation points

2025-10-17T10:05:45.4+00:00

@Smaran Thoomu Thanks for the advice. Currently, we are not using any DNS server in Azure — it’s hosted on-premises. We have created entries in the forward lookup zone for the frontend URL, backend private link URL, and the private link browser authentication URL. DNS resolution is working, and we’re able to access the URLs using private IPs. However, I’m not sure if this is the right approach to achieve the connectivity.

However, I have a question: if the ExpressRoute link goes down, since the URLs are mapped to private IPs, will the DNS query automatically fall back to public DNS, or will the packets be dropped?

We’re trying this workaround as Azure DNS Resolver is quite costly. Please suggest if there’s a better alternative solution.

Regards,

Krutibasa
Smaran Thoomu 31,910 Reputation points Microsoft External Staff Moderator

2025-10-17T12:10:11.02+00:00

krutibasa majhi Thank you for sharing the additional details - it’s great to hear that your current DNS setup is successfully resolving the Databricks private endpoint URLs to private IPs.

Regarding your question about fallback behavior if the ExpressRoute link goes down:

When the workspace URLs are statically mapped to private IPs (via on-prem DNS records), the DNS resolution will not automatically fall back to public DNS.

Since those private IPs are only routable through ExpressRoute or VPN, if the link goes down, the packets will fail to reach Azure (i.e., they’ll be dropped) - there’s no automatic DNS failover mechanism from private to public resolution.

To handle this scenario:

Option 1 – Use Conditional Forwarding with Split Resolution: Instead of static A records, use a conditional forwarder for privatelink.azuredatabricks.net that forwards to Azure (via an Azure DNS forwarder VM or lightweight DNS proxy).

When ExpressRoute is up, resolution happens through the Azure path to private IPs.

If ExpressRoute is down, the forwarder becomes unreachable, and DNS queries fall back to your public resolver - allowing access through the public endpoint (if your firewall rules permit).

This avoids manual DNS record changes.

Option 2 – Use a Cost-Effective DNS Forwarder in Azure: Instead of Azure DNS Resolver (which can be costly for smaller environments), you can deploy a small Windows or Linux VM running DNS forwarding or use Azure Firewall DNS proxy - both are viable, low-cost alternatives.

This VM can forward only Databricks-related zones to Azure’s internal resolver, minimizing management overhead.

Your current manual DNS entry approach works technically, but it’s static and doesn’t support dynamic failover. Using conditional forwarding ensures more flexibility and resiliency.

Hope this helps. Do let us know if you any further queries.

Answer 1

@Smaran Thoomu Thanks for your response .Let me explain the architecture: On-premises network is connected to the Azure Transit VNet via ExpressRoute. The Transit VNet hosts a Private DNS Resolver, is peered with the Hub VNet, and the Hub VNet is peered with all Spoke VNets, where Azure Databricks is deployed. All VNets are in different subscriptions. Can the same DNS Resolver be used for this connectivity? If yes, please guide me on how to configure it. All the vnets are in same region but different subscription.

Share via

How to resolve Azure Databricks private endpoint URLs from on-prem over ExpressRoute

1 answer

Your answer