Share via

Model deployment succeeded, but endpoint responses are inconsistent across regions

zhuzin zhuzin 40 Reputation points
2025-10-25T20:27:54.9333333+00:00

I managed to resolve the earlier “ModelNotFound” issue when deploying my custom text classification model in Azure AI Foundry — turned out it was a region mismatch between the training and endpoint resources.

Now I’m facing a new problem: while the model deploys successfully and returns correct predictions when tested directly in Foundry Studio, API calls from my application sometimes fail with inconsistent latency and occasional 503 Service Unavailable errors. This only happens when the endpoint is accessed from regions different from where the model was trained (West Europe in my case).

Is there a recommended setup or best practice for handling regional consistency and scaling for custom model endpoints in Azure AI Foundry? Should I consider replicating the model to multiple regions, or is there a configuration option to auto-route traffic for better reliability?

Azure AI Content Safety
Azure AI Content Safety
An Azure service that enables users to identify content that is potentially offensive, risky, or otherwise undesirable. Previously known as Azure Content Moderator.
0 comments
Answer accepted by question author
  1. Azar 31,055 Reputation points MVP Volunteer Moderator
    2025-10-25T20:47:26.4966667+00:00

    Hi there zhuzin zhuzin

    It sounds like your issue is related to regional availability and load balancing of the custom model endpoint. In Azure AI Foundry, deployed models are bound to the region where the deployment happens, and cross-region API calls can sometimes experience latency spikes or intermittent 503 errors. The recommended approach is to either replicate your model to multiple regions where your users or applications will access it or use Azure Front Door / Traffic Manager to route requests to the nearest healthy region. Also, check the endpoint scaling settings in Foundry — increasing the number of replicas can improve reliability for high-latency or burst traffic scenarios.

    1 person found this answer helpful.
    0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.