Azure Web Application and ChatPlayground giving different responses?

Question

Azure Web Application and ChatPlayground giving different responses?

Tanuj Verma 0

Hi,

I am working on Azure AI Search. I want to use Web Application/Chat Playground for Q/A from bunch of documents at a time. I have created azure ai search service with Basic tier package for West US Region. After this, I created skillset using Text Split Skill. Then, I have created index called azureblob-index where fields are:

1.) content (Retrievable, Searchable)

2.) metadata_storage_name (Retrievable)

3.) metadata_storage_path (Retrievable, Filterable)

4.) metadata_author (Retrievable, Filterable, Facetable, Searchable)

5.) metadata_title (Retrievable, Filterable, Facetable, Searchable)

6.) metadata_creation_date (Retrievable, Filterable, Sortable, Facetable)

7.) language (Retrievable, Filterable, Facetable, Searchable)

8.) split_text (Retrievable, Searchable)

9.) keywords (Retrievable, Searchable)

10.) summary (Retrievable, Searchable)

11.) section_titles (Retrievable, Searchable)

12.) metadata_file_type (Retrievable, Filterable, Facetable, Searchable)

13.) merged_content (Filterable, Searchable)

14.) text (Retrievable, Searchable)

15.) layoutText (Retrievable, Searchable)

After this, I have created semantic configurations where fields are:

Title_field: metadata_title

Content fields: content, merged_content

Keyword fields: keywords

After this, I have created chat playground using bring your own data. When I am asking questions, then chat playground is giving response which is not very accurate or sometimes not even matching to the question that is asked. Also, sometimes it does not give any answer by stating that this question is out of my domain/scope. But again if I ask same question after like 10-15 seconds, then I am getting answer for that question for which earlier I got out of domain/scope message. Sometimes, it gives correct answer. Same happens in the web application. Also, sometimes references in the chat playground and web application doesn't even match while question being same.

Can anyone suggest why this is happening.

1 answer

Your answer

Answer 1

Hello Tanuj !

Thank you for posting on Microsoft Learn Q&A.

In your web app code, explicitly set the same values the Playground uses and don’t rely on SDK defaults :

if you use semantic ranker, be sure it’s enabled for your service and used on each query since basic tier supports semantic ranker; it must be enabled in the service. https://free.blessedness.top/en-us/azure/search/semantic-how-to-configure
top N documents returned from search pre rerank and top K after rerank.
high strictness because more “out of scope/I don’t know” lower it to allow answers when relevance is borderline https://free.blessedness.top/en-us/azure/ai-foundry/openai/how-to/on-your-data-best-practices
query type: semantic vs simple, and query language for example en-us
captions or answers flags when using semantic ranker
filters you may have set in one place but not the other
temperature for the chat model so set 0–0.2 for deterministic answers

If you call the Azure OpenAI on your data API directly, you need to include these explicitly :

openai.responses.create({
  model: "gpt-4o",
  input: "user question",
  extra_body: {
    data_sources: [{
      type: "azure_search",
      parameters: {
        indexName: "azureblob-index",
        semanticConfiguration: "your-semantic-config",
        topNDocuments: 50,
        strictness: 2,
        roleInformation: "system prompt here",
        queryType: "semantic",
        filter: "metadata_file_type eq 'pdf'",
        // if using vectors: vectorSearchOptions, fields, etc.
      }
    }]
  },
  temperature: 0
});

Playground uses similar fields so make sure that the same numbers for topNDocuments, strictness, filters...

https://stackoverflow.com/questions/77247205/azure-openai-bring-your-own-data-fails-when-run-as-an-api-call

Right now you’ve got many text fields (content, merged_content, split_text, text, layoutText, section_titles, summary) and that often leads to uneven retrieval because only some fields are used for search and some aren't retrievable.

So pick one primary chunk text field and make it searchable and retrievable, and put it in semantic content fields and avoid having the ranker look at a different field than the one you return to the LLM. Semantic ranker works best when the informational text lives in the fields you declare in the semantic configuration.

https://free.blessedness.top/en-us/azure/search/search-get-started-semantic

Your title field (metadata_title) should be populated because missing titles degrade semantic reranking and if many docs lack titles, either populate them from filename or change the title field to a reliable fallback.

https://free.blessedness.top/en-us/azure/search/semantic-how-to-configure

and if you’re chunking via text split skill or integrated vectorization, keep chunk text only in that one field instead of splitting across split_text and content.

https://free.blessedness.top/en-us/azure/search/search-how-to-integrated-vectorization

Basic tier is fine, but if you have older basic services or too few replicas/partitions, you can see occasional timeouts or partial result sets under load. Scale to more 2 replicas for query SLAs and consider adding a partition if storage is tight.

https://free.blessedness.top/en-us/azure/search/search-limits-quotas-capacity

Share via

Azure Web Application and ChatPlayground giving different responses?

1 answer

Your answer