Edit

Share via

Create a knowledge agent in Azure AI Search

Note

This feature is currently in public preview. This preview is provided without a service-level agreement and isn't recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In Azure AI Search, a knowledge agent is a top-level resource representing a connection to a chat completion model for use in agentic retrieval workloads. A knowledge agent is used by the retrieve method in an LLM-powered information retrieval pipeline.

A knowledge agent specifies:

  • A knowledge source (one or more) that points to a searchable content
  • A chat completion model that provides reasoning capabilities for query planning and answer formulation
  • Properties for performance optimization (constrain query processing time)

After you create a knowledge agent, you can update its properties at any time. If the knowledge agent is in use, updates take effect on the next job.

Important

2025-08-01-preview introduces breaking changes for existing knowledge agents. This preview version requires one or more knowledgeSource definitions. We recommend migrating existing code to the new APIs as soon as possible.

Prerequisites

To follow the steps in this guide, we recommend Visual Studio Code with a REST client for sending preview REST API calls to Azure AI Search or the Python extension and Jupyter package.

Deploy a model for agentic retrieval

Make sure you have a supported model that Azure AI Search can access. The following instructions assume Azure AI Foundry Model as the provider.

  1. Sign in to Azure AI Foundry portal.

  2. Deploy a supported model using these instructions.

  3. Verify the search service managed identity has Cognitive Services User permissions on the Azure OpenAI resource.

    If you're testing locally, you also need Cognitive Services User permissions.

Supported models

Use Azure OpenAI or an equivalent open-source model:

  • gpt-4o
  • gpt-4o-mini
  • gpt-4.1
  • gpt-4.1-nano
  • gpt-4.1-mini
  • gpt-5
  • gpt-5-nano
  • gpt-5-mini

Configure access

Azure AI Search needs access to the chat completion model. You can use key-based or role-based authentication (recommended).

If you're using role-based authentication, on your Azure OpenAI resource, assign the Cognitive Services User role to a search service managed identity.

In Azure, you must have Owner or User Access Administrator permissions on the model provider to assign roles.

  1. Configure Azure AI Search to use a managed identity.

  2. On your model provider, such as Foundry Model, create a role assignment that gives the search service managed identity Cognitive Services User permissions. If you're testing locally, assign yourself to the same role.

  3. For local testing, follow the steps in Quickstart: Connect without keys to get a personal access token and to ensure you're logged in to a specific subscription and tenant. Paste your personal identity token into the @accessToken variable. A request that connects using your personal identity should look similar to the following example:

    @search-url=<YOUR SEARCH SERVICE URL>
@accessToken=<YOUR PERSONAL ID>

# List Indexes
GET https://{{search-url}}/indexes?api-version=2025-08-01-preview
Authorization: Bearer {{accessToken}}

Important

If you use role-based authentication, be sure to remove all references to the API key in your requests. In a request that specifies both approaches, the API key is used instead of roles.

Check for existing knowledge agents

The following request lists knowledge agents by name. Within the knowledge agents collection, all knowledge agents must be uniquely named. It's helpful to know about existing knowledge agents for reuse or for naming new agents.

# List existing knowledge agents on the search service
from azure.search.documents.indexes import SearchIndexClient

index_client = SearchIndexClient(endpoint=search_endpoint, credential=credential)

try:
    agents = {agent.name: agent for agent in index_client.list_agents(api_version=search_api_version)}
    print(f"\nKnowledge agents on search service '{search_endpoint}':")
    
    if agents:
        print(f"Found {len(agents)} knowledge agent(s):")
        for i, (name, agent) in enumerate(sorted(agents.items()), 1):
            print(f"{i}. Name: {name}")
            if agent.knowledge_sources:
                ks_names = [ks.name for ks in agent.knowledge_sources]
                print(f"   Knowledge Sources: {', '.join(ks_names)}")
            print()
    else:
        print("No knowledge agents found.")
        
except Exception as e:
    print(f"Error listing knowledge agents: {str(e)}")

You can also return a single agent by name to review its JSON definition.

# Get knowledge agent definition for earth-knowledge-agent-2
from azure.search.documents.indexes import SearchIndexClient
import json

index_client = SearchIndexClient(endpoint=search_endpoint, credential=credential)

try:
    agent_name = "earth-knowledge-agent-2"
    agent = index_client.get_agent(agent_name, api_version=search_api_version)
    
    print(f"Knowledge agent '{agent_name}':")
    print(f"Name: {agent.name}")
    
    if agent.description:
        print(f"Description: {agent.description}")
    
    if agent.models:
        print(f"\nModels ({len(agent.models)}):")
        for i, model in enumerate(agent.models, 1):
            print(f"  {i}. {type(model).__name__}")
            if hasattr(model, 'azure_open_ai_parameters'):
                params = model.azure_open_ai_parameters
                print(f"     Resource: {params.resource_url}")
                print(f"     Deployment: {params.deployment_name}")
                print(f"     Model: {params.model_name}")
    
    if agent.knowledge_sources:
        print(f"\nKnowledge Sources ({len(agent.knowledge_sources)}):")
        for i, ks in enumerate(agent.knowledge_sources, 1):
            print(f"  {i}. {ks.name} (threshold: {ks.reranker_threshold})")
    
    if agent.output_configuration:
        config = agent.output_configuration
        print(f"\nOutput: {config.modality} (activity: {config.include_activity})")
    
    # Full JSON definition
    print(f"\nJSON definition:")
    print(json.dumps(agent.as_dict(), indent=2))
    
except Exception as e:
    print(f"Error: {str(e)}")
    
    # Show available agents
    try:
        agents = {agent.name: agent for agent in index_client.list_agents(api_version=search_api_version)}
        print(f"\nAvailable agents: {list(agents.keys())}")
    except Exception:
        print("Could not list available agents.")

Create a knowledge agent

A knowledge agent drives the agentic retrieval pipeline. In application code, it's called by other agents or chat bots.

Its composition consists of connections between knowledge sources (searchable content) and chat completion models that you've deployed in Azure OpenAI. Properties on the model establish the connection. Properties on the knowledge source establish defaults that inform query execution and the response.

To create an agent, use the 2025-08-01-preview data plane REST API or an Azure SDK preview package that provides equivalent functionality.

Recall that you must have an existing knowledge source to give to the agent.

from azure.search.documents.indexes.models import KnowledgeAgent, KnowledgeAgentAzureOpenAIModel, KnowledgeSourceReference, AzureOpenAIVectorizerParameters, KnowledgeAgentOutputConfiguration, KnowledgeAgentOutputConfigurationModality
from azure.search.documents.indexes import SearchIndexClient

aoai_params = AzureOpenAIVectorizerParameters(
    resource_url=aoai_endpoint,
    deployment_name=aoai_gpt_deployment,
    model_name=aoai_gpt_model,
)

output_cfg = KnowledgeAgentOutputConfiguration(
    modality=KnowledgeAgentOutputConfigurationModality.ANSWER_SYNTHESIS,
    include_activity=True,
)

agent = KnowledgeAgent(
    name=knowledge_agent_name,
    models=[KnowledgeAgentAzureOpenAIModel(azure_open_ai_parameters=aoai_params)],
    knowledge_sources=[
        KnowledgeSourceReference(
            name=knowledge_source_name,
            reranker_threshold=2.5,
        )
    ],
    output_configuration=output_cfg,
)

index_client = SearchIndexClient(endpoint=search_endpoint, credential=credential)
index_client.create_or_update_agent(agent, api_version=search_api_version)
print(f"Knowledge agent '{knowledge_agent_name}' created or updated successfully.")

Query the knowledge agent

Call the retrieve action on the knowledge agent object to confirm the model connection and return a response. Use the 2025-08-01-preview data plane REST API or an Azure SDK preview package that provides equivalent functionality for this task. For more information about the retrieve API and the shape of the response, see Retrieve data using a knowledge agent in Azure AI Search.

Replace "where does the ocean look green?" with a query string that's valid for your search index.

Start with instructions.

instructions = """
A Q&A agent that can answer questions about the Earth at night.
If you don't have the answer, respond with "I don't know".
"""

messages = [
    {
        "role": "system",
        "content": instructions
    }
]

Then send the query.

from azure.search.documents.agent import KnowledgeAgentRetrievalClient
from azure.search.documents.agent.models import KnowledgeAgentRetrievalRequest, KnowledgeAgentMessage, KnowledgeAgentMessageTextContent, SearchIndexKnowledgeSourceParams

agent_client = KnowledgeAgentRetrievalClient(endpoint=search_endpoint, agent_name=knowledge_agent_name, credential=credential)
query_1 = """
    where does the ocean look green??
    """

messages.append({
    "role": "user",
    "content": query_1
})

req = KnowledgeAgentRetrievalRequest(
    messages=[
        KnowledgeAgentMessage(
            role=m["role"],
            content=[KnowledgeAgentMessageTextContent(text=m["content"])]
        ) for m in messages if m["role"] != "system"
    ],
    knowledge_source_params=[
        SearchIndexKnowledgeSourceParams(
            knowledge_source_name=knowledge_source_name,
        )
    ]
)

result = agent_client.retrieve(retrieval_request=req, api_version=search_api_version)
print(f"Retrieved content from '{knowledge_source_name}' successfully.")

Delete an agent

If you no longer need the agent, or if you need to rebuild it on the search service, use this request to delete the current object.

from azure.search.documents.indexes import SearchIndexClient

index_client = SearchIndexClient(endpoint=search_endpoint, credential=credential)
index_client.delete_agent(knowledge_agent_name)
print(f"Knowledge agent '{knowledge_agent_name}' deleted successfully.")

Related content