Create a blob knowledge source

2025-10-14

Note

This feature is currently in public preview. This preview is provided without a service-level agreement and isn't recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Use a blob knowledge source to index and query Azure blob content in an agentic retrieval pipeline. Knowledge sources are created independently, referenced in a knowledge agent, and used as grounding data when an agent or chatbot calls a retrieve action at query time.

Unlike a search index knowledge source, which specifies an existing and qualified index, a blob knowledge source specifies an external data source, models, and properties to automatically generate the following Azure AI Search objects:

A data source that represents a blob container.
A skillset that chunks and optionally vectorizes multimodal content from the container.
An index that stores enriched content and meets the criteria for agentic retrieval.
An indexer that uses the previous objects to drive the indexing and enrichment pipeline.

Knowledge sources are new in the 2025-08-01-preview release.

Prerequisites

Azure Storage with a blob container containing supported content types for text content. For optional image verbalization, the supported content type depends on whether your chat completion model can analyze and describe the image file.
Azure AI Search on the Basic tier or higher with semantic ranker enabled.

To try the examples in this article, we recommend Visual Studio Code with the REST Client extension for sending preview REST API calls to Azure AI Search. Currently, there's no portal support.

Check for existing knowledge sources

A knowledge source is a top-level, reusable object. All knowledge sources must be uniquely named within the knowledge sources collection. Knowing about existing knowledge sources is helpful for either reuse or naming new objects.

Use Knowledge Sources - Get (REST API) to list knowledge sources by name and type.

### List knowledge sources by name and type
GET {{search-url}}/knowledgeSources?api-version=2025-08-01-preview&$select=name,kind
api-key: {{api-key}}
Content-Type: application/json

You can also return a single knowledge source by name to review its JSON definition.

### Get a knowledge source definition
GET {{search-url}}/knowledgeSources/{{knowledge-source-name}}?api-version=2025-08-01-preview
api-key: {{api-key}}
Content-Type: application/json

The following JSON is an example response for an azureBlob knowledge source.

{
  "name": "earth-at-night-blob-ks",
  "kind": "azureBlob",
  "description": "This knowledge source pull from a blob storage container containing pages from the Earth at Night PDF.",
  "encryptionKey": null,
  "searchIndexParameters": null,
  "azureBlobParameters": {
    "connectionString": "<REDACTED>",
    "folderPath": null,
    "disableImageVerbalization": null,
    "identity": null,
    "embeddingModel": {
      "name": "demo-blob-embedding-vectorizer",
      "kind": "azureOpenAI",
      "azureOpenAIParameters": {
        "resourceUri": "<REDACTED>",
        "deploymentId": "text-embedding-ada-002",
        "apiKey": "<REDACTED>",
        "modelName": "text-embedding-ada-002",
        "authIdentity": null
      },
      "customWebApiParameters": null,
      "aiServicesVisionParameters": null,
      "amlParameters": null
    },
    "chatCompletionModel": {
      "kind": "azureOpenAI",
      "azureOpenAIParameters": {
        "resourceUri": "<REDACTED>",
        "deploymentId": "gpt-5-mini",
        "apiKey": "<REDACTED>",
        "modelName": "gpt-5-mini",
        "authIdentity": null
      }
    },
    "ingestionSchedule": null,
    "createdResources": {
      "datasource": "earth-at-night-blob-ks-datasource",
      "indexer": "earth-at-night-blob-ks-indexer",
      "skillset": "earth-at-night-blob-ks-skillset",
      "index": "earth-at-night-blob-ks-index"
    }
  },
  "webParameters": null
}

Note

Sensitive information is redacted. The generated resources appear at the end of the response. The webParameters property isn't operational in this preview and is reserved for future use.

Create a knowledge source

To create an azureBlob knowledge source:

Set environment variables at the top of your file.

@search-url = <YOUR SEARCH SERVICE URL>
@api-key = <YOUR SEARCH ADMIN API KEY>
@ks-name = <YOUR KNOWLEDGE SOURCE NAME>
@connection-string = <YOUR FULL ACCESS CONNECTION STRING TO AZURE STORAGE>
@container-name = <YOUR BLOB CONTAINER NAME>

Use the 2025-08-01-preview of Knowledge Sources - Create or Update (REST API) or an Azure SDK preview package that provides equivalent functionality to formulate the request.

PUT {{search-url}}/knowledgeSources/earth-at-night-blob-ks?api-version=2025-08-01-preview
api-key: {{api-key}}
Content-Type: application/json

{
  "name": "{{ks-name}}",
  "kind": "azureBlob",
  "description": "This knowledge source pulls from a blob storage container containing pages from the Earth at Night PDF.",
  "encryptionKey": null,
  "azureBlobParameters": {
    "connectionString": "{{connection-string}}",
    "containerName": "{{container-name}}",
    "folderPath": null,
    "disableImageVerbalization": null,
    "identity": null,
    "embeddingModel": {
      // Redacted for brevity
    },
    "chatCompletionModel": {
      // Redacted for brevity
    },
    "ingestionSchedule": {
      // Redacted for brevity
    }
  }
}

Select Send Request.

Key points:

name must be unique within the knowledge sources collection and follow the naming guidelines for objects in Azure AI Search.
kind must be azureBlob for a blob knowledge source.
encryptionKey (optional) is an encryption key in Azure Key Vault. Use this property to doubly encrypt sensitive information in both the knowledge source and the generated objects.
embeddingModel (optional) is a text embedding model that vectorizes text and image content during indexing and at query time. Use a model supported by the Azure OpenAI Embedding skill, Azure AI Vision multimodal embeddings skill, AML skill, or Custom Web API skill. The embedding skill will be included in the generated skillset, and its equivalent vectorizer will be included in the generated index.
chatCompletionModel (optional) is a chat completion model that verbalizes images or extracts content. Use a model supported by the GenAI Prompt skill, which will be included in the generated skillset. To skip image verbalization, omit this object and set "disableImageVerbalization": true.
ingestionSchedule (optional) adds scheduling information to the generated indexer. You can also add a schedule later to automate data refresh.
If you get errors, make sure the embedding and chat completion models exist at the endpoints you provided.

Review the created objects

When you create a blob knowledge source, your search service also creates an indexer, data source, skillset, and index. Exercise caution when you edit these objects, as introducing an error or incompatibility can break the pipeline.

After you create a knowledge source, the response lists the created objects. These objects are created according to a fixed template, and their names are based on the name of the knowledge source. You can't change the object names.

We recommend using the Azure portal to validate output creation. The workflow is:

Check the indexer for success or failure messages. Connection or quota errors appear here.
Check the index for searchable content. Use Search Explorer to run queries.
Check the skillset to learn how your content is chunked and optionally vectorized.
Modify the data source if you want to change connection details, such as authentication and authorization. Our example uses API keys for simplicity, but you can use Microsoft Entra ID authentication and role-based access.

Assign to a knowledge agent

If you're satisfied with the index, continue to the next step: specify the knowledge source in a knowledge agent.

Within the knowledge agent, there are more properties to set on the knowledge source that are specific to query operations.

After the knowledge agent is configured, use the retrieve action to query the knowledge source.

Delete a knowledge source

If you no longer need the knowledge source, or if you need to rebuild it on the search service, use this request to delete the current object.

Before you can delete a knowledge source, you must delete any knowledge agent that references it, or remote the references in an update action. The associated index and any indexer pipeline objects created from the knowledge source are standalone objects and don't need to be deleted or updated in tandem with the knowledge source.

If you try to delete a knowledge source that's in use, the action fails and a list of affected knowledge agents is returned.

Start by getting a list of all knowledge agents. This request returns all knowledge agents on your search service.

### Get the knowledge agent
GET {{search-endpoint}}/agents?api-version=2025-08-01-preview&$select=name
api-key: {{api-key}}
Content-Type: application/json

An example response might look like the following:

 {
     "@odata.context": "https://my-demo-search-service.search.windows.net/$metadata#agents(name)",
     "value": [
     {
         "name": "earth-blob-ka"
     },
     {
         "name": "hotels-sample-ka"
     }
     ]
 }

Get the individual knowledge agent definition to check for knowledge source references.

GET {{search-endpoint}}/agents/hotels-sample-ka?api-version=2025-08-01-preview
api-key: {{api-key}}
Content-Type: application/json

An example response might look like the following:

 {
   "name": "hotels-sample-ka",
   "description": null,
   "retrievalInstructions": null,
   "knowledgeSources": [
     {
       "name": "hotels-sample-ks",
       "alwaysQuerySource": false,
       "includeReferences": true,
       "includeReferenceSourceData": false,
       "maxSubQueries": null,
       "rerankerThreshold": null
     }
   ],
   "models": [ trimmed for brevity ],
   "outputConfiguration": { trimmed for brevity },
   "requestLimits": { trimmed for brevity},
   "encryptionKey": null
 }

Either update the knowledge agent by removing the knowledge source if you have multiple sources, or delete the knowledge agent. This example shows deletion.

### Delete knowledge agent
DELETE {{search-endpoint}}/agents/hotels-sample-ka?api-version=2025-08-01-preview
api-key: {{api-key}}
Content-Type: application/json

Delete the knowledge source.

### Delete a knowledge source definition
GET {{search-endpoint}}/knowledgeSources/hotels-sample-ks?api-version=2025-08-01-preview
api-key: {{api-key}}
Content-Type: application/json

Feedback

Was this page helpful?