Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Applies to: 
SQL Server 2025 (17.x) Preview
Azure SQL Database
Azure SQL Managed Instance
SQL database in Microsoft Fabric Preview
This article contains frequently asked questions about vectors and embeddings in the SQL Database Engine.
For samples and examples, visit the SQL AI Samples repository.
Can I create a retrieval-augmented generation (RAG) solution completely in T-SQL?
Yes, you can create a Retrieval-Augmented Generation (RAG) solution using T-SQL. This type of solution leverages the SQL Database Engine's capabilities to manage and query your data effectively. You can use T-SQL to implement the necessary data retrieval and processing logic, while also integrating with external AI services for the generation aspect. Vectors can be stored natively in SQL engine and connections to LLMs that provide natural language understanding capabilities are possible via sp_invoke_external_rest_endpoint.
- Implement a RAG solution and call OpenAI right from Azure SQL DB to ask questions about your data
- Predictable LLM results with Structured Output and sp_invoke_external_rest_endpoint
Why would I create a RAG solution completely in T-SQL?
If you want to improve an existing application without having to re-architect it to support AI capabilities, use the SQL engine built-in features to implement AI functionalities directly within your database queries. You only need to update your T-SQL code to incorporate AI features, rather than making extensive changes to your application architecture.
- Migrate and modernize Windows Server, SQL Server, and .NET workloads
- Modernize applications with Azure SQL, OpenAI, and Data API builder
Are there any end-to-end samples using Azure SQL or Fabric SQL for RAG?
Sure, you can find end-to-end samples for RAG using Azure SQL and Fabric SQL here:
Can I have RAG working on structured data, like columns and rows?
If you need to work with structured data, you can still leverage RAG by combining it with other techniques, such as using embeddings to represent your structured data in a way that can be understood by the AI model. This allows you to perform retrieval and generation tasks on structured data while still benefiting from the capabilities of RAG.
Why does sending a full, complex schema to an LLM lead to poor SQL generation—and how can I fix it?
If you have a complex and large database schema, with hundreds of tables and views, it's better to use a multi-agent approach to help to reduce the noise and allow AI models to focus on specific areas of the schema. A full description along with a working end-to-end sample is available here:
Can I connect to Azure OpenAI using Managed Identity?
Yes, you can connect to Azure OpenAI using Managed Identity. This allows you to securely authenticate and access the Azure OpenAI Service without needing to manage credentials directly. For more information, see:
- Go passwordless when calling Azure OpenAI from Azure SQL using Managed Identities
- Create an EXTERNAL MODEL with Azure OpenAI using Managed Identity
Is my data used by Microsoft for training models?
No. Data isn't used by Microsoft for training models. See the Responsible AI documentation for more information.
What data does the Azure OpenAI Service process?
Refer to the Data, privacy, and security for Azure OpenAI Service document for more information.
How can I protect my data from unauthorized AI Agent access?
Azure SQL and SQL Server provide extensive support for fine-grained access security:
- Get started with Database Engine permissions: Control access to database objects at a granular level using permissions.
- Row-Level Security (RLS): Control access to rows in a table based on the characteristics of the user executing a query. You can see RLS in action in this video.
- Dynamic data masking: Limit the exposure of sensitive data by masking it to non-privileged users.
- Always Encrypted: Protect sensitive data by encrypting it at rest and in transit, ensuring that only authorized users can access the unencrypted data.
It's also possible to audit any operation done on the database using the Audit feature in Azure SQL and SQL Server.
SQL Server Audit (Database Engine)