Cannot Create DIRECT_ACCESS on my Databricks Workspace

Eric Tan - Thanks for the question and using MS Q&A platform.

Based on the document: vector_search function this is a known limitations apply during the preview: Querying DIRECT_ACCESS index types are not supported.

𝘛𝘰 𝘴𝘵𝘢𝘺 𝘪𝘯𝘧𝘰𝘳𝘮𝘦𝘥 𝘢𝘣𝘰𝘶𝘵 𝘵𝘩𝘦 𝘭𝘢𝘵𝘦𝘴𝘵 𝘶𝘱𝘥𝘢𝘵𝘦𝘴 𝘢𝘯𝘥 𝘪𝘯𝘴𝘪𝘨𝘩𝘵𝘴 𝘰𝘯 𝘈𝘻𝘶𝘳𝘦 𝘋𝘢𝘵𝘢𝘣𝘳𝘪𝘤𝘬𝘴, 𝘥𝘢𝘵𝘢 𝘦𝘯𝘨𝘪𝘯𝘦𝘦𝘳𝘪𝘯𝘨, 𝘢𝘯𝘥 𝘤𝘭𝘰𝘶𝘥 𝘪𝘯𝘯𝘰𝘷𝘢𝘵𝘪𝘰𝘯𝘴, 𝘧𝘰𝘭𝘭𝘰𝘸 𝘮𝘦 𝘰𝘯 𝘓𝘪𝘯𝘬𝘦𝘥𝘐𝘯.

I am not querying the index, I am using VectorSearchClient.create_direct_access_index to create an index, this method has worked for me in other workspace before, just don't know why I can't the same in this new workspace.

Hi Eric Tan,

The method VectorSearchClient.create_direct_access_index internally calls the Databricks Vector Search API to create a Direct Access index. This feature is workspace-tier dependent. Your previous workspace likely had Enterprise SKU or Unity Catalog with advanced entitlements enabled. Compare the Two Workspaces: • Compare the configuration of the two workspaces where the method worked and the new one. Look at: o The Databricks runtime version. o Cluster configurations (size, type, etc.). o Available features (check if the DIRECT_ACCESS index feature is listed for both workspaces). Check API and Permissions: • Verify that your API credentials and permissions in the new workspace allow the use of VectorSearchClient.create_direct_access_index. Ensure the user roles and policies are the same as the ones that worked in the other workspace. Check for Feature Flag: • If possible, check whether the DIRECT_ACCESS feature is behind a feature flag and ensure it's enabled in the new workspace. API Version Consistency: • Check that the same API version is used in both workspaces. If one workspace was created using a newer API version, it might have additional features or limitations.

Use Delta Sync Index Instead Replace create_direct_access_index with create_delta_sync_index: Python from databricks.vector_search.client import VectorSearchClient

client = VectorSearchClient() client.create_delta_sync_index( endpoint_name="my-endpoint", index_name="my-index", primary_key="id", source_table_name="catalog.schema.table" )

This approach works on Premium workspaces and is the recommended alternative. Reference: Create and Query Vector Search Index

Upgrade or Enable Direct Access If you must use Direct Access: • Contact your Databricks account team or Azure support to upgrade the workspace to Enterprise SKU or enroll in Direct Access private preview. • Ensure Unity Catalog and serverless compute are enabled, as these are prerequisites for Direct Access.

Reference - Workspace Entitlements

Hope the above steps were helpful. If you have any other questions, please feel free to contact us.

Thanks, Vrishabh

Hi Eric Tan,

We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.

Thanks,
Vrishabh

Thanks @VRISHABHANATH PATIL

I need to use direct access index. The problem is I can't seem to be able to contact support or raise support ticket to do that, I keep trying but the Support from my azure portal will only direct me to document or search result.