本页讨论应用可以使用 Windows ML 正常处理下载和注册执行提供程序(EP)的更高级方法。 即使设备上已下载 EP,每次运行应用时都必须注册 EP,以便它们将显示在 ONNX 运行时中。
在一次呼叫中下载并注册
对于初始开发,只需调用,即可下载 EnsureAndRegisterCertifiedAsync()任何与设备和驱动程序兼容的新 EP(或新版本的 EP)(如果尚未下载),然后注册所有 EP。 请注意,首次运行时,此方法可能需要数秒甚至几分钟时间,具体取决于需要下载的网络速度和 EP。
// Get the default ExecutionProviderCatalog
var catalog = ExecutionProviderCatalog.GetDefault();
// Ensure and register all compatible execution providers with ONNX Runtime
// This downloads any necessary components and registers them
await catalog.EnsureAndRegisterCertifiedAsync();
// Get the default ExecutionProviderCatalog
winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderCatalog catalog =
winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderCatalog::GetDefault();
// Ensure and register all compatible execution providers with ONNX Runtime
catalog.EnsureAndRegisterCertifiedAsync().get();
# Please DO NOT use this API. It won't register EPs to the python ort env.
小窍门
在生产应用程序中,将 EnsureAndRegisterCertifiedAsync() 调用包装在 try-catch 块中以正常方式处理潜在的网络或下载失败。
仅注册现有提供程序
如果想要避免下载并仅注册计算机上已存在的执行提供程序:
var catalog = ExecutionProviderCatalog.GetDefault();
// Register only providers already present on the machine
// This avoids potentially long download times
await catalog.RegisterCertifiedAsync();
auto catalog = winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderCatalog::GetDefault();
// Register only providers already present on the machine
catalog.RegisterCertifiedAsync().get();
# Please DO NOT use this API. It won't register EPs to the python ort env.
发现是否有新的 EP (无需下载)
如果想要查看是否有与设备和驱动程序兼容的新 EP 可供下载,但不想开始下载,则可以使用 FindAllProviders() 该方法,然后查看是否有任何提供程序具有 ReadyState of NotPresent。 然后,你可以决定处理此问题(将用户启动到“正在更新的屏幕”中,询问他们是否要更新等)。 如果不想让用户立即等待,可以选择继续使用已下载的 EP(如 RegisterCertifiedAsync() 上所示调用)。
var catalog = ExecutionProviderCatalog.GetDefault();
// Check if there are new EPs that need to be downloaded
if (catalog.FindAllProviders().Any(provider => provider.ReadyState == ExecutionProviderReadyState.NotPresent))
{
// TODO: There are new EPs, decide how your app wants to handle that
}
else
{
// All EPs are already present, just register them
await catalog.RegisterCertifiedAsync();
}
auto catalog = winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderCatalog::GetDefault();
auto providers = catalog.FindAllProviders();
// Check if any providers need to be downloaded
bool needsDownload = false;
for (const auto& provider : providers)
{
if (provider.ReadyState() == winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderReadyState::NotPresent)
{
needsDownload = true;
break;
}
}
if (needsDownload)
{
// TODO: There are new EPs, decide how your app wants to handle that
}
else
{
// All EPs are already present, just register them
catalog.RegisterCertifiedAsync().get();
}
# winml: winui3.microsoft.windows.ai.machinelearning
catalog = winml.ExecutionProviderCatalog.get_default()
providers = catalog.find_all_providers()
if any(provider.ready_state == winml.ExecutionProviderReadyState.NOT_PRESENT for provider in providers):
# TODO: There are new EPs, decide how your app wants to handle that
pass
else:
# Register the providers one by one
for provider in providers:
provider.ensure_ready_async().get()
ort.register_execution_provider_library(provider.name, provider.library_path)
下载并注册特定的 EP
如果您的应用程序要使用特定的执行提供程序,您可以下载并注册该特定执行提供程序,而无需下载所有兼容的执行提供程序。
首先,你将使用FindAllProviders()获取所有兼容的 EP,然后可以调用特定的EnsureReadyAsync()下载特定的执行提供程序,并调用TryRegister()注册特定的执行提供程序。
var catalog = ExecutionProviderCatalog.GetDefault();
// Get the QNN provider, if present
var qnnProvider = catalog.FindAllProviders()
.FirstOrDefault(i => i.Name == "QNNExecutionProvider");
if (qnnProvider != null)
{
// Download it
var result = await qnnProvider.EnsureReadyAsync();
// If download succeeded
if (result != null && result.Status == ExecutionProviderReadyResultState.Success)
{
// Register it
bool registered = qnnProvider.TryRegister();
}
}
auto catalog = winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderCatalog::GetDefault();
// Get the QNN provider, if present
auto providers = catalog.FindAllProviders();
winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProvider qnnProvider{ nullptr };
for (auto const& p : providers)
{
if (p.Name() == L"QNNExecutionProvider")
{
qnnProvider = p;
break;
}
}
if (qnnProvider)
{
// Download required components for this provider (if not already present)
auto result = qnnProvider.EnsureReadyAsync().get();
if (result && result.Status() == winrt::Microsoft::Windows::AI::MachineLearning::ExecutionProviderReadyResultState::Success)
{
// Register the provider with ONNX Runtime
bool registered = qnnProvider.TryRegister();
}
}
catalog = winml.ExecutionProviderCatalog.get_default()
providers = catalog.find_all_providers()
qnn_provider = next((provider for provider in providers if provider.name == 'QNNExecutionProvider'), None)
if qnn_provider is not None:
# Download required components for this provider (if not already present)
result = qnn_provider.ensure_ready_async().get()
if result == winml.ExecutionProviderReadyResultState.SUCCESS:
# Register the provider with ONNX Runtime
ort.register_execution_provider_library(qnn_provider.name, qnn_provider.library_path)
生产应用示例
对于生产应用程序,下面是应用可能想要执行的作的示例,让你的用户能够控制何时发生下载。 可以检查新的执行提供程序是否可用,并有条件地下载它们:
using Microsoft.Windows.AI.MachineLearning;
var catalog = ExecutionProviderCatalog.GetDefault();
// Filter to the EPs our app supports/uses
var providers = catalog.FindAllProviders().Where(p =>
p.Name == "VitisAIExecutionProvider" ||
p.Name == "OpenVINOExecutionProvider" ||
p.Name == "QNNExecutionProvider" ||
p.Name == "NvTensorRtRtxExecutionProvider"
);
if (providers.Any(p => p.ReadyState == ExecutionProviderReadyState.NotPresent))
{
// Show UI to user asking if they want to download new execution providers
bool userWantsToDownload = await ShowDownloadDialogAsync();
if (userWantsToDownload)
{
// Download all EPs
foreach (var p in providers)
{
if (p.ReadyState == ExecutionProviderReadyState.NotPresent)
{
// Ignore result handling here; production code could inspect status
await p.EnsureReadyAsync();
}
}
// And register all EPs
await catalog.RegisterCertifiedAsync();
}
else
{
// Register only already-present EPs
await catalog.RegisterCertifiedAsync();
}
}
using namespace winrt::Microsoft::Windows::AI::MachineLearning;
auto catalog = ExecutionProviderCatalog::GetDefault();
auto allProviders = catalog.FindAllProviders();
// Filter to the EPs our app supports/uses
std::vector<ExecutionProvider> targetProviders;
for (auto const& p : allProviders)
{
auto name = p.Name();
if (name == L"VitisAIExecutionProvider" ||
name == L"OpenVINOExecutionProvider" ||
name == L"QNNExecutionProvider" ||
name == L"NvTensorRtRtxExecutionProvider")
{
targetProviders.push_back(p);
}
}
bool needsDownload = false;
for (auto const& p : targetProviders)
{
if (p.ReadyState() == ExecutionProviderReadyState::NotPresent)
{
needsDownload = true;
break;
}
}
if (needsDownload)
{
// Show UI to user or check application settings to confirm download
bool userWantsToDownload = ShowDownloadDialog();
if (userWantsToDownload)
{
// Download only the missing target providers
for (auto const& p : targetProviders)
{
if (p.ReadyState() == ExecutionProviderReadyState::NotPresent)
{
// Ignore result handling here; production code could inspect status
p.EnsureReadyAsync().get();
}
}
// Register all (both previously present and newly downloaded) providers
catalog.RegisterCertifiedAsync().get();
}
else
{
// User deferred download; register only already-present providers
catalog.RegisterCertifiedAsync().get();
}
}
else
{
// All target EPs already present
catalog.RegisterCertifiedAsync().get();
}
# remove the msvcp140.dll from the winrt-runtime package.
# So it does not cause issues with other libraries.
from pathlib import Path
from importlib import metadata
site_packages_path = Path(str(metadata.distribution('winrt-runtime').locate_file('')))
dll_path = site_packages_path / 'winrt' / 'msvcp140.dll'
if dll_path.exists():
dll_path.unlink()
from winui3.microsoft.windows.applicationmodel.dynamicdependency.bootstrap import (
InitializeOptions,
initialize
)
import winui3.microsoft.windows.ai.machinelearning as winml
import onnxruntime as ort
with initialize(options=InitializeOptions.ON_NO_MATCH_SHOW_UI):
catalog = winml.ExecutionProviderCatalog.get_default()
# Filter EPs that the app supports
providers = [provider for provider in catalog.find_all_providers() if provider.name in [
'VitisAIExecutionProvider',
'OpenVINOExecutionProvider',
'QNNExecutionProvider',
'NvTensorRtRtxExecutionProvider'
]]
# Download and make ready missing EPs if the user wants to
if any(provider.ready_state == winml.ExecutionProviderReadyState.NOT_PRESENT for provider in providers):
# Ask the user if they want to download the missing packages
if user_wants_to_download:
for provider in [provider for provider in providers if provider.ready_state == winml.ExecutionProviderReadyState.NOT_PRESENT]:
provider.ensure_ready_async().get()
# Make ready the existing EPs
for provider in [provider for provider in providers if provider.ready_state == winml.ExecutionProviderReadyState.NOT_READY]:
provider.ensure_ready_async().get()
# Register all ready EPs
for provider in [provider for provider in providers if provider.ready_state == winml.ExecutionProviderReadyState.READY]:
ort.register_execution_provider_library(provider.name, provider.library_path)