Share via

IntentResolutionEvaluator Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Evaluates intent resolution for a given query and response or a multi-turn conversation, including reasoning.

The intent resolution evaluator assesses whether the user intent was correctly identified and resolved.

Constructor

IntentResolutionEvaluator(model_config, *, threshold=3, credential=None, **kwargs)

Parameters

Name Description
model_config
Required
Union[AzureOpenAIModelConfiguration, OpenAIModelConfiguration]

Configuration for the Azure OpenAI model.

Keyword-Only Parameters

Name Description
threshold
Default value: 3
credential
Default value: None

Examples

Initialize and call IntentResolutionEvaluator using Azure AI Project URL in the following format https://{resource_name}.services.ai.azure.com/api/projects/{project_name}


   import os
   from azure.ai.evaluation import CoherenceEvaluator

   model_config = {
       "azure_endpoint": os.environ.get("AZURE_OPENAI_ENDPOINT"),  # https://<account_name>.services.ai.azure.com
       "api_key": os.environ.get("AZURE_OPENAI_KEY"),
       "azure_deployment": os.environ.get("AZURE_OPENAI_DEPLOYMENT"),
   }
   intent_resolution_evaluator = IntentResolutionEvaluator(model_config=model_config)
   intent_resolution_evaluator(
       query="What is the opening hours of the Eiffel Tower?",
       response="Opening hours of the Eiffel Tower are 9:00 AM to 11:00 PM.",
   )

Attributes

id

Evaluator identifier, experimental and to be used only with evaluation in cloud.

id = 'azureai://built-in/evaluators/intent_resolution'