Share via

GroundednessEvaluator Class

Evaluates groundedness score for a given query (optional), response, and context or a multi-turn conversation, including reasoning.

The groundedness measure assesses the correspondence between claims in an AI-generated answer and the source context, making sure that these claims are substantiated by the context. Even if the responses from LLM are factually correct, they'll be considered ungrounded if they can't be verified against the provided sources (such as your input source or your database). Use the groundedness metric when you need to verify that AI-generated responses align with and are validated by the provided context.

Groundedness scores range from 1 to 5, with 1 being the least grounded and 5 being the most grounded.

Note

To align with our support of a diverse set of models, an output key without the gpt_ prefix has been added.

To maintain backwards compatibility, the old key with the gpt_ prefix is still be present in the output;

however, it is recommended to use the new key moving forward as the old key will be deprecated in the future.

Constructor

GroundednessEvaluator(model_config, *, threshold=3, credential=None, **kwargs)

Parameters

Name Description
model_config
Required
Union[AzureOpenAIModelConfiguration, OpenAIModelConfiguration]

Configuration for the Azure OpenAI model.
threshold
Required
int

The threshold for the groundedness evaluator. Default is 3.

Keyword-Only Parameters

Name Description
threshold
Default value: 3
credential
Default value: None

Examples

Initialize and call GroundednessEvaluator using Azure AI Project URL in the following format https://{resource_name}.services.ai.azure.com/api/projects/{project_name}


   import os
   from azure.ai.evaluation import GroundednessEvaluator

   model_config = {
       "azure_endpoint": os.environ.get("AZURE_OPENAI_ENDPOINT"),  # https://<account_name>.services.ai.azure.com
       "api_key": os.environ.get("AZURE_OPENAI_KEY"),
       "azure_deployment": os.environ.get("AZURE_OPENAI_DEPLOYMENT"),
   }

   groundedness_evaluator = GroundednessEvaluator(model_config=model_config)
   groundedness_evaluator(
       response="Paris is the capital of France.",
       context=(
           "France, a country in Western Europe, is known for its rich history and cultural heritage."
           "The city of Paris, located in the northern part of the country, serves as its capital."
           "Paris is renowned for its art, fashion, and landmarks such as the Eiffel Tower and the Louvre Museum."
       ),
   )

Attributes

id

Evaluator identifier, experimental and to be used only with evaluation in cloud.

id = 'azureai://built-in/evaluators/groundedness'