Share via

AzureOpenAILabelGrader Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Wrapper class for OpenAI's label model graders.

Supplying a LabelGrader to the evaluate method will cause an asynchronous request to evaluate the grader via the OpenAI API. The results of the evaluation will then be merged into the standard evaluation results.

] :param input: The list of label-based testing criterion for this grader. Individual

values of this list are expected to be dictionaries that match the format of any of the valid (TestingCriterionLabelModelInput)[https://github.com/openai/openai-python/blob/ed53107e10e6c86754866b48f8bd862659134ca8/src/openai/types/eval_create_params.py#L125C1-L125C32] subtypes.

Constructor

AzureOpenAILabelGrader(*, model_config: AzureOpenAIModelConfiguration | OpenAIModelConfiguration, input: List[Dict[str, str]], labels: List[str], model: str, name: str, passing_labels: List[str], **kwargs: Any)

Parameters

Name Description
model_config
Required
Union[ AzureOpenAIModelConfiguration, OpenAIModelConfiguration

The model configuration to use for the grader.
labels
Required
List[str]

A list of strings representing the classification labels of this grader.
model
Required
str

The model to use for the evaluation. Must support structured outputs.
name
Required
str

The name of the grader.
passing_labels
Required
List[str]

The labels that indicate a passing result. Must be a subset of labels.
kwargs
Required
Any

Additional keyword arguments to pass to the grader.

Keyword-Only Parameters

Name Description
model_config
Required
input
Required
labels
Required
model
Required
name
Required
passing_labels
Required

Methods

get_client

Construct an appropriate OpenAI client using this grader's model configuration. Returns a slightly different client depending on whether or not this grader's model configuration is for Azure OpenAI or OpenAI.

get_client

Construct an appropriate OpenAI client using this grader's model configuration. Returns a slightly different client depending on whether or not this grader's model configuration is for Azure OpenAI or OpenAI.

get_client() -> Any

Returns

Type Description
[<xref:openai.OpenAI>, <xref:openai.AzureOpenAI>]

The OpenAI client.

Attributes

id 

id = 'azureai://built-in/evaluators/azure-openai/label_grader'