Share via

AzureOpenAIGrader Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Base class for Azure OpenAI grader wrappers, recommended only for use by experienced OpenAI API users. Combines a model configuration and any grader configuration into a singular object that can be used in evaluations.

Supplying an AzureOpenAIGrader to the evaluate method will cause an asynchronous request to evaluate the grader via the OpenAI API. The results of the evaluation will then be merged into the standard evaluation results.

] :param grader_config: The grader configuration to use for the grader. This is expected

to be formatted as a dictionary that matches the specifications of the sub-types of the TestingCriterion alias specified in (OpenAI's SDK)[https://github.com/openai/openai-python/blob/ed53107e10e6c86754866b48f8bd862659134ca8/src/openai/types/eval_create_params.py#L151].

Constructor

AzureOpenAIGrader(*, model_config: AzureOpenAIModelConfiguration | OpenAIModelConfiguration, grader_config: Dict[str, Any], **kwargs: Any)

Parameters

Name Description
model_config
Required
Union[ AzureOpenAIModelConfiguration, OpenAIModelConfiguration

The model configuration to use for the grader.
kwargs
Required
Any

Additional keyword arguments to pass to the grader.

Keyword-Only Parameters

Name Description
model_config
Required
grader_config
Required

Methods

get_client

Construct an appropriate OpenAI client using this grader's model configuration. Returns a slightly different client depending on whether or not this grader's model configuration is for Azure OpenAI or OpenAI.

get_client

Construct an appropriate OpenAI client using this grader's model configuration. Returns a slightly different client depending on whether or not this grader's model configuration is for Azure OpenAI or OpenAI.

get_client() -> Any

Returns

Type Description
[<xref:openai.OpenAI>, <xref:openai.AzureOpenAI>]

The OpenAI client.

Attributes

id 

id = 'azureai://built-in/evaluators/azure-openai/custom_grader'