Share via

AzureOpenAIStringCheckGrader Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Wrapper class for OpenAI's string check graders.

Supplying a StringCheckGrader to the evaluate method will cause an asynchronous request to evaluate the grader via the OpenAI API. The results of the evaluation will then be merged into the standard evaluation results.

] :param input: The input text. This may include template strings. :type input: str :param name: The name of the grader. :type name: str :param operation: The string check operation to perform. One of eq, ne, like, or ilike. :type operation: Literal["eq", "ne", "like", "ilike"] :param reference: The reference text. This may include template strings. :type reference: str :param kwargs: Additional keyword arguments to pass to the grader. :type kwargs: Any

Constructor

AzureOpenAIStringCheckGrader(*, model_config: AzureOpenAIModelConfiguration | OpenAIModelConfiguration, input: str, name: str, operation: Literal['eq', 'ne', 'like', 'ilike'], reference: str, **kwargs: Any)

Parameters

Name Description
model_config
Required
Union[ AzureOpenAIModelConfiguration, OpenAIModelConfiguration

The model configuration to use for the grader.

Keyword-Only Parameters

Name Description
model_config
Required
input
Required
name
Required
operation
Required
reference
Required

Methods

get_client

Construct an appropriate OpenAI client using this grader's model configuration. Returns a slightly different client depending on whether or not this grader's model configuration is for Azure OpenAI or OpenAI.

get_client

Construct an appropriate OpenAI client using this grader's model configuration. Returns a slightly different client depending on whether or not this grader's model configuration is for Azure OpenAI or OpenAI.

get_client() -> Any

Returns

Type Description
[<xref:openai.OpenAI>, <xref:openai.AzureOpenAI>]

The OpenAI client.

Attributes

id 

id = 'azureai://built-in/evaluators/azure-openai/string_check_grader'