Share via

Simulator Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Simulator for generating synthetic conversations.

Constructor

Simulator(model_config: AzureOpenAIModelConfiguration | OpenAIModelConfiguration)

Parameters

Name Description
model_config
Required
Union[AzureOpenAIModelConfiguration, OpenAIModelConfiguration]

A dictionary defining the configuration for the model. Acceptable types are AzureOpenAIModelConfiguration and OpenAIModelConfiguration.

Examples

Run a Simulator for 2 queries and 4 conversation turns.


   import os
   import asyncio
   from azure.ai.evaluation.simulator import Simulator

   model_config = {
       "azure_endpoint": os.environ.get("AZURE_OPENAI_ENDPOINT"),
       "api_key": os.environ.get("AZURE_OPENAI_KEY"),
       "azure_deployment": os.environ.get("AZURE_OPENAI_DEPLOYMENT"),
   }

   async def callback(
       messages: List[Dict],
       stream: bool = False,
       session_state: Any = None,
       context: Optional[Dict[str, Any]] = None,
   ) -> dict:
       query = messages["messages"][0]["content"]

       formatted_response = {"content": query, "role": "assistant"}
       messages["messages"].append(formatted_response)
       return {
           "messages": messages["messages"],
           "stream": stream,
           "session_state": session_state,
           "context": context,
       }

   simulator = Simulator(model_config=model_config)

   result = asyncio.run(
       simulator(
           target=callback,
           max_conversation_turns=2,
           text="some text",
           tasks=["tasks"],
           api_call_delay_sec=1,
           num_queries=1,
       )
   )