使用 Recognize 操作收集用户输入

Prerequisites

• 具有有效订阅的 Azure 帐户，有关详细信息，请参阅免费创建帐户。
• Azure 通信服务资源。 请参阅创建 Azure 通信服务资源。 请注意此资源的连接字符串。
• 使用通话自动化 SDK 创建新的 Web 服务应用程序。
• 适用于操作系统的最新 .NET 库。
• 最新的 NuGet 包。

对于 AI 功能

• 创建 Azure AI 服务并将其连接到 Azure 通信服务资源。
• 为 Azure AI 服务资源创建自定义子域。

技术规范

以下参数可用于自定义 Recognize 函数：

新建 C# 应用程序

在操作系统的控制台窗口中，使用 dotnet 命令创建新的 Web 应用程序。

dotnet new web -n MyApplication

安装 NuGet 包

从 NuGet 库 | Azure.Communication.CallAutomation 获取 NuGet 包。 按照说明安装相应的包。

建立呼叫

至此，你应已熟悉如何开始通话。 有关发起通话的详细信息，请参阅快速入门：发起传出通话。 还可以使用此处提供的代码片段来了解如何应答呼叫。

var callAutomationClient = new CallAutomationClient("<Azure Communication Services connection string>");

var answerCallOptions = new AnswerCallOptions("<Incoming call context once call is connected>", new Uri("<https://sample-callback-uri>"))  
{  
    CallIntelligenceOptions = new CallIntelligenceOptions() { CognitiveServicesEndpoint = new Uri("<Azure Cognitive Services Endpoint>") } 
};  

var answerCallResult = await callAutomationClient.AnswerCallAsync(answerCallOptions); 

调用识别动作

在你的应用程序应答呼叫时，你可以提供有关识别参与者输入和播放提示的信息。

DTMF

var maxTonesToCollect = 3;
String textToPlay = "Welcome to Contoso, please enter 3 DTMF.";
var playSource = new TextSource(textToPlay, "en-US-ElizabethNeural");
var recognizeOptions = new CallMediaRecognizeDtmfOptions(targetParticipant, maxTonesToCollect) {
  InitialSilenceTimeout = TimeSpan.FromSeconds(30),
    Prompt = playSource,
    InterToneTimeout = TimeSpan.FromSeconds(5),
    InterruptPrompt = true,
    StopTones = new DtmfTone[] {
      DtmfTone.Pound
    },
};
var recognizeResult = await callAutomationClient.GetCallConnection(callConnectionId)
  .GetCallMedia()
  .StartRecognizingAsync(recognizeOptions);

对于语音转文本流，通话自动化识别操作还支持使用自定义语音识别模型。 在生成需要侦听默认语音转文本模型可能无法理解的复杂字词的应用程序时，自定义语音识别模型等功能可能非常有用。 例如，在为远程医疗行业生成应用程序，并且虚拟代理需要能够识别医疗术语时。 可以在创建自定义语音识别项目中了解详细信息。

语音转文本选择

var choices = new List < RecognitionChoice > {
  new RecognitionChoice("Confirm", new List < string > {
    "Confirm",
    "First",
    "One"
  }) {
    Tone = DtmfTone.One
  },
  new RecognitionChoice("Cancel", new List < string > {
    "Cancel",
    "Second",
    "Two"
  }) {
    Tone = DtmfTone.Two
  }
};
String textToPlay = "Hello, This is a reminder for your appointment at 2 PM, Say Confirm to confirm your appointment or Cancel to cancel the appointment. Thank you!";

var playSource = new TextSource(textToPlay, "en-US-ElizabethNeural");
var recognizeOptions = new CallMediaRecognizeChoiceOptions(targetParticipant, choices) {
  InterruptPrompt = true,
    InitialSilenceTimeout = TimeSpan.FromSeconds(30),
    Prompt = playSource,
    OperationContext = "AppointmentReminderMenu",
    SpeechLanguages = new List<string> { "en-US", "hi-IN", "fr-FR" },
    //Only add the SpeechModelEndpointId if you have a custom speech model you would like to use
    SpeechModelEndpointId = "YourCustomSpeechModelEndpointId"
};
var recognizeResult = await callAutomationClient.GetCallConnection(callConnectionId)
  .GetCallMedia()
  .StartRecognizingAsync(recognizeOptions);

语音转文本

String textToPlay = "Hi, how can I help you today?";
var playSource = new TextSource(textToPlay, "en-US-ElizabethNeural");
var recognizeOptions = new CallMediaRecognizeSpeechOptions(targetParticipant) {
  Prompt = playSource,
    EndSilenceTimeout = TimeSpan.FromMilliseconds(1000),
    OperationContext = "OpenQuestionSpeech",
    //Only add the SpeechModelEndpointId if you have a custom speech model you would like to use
    SpeechModelEndpointId = "YourCustomSpeechModelEndpointId"
};
var recognizeResult = await callAutomationClient.GetCallConnection(callConnectionId)
  .GetCallMedia()
  .StartRecognizingAsync(recognizeOptions);

语音转文本或 DTMF

var maxTonesToCollect = 1; 
String textToPlay = "Hi, how can I help you today, you can press 0 to speak to an agent?"; 
var playSource = new TextSource(textToPlay, "en-US-ElizabethNeural"); 
var recognizeOptions = new CallMediaRecognizeSpeechOrDtmfOptions(targetParticipant, maxTonesToCollect) 
{ 
    Prompt = playSource, 
    EndSilenceTimeout = TimeSpan.FromMilliseconds(1000), 
    InitialSilenceTimeout = TimeSpan.FromSeconds(30), 
    InterruptPrompt = true, 
    OperationContext = "OpenQuestionSpeechOrDtmf",
    SpeechLanguages = new List<string> { "en-US", "hi-IN", "fr-FR" },
    //Only add the SpeechModelEndpointId if you have a custom speech model you would like to use
    SpeechModelEndpointId = "YourCustomSpeechModelEndpointId" 
}; 
var recognizeResult = await callAutomationClient.GetCallConnection(callConnectionId) 
    .GetCallMedia() 
    .StartRecognizingAsync(recognizeOptions); 

实时语言识别（预览版）

借助额外的实时语言识别，开发人员可以自动检测口语，以启用自然的、类似于人类的通信，并消除最终用户的手动语言选择。

string textToPlay = "Hi, how can I help you today?";
var playSource = new TextSource(textToPlay, "en-US-ElizabethNeural");
var recognizeOptions = new CallMediaRecognizeSpeechOptions(targetParticipant: new PhoneNumberIdentifier(targetParticipant))
{
    Prompt = playSource,
    InterruptCallMediaOperation = false,
    InterruptPrompt = false,
    InitialSilenceTimeout = TimeSpan.FromSeconds(10),
    OperationContext = "OpenQuestionSpeech",
    // Enable Language Identification
    SpeechLanguages = new List<string> { "en-US", "hi-IN", "fr-FR" },
    // Only add the SpeechModelEndpointId if you have a custom speech model you would like to use
    SpeechModelEndpointId = "YourCustomSpeechModelEndpointId"
};
var recognizeResult = await callAutomationClient.GetCallConnection(callConnectionId)
    .GetCallMedia()
    .StartRecognizingAsync(recognizeOptions);

情绪分析（预览版）

识别 API 支持使用语音输入时进行的情绪分析。 实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 它还可用于路由、个性化或分析。

string textToPlay = "Hi, how can I help you today?";
var playSource = new TextSource(textToPlay, "en-US-ElizabethNeural");
var recognizeOptions = new CallMediaRecognizeSpeechOptions(targetParticipant: new PhoneNumberIdentifier(targetParticipant))
{
    Prompt = playSource,
    InterruptCallMediaOperation = false,
    InterruptPrompt = false,
    InitialSilenceTimeout = TimeSpan.FromSeconds(10),
    OperationContext = "OpenQuestionSpeech",
    // Enable Sentiment Analysis
    IsSentimentAnalysisEnabled = true
};
var recognizeResult = await callAutomationClient.GetCallConnection(callConnectionId)
    .GetCallMedia()
    .StartRecognizingAsync(recognizeOptions);

接收识别事件更新

开发人员可以订阅有关已注册 webhook 回调的 RecognizeCompleted 和 RecognizeFailed 事件。 在应用程序中将此回调与业务逻辑配合使用可以确定当其中一个事件发生时要采取的后续步骤。

如何反序列化 RecognizeCompleted 事件的示例：

if (parsedEvent is RecognizeCompleted recognizeCompleted)
{
    logger.LogInformation($"Received call event: {recognizeCompleted.GetType()}");

    callConnectionId = recognizeCompleted.CallConnectionId;

    switch (recognizeCompleted.RecognizeResult)
    {
        case DtmfResult dtmfResult:
            var tones = dtmfResult.Tones;
            logger.LogInformation("Recognize completed successfully, tones={tones}", tones);
            break;

        case ChoiceResult choiceResult:
            var labelDetected = choiceResult.Label;
            var phraseDetected = choiceResult.RecognizedPhrase;
            var sentimentAnalysis = choiceResult.SentimentAnalysisResult;

            logger.LogInformation("Recognize completed successfully, labelDetected={labelDetected}, phraseDetected={phraseDetected}", labelDetected, phraseDetected);
            logger.LogInformation("Language Identified: {language}", choiceResult.LanguageIdentified);

            if (sentimentAnalysis != null)
            {
                logger.LogInformation("Sentiment: {sentiment}", sentimentAnalysis.Sentiment);
            }
            break;

        case SpeechResult speechResult:
            var text = speechResult.Speech;
            var speechSentiment = speechResult.SentimentAnalysisResult;

            logger.LogInformation("Recognize completed successfully, text={text}", text);
            logger.LogInformation("Language Identified: {language}", speechResult.LanguageIdentified);

            if (speechSentiment != null)
            {
                logger.LogInformation("Sentiment: {sentiment}", speechSentiment.Sentiment);
            }
            break;

        default:
            logger.LogInformation("Recognize completed successfully, recognizeResult={recognizeResult}", recognizeCompleted.RecognizeResult);
            break;
    }
}

如何反序列化 RecognizeFailed 事件的示例：

if (acsEvent is RecognizeFailed recognizeFailed) 
{ 
    if (MediaEventReasonCode.RecognizeInitialSilenceTimedOut.Equals(recognizeFailed.ReasonCode)) 
    { 
        // Take action for time out 
        logger.LogInformation("Recognition failed: initial silence time out"); 
    } 
    else if (MediaEventReasonCode.RecognizeSpeechOptionNotMatched.Equals(recognizeFailed.ReasonCode)) 
    { 
        // Take action for option not matched 
        logger.LogInformation("Recognition failed: speech option not matched"); 
    } 
    else if (MediaEventReasonCode.RecognizeIncorrectToneDetected.Equals(recognizeFailed.ReasonCode)) 
    { 
        // Take action for incorrect tone 
        logger.LogInformation("Recognition failed: incorrect tone detected"); 
    } 
    else 
    { 
        logger.LogInformation("Recognition failed, result={result}, context={context}", recognizeFailed.ResultInformation?.Message, recognizeFailed.OperationContext); 
    } 
} 

如何反序列化 RecognizeCanceled 事件的示例：

if (acsEvent is RecognizeCanceled { OperationContext: "AppointmentReminderMenu" })
        {
            logger.LogInformation($"RecognizeCanceled event received for call connection id: {@event.CallConnectionId}");
            //Take action on recognize canceled operation
           await callConnection.HangUpAsync(forEveryone: true);
        }

Prerequisites

• 具有有效订阅的 Azure 帐户，有关详细信息，请参阅免费创建帐户。
• Azure 通信服务资源。 请参阅创建 Azure 通信服务资源
• 使用通话自动化 SDK 创建新的 Web 服务应用程序。
• Java 开发工具包版本 8 或更高版本。
• Apache Maven。

对于 AI 功能

• 创建 Azure AI 服务并将其连接到 Azure 通信服务资源。
• 为 Azure AI 服务资源创建自定义子域。

技术规范

以下参数可用于自定义 Recognize 函数：

创建新的 Java 应用程序

在终端或命令窗口中，导航到要在其中创建 Java 应用程序的目录。 运行 mvn 命令以从 maven-archetype-quickstart 模板生成 Java 项目。

mvn archetype:generate -DgroupId=com.communication.quickstart -DartifactId=communication-quickstart -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false

artifactId 命令将创建一个与 mvn 参数同名的目录。 src/main/java 目录包含项目源代码。 src/test/java 目录包含测试源。

请注意，generate 步骤创建了一个与 artifactId 同名的目录。 src/main/java 目录包含源代码。 src/test/java 目录包含测试。 pom.xml 文件是项目的项目对象模型 (POM)。

将应用程序的 POM 文件更新为使用 Java 8 或更高版本。

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
</properties>

添加包引用

在 POM 文件中，为项目添加以下引用：

azure-communication-callautomation

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-communication-callautomation</artifactId>
  <version>1.0.0</version>
</dependency>

建立呼叫

至此，你应已熟悉如何开始通话。 有关发起通话的详细信息，请参阅快速入门：发起传出通话。 还可以使用此处提供的代码片段来了解如何应答呼叫。

CallIntelligenceOptions callIntelligenceOptions = new CallIntelligenceOptions().setCognitiveServicesEndpoint("https://sample-cognitive-service-resource.cognitiveservices.azure.com/"); 
answerCallOptions = new AnswerCallOptions("<Incoming call context>", "<https://sample-callback-uri>").setCallIntelligenceOptions(callIntelligenceOptions); 
Response < AnswerCallResult > answerCallResult = callAutomationClient
  .answerCallWithResponse(answerCallOptions)
  .block();

调用识别动作

在你的应用程序应答呼叫时，你可以提供有关识别参与者输入和播放提示的信息。

DTMF

var maxTonesToCollect = 3;
String textToPlay = "Welcome to Contoso, please enter 3 DTMF.";
var playSource = new TextSource() 
    .setText(textToPlay) 
    .setVoiceName("en-US-ElizabethNeural");

var recognizeOptions = new CallMediaRecognizeDtmfOptions(targetParticipant, maxTonesToCollect) 
    .setInitialSilenceTimeout(Duration.ofSeconds(30)) 
    .setPlayPrompt(playSource) 
    .setInterToneTimeout(Duration.ofSeconds(5)) 
    .setInterruptPrompt(true) 
    .setStopTones(Arrays.asList(DtmfTone.POUND));

var recognizeResponse = callAutomationClient.getCallConnectionAsync(callConnectionId) 
    .getCallMediaAsync() 
    .startRecognizingWithResponse(recognizeOptions) 
    .block(); 

log.info("Start recognizing result: " + recognizeResponse.getStatusCode()); 

对于语音转文本流，通话自动化识别操作还支持使用自定义语音识别模型。 在生成需要侦听默认语音转文本模型可能无法理解的复杂字词的应用程序时，自定义语音识别模型等功能可能非常有用。 例如，在为远程医疗行业生成应用程序，并且虚拟代理需要能够识别医疗术语时。 可以在创建自定义语音识别项目中了解详细信息。

语音转文本选择

var choices = Arrays.asList(
  new RecognitionChoice()
  .setLabel("Confirm")
  .setPhrases(Arrays.asList("Confirm", "First", "One"))
  .setTone(DtmfTone.ONE),
  new RecognitionChoice()
  .setLabel("Cancel")
  .setPhrases(Arrays.asList("Cancel", "Second", "Two"))
  .setTone(DtmfTone.TWO)
);

String textToPlay = "Hello, This is a reminder for your appointment at 2 PM, Say Confirm to confirm your appointment or Cancel to cancel the appointment. Thank you!";
var playSource = new TextSource()
  .setText(textToPlay)
  .setVoiceName("en-US-ElizabethNeural");
var recognizeOptions = new CallMediaRecognizeChoiceOptions(targetParticipant, choices)
  .setInterruptPrompt(true)
  .setInitialSilenceTimeout(Duration.ofSeconds(30))
  .setPlayPrompt(playSource)
  .setSpeechLanguages("en-US", "es-ES", "hi-IN")
  .setSentimentAnalysisEnabled(true)
  .setOperationContext("AppointmentReminderMenu")
  //Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
  .setSpeechRecognitionModelEndpointId("YourCustomSpeechModelEndpointID"); 
var recognizeResponse = callAutomationClient.getCallConnectionAsync(callConnectionId)
  .getCallMediaAsync()
  .startRecognizingWithResponse(recognizeOptions)
  .block();

语音转文本

String textToPlay = "Hi, how can I help you today?"; 
var playSource = new TextSource() 
    .setText(textToPlay) 
    .setVoiceName("en-US-ElizabethNeural"); 
var recognizeOptions = new CallMediaRecognizeSpeechOptions(targetParticipant, Duration.ofMillis(1000)) 
    .setPlayPrompt(playSource) 
    .setOperationContext("OpenQuestionSpeech")
    //Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    .setSpeechRecognitionModelEndpointId("YourCustomSpeechModelEndpointID");  
var recognizeResponse = callAutomationClient.getCallConnectionAsync(callConnectionId) 
    .getCallMediaAsync() 
    .startRecognizingWithResponse(recognizeOptions) 
    .block(); 

语音转文本或 DTMF

var maxTonesToCollect = 1; 
String textToPlay = "Hi, how can I help you today, you can press 0 to speak to an agent?"; 
var playSource = new TextSource() 
    .setText(textToPlay) 
    .setVoiceName("en-US-ElizabethNeural"); 
var recognizeOptions = new CallMediaRecognizeSpeechOrDtmfOptions(targetParticipant, maxTonesToCollect, Duration.ofMillis(1000)) 
    .setPlayPrompt(playSource) 
    .setInitialSilenceTimeout(Duration.ofSeconds(30)) 
    .setInterruptPrompt(true) 
    .setOperationContext("OpenQuestionSpeechOrDtmf")
    //Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    .setSpeechRecognitionModelEndpointId("YourCustomSpeechModelEndpointID");  
var recognizeResponse = callAutomationClient.getCallConnectionAsync(callConnectionId) 
    .getCallMediaAsync() 
    .startRecognizingWithResponse(recognizeOptions) 
    .block(); 

实时语言识别（预览版）

借助额外的实时语言识别，开发人员可以自动检测口语，以启用自然的、类似于人类的通信，并消除最终用户的手动语言选择。

String textToPlay = "Hi, how can I help you today?";
var playSource = new TextSource()
    .setText(textToPlay)
    .setVoiceName("en-US-ElizabethNeural");

var recognizeOptions = new CallMediaRecognizeSpeechOptions(participant, Duration.ofSeconds(15))
    .setPlayPrompt(playSource)
    .setInterruptPrompt(false)
    .setInitialSilenceTimeout(Duration.ofSeconds(15))
    .setSentimentAnalysisEnabled(true)
    .setSpeechLanguages("en-US", "es-ES", "hi-IN")
    .setOperationContext("OpenQuestionSpeech")
    // Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    .setSpeechRecognitionModelEndpointId("YourCustomSpeechModelEndpointID");

var recognizeResponse = callAutomationClient.getCallConnectionAsync(callConnectionId)
    .getCallMediaAsync()
    .startRecognizingWithResponse(recognizeOptions)
    .block();

情绪分析（预览版）

识别 API 支持使用语音输入时进行的情绪分析。 实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 它还可用于路由、个性化或分析。

String textToPlay = "Hi, how can I help you today?";
var playSource = new TextSource()
    .setText(textToPlay)
    .setVoiceName("en-US-ElizabethNeural");

var recognizeOptions = new CallMediaRecognizeSpeechOptions(participant, Duration.ofSeconds(15))
    .setPlayPrompt(playSource)
    .setInterruptPrompt(false)
    .setInitialSilenceTimeout(Duration.ofSeconds(15))
    .setSentimentAnalysisEnabled(true)
    .setSpeechLanguages("en-US", "es-ES", "hi-IN")
    .setOperationContext("SpeechContext");

var recognizeResponse = callAutomationClient.getCallConnectionAsync(callConnectionId)
    .getCallMediaAsync()
    .startRecognizingWithResponse(recognizeOptions)
    .block();

接收识别事件更新

开发人员可以订阅有关已注册 webhook 回调的 RecognizeCompleted 和 RecognizeFailed 事件。 在应用程序中将此回调与业务逻辑配合使用可以确定当其中一个事件发生时要采取的后续步骤。

如何反序列化 RecognizeCompleted 事件的示例：

if (acsEvent instanceof RecognizeCompleted) { 
    RecognizeCompleted event = (RecognizeCompleted) acsEvent; 
    RecognizeResult recognizeResult = event.getRecognizeResult().get(); 
    if (recognizeResult instanceof DtmfResult) { 
        // Take action on collect tones 
        DtmfResult dtmfResult = (DtmfResult) recognizeResult; 
        List<DtmfTone> tones = dtmfResult.getTones(); 
        log.info("Recognition completed, tones=" + tones + ", context=" + event.getOperationContext()); 
    } else if (recognizeResult instanceof ChoiceResult) { 
        ChoiceResult collectChoiceResult = (ChoiceResult) recognizeResult; 
        String labelDetected = collectChoiceResult.getLabel(); 
        String phraseDetected = collectChoiceResult.getRecognizedPhrase();
        String languageIdentified = collectChoiceResult.getLanguageIdentified();
        log.info("Recognition completed, labelDetected=" + labelDetected + ", phraseDetected=" + phraseDetected + ", context=" + event.getOperationContext());
        log.info("Language Identified: " + languageIdentified);
        if (choiceResult.getSentimentAnalysisResult() != null) {
            log.info("Sentiment: " + choiceResult.getSentimentAnalysisResult().getSentiment());
        }
    } else if (recognizeResult instanceof SpeechResult) { 
        SpeechResult speechResult = (SpeechResult) recognizeResult; 
        String text = speechResult.getSpeech();
        String languageIdentified = speechResult.getLanguageIdentified();
        log.info("Recognition completed, text=" + text + ", context=" + event.getOperationContext());
        log.info("Language Identified: " + languageIdentified);
        if (speechResult.getSentimentAnalysisResult() != null) {
            log.info("Sentiment: " + speechResult.getSentimentAnalysisResult().getSentiment());
        }
    } else { 
        log.info("Recognition completed, result=" + recognizeResult + ", context=" + event.getOperationContext()); 
    } 
} 

如何反序列化 RecognizeFailed 事件的示例：

if (acsEvent instanceof RecognizeFailed) { 
    RecognizeFailed event = (RecognizeFailed) acsEvent; 
    if (ReasonCode.Recognize.INITIAL_SILENCE_TIMEOUT.equals(event.getReasonCode())) { 
        // Take action for time out 
        log.info("Recognition failed: initial silence time out"); 
    } else if (ReasonCode.Recognize.SPEECH_OPTION_NOT_MATCHED.equals(event.getReasonCode())) { 
        // Take action for option not matched 
        log.info("Recognition failed: speech option not matched"); 
    } else if (ReasonCode.Recognize.DMTF_OPTION_MATCHED.equals(event.getReasonCode())) { 
        // Take action for incorrect tone 
        log.info("Recognition failed: incorrect tone detected"); 
    } else { 
        log.info("Recognition failed, result=" + event.getResultInformation().getMessage() + ", context=" + event.getOperationContext()); 
    } 
} 

如何反序列化 RecognizeCanceled 事件的示例：

if (acsEvent instanceof RecognizeCanceled) { 
    RecognizeCanceled event = (RecognizeCanceled) acsEvent; 
    log.info("Recognition canceled, context=" + event.getOperationContext()); 
}

Prerequisites

• 具有有效订阅的 Azure 帐户。有关详细信息，请参阅免费创建帐户。
• Azure 通信服务资源。 请参阅创建 Azure 通信服务资源。 请注意此资源的连接字符串。
• 使用通话自动化 SDK 创建新的 Web 服务应用程序。
• 安装 Node.js，可以从其官方网站进行安装。

对于 AI 功能

• 创建 Azure AI 服务并将其连接到 Azure 通信服务资源。
• 为 Azure AI 服务资源创建自定义子域。

技术规范

以下参数可用于自定义 Recognize 函数：

创建新的 JavaScript 应用程序

在项目目录中创建新的 JavaScript 应用程序。 使用以下命令，初始化新的 Node.js 项目。 这会为项目创建一个 package.json 文件，该文件管理项目的依赖项。

npm init -y

安装 Azure 通信服务呼叫自动化包

npm install @azure/communication-call-automation

例如，在项目目录中创建新的 JavaScript 文件并将其命名为 app.js。 在此文件中编写 JavaScript 代码。

通过以下命令，使用 Node.js 运行你的应用程序。

node app.js

建立呼叫

至此，你应已熟悉如何开始通话。 有关发起通话的详细信息，请参阅快速入门：发起传出通话。

调用识别动作

在你的应用程序应答呼叫时，你可以提供有关识别参与者输入和播放提示的信息。

DTMF

const maxTonesToCollect = 3; 
const textToPlay = "Welcome to Contoso, please enter 3 DTMF."; 
const playSource: TextSource = { text: textToPlay, voiceName: "en-US-ElizabethNeural", kind: "textSource" }; 
const recognizeOptions: CallMediaRecognizeDtmfOptions = { 
    maxTonesToCollect: maxTonesToCollect, 
    initialSilenceTimeoutInSeconds: 30, 
    playPrompt: playSource, 
    interToneTimeoutInSeconds: 5, 
    interruptPrompt: true, 
    stopDtmfTones: [ DtmfTone.Pound ], 
    kind: "callMediaRecognizeDtmfOptions" 
}; 

await callAutomationClient.getCallConnection(callConnectionId) 
    .getCallMedia() 
    .startRecognizing(targetParticipant, recognizeOptions); 

对于语音转文本流，通话自动化识别操作还支持使用自定义语音识别模型。 在生成需要侦听默认语音转文本模型可能无法理解的复杂字词的应用程序时，自定义语音识别模型等功能可能非常有用。 例如，在为远程医疗行业生成应用程序，并且虚拟代理需要能够识别医疗术语时。 可以在创建自定义语音识别项目中了解详细信息。

语音转文本选择

const choices = [ 
    {  
        label: "Confirm", 
        phrases: [ "Confirm", "First", "One" ], 
        tone: DtmfTone.One 
    }, 
    { 
        label: "Cancel", 
        phrases: [ "Cancel", "Second", "Two" ], 
        tone: DtmfTone.Two 
    } 
]; 

const textToPlay = "Hello, This is a reminder for your appointment at 2 PM, Say Confirm to confirm your appointment or Cancel to cancel the appointment. Thank you!"; 
const playSource: TextSource = { text: textToPlay, voiceName: "en-US-ElizabethNeural", kind: "textSource" }; 
const recognizeOptions: CallMediaRecognizeChoiceOptions = { 
    choices: choices, 
    interruptPrompt: true, 
    initialSilenceTimeoutInSeconds: 30, 
    playPrompt: playSource, 
    operationContext: "AppointmentReminderMenu", 
    kind: "callMediaRecognizeChoiceOptions",
    //Only add the speechRecognitionModelEndpointId if you have a custom speech model you would like to use
    speechRecognitionModelEndpointId: "YourCustomSpeechEndpointId"
}; 

await callAutomationClient.getCallConnection(callConnectionId) 
    .getCallMedia() 
    .startRecognizing(targetParticipant, recognizeOptions); 

语音转文本

const textToPlay = "Hi, how can I help you today?"; 
const playSource: TextSource = { text: textToPlay, voiceName: "en-US-ElizabethNeural", kind: "textSource" }; 
const recognizeOptions: CallMediaRecognizeSpeechOptions = { 
    endSilenceTimeoutInSeconds: 1, 
    playPrompt: playSource, 
    operationContext: "OpenQuestionSpeech", 
    kind: "callMediaRecognizeSpeechOptions",
    //Only add the speechRecognitionModelEndpointId if you have a custom speech model you would like to use
    speechRecognitionModelEndpointId: "YourCustomSpeechEndpointId"
}; 

await callAutomationClient.getCallConnection(callConnectionId) 
    .getCallMedia() 
    .startRecognizing(targetParticipant, recognizeOptions); 

语音转文本或 DTMF

const maxTonesToCollect = 1; 
const textToPlay = "Hi, how can I help you today, you can press 0 to speak to an agent?"; 
const playSource: TextSource = { text: textToPlay, voiceName: "en-US-ElizabethNeural", kind: "textSource" }; 
const recognizeOptions: CallMediaRecognizeSpeechOrDtmfOptions = { 
    maxTonesToCollect: maxTonesToCollect, 
    endSilenceTimeoutInSeconds: 1, 
    playPrompt: playSource, 
    initialSilenceTimeoutInSeconds: 30, 
    interruptPrompt: true, 
    operationContext: "OpenQuestionSpeechOrDtmf", 
    kind: "callMediaRecognizeSpeechOrDtmfOptions",
    //Only add the speechRecognitionModelEndpointId if you have a custom speech model you would like to use
    speechRecognitionModelEndpointId: "YourCustomSpeechEndpointId"
}; 

await callAutomationClient.getCallConnection(callConnectionId) 
    .getCallMedia() 
    .startRecognizing(targetParticipant, recognizeOptions); 

实时语言识别（预览版）

借助额外的实时语言识别，开发人员可以自动检测口语，以启用自然的、类似于人类的通信，并消除最终用户的手动语言选择。

const textToPlay = "Hi, how can I help you today?";
const playSource: TextSource = {
  text: textToPlay,
  voiceName: "en-US-ElizabethNeural",
  kind: "textSource"
};
const recognizeOptions: CallMediaRecognizeSpeechOptions = {
  endSilenceTimeoutInSeconds: 30,
  playPrompt: playSource,
  operationContext: "speechContext",
  kind: "callMediaRecognizeSpeechOptions",
  // Enable Language Identification
  speechLanguages: ["en-US", "hi-IN", "fr-FR"],
  // Only add the speechRecognitionModelEndpointId if you have a custom speech model you would like to use
  speechRecognitionModelEndpointId: "YourCustomSpeechEndpointId"
};
await callAutomationClient.getCallConnection(callConnectionId)
  .getCallMedia()
  .startRecognizing(targetParticipant, recognizeOptions);

情绪分析（预览版）

识别 API 支持使用语音输入时进行的情绪分析。 实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 它还可用于路由、个性化或分析。

const textToPlay = "Hi, how can I help you today?";
const playSource: TextSource = {
  text: textToPlay,
  voiceName: "en-US-ElizabethNeural",
  kind: "textSource"
};

const recognizeOptions: CallMediaRecognizeSpeechOptions = {
  endSilenceTimeoutInSeconds: 30,
  playPrompt: playSource,
  operationContext: "speechContext",
  kind: "callMediaRecognizeSpeechOptions",

  // Enable Sentiment Analysis
  enableSentimentAnalysis: true
};

await callAutomationClient.getCallConnection(callConnectionId)
  .getCallMedia()
  .startRecognizing(targetParticipant, recognizeOptions);

接收识别事件更新

开发人员可以订阅有关已注册 webhook 回调的 RecognizeCompleted 和 RecognizeFailed 事件。 在应用程序中将此回调与业务逻辑配合使用可以确定当其中一个事件发生时要采取的后续步骤。

如何反序列化 RecognizeCompleted 事件的示例：

if (event.type === "Microsoft.Communication.RecognizeCompleted") {
  console.log("Received RecognizeCompleted event");

  const callConnectionId = eventData.callConnectionId;

  if (eventData.recognitionType === "choices") {
    const labelDetected = eventData.choiceResult.label;
    console.log(`Detected label: ${labelDetected}`);
    console.log("Choice Result:", JSON.stringify(eventData.choiceResult, null, 2));
    console.log(`Language Identified: ${eventData.choiceResult.languageIdentified}`);

    if (eventData.choiceResult?.sentimentAnalysisResult !== undefined) {
      console.log(`Sentiment: ${eventData.choiceResult.sentimentAnalysisResult.sentiment}`);
    }
  }

  if (eventData.recognitionType === "dtmf") {
    const tones = eventData.dtmfResult.tones;
    console.log(`DTMF Tones: ${tones}`);
    console.log(`Current Context: ${eventData.operationContext}`);
  }

  if (eventData.recognitionType === "speech") {
    const text = eventData.speechResult.speech;
    console.log(`Recognition completed, text: ${text}, context: ${eventData.operationContext}`);
    console.log(`Language Identified: ${eventData.speechResult.languageIdentified}`);

    if (eventData.speechResult?.sentimentAnalysisResult !== undefined) {
      console.log(`Sentiment: ${eventData.speechResult.sentimentAnalysisResult.sentiment}`);
    }
  }
}

如何反序列化 RecognizeFailed 事件的示例：

if (event.type === "Microsoft.Communication.RecognizeFailed") {
    console.log("Recognize failed: data=%s", JSON.stringify(eventData, null, 2));
}

如何反序列化 RecognizeCanceled 事件的示例：

if (event.type === "Microsoft.Communication.RecognizeCanceled") {
    console.log("Recognize canceled, context=%s", eventData.operationContext);
}

Prerequisites

• 具有有效订阅的 Azure 帐户。有关详细信息，请参阅免费创建帐户。
• Azure 通信服务资源。 请参阅创建 Azure 通信服务资源。 请注意此资源的连接字符串。
• 使用通话自动化 SDK 创建新的 Web 服务应用程序。
• 从 Python.org 安装 Python。

对于 AI 功能

• 创建 Azure AI 服务并将其连接到 Azure 通信服务资源。
• 为 Azure AI 服务资源创建自定义子域。

技术规范

以下参数可用于自定义 Recognize 函数：

创建新的 Python 应用程序

为你的项目设置 Python 虚拟环境

python -m venv play-audio-app

激活虚拟环境

在 Windows 上使用以下命令：

.\ play-audio-quickstart \Scripts\activate

在 Unix 上，使用以下命令：

source play-audio-quickstart /bin/activate

安装 Azure 通信服务呼叫自动化包

pip install azure-communication-callautomation

例如，在项目目录中创建应用程序文件并将其命名为 app.py。 在此文件中编写 Python 代码。

通过以下命令，使用 Python 运行你的应用程序。

python app.py

建立呼叫

至此，你应已熟悉如何开始通话。 有关发起通话的详细信息，请参阅快速入门：发起传出通话。

调用识别动作

在你的应用程序应答呼叫时，你可以提供有关识别参与者输入和播放提示的信息。

DTMF

max_tones_to_collect = 3 
text_to_play = "Welcome to Contoso, please enter 3 DTMF." 
play_source = TextSource(text=text_to_play, voice_name="en-US-ElizabethNeural") 
call_automation_client.get_call_connection(call_connection_id).start_recognizing_media( 
    dtmf_max_tones_to_collect=max_tones_to_collect, 
    input_type=RecognizeInputType.DTMF, 
    target_participant=target_participant, 
    initial_silence_timeout=30, 
    play_prompt=play_source, 
    dtmf_inter_tone_timeout=5, 
    interrupt_prompt=True, 
    dtmf_stop_tones=[ DtmfTone.Pound ]) 

对于语音转文本流，通话自动化识别操作还支持使用自定义语音识别模型。 在生成需要侦听默认语音转文本模型可能无法理解的复杂字词的应用程序时，自定义语音识别模型等功能可能非常有用。 例如，在为远程医疗行业生成应用程序，并且虚拟代理需要能够识别医疗术语时。 可以在创建自定义语音识别项目中了解详细信息。

语音转文本选择

choices = [ 
    RecognitionChoice( 
        label="Confirm", 
        phrases=[ "Confirm", "First", "One" ], 
        tone=DtmfTone.ONE 
    ), 
    RecognitionChoice( 
        label="Cancel", 
        phrases=[ "Cancel", "Second", "Two" ], 
        tone=DtmfTone.TWO 
    ) 
] 
text_to_play = "Hello, This is a reminder for your appointment at 2 PM, Say Confirm to confirm your appointment or Cancel to cancel the appointment. Thank you!" 
play_source = TextSource(text=text_to_play, voice_name="en-US-ElizabethNeural") 
call_automation_client.get_call_connection(call_connection_id).start_recognizing_media( 
    input_type=RecognizeInputType.CHOICES, 
    target_participant=target_participant, 
    choices=choices, 
    interrupt_prompt=True, 
    initial_silence_timeout=30, 
    play_prompt=play_source, 
    operation_context="AppointmentReminderMenu",
    # Only add the speech_recognition_model_endpoint_id if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id="YourCustomSpeechModelEndpointId")  

语音转文本

text_to_play = "Hi, how can I help you today?" 
play_source = TextSource(text=text_to_play, voice_name="en-US-ElizabethNeural") 
call_automation_client.get_call_connection(call_connection_id).start_recognizing_media( 
    input_type=RecognizeInputType.SPEECH, 
    target_participant=target_participant, 
    end_silence_timeout=1, 
    play_prompt=play_source, 
    operation_context="OpenQuestionSpeech",
    # Only add the speech_recognition_model_endpoint_id if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id="YourCustomSpeechModelEndpointId") 

语音转文本或 DTMF

max_tones_to_collect = 1 
text_to_play = "Hi, how can I help you today, you can also press 0 to speak to an agent." 
play_source = TextSource(text=text_to_play, voice_name="en-US-ElizabethNeural") 
call_automation_client.get_call_connection(call_connection_id).start_recognizing_media( 
    dtmf_max_tones_to_collect=max_tones_to_collect, 
    input_type=RecognizeInputType.SPEECH_OR_DTMF, 
    target_participant=target_participant, 
    end_silence_timeout=1, 
    play_prompt=play_source, 
    initial_silence_timeout=30, 
    interrupt_prompt=True, 
    operation_context="OpenQuestionSpeechOrDtmf",
    # Only add the speech_recognition_model_endpoint_id if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id="YourCustomSpeechModelEndpointId")  
app.logger.info("Start recognizing") 

实时语言识别（预览版）

借助额外的实时语言识别，开发人员可以自动检测口语，以启用自然的、类似于人类的通信，并消除最终用户的手动语言选择。

text_to_play = "Hi, how can I help you today?"
play_source = TextSource(text=text_to_play, voice_name="en-US-ElizabethNeural")

connection_client = call_automation_client.get_call_connection(call_connection_id)

recognize_result = await connection_client.start_recognizing_media(
    input_type=RecognizeInputType.SPEECH,
    target_participant=PhoneNumberIdentifier(caller_id),
    end_silence_timeout=15,
    play_prompt=play_source,
    operation_context="OpenQuestionSpeech",

    # Enable language identification
    speech_language=["en-US", "es-ES", "hi-IN"],

    # Only add the speech_recognition_model_endpoint_id if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id="YourCustomSpeechModelEndpointId"
)

情绪分析（预览版）

识别 API 支持使用语音输入时进行的情绪分析。 实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 它还可用于路由、个性化或分析。

text_to_play = "Hi, how can I help you today?"
play_source = TextSource(text=text_to_play, voice_name="en-US-ElizabethNeural")

connection_client = call_automation_client.get_call_connection(call_connection_id)

recognize_result = await connection_client.start_recognizing_media(
    input_type=RecognizeInputType.SPEECH,
    target_participant=PhoneNumberIdentifier(caller_id),
    end_silence_timeout=15,
    play_prompt=play_source,
    operation_context="OpenQuestionSpeech",
    
    # Enable sentiment analysis
    IsSentimentAnalysisEnabled = true,

    # Only add the speech_recognition_model_endpoint_id if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id="YourCustomSpeechModelEndpointId"
)

接收识别事件更新

开发人员可以订阅有关已注册 webhook 回调的 RecognizeCompleted 和 RecognizeFailed 事件。 在应用程序中将此回调与业务逻辑配合使用可以确定当其中一个事件发生时要采取的后续步骤。

如何反序列化 RecognizeCompleted 事件的示例：

if event.type == "Microsoft.Communication.RecognizeCompleted":
    print(f"Received RecognizeCompleted event for connection id: {call_connection_id}")

    recognition_type = event.data.get("recognitionType")

    if recognition_type == "dtmf":
        tones = event.data["dtmfResult"]["tones"]
        context = event.data["operationContext"]
        print(f"Recognition completed, tones={tones}, context={context}")

    elif recognition_type == "choices":
        choice_result = event.data["choiceResult"]
        label_detected = choice_result["label"]
        phrase_detected = choice_result["recognizedPhrase"]
        language_identified = choice_result.get("languageIdentified")
        sentiment = choice_result.get("sentimentAnalysisResult", {}).get("sentiment")

        print(f"Recognition completed, labelDetected={label_detected}, phraseDetected={phrase_detected}, context={event.data['operationContext']}")
        print(f"Language Identified: {language_identified}")
        print(f"Sentiment: {sentiment}")

    elif recognition_type == "speech":
        speech_result = event.data["speechResult"]
        text = speech_result["speech"]
        language_identified = speech_result.get("languageIdentified")
        sentiment = speech_result.get("sentimentAnalysisResult", {}).get("sentiment")

        print(f"Recognition completed, text={text}, context={event.data['operationContext']}")
        print(f"Language Identified: {language_identified}")
        print(f"Sentiment: {sentiment}")

    else:
        print(f"Recognition completed: data={event.data}")

如何反序列化 RecognizeFailed 事件的示例：

if event.type == "Microsoft.Communication.RecognizeFailed": 
    app.logger.info("Recognize failed: data=%s", event.data); 

如何反序列化 RecognizeCanceled 事件的示例：

if event.type == "Microsoft.Communication.RecognizeCanceled":
    # Handle the RecognizeCanceled event according to your application logic