将实时听录添加到应用程序中

创建通话并提供听录详细信息

为 ACS 定义 TranscriptionOptions，以指定何时开始听录、听录的区域设置以及用于发送口述文本的 Websocket 连接。

var createCallOptions = new CreateCallOptions(callInvite, callbackUri)
{
    CallIntelligenceOptions = new CallIntelligenceOptions() { CognitiveServicesEndpoint = new Uri(cognitiveServiceEndpoint) },
    TranscriptionOptions = new TranscriptionOptions(new Uri(""), "en-US", false, TranscriptionTransport.Websocket)
};
CreateCallResult createCallResult = await callAutomationClient.CreateCallAsync(createCallOptions);

情绪分析（预览版）

实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 提供公共预览版，可通过 createCall、answerCall 和 startTranscription 使用。

创建已启用情绪分析的呼叫

// Define transcription options with sentiment analysis enabled
var transcriptionOptions = new TranscriptionOptions
{
    IsSentimentAnalysisEnabled = true
};

var callIntelligenceOptions = new CallIntelligenceOptions
{
    CognitiveServicesEndpoint = new Uri(cognitiveServiceEndpoint)
};

var createCallOptions = new CreateCallOptions(callInvite, new Uri("https://test"))
{
    CallIntelligenceOptions = callIntelligenceOptions,
    TranscriptionOptions = transcriptionOptions
};

CreateCallResult createCallResult = await callAutomationClient.CreateCallAsync(createCallOptions);

接听已启用情绪分析的呼叫

// Define transcription options with sentiment analysis enabled
var transcriptionOptions = new TranscriptionOptions
{
    IsSentimentAnalysisEnabled = true
};

var answerCallOptions = new AnswerCallOptions(incomingCallContext, callbackUri)
{
    TranscriptionOptions = transcriptionOptions
};

var answerCallResult = await client.AnswerCallAsync(answerCallOptions);

PII 编修（预览版）

自动识别和屏蔽敏感信息（例如姓名、地址或标识号），以确保隐私和法规合规性。 在 createCall、answerCall 和 startTranscription 中可用。

接听已启用 PII 编修的呼叫

var transcriptionOptions = new TranscriptionOptions 
{ 
   PiiRedactionOptions = new PiiRedactionOptions 
   { 
       IsEnabled = true, 
       RedactionType = RedactionType.MaskWithCharacter 
   },  
}; 
 
var options = new AnswerCallOptions(incomingCallContext, callbackUri) 
{ 
   TranscriptionOptions = transcriptionOptions, 
}; 
 
//Answer call request 
var answerCallResult = await client.AnswerCallAsync(options); 

实时语言检测（预览版）

自动检测口语，以实现自然、类人的沟通并消除手动语言选择。 在 createCall、answerCall 和 startTranscription 中可用。

创建已启用实时语言检测的呼叫

var transcriptionOptions = new TranscriptionOptions 
{ 
   Locales = new List<string> { "en-US", "fr-FR", "hi-IN" }
};

var createCallOptions = new CreateCallOptions(callInviteOption, new Uri("https://test")) 
{ 
    TranscriptionOptions = transcriptionOptions 
}; 
 
//CreateCall request 
var createCallRequest = await client.CreateCallAsync(createCallOptions);

连接到会议室通话并提供听录详细信息

如果要连接到 ACS 会议室并想要使用听录，请按如下所示配置听录选项：

var transcriptionOptions = new TranscriptionOptions(
    transportUri: new Uri(""),
    locale: "en-US", 
    startTranscription: false,
    transcriptionTransport: TranscriptionTransport.Websocket,
    //Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    SpeechRecognitionModelEndpointId = "YourCustomSpeechRecognitionModelEndpointId"
);

var connectCallOptions = new ConnectCallOptions(new RoomCallLocator("roomId"), callbackUri)
{
    CallIntelligenceOptions = new CallIntelligenceOptions() 
    { 
        CognitiveServicesEndpoint = new Uri(cognitiveServiceEndpoint) 
    },
    TranscriptionOptions = transcriptionOptions
};

var connectResult = await client.ConnectCallAsync(connectCallOptions);

开始听录

准备好开始听录后，可以明确调用通话自动化来开始听录通话。

// Start transcription with options
var transcriptionOptions = new StartTranscriptionOptions
{
    OperationContext = "startMediaStreamingContext",
    IsSentimentAnalysisEnabled = true,

    // Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    SpeechRecognitionModelEndpointId = "YourCustomSpeechRecognitionModelEndpointId"
};

// Start transcription
await callMedia.StartTranscriptionAsync(transcriptionOptions);

// Alternative: Start transcription without options
// await callMedia.StartTranscriptionAsync();

获取通话中途汇总（预览版）

使用实时汇总功能增强呼叫工作流。 通过在听录选项中启用汇总，ACS 可在通话中途自动生成简洁的回顾（包括决策、操作项和关键讨论点），而无需等待通话结束。 这有助于团队保持一致，并在实时对话期间更快跟进。

// Define transcription options with call summarization enabled
var transcriptionOptions = new TranscriptionOptions
{
    SummarizationOptions = new SummarizationOptions
    {
        Locale = "en-US"
    }
};

// Answer call with transcription options
var answerCallOptions = new AnswerCallOptions(incomingCallContext, callbackUri)
{
    TranscriptionOptions = transcriptionOptions
};

var answerCallResult = await client.AnswerCallAsync(answerCallOptions);

其他标头：

相关 ID 和呼叫连接 ID 现在包含在 WebSocket 标头中，以提高可跟踪性，x-ms-call-correlation-id 和 x-ms-call-connection-id。

接收听录流

听录开始时，Websocket 会接收听录元数据有效负载，将其作为第一个数据包。

{
    "kind": "TranscriptionMetadata",
    "transcriptionMetadata": {
        "subscriptionId": "aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e",
        "locale": "en-us",
        "callConnectionId": "65c57654=f12c-4975-92a4-21668e61dd98",
        "correlationId": "65c57654=f12c-4975-92a4-21668e61dd98"
    }
}

接收听录数据

在元数据之后，Websocket 接收的下一个数据包将是听录音频的听录数据。

{
    "kind": "TranscriptionData",
    "transcriptionData": {
        "text": "Testing transcription.",
        "format": "display",
        "confidence": 0.695223331451416,
        "offset": 2516998782481234400,
        "words": [
            {
                "text": "testing",
                "offset": 2516998782481234400
            },
            {
                "text": "testing",
                "offset": 2516998782481234400
            }
        ],
        "participantRawID": "8:acs:",
        "resultStatus": "Final"
    }
}

在已启用 AI 功能的情况下接收听录流（预览版）

在通话期间启用听录时，Azure 通信服务会发出描述听录会话的配置和上下文的元数据。 其中包括区域设置、呼叫连接 ID、情绪分析设置和 PII 编修首选项等详细信息。 开发人员可以使用此有效负载来验证听录设置、审核配置或排查与 AI 增强的实时听录功能相关的问题。

{
  "kind": "TranscriptionMetadata",
  "transcriptionMetadata": {
    "subscriptionId": "863b5e55-de0d-4fc3-8e58-2d68e976b5ad",
    "locale": "en-US",
    "callConnectionId": "02009180-9dc2-429b-a3eb-d544b7b6a0e1",
    "correlationId": "62c8215b-5276-4d3c-bb6d-06a1b114651b",
    "speechModelEndpointId": null,
    "locales": [],
    "enableSentimentAnalysis": true,
    "piiRedactionOptions": {
      "enable": true,
      "redactionType": "MaskWithCharacter"
    }
  }
}

在已启用 AI 功能的情况下接收听录数据（预览版）

初始元数据数据包后，WebSocket 连接将开始接收每个转录音频段的 TranscriptionData 事件。 这些数据包中包含转录的文本、置信度分数、计时信息，若已启用情绪分析和 PII 编修，则还包含这两项信息。 此数据可用于在通话期间生成实时仪表板、触发工作流或分析聊天动态。

{
  "kind": "TranscriptionData",
  "transcriptionData": {
    "text": "My date of birth is *********.",
    "format": "display",
    "confidence": 0.8726407289505005,
    "offset": 309058340,
    "duration": 31600000,
    "words": [],
    "participantRawID": "4:+917020276722",
    "resultStatus": "Final",
    "sentimentAnalysisResult": {
      "sentiment": "neutral"
    }
  }
}

处理 Websocket 服务器中的听录流

using WebServerApi;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();
app.UseWebSockets();
app.Map("/ws", async context =>
{
    if (context.WebSockets.IsWebSocketRequest)
    {
        using var webSocket = await context.WebSockets.AcceptWebSocketAsync();
        await HandleWebSocket.Echo(webSocket);
    }
    else
    {
        context.Response.StatusCode = StatusCodes.Status400BadRequest;
    }
});

app.Run();

Websocket 处理程序的代码更新

using Azure.Communication.CallAutomation;
using System.Net.WebSockets;
using System.Text;

namespace WebServerApi
{
    public class HandleWebSocket
    {
        public static async Task Echo(WebSocket webSocket)
        {
            var buffer = new byte[1024 * 4];
            var receiveResult = await webSocket.ReceiveAsync(
                new ArraySegment(buffer), CancellationToken.None);

            while (!receiveResult.CloseStatus.HasValue)
            {
                string msg = Encoding.UTF8.GetString(buffer, 0, receiveResult.Count);
                var response = StreamingDataParser.Parse(msg);

                if (response != null)
                {
                    if (response is AudioMetadata audioMetadata)
                    {
                        Console.WriteLine("***************************************************************************************");
                        Console.WriteLine("MEDIA SUBSCRIPTION ID-->"+audioMetadata.MediaSubscriptionId);
                        Console.WriteLine("ENCODING-->"+audioMetadata.Encoding);
                        Console.WriteLine("SAMPLE RATE-->"+audioMetadata.SampleRate);
                        Console.WriteLine("CHANNELS-->"+audioMetadata.Channels);
                        Console.WriteLine("LENGTH-->"+audioMetadata.Length);
                        Console.WriteLine("***************************************************************************************");
                    }
                    if (response is AudioData audioData)
                    {
                        Console.WriteLine("***************************************************************************************");
                        Console.WriteLine("DATA-->"+audioData.Data);
                        Console.WriteLine("TIMESTAMP-->"+audioData.Timestamp);
                        Console.WriteLine("IS SILENT-->"+audioData.IsSilent);
                        Console.WriteLine("***************************************************************************************");
                    }

                    if (response is TranscriptionMetadata transcriptionMetadata)
                    {
                        Console.WriteLine("***************************************************************************************");
                        Console.WriteLine("TRANSCRIPTION SUBSCRIPTION ID-->"+transcriptionMetadata.TranscriptionSubscriptionId);
                        Console.WriteLine("LOCALE-->"+transcriptionMetadata.Locale);
                        Console.WriteLine("CALL CONNECTION ID--?"+transcriptionMetadata.CallConnectionId);
                        Console.WriteLine("CORRELATION ID-->"+transcriptionMetadata.CorrelationId);
                        Console.WriteLine("LOCALES-->" + transcriptionMetadata.Locales);  
                        Console.WriteLine("PII REDACTION OPTIONS ISENABLED-->" + transcriptionMetadata.PiiRedactionOptions?.IsEnabled);  
                        Console.WriteLine("PII REDACTION OPTIONS - REDACTION TYPE-->" + transcriptionMetadata.PiiRedactionOptions?.RedactionType); 
                        Console.WriteLine("***************************************************************************************");
                    }
                    if (response is TranscriptionData transcriptionData)
                    {
                        Console.WriteLine("***************************************************************************************");
                        Console.WriteLine("TEXT-->"+transcriptionData.Text);
                        Console.WriteLine("FORMAT-->"+transcriptionData.Format);
                        Console.WriteLine("OFFSET-->"+transcriptionData.Offset);
                        Console.WriteLine("DURATION-->"+transcriptionData.Duration);
                        Console.WriteLine("PARTICIPANT-->"+transcriptionData.Participant.RawId);
                        Console.WriteLine("CONFIDENCE-->"+transcriptionData.Confidence);
                        Console.WriteLine("SENTIMENT ANALYSIS RESULT-->" + transcriptionData.SentimentAnalysisResult?.Sentiment);

                        foreach (var word in transcriptionData.Words)
                        {
                            Console.WriteLine("TEXT-->"+word.Text);
                            Console.WriteLine("OFFSET-->"+word.Offset);
                            Console.WriteLine("DURATION-->"+word.Duration);
                        }
                        Console.WriteLine("***************************************************************************************");
                    }
                }

                await webSocket.SendAsync(
                    new ArraySegment(buffer, 0, receiveResult.Count),
                    receiveResult.MessageType,
                    receiveResult.EndOfMessage,
                    CancellationToken.None);

                receiveResult = await webSocket.ReceiveAsync(
                    new ArraySegment(buffer), CancellationToken.None);
            }

            await webSocket.CloseAsync(
                receiveResult.CloseStatus.Value,
                receiveResult.CloseStatusDescription,
                CancellationToken.None);
        }
    }
}

更新听录

如果应用程序允许用户选择其首选语言，你可能还需要捕获该语言版本的听录。 为此，通话自动化 SDK 允许更新听录区域设置。

UpdateTranscriptionOptions updateTranscriptionOptions = new UpdateTranscriptionOptions(locale)
{
OperationContext = "UpdateTranscriptionContext",
//Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
SpeechRecognitionModelEndpointId = "YourCustomSpeechRecognitionModelEndpointId"
};

await client.GetCallConnection(callConnectionId).GetCallMedia().UpdateTranscriptionAsync(updateTranscriptionOptions);

停止听录

当应用程序需要停止收听听录时，可以使用 StopTranscription 请求让通话自动化知道停止向 websocket 发送听录数据。

StopTranscriptionOptions stopOptions = new StopTranscriptionOptions()
{
    OperationContext = "stopTranscription"
};

await callMedia.StopTranscriptionAsync(stopOptions);

创建通话并提供听录详细信息

为 ACS 定义 TranscriptionOptions，以指定何时开始听录、听录的区域设置以及用于发送口述文本的 Websocket 连接。

CallInvite callInvite = new CallInvite(target, caller); 

CallIntelligenceOptions callIntelligenceOptions = new CallIntelligenceOptions()
    .setCognitiveServicesEndpoint(appConfig.getCognitiveServiceEndpoint()); 

TranscriptionOptions transcriptionOptions = new TranscriptionOptions(
    appConfig.getWebSocketUrl(), 
    TranscriptionTransport.WEBSOCKET, 
    "en-US", 
    false,
    "your-endpoint-id-here" // speechRecognitionEndpointId
); 

CreateCallOptions createCallOptions = new CreateCallOptions(callInvite, appConfig.getCallBackUri());
createCallOptions.setCallIntelligenceOptions(callIntelligenceOptions); 
createCallOptions.setTranscriptionOptions(transcriptionOptions); 

Response result = client.createCallWithResponse(createCallOptions, Context.NONE); 
return result.getValue().getCallConnectionProperties().getCallConnectionId(); 

情绪分析（预览版）

实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 提供公共预览版，可通过 createCall、answerCall 和 startTranscription 使用。

创建已启用情绪分析的呼叫

CallInvite callInvite = new CallInvite(target, caller);

CallIntelligenceOptions callIntelligenceOptions = new CallIntelligenceOptions()
    .setCognitiveServicesEndpoint(cognitiveServicesEndpoint);

TranscriptionOptions transcriptionOptions = new TranscriptionOptions("en-ES")
    .setTransportUrl(websocketUriHost)
    .setEnableSentimentAnalysis(true) // Enable sentiment analysis
    .setLocales(locales);

CreateCallOptions createCallOptions = new CreateCallOptions(callInvite, callbackUri.toString())
    .setCallIntelligenceOptions(callIntelligenceOptions)
    .setTranscriptionOptions(transcriptionOptions);

// Create call request
Response<CreateCallResult> result = client.createCallWithResponse(createCallOptions, Context.NONE);

接听已启用情绪分析的呼叫

TranscriptionOptions transcriptionOptions = new TranscriptionOptions("en-ES")
    .setTransportUrl(websocketUriHost)
    .setEnableSentimentAnalysis(true) // Enable sentiment analysis
    .setLocales(locales);

AnswerCallOptions answerCallOptions = new AnswerCallOptions(data.getString("incomingCallContext"), callbackUri)
    .setCallIntelligenceOptions(callIntelligenceOptions)
    .setTranscriptionOptions(transcriptionOptions);

// Answer call request
Response<AnswerCallResult> answerCallResponse = client.answerCallWithResponse(answerCallOptions, Context.NONE);

PII 编修（预览版）

自动识别和屏蔽敏感信息（例如姓名、地址或标识号），以确保隐私和法规合规性。 在 createCall、answerCall 和 startTranscription 中可用。

接听已启用 PII 编修的呼叫

PiiRedactionOptions piiRedactionOptions = new PiiRedactionOptions()
    .setEnabled(true)
    .setRedactionType(RedactionType.MASK_WITH_CHARACTER);

TranscriptionOptions transcriptionOptions = new TranscriptionOptions("en-ES")
    .setTransportUrl(websocketUriHost)
    .setPiiRedactionOptions(piiRedactionOptions)
    .setLocales(locales);

AnswerCallOptions answerCallOptions = new AnswerCallOptions(data.getString("incomingCallContext"), callbackUri)
    .setCallIntelligenceOptions(callIntelligenceOptions)
    .setTranscriptionOptions(transcriptionOptions);

// Answer call request
Response<AnswerCallResult> answerCallResponse = client.answerCallWithResponse(answerCallOptions, Context.NONE);

实时语言检测（预览版）

自动检测口语，以实现自然、类人的沟通并消除手动语言选择。 在 createCall、answerCall 和 startTranscription 中可用。

创建已启用实时语言检测的呼叫

var transcriptionOptions = new TranscriptionOptions
{
    Locales = new List<string> { "en-US", "fr-FR", "hi-IN" },
};

var createCallOptions = new CreateCallOptions(callInviteOption, new Uri("https://test"))
{
    TranscriptionOptions = transcriptionOptions
};

var createCallResult = await client.CreateCallAsync(createCallOptions);

连接到会议室通话并提供听录详细信息

如果要连接到 ACS 会议室并想要使用听录，请按如下所示配置听录选项：

TranscriptionOptions transcriptionOptions = new TranscriptionOptions(
    appConfig.getWebSocketUrl(), 
    TranscriptionTransport.WEBSOCKET, 
    "en-US", 
    false,
    "your-endpoint-id-here" // speechRecognitionEndpointId
);

ConnectCallOptions connectCallOptions = new ConnectCallOptions(new RoomCallLocator("roomId"), appConfig.getCallBackUri())
    .setCallIntelligenceOptions(
        new CallIntelligenceOptions()
            .setCognitiveServicesEndpoint(appConfig.getCognitiveServiceEndpoint())
    )
    .setTranscriptionOptions(transcriptionOptions);

ConnectCallResult connectCallResult = Objects.requireNonNull(client
    .connectCallWithResponse(connectCallOptions)
    .block())
    .getValue();

开始听录

准备好开始听录后，可以明确调用通话自动化来开始听录通话。

//Option 1: Start transcription with options
StartTranscriptionOptions transcriptionOptions = new StartTranscriptionOptions()
    .setOperationContext("startMediaStreamingContext"); 

client.getCallConnection(callConnectionId)
    .getCallMedia()
    .startTranscriptionWithResponse(transcriptionOptions, Context.NONE); 

// Alternative: Start transcription without options
// client.getCallConnection(callConnectionId)
//     .getCallMedia()
//     .startTranscription();

获取通话中途汇总（预览版）

使用实时汇总功能增强呼叫工作流。 通过在听录选项中启用汇总，ACS 可在通话中途自动生成简洁的回顾（包括决策、操作项和关键讨论点），而无需等待通话结束。 这有助于团队保持一致，并在实时对话期间更快跟进。

SummarizationOptions summarizationOptions = new SummarizationOptions()
    .setEnableEndCallSummary(true)
    .setLocale("en-US");

TranscriptionOptions transcriptionOptions = new TranscriptionOptions("en-ES")
    .setTransportUrl(websocketUriHost)
    .setSummarizationOptions(summarizationOptions)
    .setLocales(locales);

AnswerCallOptions answerCallOptions = new AnswerCallOptions(data.getString("incomingCallContext"), callbackUri)
    .setCallIntelligenceOptions(callIntelligenceOptions)
    .setTranscriptionOptions(transcriptionOptions);

// Answer call request
Response<AnswerCallResult> answerCallResponse = client.answerCallWithResponse(answerCallOptions, Context.NONE);

其他标头：

相关 ID 和呼叫连接 ID 现在包含在 WebSocket 标头中，以提高可跟踪性，x-ms-call-correlation-id 和 x-ms-call-connection-id。

接收听录流

听录开始时，Websocket 会接收听录元数据有效负载，将其作为第一个数据包。

{
    "kind": "TranscriptionMetadata",
    "transcriptionMetadata": {
        "subscriptionId": "aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e",
        "locale": "en-us",
        "callConnectionId": "65c57654=f12c-4975-92a4-21668e61dd98",
        "correlationId": "65c57654=f12c-4975-92a4-21668e61dd98"
    }
}

接收听录数据

在元数据之后，Websocket 接收的下一个数据包将是听录音频的听录数据。

{
    "kind": "TranscriptionData",
    "transcriptionData": {
        "text": "Testing transcription.",
        "format": "display",
        "confidence": 0.695223331451416,
        "offset": 2516998782481234400,
        "words": [
            {
                "text": "testing",
                "offset": 2516998782481234400
            },
            {
                "text": "testing",
                "offset": 2516998782481234400
            }
        ],
        "participantRawID": "8:acs:",
        "resultStatus": "Final"
    }
}

在已启用 AI 功能的情况下接收听录流（预览版）

在通话期间启用听录时，Azure 通信服务会发出描述听录会话的配置和上下文的元数据。 其中包括区域设置、呼叫连接 ID、情绪分析设置和 PII 编修首选项等详细信息。 开发人员可以使用此有效负载来验证听录设置、审核配置或排查与 AI 增强的实时听录功能相关的问题。

{
  "kind": "TranscriptionMetadata",
  "transcriptionMetadata": {
    "subscriptionId": "863b5e55-de0d-4fc3-8e58-2d68e976b5ad",
    "locale": "en-US",
    "callConnectionId": "02009180-9dc2-429b-a3eb-d544b7b6a0e1",
    "correlationId": "62c8215b-5276-4d3c-bb6d-06a1b114651b",
    "speechModelEndpointId": null,
    "locales": [],
    "enableSentimentAnalysis": true,
    "piiRedactionOptions": {
      "enable": true,
      "redactionType": "MaskWithCharacter"
    }
  }
}

在已启用 AI 功能的情况下接收听录数据（预览版）

初始元数据数据包后，WebSocket 连接将开始接收每个转录音频段的 TranscriptionData 事件。 这些数据包中包含转录的文本、置信度分数、计时信息，若已启用情绪分析和 PII 编修，则还包含这两项信息。 此数据可用于在通话期间生成实时仪表板、触发工作流或分析聊天动态。

{
  "kind": "TranscriptionData",
  "transcriptionData": {
    "text": "My date of birth is *********.",
    "format": "display",
    "confidence": 0.8726407289505005,
    "offset": 309058340,
    "duration": 31600000,
    "words": [],
    "participantRawID": "4:+917020276722",
    "resultStatus": "Final",
    "sentimentAnalysisResult": {
      "sentiment": "neutral"
    }
  }
}

处理 Websocket 服务器中的听录流

package com.example;

import org.glassfish.tyrus.server.Server;

import java.io.BufferedReader;
import java.io.InputStreamReader;

public class App {
    public static void main(String[] args) {
        Server server = new Server("localhost", 8081, "/ws", null, WebSocketServer.class);

        try {
            server.start();
            System.out.println("Web socket running on port 8081...");
            System.out.println("wss://localhost:8081/ws/server");
            BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
            reader.readLine();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            server.stop();
        }
    }
}

Websocket 处理程序的代码更新

package com.example;

import javax.websocket.OnMessage;
import javax.websocket.Session;
import javax.websocket.server.ServerEndpoint;

import com.azure.communication.callautomation.models.streaming.StreamingData;
import com.azure.communication.callautomation.models.streaming.StreamingDataParser;
import com.azure.communication.callautomation.models.streaming.media.AudioData;
import com.azure.communication.callautomation.models.streaming.media.AudioMetadata;
import com.azure.communication.callautomation.models.streaming.transcription.TranscriptionData;
import com.azure.communication.callautomation.models.streaming.transcription.TranscriptionMetadata;
import com.azure.communication.callautomation.models.streaming.transcription.Word;

@ServerEndpoint("/server")
public class WebSocketServer {
    @OnMessage
    public void onMessage(String message, Session session) {
        StreamingData data = StreamingDataParser.parse(message);

        if (data instanceof AudioMetadata) {
            AudioMetadata audioMetaData = (AudioMetadata) data;
            System.out.println("----------------------------------------------------------------");
            System.out.println("SUBSCRIPTION ID: --> " + audioMetaData.getMediaSubscriptionId());
            System.out.println("ENCODING: --> " + audioMetaData.getEncoding());
            System.out.println("SAMPLE RATE: --> " + audioMetaData.getSampleRate());
            System.out.println("CHANNELS: --> " + audioMetaData.getChannels());
            System.out.println("LENGTH: --> " + audioMetaData.getLength());
            System.out.println("----------------------------------------------------------------");
        }

        if (data instanceof AudioData) {
            AudioData audioData = (AudioData) data;
            System.out.println("----------------------------------------------------------------");
            System.out.println("DATA: --> " + audioData.getData());
            System.out.println("TIMESTAMP: --> " + audioData.getTimestamp());
            System.out.println("IS SILENT: --> " + audioData.isSilent());
            System.out.println("----------------------------------------------------------------");
        }

        if (data instanceof TranscriptionMetadata) {
            TranscriptionMetadata transcriptionMetadata = (TranscriptionMetadata) data;
        
            System.out.println("----------------------------------------------------------------");
            System.out.println("TRANSCRIPTION SUBSCRIPTION ID: --> " + transcriptionMetadata.getTranscriptionSubscriptionId());
            System.out.println("LOCALE: --> " + transcriptionMetadata.getLocale());
            System.out.println("CALL CONNECTION ID: --> " + transcriptionMetadata.getCallConnectionId());
            System.out.println("CORRELATION ID: --> " + transcriptionMetadata.getCorrelationId());
        
            // Check for PII Redaction Options locale
            if (transcriptionMetadata.getPiiRedactionOptions() != null &&
                transcriptionMetadata.getPiiRedactionOptions().getLocale() != null) {
                System.out.println("PII Redaction Locale: --> " + transcriptionMetadata.getPiiRedactionOptions().getLocale());
            }
        
            // Check for detected locales
            if (transcriptionMetadata.getLocales() != null) {
                System.out.println("Detected Locales: --> " + transcriptionMetadata.getLocales());
            }
        
            System.out.println("----------------------------------------------------------------");
        }

        if (data instanceof TranscriptionData) {
            TranscriptionData transcriptionData = (TranscriptionData) data;
            System.out.println("----------------------------------------------------------------");
            System.out.println("TEXT: --> " + transcriptionData.getText());
            System.out.println("FORMAT: --> " + transcriptionData.getFormat());
            System.out.println("CONFIDENCE: --> " + transcriptionData.getConfidence());
            System.out.println("OFFSET: --> " + transcriptionData.getOffset());
            System.out.println("DURATION: --> " + transcriptionData.getDuration());
            System.out.println("RESULT STATUS: --> " + transcriptionData.getResultStatus());
            for (Word word : transcriptionData.getWords()) {
                System.out.println("Text: --> " + word.getText());
                System.out.println("Offset: --> " + word.getOffset());
                System.out.println("Duration: --> " + word.getDuration());
            }
            System.out.println("SENTIMENT:-->" + transcriptionData.getSentimentAnalysisResult().getSentiment()); 
            System.out.println("LANGUAGE IDENTIFIED:-->" + transcriptionData.getLanguageIdentified()); 
            System.out.println("----------------------------------------------------------------");
        }
    }
}

更新听录

如果应用程序允许用户选择其首选语言，你可能还需要捕获该语言版本的听录。 为此，通话自动化 SDK 允许更新听录区域设置。

UpdateTranscriptionOptions transcriptionOptions = new UpdateTranscriptionOptions()
    .setLocale(newLocale)
    .setOperationContext("transcriptionContext")
    .setSpeechRecognitionEndpointId("your-endpoint-id-here");

client.getCallConnection(callConnectionId)
    .getCallMedia()
    .updateTranscriptionWithResponse(transcriptionOptions, Context.NONE);

停止听录

当应用程序需要停止收听听录时，可以使用 StopTranscription 请求让通话自动化知道停止向 websocket 发送听录数据。

// Option 1: Stop transcription with options
StopTranscriptionOptions stopTranscriptionOptions = new StopTranscriptionOptions()
    .setOperationContext("stopTranscription");

client.getCallConnection(callConnectionId)
    .getCallMedia()
    .stopTranscriptionWithResponse(stopTranscriptionOptions, Context.NONE);

// Alternative: Stop transcription without options
// client.getCallConnection(callConnectionId)
//     .getCallMedia()
//     .stopTranscription();

创建通话并提供听录详细信息

为 ACS 定义 TranscriptionOptions，以指定何时开始听录、听录的区域设置以及用于发送口述文本的 Websocket 连接。

const transcriptionOptions = {
    transportUrl: "",
    transportType: "websocket",
    locale: "en-US",
    startTranscription: false,
    speechRecognitionModelEndpointId: "YOUR_CUSTOM_SPEECH_RECOGNITION_MODEL_ID"
};

const options = {
    callIntelligenceOptions: {
        cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
    },
    transcriptionOptions: transcriptionOptions
};

console.log("Placing outbound call...");
acsClient.createCall(callInvite, process.env.CALLBACK_URI + "/api/callbacks", options);

情绪分析（预览版）

实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 提供公共预览版，可通过 createCall、answerCall 和 startTranscription 使用。

创建已启用情绪分析的呼叫

const transcriptionOptions = {
    transportUrl: "",
    transportType: "websocket",
    locale: "en-US",
    startTranscription: false,
    enableSentimentAnalysis: true,
    speechRecognitionModelEndpointId: "YOUR_CUSTOM_SPEECH_RECOGNITION_MODEL_ID"
};

const options = {
    callIntelligenceOptions: {
        cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
    },
    transcriptionOptions: transcriptionOptions
};

console.log("Placing outbound call...");
acsClient.createCall(callInvite, process.env.CALLBACK_URI + "/api/callbacks", options);

接听已启用情绪分析的呼叫

const transcriptionOptions: TranscriptionOptions = {
  transportUrl: transportUrl,
  transportType: "websocket",
  startTranscription: true,
  enableSentimentAnalysis: true
};

const answerCallOptions: AnswerCallOptions = {
  callIntelligenceOptions: {
    cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
  },
  transcriptionOptions: transcriptionOptions,
  enableLoopbackAudio: true
};

await acsClient.answerCall(incomingCallContext, callbackUri, answerCallOptions);

PII 编修（预览版）

自动识别和屏蔽敏感信息（例如姓名、地址或标识号），以确保隐私和法规合规性。 在 createCall、answerCall 和 startTranscription 中可用。

接听已启用 PII 编修的呼叫

const transcriptionOptions: TranscriptionOptions = {
  transportUrl: transportUrl,
  transportType: "websocket",
  startTranscription: true,
  piiRedactionOptions: {
    enable: true,
    redactionType: "maskWithCharacter"
  }
};

const answerCallOptions: AnswerCallOptions = {
  callIntelligenceOptions: {
    cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
  },
  transcriptionOptions: transcriptionOptions,
  enableLoopbackAudio: true
};

await acsClient.answerCall(incomingCallContext, callbackUri, answerCallOptions);

实时语言检测（预览版）

自动检测口语，以实现自然、类人的沟通并消除手动语言选择。 在 createCall、answerCall 和 startTranscription 中可用。

创建已启用实时语言检测的呼叫

const transcriptionOptions: TranscriptionOptions = {
  transportUrl: transportUrl,
  transportType: "websocket",
  startTranscription: true,
  locales: ["es-ES", "en-US"]
};

const createCallOptions: CreateCallOptions = {
  callIntelligenceOptions: {
    cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
  },
  transcriptionOptions: transcriptionOptions,
  operationContext: "CreatPSTNCallContext",
  enableLoopbackAudio: true
};

连接到会议室通话并提供听录详细信息

如果要连接到 ACS 会议室并想要使用听录，请按如下所示配置听录选项：

const transcriptionOptions = {
    transportUri: "",
    locale: "en-US",
    transcriptionTransport: "websocket",
    startTranscription: false,
    speechRecognitionModelEndpointId: "YOUR_CUSTOM_SPEECH_RECOGNITION_MODEL_ID"
};

const callIntelligenceOptions = {
    cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
};

const connectCallOptions = {
    callIntelligenceOptions: callIntelligenceOptions,
    transcriptionOptions: transcriptionOptions
};

const callLocator = {
    id: roomId,
    kind: "roomCallLocator"
};

const connectResult = await client.connectCall(callLocator, callBackUri, connectCallOptions);

开始听录

准备好开始听录后，可以明确调用通话自动化来开始听录通话。

const startTranscriptionOptions = {
    locale: "en-AU",
    operationContext: "startTranscriptionContext"
};

// Start transcription with options
await callMedia.startTranscription(startTranscriptionOptions);

// Alternative: Start transcription without options
// await callMedia.startTranscription();

获取通话中途汇总（预览版）

使用实时汇总功能增强呼叫工作流。 通过在听录选项中启用汇总，ACS 可在通话中途自动生成简洁的回顾（包括决策、操作项和关键讨论点），而无需等待通话结束。 这有助于团队保持一致，并在实时对话期间更快跟进。

const transcriptionOptions: TranscriptionOptions = {
  transportUrl: transportUrl,
  transportType: "websocket",
  startTranscription: true,
  summarizationOptions: {
    enableEndCallSummary: true,
    locale: "es-ES"
  }
};

const answerCallOptions: AnswerCallOptions = {
  callIntelligenceOptions: {
    cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICES_ENDPOINT
  },
  transcriptionOptions: transcriptionOptions,
  enableLoopbackAudio: true
};

await acsClient.answerCall(incomingCallContext, callbackUri, answerCallOptions);

其他标头：

相关 ID 和呼叫连接 ID 现在包含在 WebSocket 标头中，以提高可跟踪性，x-ms-call-correlation-id 和 x-ms-call-connection-id。

接收听录流

听录开始时，Websocket 会接收听录元数据有效负载，将其作为第一个数据包。

{
    "kind": "TranscriptionMetadata",
    "transcriptionMetadata": {
        "subscriptionId": "aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e",
        "locale": "en-us",
        "callConnectionId": "65c57654=f12c-4975-92a4-21668e61dd98",
        "correlationId": "65c57654=f12c-4975-92a4-21668e61dd98"
    }
}

接收听录数据

在元数据之后，Websocket 接收的下一个数据包将是听录音频的听录数据。

{
    "kind": "TranscriptionData",
    "transcriptionData": {
        "text": "Testing transcription.",
        "format": "display",
        "confidence": 0.695223331451416,
        "offset": 2516998782481234400,
        "words": [
            {
                "text": "testing",
                "offset": 2516998782481234400
            },
            {
                "text": "testing",
                "offset": 2516998782481234400
            }
        ],
        "participantRawID": "8:acs:",
        "resultStatus": "Final"
    }
}

在已启用 AI 功能的情况下接收听录流（预览版）

在通话期间启用听录时，Azure 通信服务会发出描述听录会话的配置和上下文的元数据。 其中包括区域设置、呼叫连接 ID、情绪分析设置和 PII 编修首选项等详细信息。 开发人员可以使用此有效负载来验证听录设置、审核配置或排查与 AI 增强的实时听录功能相关的问题。

{
  "kind": "TranscriptionMetadata",
  "transcriptionMetadata": {
    "subscriptionId": "863b5e55-de0d-4fc3-8e58-2d68e976b5ad",
    "locale": "en-US",
    "callConnectionId": "02009180-9dc2-429b-a3eb-d544b7b6a0e1",
    "correlationId": "62c8215b-5276-4d3c-bb6d-06a1b114651b",
    "speechModelEndpointId": null,
    "locales": [],
    "enableSentimentAnalysis": true,
    "piiRedactionOptions": {
      "enable": true,
      "redactionType": "MaskWithCharacter"
    }
  }
}

在已启用 AI 功能的情况下接收听录数据（预览版）

初始元数据数据包后，WebSocket 连接将开始接收每个转录音频段的 TranscriptionData 事件。 这些数据包中包含转录的文本、置信度分数、计时信息，若已启用情绪分析和 PII 编修，则还包含这两项信息。 此数据可用于在通话期间生成实时仪表板、触发工作流或分析聊天动态。

{
  "kind": "TranscriptionData",
  "transcriptionData": {
    "text": "My date of birth is *********.",
    "format": "display",
    "confidence": 0.8726407289505005,
    "offset": 309058340,
    "duration": 31600000,
    "words": [],
    "participantRawID": "4:+917020276722",
    "resultStatus": "Final",
    "sentimentAnalysisResult": {
      "sentiment": "neutral"
    }
  }
}

处理 Websocket 服务器中的听录流

import WebSocket from 'ws';
import { streamingData } from '@azure/communication-call-automation/src/util/streamingDataParser';

const wss = new WebSocket.Server({ port: 8081 });

wss.on('connection', (ws) => {
  console.log('Client connected');

  ws.on('message', (packetData) => {
    const decoder = new TextDecoder();
    const stringJson = decoder.decode(packetData);
    console.log("STRING JSON =>", stringJson);

    const response = streamingData(packetData);
    const kind = response?.kind;

    if (kind === "TranscriptionMetadata") {
      console.log("--------------------------------------------");
      console.log("Transcription Metadata");
      console.log("CALL CONNECTION ID: -->", response.callConnectionId);
      console.log("CORRELATION ID: -->", response.correlationId);
      console.log("LOCALE: -->", response.locale);
      console.log("SUBSCRIPTION ID: -->", response.subscriptionId);
      console.log("SPEECH MODEL ENDPOINT: -->", response.speechRecognitionModelEndpointId);
      console.log("IS SENTIMENT ANALYSIS ENABLED: -->", response.enableSentimentAnalysis);

      if (response.piiRedactionOptions) {
        console.log("PII REDACTION ENABLED: -->", response.piiRedactionOptions.enable);
        console.log("PII REDACTION TYPE: -->", response.piiRedactionOptions.redactionType);
      }

      if (response.locales) {
        response.locales.forEach((language) => {
          console.log("LOCALE DETECTED: -->", language);
        });
      }

      console.log("--------------------------------------------");
    } else if (kind === "TranscriptionData") {
      console.log("--------------------------------------------");
      console.log("Transcription Data");
      console.log("TEXT: -->", response.text);
      console.log("FORMAT: -->", response.format);
      console.log("CONFIDENCE: -->", response.confidence);
      console.log("OFFSET IN TICKS: -->", response.offsetInTicks);
      console.log("DURATION IN TICKS: -->", response.durationInTicks);
      console.log("RESULT STATE: -->", response.resultState);

      if (response.participant?.phoneNumber) {
        console.log("PARTICIPANT PHONE NUMBER: -->", response.participant.phoneNumber);
      }

      if (response.participant?.communicationUserId) {
        console.log("PARTICIPANT USER ID: -->", response.participant.communicationUserId);
      }

      if (response.words?.length) {
        response.words.forEach((word) => {
          console.log("WORD TEXT: -->", word.text);
          console.log("WORD DURATION IN TICKS: -->", word.durationInTicks);
          console.log("WORD OFFSET IN TICKS: -->", word.offsetInTicks);
        });
      }

      if (response.sentimentAnalysisResult) {
        console.log("SENTIMENT: -->", response.sentimentAnalysisResult.sentiment);
      }

      console.log("LANGUAGE IDENTIFIED: -->", response.languageIdentified);
      console.log("--------------------------------------------");
    }
  });

  ws.on('close', () => {
    console.log('Client disconnected');
  });
});

console.log('WebSocket server running on port 8081');

更新听录

如果应用程序允许用户选择其首选语言，你可能还需要捕获该语言版本的听录。 为此，通话自动化 SDK 允许更新听录区域设置。

async function updateTranscriptionAsync() {
  const options: UpdateTranscriptionOptions = {
operationContext: "updateTranscriptionContext",
speechRecognitionModelEndpointId: "YOUR_CUSTOM_SPEECH_RECOGNITION_MODEL_ID"
  };
  await acsClient
.getCallConnection(callConnectionId)
.getCallMedia()
.updateTranscription("en-au", options);
}

停止听录

当应用程序需要停止收听听录时，可以使用 StopTranscription 请求让通话自动化知道停止向 websocket 发送听录数据。

const stopTranscriptionOptions = {
    operationContext: "stopTranscriptionContext"
};

// Stop transcription with options
await callMedia.stopTranscription(stopTranscriptionOptions);

// Alternative: Stop transcription without options
// await callMedia.stopTranscription();

创建通话并提供听录详细信息

为 ACS 定义 TranscriptionOptions，以指定何时开始听录、听录的区域设置以及用于发送口述文本的 Websocket 连接。

transcription_options = TranscriptionOptions(
    transport_url="WEBSOCKET_URI_HOST",
    transport_type=TranscriptionTransportType.WEBSOCKET,
    locale="en-US",
    start_transcription=False,
    #Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id = "YourCustomSpeechRecognitionModelEndpointId"
);
)

call_connection_properties = call_automation_client.create_call(
    target_participant,
    CALLBACK_EVENTS_URI,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    source_caller_id_number=source_caller,
    transcription=transcription_options
)

情绪分析（预览版）

实时跟踪对话的情感基调以支持客户和代理交互，并在必要时允许主管进行干预。 提供公共预览版，可通过 createCall、answerCall 和 startTranscription 使用。

创建已启用情绪分析的呼叫

transcription_options = TranscriptionOptions(
    transport_url=self.transport_url,
    transport_type=StreamingTransportType.WEBSOCKET,
    locale="en-US",
    start_transcription=False,
    enable_sentiment_analysis=True
)

call_connection_properties = await call_automation_client.create_call(
    target_participant=[target_participant],
    callback_url=CALLBACK_EVENTS_URI,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    source_caller_id_number=source_caller,
    transcription=transcription_options
)

接听已启用情绪分析的呼叫

transcription_options = TranscriptionOptions(
    transport_url=self.transport_url,
    transport_type=StreamingTransportType.WEBSOCKET,
    locale="en-US",
    start_transcription=False,
    enable_sentiment_analysis=True
)

answer_call_result = await call_automation_client.answer_call(
    incoming_call_context=incoming_call_context,
    transcription=transcription_options,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    callback_url=callback_uri,
    enable_loopback_audio=True,
    operation_context="answerCallContext"
)

PII 编修（预览版）

自动识别和屏蔽敏感信息（例如姓名、地址或标识号），以确保隐私和法规合规性。 在 createCall、answerCall 和 startTranscription 中可用。

接听已启用 PII 编修的呼叫

transcription_options = TranscriptionOptions(
    transport_url=self.transport_url,
    transport_type=StreamingTransportType.WEBSOCKET,
    locale=["en-US", "es-ES"],
    start_transcription=False,
    pii_redaction=PiiRedactionOptions(
        enable=True,
        redaction_type=RedactionType.MASK_WITH_CHARACTER
    )
)

answer_call_result = await call_automation_client.answer_call(
    incoming_call_context=incoming_call_context,
    transcription=transcription_options,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    callback_url=callback_uri,
    enable_loopback_audio=True,
    operation_context="answerCallContext"
)

实时语言检测（预览版）

自动检测口语，以实现自然、类人的沟通并消除手动语言选择。 在 createCall、answerCall 和 startTranscription 中可用。

创建已启用实时语言检测的呼叫

transcription_options = TranscriptionOptions(
    transport_url=self.transport_url,
    transport_type=StreamingTransportType.WEBSOCKET,
    locale=["en-US", "es-ES","hi-IN"],
    start_transcription=False,
    enable_sentiment_analysis=True,
)

call_connection_properties = await call_automation_client.create_call(
    target_participant=[target_participant],
    callback_url=CALLBACK_EVENTS_URI,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    source_caller_id_number=source_caller,
    transcription=transcription_options
)

连接到会议室通话并提供听录详细信息

如果要连接到 ACS 会议室并想要使用听录，请按如下所示配置听录选项：

transcription_options = TranscriptionOptions(
    transport_url="",
    transport_type=TranscriptionTransportType.WEBSOCKET,
    locale="en-US",
    start_transcription=False,
    #Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id = "YourCustomSpeechRecognitionModelEndpointId"
)

connect_result = client.connect_call(
    room_id="roomid",
    CALLBACK_EVENTS_URI,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    operation_context="connectCallContext",
    transcription=transcription_options
)

开始听录

准备好开始听录后，可以明确调用通话自动化来开始听录通话。

# Start transcription without options
call_connection_client.start_transcription()

# Option 1: Start transcription with locale and operation context
# call_connection_client.start_transcription(locale="en-AU", operation_context="startTranscriptionContext")

# Option 2: Start transcription with operation context
# call_connection_client.start_transcription(operation_context="startTranscriptionContext")

获取通话中途汇总（预览版）

使用实时汇总功能增强呼叫工作流。 通过在听录选项中启用汇总，ACS 可在通话中途自动生成简洁的回顾（包括决策、操作项和关键讨论点），而无需等待通话结束。 这有助于团队保持一致，并在实时对话期间更快跟进。

transcription_options = TranscriptionOptions(
    transport_url=self.transport_url,
    transport_type=StreamingTransportType.WEBSOCKET,
    locale="en-US",
    start_transcription=False,
    summarization=SummarizationOptions(
        enable_end_call_summary=True,
        locale="en-US"
    )
)

answer_call_result = await call_automation_client.answer_call(
    incoming_call_context=incoming_call_context,
    transcription=transcription_options,
    cognitive_services_endpoint=COGNITIVE_SERVICES_ENDPOINT,
    callback_url=callback_uri,
    enable_loopback_audio=True,
    operation_context="answerCallContext"
)


await call_connection_client.summarize_call(
    operation_context=self.operation_context,
    operation_callback_url=self.operation_callback_url,
    summarization=transcription_options.summarization
)

其他标头：

相关 ID 和呼叫连接 ID 现在包含在 WebSocket 标头中，以提高可跟踪性，x-ms-call-correlation-id 和 x-ms-call-connection-id。

接收听录流

听录开始时，Websocket 会接收听录元数据有效负载，将其作为第一个数据包。

{
    "kind": "TranscriptionMetadata",
    "transcriptionMetadata": {
        "subscriptionId": "aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e",
        "locale": "en-us",
        "callConnectionId": "65c57654=f12c-4975-92a4-21668e61dd98",
        "correlationId": "65c57654=f12c-4975-92a4-21668e61dd98"
    }
}

接收听录数据

在元数据之后，Websocket 接收的下一个数据包将是听录音频的听录数据。

{
    "kind": "TranscriptionData",
    "transcriptionData": {
        "text": "Testing transcription.",
        "format": "display",
        "confidence": 0.695223331451416,
        "offset": 2516998782481234400,
        "words": [
            {
                "text": "testing",
                "offset": 2516998782481234400
            },
            {
                "text": "testing",
                "offset": 2516998782481234400
            }
        ],
        "participantRawID": "8:acs:",
        "resultStatus": "Final"
    }
}

在已启用 AI 功能的情况下接收听录流（预览版）

在通话期间启用听录时，Azure 通信服务会发出描述听录会话的配置和上下文的元数据。 其中包括区域设置、呼叫连接 ID、情绪分析设置和 PII 编修首选项等详细信息。 开发人员可以使用此有效负载来验证听录设置、审核配置或排查与 AI 增强的实时听录功能相关的问题。

{
  "kind": "TranscriptionMetadata",
  "transcriptionMetadata": {
    "subscriptionId": "863b5e55-de0d-4fc3-8e58-2d68e976b5ad",
    "locale": "en-US",
    "callConnectionId": "02009180-9dc2-429b-a3eb-d544b7b6a0e1",
    "correlationId": "62c8215b-5276-4d3c-bb6d-06a1b114651b",
    "speechModelEndpointId": null,
    "locales": [],
    "enableSentimentAnalysis": true,
    "piiRedactionOptions": {
      "enable": true,
      "redactionType": "MaskWithCharacter"
    }
  }
}

在已启用 AI 功能的情况下接收听录数据（预览版）

初始元数据数据包后，WebSocket 连接将开始接收每个转录音频段的 TranscriptionData 事件。 这些数据包中包含转录的文本、置信度分数、计时信息，若已启用情绪分析和 PII 编修，则还包含这两项信息。 此数据可用于在通话期间生成实时仪表板、触发工作流或分析聊天动态。

{
  "kind": "TranscriptionData",
  "transcriptionData": {
    "text": "My date of birth is *********.",
    "format": "display",
    "confidence": 0.8726407289505005,
    "offset": 309058340,
    "duration": 31600000,
    "words": [],
    "participantRawID": "4:+917020276722",
    "resultStatus": "Final",
    "sentimentAnalysisResult": {
      "sentiment": "neutral"
    }
  }
}

处理 Websocket 服务器中的听录流

import asyncio
import json
import websockets
from azure.communication.callautomation._shared.models import identifier_from_raw_id

async def handle_client(websocket, path):
    print("Client connected")
    try:
        async for message in websocket:
            json_object = json.loads(message)
            kind = json_object['kind']
            if kind == 'TranscriptionMetadata':
                print("Transcription metadata")
                print("-------------------------")
                print("Subscription ID:", json_object['transcriptionMetadata']['subscriptionId'])
                print("Locale:", json_object['transcriptionMetadata']['locale'])
                print("Call Connection ID:", json_object['transcriptionMetadata']['callConnectionId'])
                print("Correlation ID:", json_object['transcriptionMetadata']['correlationId'])
                print("Locales:", json_object['transcriptionMetadata']['locales']) 
                print("PII Redaction Options:", json_object['transcriptionMetadata']['piiRedactionOptions']) 
            if kind == 'TranscriptionData':
                participant = identifier_from_raw_id(json_object['transcriptionData']['participantRawID'])
                word_data_list = json_object['transcriptionData']['words']
                print("Transcription data")
                print("-------------------------")
                print("Text:", json_object['transcriptionData']['text'])
                print("Format:", json_object['transcriptionData']['format'])
                print("Confidence:", json_object['transcriptionData']['confidence'])
                print("Offset:", json_object['transcriptionData']['offset'])
                print("Duration:", json_object['transcriptionData']['duration'])
                print("Participant:", participant.raw_id)
                print("Result Status:", json_object['transcriptionData']['resultStatus']) 
                print("Sentiment Analysis Result:", json_object['transcriptionData']['sentimentAnalysisResult']) 
                print("Result Status:", json_object['transcriptionData']['resultStatus'])
                for word in word_data_list:
                    print("Word:", word['text'])
                    print("Offset:", word['offset'])
                    print("Duration:", word['duration'])
            
    except websockets.exceptions.ConnectionClosedOK:
        print("Client disconnected")
    except websockets.exceptions.ConnectionClosedError as e:
        print("Connection closed with error: %s", e)
    except Exception as e:
        print("Unexpected error: %s", e)

start_server = websockets.serve(handle_client, "localhost", 8081)

print('WebSocket server running on port 8081')

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

更新听录

如果应用程序允许用户选择其首选语言，你可能还需要捕获该语言版本的听录。 为此，通话自动化 SDK 允许更新听录区域设置。

await call_automation_client.get_call_connection(
    call_connection_id=call_connection_id
).update_transcription(
    operation_context="UpdateTranscriptionContext",
    locale="en-au",
    #Only add the SpeechRecognitionModelEndpointId if you have a custom speech model you would like to use
    speech_recognition_model_endpoint_id = "YourCustomSpeechRecognitionModelEndpointId"
)

停止听录

当应用程序需要停止收听听录时，可以使用 StopTranscription 请求让通话自动化知道停止向 websocket 发送听录数据。

# Stop transcription without options
call_connection_client.stop_transcription()

# Alternative: Stop transcription with operation context
# call_connection_client.stop_transcription(operation_context="stopTranscriptionContext")