客户端事件
本组目录
session.update
收到 session.created 后设置当前会话;需空闲、无缓冲音频及待完成工具。失败不提交配置。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="session.update" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
session | SessionUpdate | 是 | |
完整 JSON 示例
{
"type": "session.update",
"session": {
"type": "realtime",
"instructions": "你是客服,请简短回答。",
"output_modalities": [
"text"
],
"tools": [],
"tool_choice": "auto"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "session.update"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"session": {
"$ref": "#/$defs/SessionUpdate"
}
},
"required": [
"type",
"session"
],
"description": "收到 session.created 后设置当前会话;需空闲、无缓冲音频及待完成工具。失败不提交配置。"
}
conversation.item.create
追加用户文本、Function 结果或 MCP 审批;不支持插入历史位置。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.create" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item | oneOf | 是 | |
完整 JSON 示例
{
"type": "conversation.item.create",
"item": {
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "你好"
}
]
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.create"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item": {
"oneOf": [
{
"$ref": "#/$defs/UserItem"
},
{
"$ref": "#/$defs/FunctionOutput"
},
{
"$ref": "#/$defs/Approval"
}
]
}
},
"required": [
"type",
"item"
],
"description": "追加用户文本、Function 结果或 MCP 审批;不支持插入历史位置。"
}
conversation.item.retrieve
读取当前连接中的 item。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.retrieve" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
完整 JSON 示例
{
"type": "conversation.item.retrieve",
"item_id": "item_demo"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.retrieve"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
}
},
"required": [
"type",
"item_id"
],
"description": "读取当前连接中的 item。"
}
conversation.item.truncate
截断 assistant 音频历史;content_index 必须为 0;audio_end_ms 不得超过已生成时长。当前实现清空该 item 的历史转写,不承诺逐字对齐。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.truncate" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
content_index | const | 是 | const=0 |
audio_end_ms | integer | 是 | minimum=0 |
完整 JSON 示例
{
"type": "conversation.item.truncate",
"item_id": "item_demo",
"content_index": 0,
"audio_end_ms": 500
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.truncate"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"content_index": {
"const": 0
},
"audio_end_ms": {
"type": "integer",
"minimum": 0
}
},
"required": [
"type",
"item_id",
"content_index",
"audio_end_ms"
],
"description": "截断 assistant 音频历史;content_index 必须为 0;audio_end_ms 不得超过已生成时长。当前实现清空该 item 的历史转写,不承诺逐字对齐。"
}
response.create
创建一次响应;有待完成工具时拒绝。自动 VAD 已触发时不要重复发送。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.create" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response | ResponseOptions | 否 | |
完整 JSON 示例
{
"type": "response.create"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.create"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response": {
"$ref": "#/$defs/ResponseOptions"
}
},
"required": [
"type"
],
"description": "创建一次响应;有待完成工具时拒绝。自动 VAD 已触发时不要重复发送。"
}
response.cancel
取消活动响应;可指定 response_id 防止取消错误响应。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.cancel" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 否 | |
完整 JSON 示例
{
"type": "response.cancel"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.cancel"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
}
},
"required": [
"type"
],
"description": "取消活动响应;可指定 response_id 防止取消错误响应。"
}
input_audio_buffer.append
追加 Base64 PCM16LE 单声道 24kHz;单包最多 10 秒、缓冲最多 120 秒。示例为两个静音采样,实际提交至少 100ms。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.append" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
audio | string | 是 | |
完整 JSON 示例
{
"type": "input_audio_buffer.append",
"audio": "AAAAAA=="
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.append"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"audio": {
"type": "string"
}
},
"required": [
"type",
"audio"
],
"description": "追加 Base64 PCM16LE 单声道 24kHz;单包最多 10 秒、缓冲最多 120 秒。示例为两个静音采样,实际提交至少 100ms。"
}
input_audio_buffer.commit
手动提交至少 100ms 音频;等待转写,再 response.create。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.commit" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
完整 JSON 示例
{
"type": "input_audio_buffer.commit"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.commit"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
}
},
"required": [
"type"
],
"description": "手动提交至少 100ms 音频;等待转写,再 response.create。"
}
input_audio_buffer.clear
丢弃尚未提交的音频缓冲。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.clear" |
event_id | string | 否 | 客户端可选,用于关联错误;服务端事件始终携带。 |
完整 JSON 示例
{
"type": "input_audio_buffer.clear"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.clear"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
}
},
"required": [
"type"
],
"description": "丢弃尚未提交的音频缓冲。"
}
服务端事件
本组目录
session.created
返回当前生效会话配置;工具凭据不回显。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="session.created" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
session | Session | 是 | |
完整 JSON 示例
{
"type": "session.created",
"event_id": "event_demo",
"session": {
"id": "sess_demo",
"object": "realtime.session",
"model": "latentx-realtime-2.5-mini",
"type": "realtime",
"instructions": "你是客服,请简短回答。",
"output_modalities": [
"text"
],
"tools": [],
"tool_choice": "auto",
"audio": {
"input": {
"format": {
"type": "audio/pcm",
"rate": 24000
},
"turn_detection": {
"type": "server_vad",
"silence_duration_ms": 300,
"create_response": true,
"interrupt_response": true
}
},
"output": {
"format": {
"type": "audio/pcm",
"rate": 24000
},
"voice": "female-default"
}
},
"latentx": {
"usage_mode": "transcription"
}
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "session.created"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"session": {
"$ref": "#/$defs/Session"
}
},
"required": [
"type",
"event_id",
"session"
],
"description": "返回当前生效会话配置;工具凭据不回显。"
}
session.updated
返回当前生效会话配置;工具凭据不回显。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="session.updated" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
session | Session | 是 | |
完整 JSON 示例
{
"type": "session.updated",
"event_id": "event_demo",
"session": {
"id": "sess_demo",
"object": "realtime.session",
"model": "latentx-realtime-2.5-mini",
"type": "realtime",
"instructions": "你是客服,请简短回答。",
"output_modalities": [
"text"
],
"tools": [],
"tool_choice": "auto",
"audio": {
"input": {
"format": {
"type": "audio/pcm",
"rate": 24000
},
"turn_detection": {
"type": "server_vad",
"silence_duration_ms": 300,
"create_response": true,
"interrupt_response": true
}
},
"output": {
"format": {
"type": "audio/pcm",
"rate": 24000
},
"voice": "female-default"
}
},
"latentx": {
"usage_mode": "transcription"
}
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "session.updated"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"session": {
"$ref": "#/$defs/Session"
}
},
"required": [
"type",
"event_id",
"session"
],
"description": "返回当前生效会话配置;工具凭据不回显。"
}
conversation.item.added
会话 item 新增、完成或读取结果;MCP item 的字段由 type 决定。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.added" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item | Item | 是 | |
previous_item_id | ['string', 'null'] | 否 | |
完整 JSON 示例
{
"type": "conversation.item.added",
"event_id": "event_demo",
"item": {
"id": "item_demo",
"object": "realtime.item",
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "你好"
}
],
"status": "completed"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.added"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item": {
"$ref": "#/$defs/Item"
},
"previous_item_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"type",
"event_id",
"item"
],
"description": "会话 item 新增、完成或读取结果;MCP item 的字段由 type 决定。"
}
conversation.item.done
会话 item 新增、完成或读取结果;MCP item 的字段由 type 决定。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item | Item | 是 | |
previous_item_id | ['string', 'null'] | 否 | |
完整 JSON 示例
{
"type": "conversation.item.done",
"event_id": "event_demo",
"item": {
"id": "item_demo",
"object": "realtime.item",
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "你好"
}
],
"status": "completed"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item": {
"$ref": "#/$defs/Item"
},
"previous_item_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"type",
"event_id",
"item"
],
"description": "会话 item 新增、完成或读取结果;MCP item 的字段由 type 决定。"
}
conversation.item.retrieved
会话 item 新增、完成或读取结果;MCP item 的字段由 type 决定。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.retrieved" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item | Item | 是 | |
previous_item_id | ['string', 'null'] | 否 | |
完整 JSON 示例
{
"type": "conversation.item.retrieved",
"event_id": "event_demo",
"item": {
"id": "item_demo",
"object": "realtime.item",
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "你好"
}
],
"status": "completed"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.retrieved"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item": {
"$ref": "#/$defs/Item"
},
"previous_item_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"type",
"event_id",
"item"
],
"description": "会话 item 新增、完成或读取结果;MCP item 的字段由 type 决定。"
}
conversation.item.truncated
确认历史截断。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.truncated" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
content_index | integer | 是 | |
audio_end_ms | integer | 是 | |
完整 JSON 示例
{
"type": "conversation.item.truncated",
"event_id": "event_demo",
"item_id": "item_demo",
"content_index": 0,
"audio_end_ms": 500
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.truncated"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"content_index": {
"type": "integer"
},
"audio_end_ms": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"item_id",
"content_index",
"audio_end_ms"
],
"description": "确认历史截断。"
}
input_audio_buffer.speech_started
VAD 检测到语音边界;毫秒位置属于当前音频时间线。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.speech_started" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
audio_start_ms | integer | 是 | |
完整 JSON 示例
{
"type": "input_audio_buffer.speech_started",
"event_id": "event_demo",
"item_id": "item_demo",
"audio_start_ms": 500
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.speech_started"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"audio_start_ms": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"item_id",
"audio_start_ms"
],
"description": "VAD 检测到语音边界;毫秒位置属于当前音频时间线。"
}
input_audio_buffer.speech_stopped
VAD 检测到语音边界;毫秒位置属于当前音频时间线。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.speech_stopped" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
audio_end_ms | integer | 是 | |
完整 JSON 示例
{
"type": "input_audio_buffer.speech_stopped",
"event_id": "event_demo",
"item_id": "item_demo",
"audio_end_ms": 500
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.speech_stopped"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"audio_end_ms": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"item_id",
"audio_end_ms"
],
"description": "VAD 检测到语音边界;毫秒位置属于当前音频时间线。"
}
input_audio_buffer.committed
音频转写已提交为用户 item。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.committed" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
previous_item_id | ['string', 'null'] | 是 | |
完整 JSON 示例
{
"type": "input_audio_buffer.committed",
"event_id": "event_demo",
"item_id": "item_demo",
"previous_item_id": null
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.committed"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"previous_item_id": {
"type": [
"string",
"null"
]
}
},
"required": [
"type",
"event_id",
"item_id",
"previous_item_id"
],
"description": "音频转写已提交为用户 item。"
}
input_audio_buffer.cleared
缓冲已清空。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="input_audio_buffer.cleared" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
完整 JSON 示例
{
"type": "input_audio_buffer.cleared",
"event_id": "event_demo"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "input_audio_buffer.cleared"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
}
},
"required": [
"type",
"event_id"
],
"description": "缓冲已清空。"
}
conversation.item.input_audio_transcription.completed
转写完成。仅 transcription 模式单列 usage;即使没有模型响应,也可收到本事件。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="conversation.item.input_audio_transcription.completed" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
content_index | integer | 是 | |
transcript | string | 是 | |
usage | TranscriptionUsage | 否 | |
完整 JSON 示例
{
"type": "conversation.item.input_audio_transcription.completed",
"event_id": "event_demo",
"item_id": "item_demo",
"content_index": 0,
"transcript": "你好",
"usage": {
"type": "duration",
"seconds": 1.25
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "conversation.item.input_audio_transcription.completed"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"content_index": {
"type": "integer"
},
"transcript": {
"type": "string"
},
"usage": {
"$ref": "#/$defs/TranscriptionUsage"
}
},
"required": [
"type",
"event_id",
"item_id",
"content_index",
"transcript"
],
"description": "转写完成。仅 transcription 模式单列 usage;即使没有模型响应,也可收到本事件。"
}
response.created
响应开始或结束;只有 done 且 status=completed 才表示模型响应成功,MCP 执行仍可能未结束。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.created" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response | Response | 是 | |
完整 JSON 示例
{
"type": "response.created",
"event_id": "event_demo",
"response": {
"id": "resp_demo",
"object": "realtime.response",
"status": "in_progress",
"status_details": null,
"output": [],
"conversation_id": "sess_demo",
"output_modalities": [
"text"
],
"usage": {
"input_tokens": 10,
"output_tokens": 2,
"total_tokens": 12,
"input_token_details": {
"text_tokens": 10,
"audio_tokens": 0,
"cached_tokens": 0,
"cached_tokens_details": {
"text_tokens": 0,
"audio_tokens": 0
}
},
"output_token_details": {
"text_tokens": 2,
"audio_tokens": 0
}
}
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.created"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response": {
"$ref": "#/$defs/Response"
}
},
"required": [
"type",
"event_id",
"response"
],
"description": "响应开始或结束;只有 done 且 status=completed 才表示模型响应成功,MCP 执行仍可能未结束。"
}
response.done
响应开始或结束;只有 done 且 status=completed 才表示模型响应成功,MCP 执行仍可能未结束。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response | Response | 是 | |
完整 JSON 示例
{
"type": "response.done",
"event_id": "event_demo",
"response": {
"id": "resp_demo",
"object": "realtime.response",
"status": "completed",
"status_details": null,
"output": [],
"conversation_id": "sess_demo",
"output_modalities": [
"text"
],
"usage": {
"input_tokens": 10,
"output_tokens": 2,
"total_tokens": 12,
"input_token_details": {
"text_tokens": 10,
"audio_tokens": 0,
"cached_tokens": 0,
"cached_tokens_details": {
"text_tokens": 0,
"audio_tokens": 0
}
},
"output_token_details": {
"text_tokens": 2,
"audio_tokens": 0
}
}
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response": {
"$ref": "#/$defs/Response"
}
},
"required": [
"type",
"event_id",
"response"
],
"description": "响应开始或结束;只有 done 且 status=completed 才表示模型响应成功,MCP 执行仍可能未结束。"
}
response.output_item.added
响应输出项开始或结束。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_item.added" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
output_index | integer | 是 | |
item | Item | 是 | |
完整 JSON 示例
{
"type": "response.output_item.added",
"event_id": "event_demo",
"response_id": "resp_demo",
"output_index": 0,
"item": {
"id": "item_demo",
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "你好"
}
]
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_item.added"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"item": {
"$ref": "#/$defs/Item"
}
},
"required": [
"type",
"event_id",
"response_id",
"output_index",
"item"
],
"description": "响应输出项开始或结束。"
}
response.output_item.done
响应输出项开始或结束。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_item.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
output_index | integer | 是 | |
item | Item | 是 | |
完整 JSON 示例
{
"type": "response.output_item.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"output_index": 0,
"item": {
"id": "item_demo",
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "你好"
}
]
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_item.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"item": {
"$ref": "#/$defs/Item"
}
},
"required": [
"type",
"event_id",
"response_id",
"output_index",
"item"
],
"description": "响应输出项开始或结束。"
}
response.content_part.added
消息内容部分开始或结束。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.content_part.added" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
part | Content | 是 | |
完整 JSON 示例
{
"type": "response.content_part.added",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"part": {
"type": "output_text",
"text": "你好"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.content_part.added"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"part": {
"$ref": "#/$defs/Content"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"part"
],
"description": "消息内容部分开始或结束。"
}
response.content_part.done
消息内容部分开始或结束。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.content_part.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
part | Content | 是 | |
完整 JSON 示例
{
"type": "response.content_part.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"part": {
"type": "output_text",
"text": "你好"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.content_part.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"part": {
"$ref": "#/$defs/Content"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"part"
],
"description": "消息内容部分开始或结束。"
}
response.output_text.delta
增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_text.delta" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
delta | string | 是 | |
完整 JSON 示例
{
"type": "response.output_text.delta",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"delta": "你好"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_text.delta"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"delta": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"delta"
],
"description": "增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。"
}
response.output_text.done
增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_text.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
text | string | 是 | |
完整 JSON 示例
{
"type": "response.output_text.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"text": "你好"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_text.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"text": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"text"
],
"description": "增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。"
}
response.output_audio.delta
增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_audio.delta" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
delta | string | 是 | |
完整 JSON 示例
{
"type": "response.output_audio.delta",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"delta": "AAAAAA=="
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_audio.delta"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"delta": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"delta"
],
"description": "增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。"
}
response.output_audio_transcript.delta
增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_audio_transcript.delta" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
delta | string | 是 | |
完整 JSON 示例
{
"type": "response.output_audio_transcript.delta",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"delta": "你好"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_audio_transcript.delta"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"delta": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"delta"
],
"description": "增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。"
}
response.output_audio_transcript.done
增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_audio_transcript.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
transcript | string | 是 | |
完整 JSON 示例
{
"type": "response.output_audio_transcript.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0,
"transcript": "你好"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_audio_transcript.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
},
"transcript": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index",
"transcript"
],
"description": "增量按到达顺序累积;音频 delta 为 Base64 PCM,done 文本为本项完整内容。"
}
response.output_audio.done
本项音频生成结束,不代表客户端已播放完。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.output_audio.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
content_index | integer | 是 | |
完整 JSON 示例
{
"type": "response.output_audio.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "item_demo",
"output_index": 0,
"content_index": 0
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.output_audio.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"content_index": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"content_index"
],
"description": "本项音频生成结束,不代表客户端已播放完。"
}
response.function_call_arguments.delta
客户执行的 Function 参数;等待 response.done 后执行并回传全部结果。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.function_call_arguments.delta" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
call_id | string | 是 | |
delta | string | 是 | |
完整 JSON 示例
{
"type": "response.function_call_arguments.delta",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "call_demo",
"output_index": 0,
"call_id": "call_demo",
"delta": "{\"order_id\":\"DEMO-1\"}"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.function_call_arguments.delta"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"call_id": {
"type": "string"
},
"delta": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"call_id",
"delta"
],
"description": "客户执行的 Function 参数;等待 response.done 后执行并回传全部结果。"
}
response.function_call_arguments.done
客户执行的 Function 参数;等待 response.done 后执行并回传全部结果。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.function_call_arguments.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
call_id | string | 是 | |
arguments | string | 是 | |
name | string | 是 | |
完整 JSON 示例
{
"type": "response.function_call_arguments.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "call_demo",
"output_index": 0,
"call_id": "call_demo",
"arguments": "{\"order_id\":\"DEMO-1\"}",
"name": "lookup_order"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.function_call_arguments.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"call_id": {
"type": "string"
},
"arguments": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"call_id",
"arguments",
"name"
],
"description": "客户执行的 Function 参数;等待 response.done 后执行并回传全部结果。"
}
response.mcp_call_arguments.delta
MCP 参数;工具由平台执行,客户只处理需要的审批。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.mcp_call_arguments.delta" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
delta | string | 是 | |
完整 JSON 示例
{
"type": "response.mcp_call_arguments.delta",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "mcp_demo",
"output_index": 0,
"delta": "{\"order_id\":\"DEMO-1\"}"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.mcp_call_arguments.delta"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"delta": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"delta"
],
"description": "MCP 参数;工具由平台执行,客户只处理需要的审批。"
}
response.mcp_call_arguments.done
MCP 参数;工具由平台执行,客户只处理需要的审批。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.mcp_call_arguments.done" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
response_id | string | 是 | |
item_id | string | 是 | |
output_index | integer | 是 | |
arguments | string | 是 | |
完整 JSON 示例
{
"type": "response.mcp_call_arguments.done",
"event_id": "event_demo",
"response_id": "resp_demo",
"item_id": "mcp_demo",
"output_index": 0,
"arguments": "{\"order_id\":\"DEMO-1\"}"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.mcp_call_arguments.done"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"response_id": {
"type": "string"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
},
"arguments": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"response_id",
"item_id",
"output_index",
"arguments"
],
"description": "MCP 参数;工具由平台执行,客户只处理需要的审批。"
}
mcp_list_tools.in_progress
工具发现状态;失败会导致配置更新失败。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="mcp_list_tools.in_progress" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
完整 JSON 示例
{
"type": "mcp_list_tools.in_progress",
"event_id": "event_demo",
"item_id": "mcp_list_demo"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "mcp_list_tools.in_progress"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"item_id"
],
"description": "工具发现状态;失败会导致配置更新失败。"
}
response.mcp_call.in_progress
MCP 执行状态;最终 output/error 在对应 mcp_call item 中。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.mcp_call.in_progress" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
output_index | integer | 是 | |
完整 JSON 示例
{
"type": "response.mcp_call.in_progress",
"event_id": "event_demo",
"item_id": "mcp_demo",
"output_index": 0
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.mcp_call.in_progress"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"item_id",
"output_index"
],
"description": "MCP 执行状态;最终 output/error 在对应 mcp_call item 中。"
}
mcp_list_tools.completed
工具发现状态;失败会导致配置更新失败。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="mcp_list_tools.completed" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
完整 JSON 示例
{
"type": "mcp_list_tools.completed",
"event_id": "event_demo",
"item_id": "mcp_list_demo"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "mcp_list_tools.completed"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"item_id"
],
"description": "工具发现状态;失败会导致配置更新失败。"
}
response.mcp_call.completed
MCP 执行状态;最终 output/error 在对应 mcp_call item 中。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.mcp_call.completed" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
output_index | integer | 是 | |
完整 JSON 示例
{
"type": "response.mcp_call.completed",
"event_id": "event_demo",
"item_id": "mcp_demo",
"output_index": 0
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.mcp_call.completed"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"item_id",
"output_index"
],
"description": "MCP 执行状态;最终 output/error 在对应 mcp_call item 中。"
}
mcp_list_tools.failed
工具发现状态;失败会导致配置更新失败。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="mcp_list_tools.failed" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
完整 JSON 示例
{
"type": "mcp_list_tools.failed",
"event_id": "event_demo",
"item_id": "mcp_list_demo"
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "mcp_list_tools.failed"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
}
},
"required": [
"type",
"event_id",
"item_id"
],
"description": "工具发现状态;失败会导致配置更新失败。"
}
response.mcp_call.failed
MCP 执行状态;最终 output/error 在对应 mcp_call item 中。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="response.mcp_call.failed" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
item_id | string | 是 | |
output_index | integer | 是 | |
完整 JSON 示例
{
"type": "response.mcp_call.failed",
"event_id": "event_demo",
"item_id": "mcp_demo",
"output_index": 0
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "response.mcp_call.failed"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"item_id": {
"type": "string"
},
"output_index": {
"type": "integer"
}
},
"required": [
"type",
"event_id",
"item_id",
"output_index"
],
"description": "MCP 执行状态;最终 output/error 在对应 mcp_call item 中。"
}
error
协议错误;用 error.event_id 关联客户端事件。配置失败不会覆盖旧配置。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="error" |
event_id | string | 是 | 客户端可选,用于关联错误;服务端事件始终携带。 |
error | object | 是 | |
完整 JSON 示例
{
"type": "error",
"event_id": "event_demo",
"error": {
"type": "invalid_request_error",
"code": "invalid_session",
"message": "session field not supported: temperature",
"event_id": "client_demo"
}
}
事件 Schema
{
"type": "object",
"properties": {
"type": {
"const": "error"
},
"event_id": {
"type": "string",
"description": "客户端可选,用于关联错误;服务端事件始终携带。"
},
"error": {
"type": "object",
"properties": {
"type": {
"type": "string"
},
"code": {
"type": "string",
"description": "Public codes include invalid_tool_arguments, invalid_knowledge_id, tool_timeout, tool_cancelled, tool_execution_failed, conversation_storage_unavailable, realtime_key_invalid, realtime_quota_rejected, invalid_realtime_event, invalid_realtime_input, invalid_response_done, missing_realtime_usage, usage_estimation_failed and voice_operation_failed. Branch on code, not message."
},
"message": {
"type": "string"
},
"event_id": {
"type": "string"
}
},
"required": [
"type",
"code",
"message"
]
}
},
"required": [
"type",
"event_id",
"error"
],
"description": "协议错误;用 error.event_id 关联客户端事件。配置失败不会覆盖旧配置。"
}
New sessions record events by default. GET /voice/v1/conversations/{id}/events returns the same public Realtime server event IDs, types and data, except audio Base64 deltas. Check capture_state and follow opaque next_cursor even on empty pages. Recording is independently enabled; query /recordings for tracks. Sending does not prove client receipt or playback.
对象字段
字节上限按 UTF-8 计算。Schema 中 oneOf 表示择一;完整嵌套结构可展开查看。响应中的可选字段可能省略。
KnowledgeRef
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
id | string | 是 | |
version | string | 是 | |
完整 KnowledgeRef Schema
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"version": {
"type": "string"
}
},
"required": [
"id",
"version"
],
"additionalProperties": false
}
Skill
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
id | string | 是 | |
version | string | 是 | |
preload | boolean | 否 | true 全量预加载;false 仅描述,按需读取。 default=false |
markdown | string | 否 | 内联正文,仅用于 Realtime session.latentx.skills。自定义模型 config.skills 只接受已保存版本引用 {id,version,preload}:先 PUT /voice/v1/skills/{id}/versions/{version},再引用;带 markdown、resources 或 required_tools 时校验返回400。 |
resources | object | 否 | |
required_tools | array | 否 | |
完整 Skill Schema
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"version": {
"type": "string"
},
"preload": {
"type": "boolean",
"default": false,
"description": "true 全量预加载;false 仅描述,按需读取。"
},
"markdown": {
"type": "string",
"description": "内联正文,仅用于 Realtime session.latentx.skills。自定义模型 config.skills 只接受已保存版本引用 {id,version,preload}:先 PUT /voice/v1/skills/{id}/versions/{version},再引用;带 markdown、resources 或 required_tools 时校验返回400。"
},
"resources": {
"type": "object",
"additionalProperties": {
"type": "string"
}
},
"required_tools": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"id",
"version"
],
"additionalProperties": false
}
FunctionTool
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="function" |
name | string | 是 | |
description | string | 否 | |
parameters | object | 是 | 工具参数的 JSON Schema;由客户执行前再次校验。 |
完整 FunctionTool Schema
{
"type": "object",
"properties": {
"type": {
"const": "function"
},
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"parameters": {
"type": "object",
"description": "工具参数的 JSON Schema;由客户执行前再次校验。"
}
},
"required": [
"type",
"name",
"parameters"
],
"additionalProperties": false
}
MCPTool
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="mcp" |
server_label | string | 是 | |
server_url | string | 否 | 公网 HTTPS:443,无 query、凭据或重定向;同会话成功导入后可仅传 label 复用。 |
server_description | string | 否 | |
authorization | string | 否 | 仅写入:Bearer token;响应不回显。 |
headers | object | 否 | 仅写入;响应不回显。 |
allowed_tools | oneOf | 否 | |
require_approval | oneOf | 否 | default="always" |
完整 MCPTool Schema
{
"type": "object",
"properties": {
"type": {
"const": "mcp"
},
"server_label": {
"type": "string"
},
"server_url": {
"type": "string",
"description": "公网 HTTPS:443,无 query、凭据或重定向;同会话成功导入后可仅传 label 复用。"
},
"server_description": {
"type": "string"
},
"authorization": {
"type": "string",
"description": "仅写入:Bearer token;响应不回显。"
},
"headers": {
"type": "object",
"additionalProperties": {
"type": "string"
},
"description": "仅写入;响应不回显。"
},
"allowed_tools": {
"oneOf": [
{
"type": "array",
"items": {
"type": "string"
}
},
{
"type": "object",
"properties": {
"tool_names": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"tool_names"
],
"additionalProperties": false
}
]
},
"require_approval": {
"oneOf": [
{
"enum": [
"always",
"never"
]
},
{
"type": "object",
"properties": {
"always": {
"type": "object",
"properties": {
"tool_names": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"tool_names"
]
},
"never": {
"type": "object",
"properties": {
"tool_names": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"tool_names"
]
}
},
"required": []
}
],
"default": "always"
}
},
"required": [
"type",
"server_label"
],
"additionalProperties": false
}
Tool
完整 Tool Schema
{
"oneOf": [
{
"$ref": "#/$defs/FunctionTool"
},
{
"$ref": "#/$defs/MCPTool"
}
]
}
ToolChoice
完整 ToolChoice Schema
{
"oneOf": [
{
"enum": [
"auto",
"none",
"required"
]
},
{
"type": "object",
"properties": {
"type": {
"const": "function"
},
"name": {
"type": "string"
}
},
"required": [
"type",
"name"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"type": {
"const": "mcp"
},
"server_label": {
"type": "string"
},
"name": {
"type": "string"
}
},
"required": [
"type",
"server_label",
"name"
],
"additionalProperties": false
}
]
}
Modalities
只能选一种;audio 同时返回音频与转写文本。
完整 Modalities Schema
{
"enum": [
[
"text"
],
[
"audio"
]
],
"description": "只能选一种;audio 同时返回音频与转写文本。"
}
Format
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="audio/pcm" |
rate | const | 是 | const=24000 |
完整 Format Schema
{
"type": "object",
"properties": {
"type": {
"const": "audio/pcm"
},
"rate": {
"const": 24000
}
},
"required": [
"type",
"rate"
],
"additionalProperties": false
}
TurnDetection
null 为手动 commit + response.create;默认 server_vad。
完整 TurnDetection Schema
{
"oneOf": [
{
"type": "null"
},
{
"type": "object",
"properties": {
"type": {
"const": "server_vad"
},
"silence_duration_ms": {
"type": "integer",
"minimum": 200,
"maximum": 2000,
"default": 300
},
"create_response": {
"type": "boolean",
"default": true
},
"interrupt_response": {
"type": "boolean",
"default": true
}
},
"required": [],
"additionalProperties": false
}
],
"description": "null 为手动 commit + response.create;默认 server_vad。"
}
Audio
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
input | object | 否 | |
output | object | 否 | |
完整 Audio Schema
{
"type": "object",
"properties": {
"input": {
"type": "object",
"properties": {
"format": {
"$ref": "#/$defs/Format"
},
"turn_detection": {
"$ref": "#/$defs/TurnDetection"
},
"transcription": {
"type": "null",
"description": "仅允许 null;识别使用所选模型的 ASR,不支持会话覆盖。"
}
},
"required": [],
"additionalProperties": false
},
"output": {
"type": "object",
"properties": {
"format": {
"$ref": "#/$defs/Format"
},
"voice": {
"type": "string",
"description": "从 /voice/v1/voices 取得;首次输出音频后锁定。"
}
},
"required": [],
"additionalProperties": false
}
},
"required": [],
"additionalProperties": false
}
Resources
LatenTX 扩展。省略保留原配置,[] 清空。仅当前会话生效;引用必须属于当前租户。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
skills | array | 否 | maxItems=16 items: Skill |
knowledge_refs | array | 否 | maxItems=16 items: KnowledgeRef |
mcp_refs | array | 否 | |
完整 Resources Schema
{
"type": "object",
"properties": {
"skills": {
"type": "array",
"items": {
"$ref": "#/$defs/Skill"
},
"maxItems": 16
},
"knowledge_refs": {
"type": "array",
"items": {
"$ref": "#/$defs/KnowledgeRef"
},
"maxItems": 16
},
"mcp_refs": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [],
"additionalProperties": false,
"description": "LatenTX 扩展。省略保留原配置,[] 清空。仅当前会话生效;引用必须属于当前租户。"
}
SessionUpdate
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 否 | const="realtime" |
model | string | 否 | 如传入必须与连接的 model 一致,不能切换模型。 |
instructions | string | 否 | 替换当前会话的业务提示词;空字符串清空业务提示词;平台输出规则仍生效;UTF-8 最多 64 KiB。 |
output_modalities | Modalities | 否 | |
tools | array | 否 | maxItems=32 items: Tool |
tool_choice | ToolChoice | 否 | |
audio | Audio | 否 | |
latentx | Resources | 否 | |
完整 SessionUpdate Schema
{
"type": "object",
"properties": {
"type": {
"const": "realtime"
},
"model": {
"type": "string",
"description": "如传入必须与连接的 model 一致,不能切换模型。"
},
"instructions": {
"type": "string",
"description": "替换当前会话的业务提示词;空字符串清空业务提示词;平台输出规则仍生效;UTF-8 最多 64 KiB。"
},
"output_modalities": {
"$ref": "#/$defs/Modalities"
},
"tools": {
"type": "array",
"items": {
"$ref": "#/$defs/Tool"
},
"maxItems": 32
},
"tool_choice": {
"$ref": "#/$defs/ToolChoice"
},
"audio": {
"$ref": "#/$defs/Audio"
},
"latentx": {
"$ref": "#/$defs/Resources"
}
},
"required": [],
"additionalProperties": false
}
ResponseOptions
只覆盖本次响应;不会写回会话。max_output_tokens 等未列字段不支持。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
instructions | string | 否 | 只覆盖本次响应的业务提示词;平台输出规则仍生效;UTF-8 最多 64 KiB。 |
output_modalities | Modalities | 否 | |
tools | array | 否 | maxItems=32 items: Tool |
tool_choice | ToolChoice | 否 | |
完整 ResponseOptions Schema
{
"type": "object",
"properties": {
"instructions": {
"type": "string",
"description": "只覆盖本次响应的业务提示词;平台输出规则仍生效;UTF-8 最多 64 KiB。"
},
"output_modalities": {
"$ref": "#/$defs/Modalities"
},
"tools": {
"type": "array",
"items": {
"$ref": "#/$defs/Tool"
},
"maxItems": 32
},
"tool_choice": {
"$ref": "#/$defs/ToolChoice"
}
},
"required": [],
"additionalProperties": false,
"description": "只覆盖本次响应;不会写回会话。max_output_tokens 等未列字段不支持。"
}
Content
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | enum | 是 | enum=["input_text", "input_audio", "output_text", "output_audio"] |
text | string | 否 | |
transcript | string | 否 | |
完整 Content Schema
{
"type": "object",
"properties": {
"type": {
"enum": [
"input_text",
"input_audio",
"output_text",
"output_audio"
]
},
"text": {
"type": "string"
},
"transcript": {
"type": "string"
}
},
"required": [
"type"
]
}
UserItem
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="message" |
id | string | 否 | |
role | const | 是 | const="user" |
content | array | 是 | |
完整 UserItem Schema
{
"type": "object",
"properties": {
"type": {
"const": "message"
},
"id": {
"type": "string"
},
"role": {
"const": "user"
},
"content": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {
"const": "input_text"
},
"text": {
"type": "string",
"description": "非空;合计最多 64 KiB。"
}
},
"required": [
"type",
"text"
]
},
"minItems": 1
}
},
"required": [
"type",
"role",
"content"
]
}
FunctionOutput
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="function_call_output" |
id | string | 否 | |
call_id | string | 是 | |
output | string | 是 | 字符串;建议 JSON 字符串,最多 64 KiB。 |
完整 FunctionOutput Schema
{
"type": "object",
"properties": {
"type": {
"const": "function_call_output"
},
"id": {
"type": "string"
},
"call_id": {
"type": "string"
},
"output": {
"type": "string",
"description": "字符串;建议 JSON 字符串,最多 64 KiB。"
}
},
"required": [
"type",
"call_id",
"output"
]
}
Approval
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="mcp_approval_response" |
id | string | 否 | |
approval_request_id | string | 是 | |
approve | boolean | 是 | |
reason | string | 否 | |
完整 Approval Schema
{
"type": "object",
"properties": {
"type": {
"const": "mcp_approval_response"
},
"id": {
"type": "string"
},
"approval_request_id": {
"type": "string"
},
"approve": {
"type": "boolean"
},
"reason": {
"type": "string"
}
},
"required": [
"type",
"approval_request_id",
"approve"
]
}
Item
按 type 使用对应字段;非所有字段都会同时返回。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
id | string | 是 | |
object | string | 否 | 通常为 realtime.item;部分 MCP 事件省略。 |
type | enum | 是 | enum=["message", "function_call", "function_call_output", "mcp_list_tools", "mcp_call", "mcp_approval_request", "mcp_approval_response"] |
status | string | 否 | |
role | string | 否 | |
content | array | 否 | items: Content |
call_id | string | 否 | |
name | string | 否 | |
arguments | string | 否 | 累积后的 JSON 字符串。 |
output | string | 否 | |
server_label | string | 否 | |
tools | array | 否 | |
error | any | 否 | |
approval_request_id | string | 否 | |
approve | boolean | 否 | |
reason | string | 否 | |
完整 Item Schema
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"object": {
"type": "string",
"description": "通常为 realtime.item;部分 MCP 事件省略。"
},
"type": {
"enum": [
"message",
"function_call",
"function_call_output",
"mcp_list_tools",
"mcp_call",
"mcp_approval_request",
"mcp_approval_response"
]
},
"status": {
"type": "string"
},
"role": {
"type": "string"
},
"content": {
"type": "array",
"items": {
"$ref": "#/$defs/Content"
}
},
"call_id": {
"type": "string"
},
"name": {
"type": "string"
},
"arguments": {
"type": "string",
"description": "累积后的 JSON 字符串。"
},
"output": {
"type": "string"
},
"server_label": {
"type": "string"
},
"tools": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"input_schema": {
"type": "object"
}
},
"required": []
}
},
"error": {},
"approval_request_id": {
"type": "string"
},
"approve": {
"type": "boolean"
},
"reason": {
"type": "string"
}
},
"required": [
"id",
"type"
],
"description": "按 type 使用对应字段;非所有字段都会同时返回。"
}
Usage
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
total_tokens | integer | 是 | |
input_tokens | integer | 是 | |
output_tokens | integer | 是 | |
input_token_details | object | 是 | |
output_token_details | object | 是 | |
完整 Usage Schema
{
"type": "object",
"properties": {
"total_tokens": {
"type": "integer"
},
"input_tokens": {
"type": "integer"
},
"output_tokens": {
"type": "integer"
},
"input_token_details": {
"type": "object",
"properties": {
"text_tokens": {
"type": "integer"
},
"audio_tokens": {
"type": "integer"
},
"cached_tokens": {
"type": "integer"
},
"cached_tokens_details": {
"type": "object",
"properties": {
"text_tokens": {
"type": "integer"
},
"audio_tokens": {
"type": "integer"
}
},
"required": []
}
},
"required": []
},
"output_token_details": {
"type": "object",
"properties": {
"text_tokens": {
"type": "integer"
},
"audio_tokens": {
"type": "integer"
}
},
"required": []
}
},
"required": [
"total_tokens",
"input_tokens",
"output_tokens",
"input_token_details",
"output_token_details"
]
}
Response
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
id | string | 是 | |
object | const | 是 | const="realtime.response" |
status | enum | 是 | enum=["in_progress", "completed", "cancelled", "failed"] |
status_details | any | 否 | 正常为 null;失败或取消时可能包含原因。 |
output | array | 是 | items: Item |
conversation_id | string | 否 | |
output_modalities | Modalities | 否 | |
usage | Usage | 是 | |
完整 Response Schema
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"object": {
"const": "realtime.response"
},
"status": {
"enum": [
"in_progress",
"completed",
"cancelled",
"failed"
]
},
"status_details": {
"description": "正常为 null;失败或取消时可能包含原因。"
},
"output": {
"type": "array",
"items": {
"$ref": "#/$defs/Item"
}
},
"conversation_id": {
"type": "string"
},
"output_modalities": {
"$ref": "#/$defs/Modalities"
},
"usage": {
"$ref": "#/$defs/Usage"
}
},
"required": [
"id",
"object",
"status",
"output",
"usage"
]
}
SessionResources
服务端回显,不可通过 session.update 写入 usage_mode。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
skills | array | 否 | maxItems=16 items: Skill |
knowledge_refs | array | 否 | maxItems=16 items: KnowledgeRef |
mcp_refs | array | 否 | |
usage_mode | const | 否 | 开启独立转写计量时返回;默认 response 模式可省略此字段。 const="transcription" |
完整 SessionResources Schema
{
"type": "object",
"properties": {
"skills": {
"type": "array",
"items": {
"$ref": "#/$defs/Skill"
},
"maxItems": 16
},
"knowledge_refs": {
"type": "array",
"items": {
"$ref": "#/$defs/KnowledgeRef"
},
"maxItems": 16
},
"mcp_refs": {
"type": "array",
"items": {
"type": "string"
}
},
"usage_mode": {
"const": "transcription",
"readOnly": true,
"description": "开启独立转写计量时返回;默认 response 模式可省略此字段。"
}
},
"required": [],
"description": "服务端回显,不可通过 session.update 写入 usage_mode。"
}
TranscriptionUsage
transcription 模式下,此条转写的计量秒数,可以含小数。
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
type | const | 是 | const="duration" |
seconds | number | 是 | minimum=0 |
完整 TranscriptionUsage Schema
{
"type": "object",
"properties": {
"type": {
"const": "duration"
},
"seconds": {
"type": "number",
"minimum": 0
}
},
"required": [
"type",
"seconds"
],
"description": "transcription 模式下,此条转写的计量秒数,可以含小数。"
}
Session
| 字段 | 类型 | 必填 | 约束与说明 |
|---|
id | string | 是 | |
object | const | 是 | const="realtime.session" |
type | const | 否 | const="realtime" |
model | string | 是 | 如传入必须与连接的 model 一致,不能切换模型。 |
instructions | string | 是 | 替换当前会话的业务提示词;空字符串清空业务提示词;平台输出规则仍生效;UTF-8 最多 64 KiB。 |
output_modalities | Modalities | 是 | |
tools | array | 是 | maxItems=32 items: Tool |
tool_choice | ToolChoice | 否 | |
audio | Audio | 是 | |
latentx | SessionResources | 否 | |
完整 Session Schema
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"object": {
"const": "realtime.session"
},
"type": {
"const": "realtime"
},
"model": {
"type": "string",
"description": "如传入必须与连接的 model 一致,不能切换模型。"
},
"instructions": {
"type": "string",
"description": "替换当前会话的业务提示词;空字符串清空业务提示词;平台输出规则仍生效;UTF-8 最多 64 KiB。"
},
"output_modalities": {
"$ref": "#/$defs/Modalities"
},
"tools": {
"type": "array",
"items": {
"$ref": "#/$defs/Tool"
},
"maxItems": 32
},
"tool_choice": {
"$ref": "#/$defs/ToolChoice"
},
"audio": {
"$ref": "#/$defs/Audio"
},
"latentx": {
"$ref": "#/$defs/SessionResources"
}
},
"required": [
"id",
"object",
"model",
"instructions",
"output_modalities",
"tools",
"audio"
]
}
以下为输入被接纳时的一种常见时序,并非每段声音都会生成回复。自动 VAD:append → speech_started → speech_stopped → committed → transcription.completed → 自动响应(create_response=true)。手动模式:turn_detection=null → append → commit → transcription.completed → response.create。自动模式无需重复触发响应。输出音频与转写事件可能交错。
默认按 response.done.response.usage 读取用量。单列转写时,在握手使用 X-LatenTX-Usage-Mode: transcription 或查询参数 usage_mode=transcription,在 session.created 中确认 session.latentx.usage_mode;同一条 ASR 输入不会再次计入响应。参阅两种计量模式与对账示例。
Function:arguments.done → response.done → 客户校验并执行 → function_call_output(全部回传)→ response.create。
MCP:可能先出现 mcp_approval_request item;等待 response.done → 回传 mcp_approval_response → 平台执行 → mcp_call 终态 item。免审批也要等待执行完成,再 response.create。模型响应结束并不等于 MCP 已结束;不要仅根据一个 completed 事件推断所有调用完成。
session.latentx.knowledge_refs 只引用当前租户的知识版本。平台挂载 search_knowledge 并在服务端执行,搜索范围仅限当前挂载知识。不要把它作为客户执行 Function 回传,也不要再次注册同名 MCP。Skill 用于流程,知识用于事实与 QA;正文无需固定 QA 格式,清晰标题和问答结构有助于检索。已保存资源不会自动热更新会话。
先检查握手 HTTP 状态,再处理 WS error 和 response.done.response.status。连接断开且尚无成功终态时,应记为未完成;当前没有会话断线恢复协议,不自动重放有副作用的工具。
对外事件保留会话、消息、语音、工具和用量语义。配置回显只包含已确认的客户配置;平台默认指令及内部执行配置不回显。工具连接凭证及平台内部地址不回显。历史查询与实时使用一致披露规则,但不承诺逐网络包重放;音频通过录音查询。内部成本、供应商信息和原始错误不返回。
会话最长20分钟;单响应最长120秒;单消息1MiB;指令及工具结果各64KiB;历史512KiB、最多512个item。最多32个工具定义、4个MCP服务、64个工具schema,总schema128KiB;最多16个Skill、16个知识资源。MCP发现15秒、执行20秒超时。更多约束见接入指南。
不支持 session.temperature、response.max_output_tokens、原生 OpenAI WebRTC/client_secrets、图像输入、任意codec、会话模型切换或输出JSON Schema。ASR/TTS语言在自定义模型或网页/电话配置中选择,不是 session.audio 的可写 language 字段。