Session logging（会话日志记录 (Session logging)）¶

Chatbot executions in AIP Chatbot Studio are logged as structured events that can be exported to a Foundry streaming dataset for monitoring and analysis. Each message sent to a chatbot counts as one execution, and every execution is assigned a unique trace identifier that connects all related log entries.

:::callout{theme="neutral"} Chatbot session log exporting uses the configure logging feature. Event schemas and event types are subject to change. New events may be added and existing schemas may be modified. :::

Prerequisites¶

Exporting chatbot session logs requires the log export feature. See export permissions for the required roles. Chatbot execution logs contain user identifiers that can be traced back to the user who sent the message, so access to exported log data should be carefully managed.

To set up log exporting, follow the steps in the Configure logging documentation, including the recommended Apply markings step to apply security markings to the output streaming dataset to control access to log data.

Learn how to view logs in the platform with AIP observability.

Understanding chatbot execution logs¶

Each log entry contains common fields provided by the log schema, as well as chatbot-specific event data. The chatbot-specific data consists of an event_name and a payload describing what occurred during the execution.

Common log fields¶

The following fields are included in every log entry and are useful for filtering and correlating logs. Logs from products across the execution flow, such as function executions and language model usage, also share these fields, so you can use them to trace the complete execution request:

Field	Type	Description
`traceId`	String	A Foundry trace identifier assigned to each execution. All log entries from the same execution share the same `traceId`. Use this to follow a request end-to-end.
`uid`	String	The identifier of the user who sent the message.
`owning_rid`	String	The resource identifier of the source executor that initiated the execution, such as a chatbot, function, or Workshop application. For example, if Chatbot A calls a function that invokes Chatbot B, all resulting logs have Chatbot A's RID as the `owning_rid`. Use this to find all logs originating from a specific source executor, regardless of how deep the call chain goes.

Chatbot event data¶

Each chatbot event includes an event_name identifying the type of event and a payload with event-specific fields. Every event payload includes a session_rid, which identifies the chatbot session. Use this to group all events from a single conversation.

Sample log structure¶

The following is a simplified example of a log entry for a user_request event:

{
  "time": "2024-01-15T09:30:00.000Z",
  "uid": "<USER_ID>",
  "traceId": "<TRACE_ID>",
  "content": {
    "event_name": "user_request",
    "payload": {
      "session_rid": "ri.aip-agents..session.<SESSION_ID>",
      "user_query": {
        "content": [
          {
            "text": {
              "content": "Summarize the key points from the latest press conference transcript."
            }
          }
        ]
      }
    }
  }
}

Event types¶

The following event types are logged during a chatbot execution:

Event name	Description
`session_metadata`	Records the chatbot RID, chatbot version, session RID, and caller identifier at the start of an execution.
`user_request`	Captures the user's message along with any retrieval context and application variables.
`system_chat_message`	Contains the compiled system prompt sent to the LLM, including tool definitions.
`user_chat_message`	Contains the user message content as sent to the LLM.
`assistant_chat_message`	Contains the LLM's response content, which may include text or tool calls.
`tool_call`	Records a tool invocation, including the tool name and parsed input.
`tool_call_result`	Records the result of a tool invocation, including whether it succeeded or failed, and how long it took.
`final_response`	Contains the chatbot's final response returned to the user.
`execution_error`	Records an error that occurred during execution.

Session metadata¶

Logged once at the start of each execution with identifying information about the chatbot and session.

Field	Type	Description
`agent_rid`	String	The resource identifier of the chatbot.
`agent_version`	String	The version of the chatbot.
`session_rid`	String	The resource identifier of the session.
`caller_identifier`	String	An identifier for the caller that initiated the execution.

User request¶

Captures the full context of a user's message, including any retrieval context and application variables that were resolved for this execution.

Field	Type	Description
`session_rid`	String	The session identifier.
`user_query`	UserQuery	The user's message content, which may include text and media.
`contexts_from_profile`	List\	Retrieval contexts resolved from the chatbot's configured context sources. Each context represents a specific context type, such as Ontology, document, function-backed, or custom documentation.
`contexts_from_user_input`	List\	Retrieval contexts resolved from the user's input.
`application_variables`	List\	The application variables and their values at the time of the request. Each variable includes its identifier, name, value, and prompt visibility setting.
`node_id`	String	An identifier for the execution node.
`parent_node_id`	String (optional)	The identifier for the session node this execution was continued from.

Chat messages¶

The system_chat_message, user_chat_message, and assistant_chat_message events share a similar structure.

Field	Type	Description
`session_rid`	String	The session identifier.
`content`	List\	The message content, which may include text, media references, or tool calls.
`native_tools`	List\	The tools available to the LLM when using native tool calling mode. Only present in `system_chat_message`. Tools configured in prompted tool calling mode are included in the `content` field instead.

Tool calls¶

The tool_call event records when the chatbot invokes a tool.

Field	Type	Description
`session_rid`	String	The session identifier.
`tool_name`	String	The name of the tool that was called.
`parsed_tool_input`	Map\	A map of input parameter names to their values.

Tool call results¶

The tool_call_result event records the outcome of a tool invocation.

Field	Type	Description
`session_rid`	String	The session identifier.
`tool_name`	String	The name of the tool that was called.
`tool_call_id`	String (optional)	An identifier for the specific tool call.
`duration_milliseconds`	Long	How long the tool call took to execute, in milliseconds.
`result`	ToolResult	The result of the tool call, which is one of the following:

The ToolResult type is one of:

Success: Contains an llm_value (String) with the value returned to the LLM, and a variable_updates list with the variable name and new value for any application variables that were updated.
Failure: Contains an error_message (String) and a reason (String) describing why the tool call failed.

Final response¶

Contains the chatbot's final response to the user.

Field	Type	Description
`session_rid`	String	The session identifier.
`response`	FinalResponse	The response content, which is one of the following:

The FinalResponse type is one of:

Chatbot response: Contains a list of content items, which may include text and media.
Client tool call: Indicates the chatbot is deferring to a client-side action rather than returning a direct response.

Execution errors¶

Logged when an error occurs during execution.

Field	Type	Description
`session_rid`	String	The session identifier.
`error_name`	String	The name or type of the error.

Examples¶

Reconstruct a conversation¶

To reconstruct a chat conversation with a specific chatbot:

Filter logs where owning_rid matches the chatbot's RID.
Filter further by session_rid to isolate a single conversation.
For user messages, find entries where event_name is user_request and extract user_query from the payload.
For chatbot replies, find entries where event_name is final_response and extract response from the payload.

Evaluate tool performance¶

To analyze how tools are performing:

Find entries where event_name is tool_call_result. Each entry includes the tool_name, duration_milliseconds, and whether the call succeeded or failed.
To measure reliability, compare the count of success results versus failure results per tool.
To identify slow tools, sort by duration_milliseconds.
To detect retries, look for multiple tool_call and tool_call_result pairs with the same tool_name within a single traceId. A failure followed by another call to the same tool indicates the chatbot retried.

中文翻译¶

会话日志记录 (Session logging)¶

AIP Chatbot Studio 中的聊天机器人执行过程会以结构化事件的形式记录日志，这些日志可以导出到 Foundry 流式数据集 (streaming dataset) 中，用于监控和分析。每次向聊天机器人发送消息计为一次执行，每次执行都会被分配一个唯一的追踪标识符 (trace identifier)，用于关联所有相关的日志条目。

:::callout{theme="neutral"} 聊天机器人会话日志导出功能使用了配置日志记录特性。事件模式 (Event schemas) 和事件类型 (event types) 可能会发生变化。可能会新增事件，也可能会修改现有模式。 :::

前提条件 (Prerequisites)¶

导出聊天机器人会话日志需要启用日志导出功能。有关所需角色，请参阅导出权限。聊天机器人执行日志包含可追溯到发送消息用户的用户标识符 (user identifiers)，因此应谨慎管理对导出日志数据的访问权限。

要设置日志导出，请按照配置日志记录文档中的步骤操作，包括建议的应用标记 (Apply markings) 步骤，以对输出流式数据集应用安全标记 (security markings)，从而控制对日志数据的访问。

了解如何通过 AIP 可观测性 (AIP observability) 在平台中查看日志。

理解聊天机器人执行日志 (Understanding chatbot execution logs)¶

每条日志条目都包含日志模式 (log schema) 提供的通用字段，以及聊天机器人特有的事件数据。聊天机器人特有的数据包括一个 event_name 和一个描述执行过程中发生情况的 payload。

通用日志字段 (Common log fields)¶

以下字段包含在每条日志条目中，对于过滤和关联日志非常有用。来自执行流程中各个产品（如函数执行和语言模型使用）的日志也共享这些字段，因此您可以使用它们来追踪完整的执行请求：

字段	类型	描述
`traceId`	字符串 (String)	分配给每次执行的 Foundry 追踪标识符。来自同一次执行的所有日志条目共享相同的 `traceId`。使用此字段可以端到端地跟踪一个请求。
`uid`	字符串 (String)	发送消息的用户标识符。
`owning_rid`	字符串 (String)	发起执行的源执行器 (source executor) 的资源标识符，例如聊天机器人、函数或 Workshop 应用程序。例如，如果聊天机器人 A 调用了一个函数，该函数又调用了聊天机器人 B，那么所有生成的日志都将以聊天机器人 A 的 RID 作为 `owning_rid`。使用此字段可以查找源自特定源执行器的所有日志，无论调用链有多深。

聊天机器人事件数据 (Chatbot event data)¶

每个聊天机器人事件都包含一个标识事件类型的 event_name 和一个包含事件特定字段的 payload。每个事件负载都包含一个 session_rid，用于标识聊天机器人会话。使用此字段可以将来自单个对话的所有事件分组。

示例日志结构 (Sample log structure)¶

以下是一个 user_request 事件日志条目的简化示例：

{
  "time": "2024-01-15T09:30:00.000Z",
  "uid": "<USER_ID>",
  "traceId": "<TRACE_ID>",
  "content": {
    "event_name": "user_request",
    "payload": {
      "session_rid": "ri.aip-agents..session.<SESSION_ID>",
      "user_query": {
        "content": [
          {
            "text": {
              "content": "总结最新新闻发布会记录中的关键点。"
            }
          }
        ]
      }
    }
  }
}

事件类型 (Event types)¶

在聊天机器人执行过程中会记录以下事件类型：

事件名称 (Event name)	描述 (Description)
`session_metadata`	在执行开始时记录聊天机器人 RID、聊天机器人版本、会话 RID 和调用者标识符。
`user_request`	捕获用户的消息以及任何检索上下文 (retrieval context) 和应用程序变量 (application variables)。
`system_chat_message`	包含发送给 LLM 的已编译系统提示 (system prompt)，包括工具定义。
`user_chat_message`	包含发送给 LLM 的用户消息内容。
`assistant_chat_message`	包含 LLM 的响应内容，可能包括文本或工具调用 (tool calls)。
`tool_call`	记录一次工具调用，包括工具名称和解析后的输入。
`tool_call_result`	记录一次工具调用的结果，包括是否成功或失败，以及耗时。
`final_response`	包含返回给用户的聊天机器人最终响应。
`execution_error`	记录执行过程中发生的错误。

会话元数据 (Session metadata)¶

在每次执行开始时记录一次，包含聊天机器人和会话的标识信息。

字段	类型	描述
`agent_rid`	字符串 (String)	聊天机器人的资源标识符。
`agent_version`	字符串 (String)	聊天机器人的版本。
`session_rid`	字符串 (String)	会话的资源标识符。
`caller_identifier`	字符串 (String)	发起执行的调用者的标识符。

用户请求 (User request)¶

捕获用户消息的完整上下文，包括为此执行解析的任何检索上下文和应用程序变量。

字段	类型	描述
`session_rid`	字符串 (String)	会话标识符。
`user_query`	UserQuery	用户的消息内容，可能包含文本和媒体。
`contexts_from_profile`	列表\	从聊天机器人配置的上下文源解析出的检索上下文。每个上下文代表一个特定的上下文类型，例如本体 (Ontology)、文档、函数支持或自定义文档。
`contexts_from_user_input`	列表\	从用户输入解析出的检索上下文。
`application_variables`	列表\	请求时的应用程序变量及其值。每个变量包括其标识符、名称、值和提示可见性设置。
`node_id`	字符串 (String)	执行节点的标识符。
`parent_node_id`	字符串 (String，可选)	此执行所延续的会话节点的标识符。

聊天消息 (Chat messages)¶

system_chat_message、user_chat_message 和 assistant_chat_message 事件共享相似的结构。

字段	类型	描述
`session_rid`	字符串 (String)	会话标识符。
`content`	列表\	消息内容，可能包含文本、媒体引用或工具调用。
`native_tools`	列表\	使用原生工具调用 (native tool calling) 模式时，LLM 可用的工具。仅存在于 `system_chat_message` 中。配置为提示工具调用 (prompted tool calling) 模式的工具则包含在 `content` 字段中。

工具调用 (Tool calls)¶

tool_call 事件记录聊天机器人调用工具的时刻。

字段	类型	描述
`session_rid`	字符串 (String)	会话标识符。
`tool_name`	字符串 (String)	被调用的工具名称。
`parsed_tool_input`	映射\	输入参数名称到其值的映射。

工具调用结果 (Tool call results)¶

tool_call_result 事件记录一次工具调用的结果。

字段	类型	描述
`session_rid`	字符串 (String)	会话标识符。
`tool_name`	字符串 (String)	被调用的工具名称。
`tool_call_id`	字符串 (String，可选)	特定工具调用的标识符。
`duration_milliseconds`	长整型 (Long)	工具调用执行所花费的时间，以毫秒为单位。
`result`	ToolResult	工具调用的结果，为以下之一：

ToolResult 类型为以下之一：

成功 (Success): 包含一个 llm_value（字符串），即返回给 LLM 的值，以及一个 variable_updates 列表，其中包含已更新的应用程序变量的变量名和新值。
失败 (Failure): 包含一个 error_message（字符串）和一个 reason（字符串），描述工具调用失败的原因。

最终响应 (Final response)¶

包含聊天机器人给用户的最终响应。

字段	类型	描述
`session_rid`	字符串 (String)	会话标识符。
`response`	FinalResponse	响应内容，为以下之一：

FinalResponse 类型为以下之一：

聊天机器人响应 (Chatbot response): 包含一个内容项列表，可能包括文本和媒体。
客户端工具调用 (Client tool call): 表示聊天机器人正在委托给客户端操作，而不是返回直接响应。

执行错误 (Execution errors)¶

在执行过程中发生错误时记录。

字段	类型	描述
`session_rid`	字符串 (String)	会话标识符。
`error_name`	字符串 (String)	错误的名称或类型。

示例 (Examples)¶

重建对话 (Reconstruct a conversation)¶

要重建与特定聊天机器人的聊天对话：

过滤 owning_rid 与聊天机器人 RID 匹配的日志。
进一步按 session_rid 过滤，以隔离单个对话。
对于用户消息，查找 event_name 为 user_request 的条目，并从 payload 中提取 user_query。
对于聊天机器人回复，查找 event_name 为 final_response 的条目，并从 payload 中提取 response。

评估工具性能 (Evaluate tool performance)¶

要分析工具的性能：

查找 event_name 为 tool_call_result 的条目。每个条目都包含 tool_name、duration_milliseconds 以及调用是否成功或失败。
要衡量可靠性，比较每个工具的成功结果与失败结果的数量。
要识别慢速工具，按 duration_milliseconds 排序。
要检测重试，查找在单个 traceId 内具有相同 tool_name 的多个 tool_call 和 tool_call_result 对。一次失败后紧接着对同一工具的另一次调用表明聊天机器人进行了重试。