xiaoski/PicoBot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: xiaoski:main
Branches Tags
xiaoski:main
xiaoski:scheduled-run-design
xiaoski:feat/webui-p1
xiaoski:agent/tool-media-input
..
compare: xiaoski:agent/tool-media-input
Branches Tags
xiaoski:main
xiaoski:scheduled-run-design
xiaoski:feat/webui-p1
xiaoski:agent/tool-media-input

No commits in common. "main" and "agent/tool-media-input" have entirely different histories.
main ... agent/tool

177 changed files with 8546 additions and 51889 deletions
Show Stats Expand all files Collapse all files

2
.gitignore vendored
View File

@ -2,11 +2,9 @@
/webui/node_modules/
/webui/dist/
docker_build/
picobot-*.tar
reference/**
.env
*.env
Cargo.lock
.worktrees/
.superpowers/
design

70
AGENTS.md
View File

@ -7,10 +7,6 @@ This file is the operational contract for coding agents working in this reposito
- `cargo build` — build the binary
- `cargo run -- gateway` — start gateway server (binds `127.0.0.1:19876` by default)
- `cargo run -- chat` — connect to gateway as CLI client (default `ws://127.0.0.1:19876/ws`)
- `cargo run -- run "prompt"` — send one prompt through Gateway, print the terminal Turn, and exit; stdin, JSON, verbose progress, and timeout modes are available
- `cargo run -- reload` — validate and gracefully reload a running Gateway's configuration
- `cargo run -- health [--json]` — check core, configuration-dependent, and optional runtime dependencies without starting Gateway
- `docker compose up -d` — start the container with Gateway bound/published on `0.0.0.0:19876`; override `PICOBOT_GATEWAY_HOST`, `PICOBOT_PUBLISH_HOST`, or `PICOBOT_GATEWAY_PORT` as needed
- WebUI — start Gateway, then open `http://127.0.0.1:19876/`; no separate frontend build is required
- `cd webui && npm ci && npm run check && npm run build` — validate the Svelte WebUI independently (Node.js 20+); its local `dist/` is ignored
- `cargo build` automatically runs an incremental WebUI production build into Cargo `OUT_DIR`; it runs `npm ci` only when `package-lock.json` is not represented by the installed dependency stamp
@ -19,11 +15,9 @@ This file is the operational contract for coding agents working in this reposito
## Config
- Config load order: `~/.picobot/config.json` then fallback to `./config.json` (`Config::load_default` in `src/config/mod.rs`)
- `.env` files use a custom parser, not dotenv: load `<config-dir>/.env`, then `<workspace_dir>/.env`, while pre-existing process variables remain highest priority; config placeholders `<VAR_NAME>` use the merged values
- `.env` (cwd) is loaded with a custom parser, not via dotenv crate; env var placeholders `<VAR_NAME>` in config JSON are substituted
- Config example: `resources/templates/config.example.json` (released to `~/.picobot/` on first run)
- Runtime config loading ignores recoverable unknown/type-mismatched fields and invalid non-core named entries while retaining diagnostics and the raw file revision; malformed JSON, an unusable `default` Agent chain, and unsafe runtime construction remain fatal. WebUI writes are strict, and backend cleanup may remove only diagnosed paths from the same revision
- CLI TUI identity is stored in `~/.picobot/tui_client_id`; it is a non-secret stable chat scope used to restore dialogs across reconnects
- One-shot `run` uses a unique chat scope per invocation; for loopback Gateway URLs it authenticates `/ws` with `~/.picobot/web_admin_token`, while remote URLs use the existing paired CLI token
## Tests
@ -44,18 +38,15 @@ This file is the operational contract for coding agents working in this reposito
- **Gateway mode** (`cargo run -- gateway`): HTTP/WebSocket server; owns `GatewayState` which holds all services
- **Client mode** (`cargo run -- chat`): TUI chat client; connects to gateway via WebSocket, purely for user interaction
- **One-shot client mode** (`cargo run -- run "prompt"`): isolated CLI chat scope; connects to Gateway, waits for a terminal Turn, prints it, and exits
### Core Data Flow
```
Channel → MessageBus.inbound → Gateway inbound router/lane → SessionManager → per-session worker → AgentLoop
Channel → MessageBus.inbound → Gateway processor → SessionManager → per-session worker → AgentLoop
↑ │
└── Channel ← OutboundDispatcher ← per-conversation lane ← MessageBus.outbound
AgentLoop → TurnEvent → Session TurnController → latest TurnSnapshot → DeliveryCoordinator → per-turn TurnSink → Channel
WebSocket/Channel → MessageBus.control → Gateway control router → SessionManager (dialog operations)
WebSocket/Channel → MessageBus.control → Gateway processor → SessionManager (dialog operations)
Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler delivery policy → SessionManager/MessageBus
```
@ -67,12 +58,10 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
| `client` | TUI rendering, WebSocket client for CLI chat | `App`, `run()` |
| `channels` | External integrations (Feishu, CLI chat) | `ChannelManager`, `Channel` trait |
| `bus` | Bounded async queues and ordered outbound delivery lanes | `MessageBus`, `OutboundDispatcher`, `InboundMessage`, `OutboundMessage`, `ControlMessage` |
| `session` | Conversation lifecycle, dialog operations, per-session serialization, Turn state, persistence coordination | `SessionManager`, `Session`, `TurnController` |
| `agent` | LLM call loop, tool execution, context compression, semantic Turn events | `AgentLoop`, `TurnEvent` |
| `providers` | Native LLM streams normalized into text/reasoning/tool/usage chunks | `LLMProvider`, `ProviderChunk`, `create_provider()` |
| `delivery` | Snapshot projection, latest-wins throttling, terminal retry, per-turn sink lifecycle | `DeliveryCoordinator`, `TurnDeliveryService`, `PresentationPolicy` |
| `tools` | Agent tools and external adapters (bash, files, HTTP, browser, health) | `ToolRegistry`, `Tool`, `ToolExecutionContext` |
| `health` | Shared read-only dependency diagnostics for CLI/tool/slash entry points | `HealthService`, `HealthReport` |
| `session` | Conversation lifecycle, dialog operations, per-session serialization, persistence coordination | `SessionManager`, `Session` |
| `agent` | LLM call loop, tool execution, context compression | `AgentLoop` |
| `providers` | LLM API clients (OpenAI-compatible, Anthropic) | `LLMProvider` trait, factory `create_provider()` |
| `tools` | Agent tools (bash, file ops, http, web, get_skill) | `ToolRegistry`, `Tool` trait |
| `skills` | Skills loading, management, and prompt building | `SkillsLoader`, `Skill` |
| `storage` | SQLite persistence for sessions and messages | `Storage`, `SessionMeta`, `MessageMeta` |
| `scheduler` | Cron-based job scheduling, next-run computation | `Scheduler`, `Schedule`, `next_run_for_schedule()` |
@ -85,48 +74,26 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
### Functional Boundaries
- **Channels** publish inbound messages through `MessageBus`; outbound writes arrive through `OutboundDispatcher` or a per-turn `TurnSink`. They know nothing about sessions or LLM
- **Inbound contract** carries normalized sender/time/media plus `ChannelContext`; core routing may interpret `reply_to` but must treat platform-private context as opaque reply data
- **Channels** only send/receive messages via `MessageBus`; they know nothing about sessions or LLM
- **MessageBus** owns bounded inbound/outbound/control queues; outbound routing and retries belong to `OutboundDispatcher`, not the queue
- **SessionManager** owns session state, dialog operations, context construction, per-session work queues, active-Turn steering admission, and persistence coordination
- **TurnController** is the only owner of active Turn state; snapshots are complete latest-wins values, not token queues, and `Completed` is published only after atomic persistence succeeds
- **DeliveryCoordinator** projects active Turn snapshots without mutating history; it owns `TurnSink` lifecycle but no platform message IDs, which remain private to each sink
- **SessionManager** owns session state, dialog operations, context construction, per-session work queues, and persistence coordination
- **WorkManager** owns the single active plan per session, item state transitions, plan versions, and plan-change events; plans are optional and absent from ordinary chat context
- **Scheduler** owns one unified Scheduled Run path: claim-time JobRun snapshots, isolated Root/named Agent execution, exactly-once `complete_scheduled_run`, structured outcome, and policy-driven outbox delivery. Scheduled origin propagates to descendants, forces background delegation to foreground, and disables direct messaging, signals, Inbox completion slots, and cron/config management tools; `on_alert` suppresses only structured `ok`
- **AgentLoop** is stateless across turns; it receives prepared history, drains same-Turn steering only at safe model boundaries, calls LLM providers, executes tools, and returns one result
- **Context overflow recovery** is type-driven: before tool progress Session may commit one checkpoint and retry once; after any tool batch AgentLoop may retry the current Provider step once from its in-memory transcript, preserving current tool calls/results, and Session must never restart that Turn from durable history
- **Context compaction** keeps `messages` append-only and uses one active checkpoint per Session (`summary + first_retained_seq`) for deterministic Provider projection; `/compact`, Turn-boundary auto compaction, and overflow share the same compactor/CAS commit path, Session restoration never derives context from Timeline or calls a Provider, the Model `token_limit` (default 128K) is the hard window ceiling and an optional Agent `token_limit` can only narrow it via `min(agent, model)`, summary input is bounded from that effective window rather than a fixed cap, and the only automatic threshold is `context_tokens > context_window - effective_reserve`
- **AgentCatalog** is immutable per runtime generation; candidate preparation strictly validates trusted Markdown definitions, Provider profiles, tool/Skill allowlists, and delegation edges before activation. A definition that fails per-file validation (bad YAML, unknown provider/profile/model/tool/skill, or an explicit delegate edge to an absent target) is disabled for that generation only and reported via `load_errors` (exposed by `GET /api/agents`), never blocking startup or reload; config- and directory-trust-level failures remain fatal. Sub-Agent orchestration is an intrinsic, always-on mechanism (no feature switch). Named Agents support foreground (single/batch) and Root-initiated single background execution with durable run/inbox delivery; a built-in general-purpose definition is released to `~/.picobot/agents/` on first run. Background batches and nested background remain restricted
- **Scheduler** supports legacy direct-delivery jobs and managed `task`/`monitor` jobs; managed agents cannot call `send_message`, and `on_alert` suppresses only healthy informational results
- **AgentLoop** is stateless across turns; it receives prepared history, calls LLM providers, executes tools, and returns one result
- **WebUI management APIs** only expose allowlisted config/profile/log/storage operations; keep response limits, secret redaction, atomic config writes, and profile path allowlists intact
- **Configuration recovery** builds an effective typed `Config` from a request-local copy and never rewrites the source automatically; diagnostics use raw RFC 6901 paths, array recovery preserves original indexes, ordinary writes reject ignored fields, and cleanup requires the exact source SHA-256 revision before atomically deleting diagnosed paths
- **WebUI styling** uses the local Fluent 2 semantic tokens in `webui/src/styles.css`; page and component styles should consume the aliases instead of introducing independent hard-coded palettes, and must preserve selectable light/dark and brand-color themes, keyboard focus, responsive layout, and reduced-motion behavior. Browser-only appearance settings belong in `lib/theme.js`/`localStorage`, and `public/theme-init.js` must restore them before Svelte mounts
- **Gateway config reload** validates and prepares a complete next runtime generation before retiring the old one; close runtime admission before draining inbound lanes, Sessions, Scheduler jobs, and background sub-agents; MCP connects only during activation; host/port, workspace, and effective storage path remain restart-only, and runtime reload must never mutate process environment variables
- **WebUI slash completion** must consume the existing `get_slash_commands` WebSocket response; do not duplicate the backend command list in frontend source
- **Session token statistics** persist Provider-reported usage atomically with each completed Turn; WebUI `session_stats` and `/info [--json]` must consume the same SessionStats projection, and context occupancy must use the final request's prompt usage rather than accumulated Turn totals
- **WebUI chat rendering** sanitizes Markdown before inserting HTML; durable `turn_committed` deltas calibrate normal terminal Turns without a full history reload, and history must preserve structured tool-call metadata so calls and results remain independently collapsible
- **WebUI chat rendering** sanitizes Markdown before inserting HTML; session history must preserve structured tool-call metadata so calls and results remain independently collapsible
- **WebUI/TUI file transfer** streams bytes over authenticated HTTP and sends only short-lived upload IDs/attachment metadata over WebSocket; messages persist local media paths without guaranteeing later availability, and client responses must never expose those paths
- **WebUI/TUI same-turn media delivery** stages same-session `send_message(files=...)` media on the active Turn and commits it on the final assistant message, after durable tool-call history; safe raster formats should render as an inline preview with download fallback
- **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token. The same conjunction may authenticate only `/ws` for local one-shot `run`; it must never authorize management APIs. Never put bearer or admin tokens in URLs or logs
- **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token; never put bearer tokens in URLs or logs
- **Providers** are pure HTTP clients; no bus/session/channel awareness
- **Provider reasoning state** is private replay data: persist it, replay it only to the matching provider, and never expose it to clients, channels, or logs
- **Tools** are executed by `AgentLoop`; every invocation is normalized to `ToolOutput` and passes through `ToolOutputProcessor`. Plain `ToolResult` implementations use the default conversion, while artifact-producing tools declare model/user audience explicitly; model capability checks, final-reply attachment, channel delivery, and Provider serialization stay outside tools
- **Delegated tool access**: a named Agent's tool set is decided solely by its definition file (admin-authored). `delegate`, `emit_signal`, `get_skill` and `agent_task` are runtime-injected and must never be declared in `tools` (`get_skill` is the scoped-skill switch); `allowed_tools` can only narrow the definition, never expand it
- **No foreground wait tool**: Agents wait for asynchronous work by ending the Turn and letting queued completions/signals open a continuation Turn, or by polling status tools; there is no model-callable `sleep`/wait tool. Cancelling a Turn must still normalize active tool blocks to `Cancelled`
- **Skill enable/disable**: skills default to enabled; user-disabled skill names are persisted in `<config_dir>/skills_state.json` (skill files are never modified), and disabled skills are excluded from prompts, listings, and `get_skill` at load time
- **MCP enable/disable**: `mcp.servers[].enabled` defaults to true; disabled servers are skipped at activation (no connection attempt) and by health checks
- **Stateful tools** receive `ToolExecutionContext`; browser calls without `persistent_id` map each PicoBot dialog to an opaque transient agent-browser session, whose idle daemon timeout is controlled by `browser.idle_timeout_secs` (one hour by default). For long-running work an Agent may autonomously create a persistent identity and must pass its validated ID on every related action; persistent browser daemons disable idle auto-close, the same ID shares one agent-browser session and serialization gate across dialogs, and different IDs have independent sessions/gates. `browser_profiles` may create IDs, persist bounded semantic labels, list, or delete only validated IDs beneath the configured profile root. Do not introduce a global persistence mode switch, a default persistent ID, automatic per-dialog persistent Profiles, Fantoccini, ChromeDriver, WebDriver, model-controlled raw agent-browser session IDs, or arbitrary Profile paths
- **Health diagnostics** are read-only and share `HealthService` across `picobot health`, the `health` tool, `/health`, and authenticated WebUI `GET /api/health`; the public `GET /health` remains a lightweight liveness/version probe, while full checks run only on demand and must not install/fix dependencies, call Provider APIs, or expose secrets. Treat `fd`/`fdfind` as equivalent preferred backends; browser launch diagnostics must isolate their socket namespace from active agent-browser sessions and must not use `--quick`, which skips the live launch probe
- **Tools** are executed by `AgentLoop`; they receive raw arguments and normally return text. Tools that produce model-consumable media use the structured `execute_with_media` side channel; model capability checks and provider content-block serialization stay outside tools
### Concurrency and Lifecycle Invariants
- One session runs at most one Turn; ordinary input steers its active Turn by default, `/queue` explicitly waits for the next Turn, and different sessions may run concurrently
- Steering admission, final close, fallback, and `/stop` must be lossless and mutually exclusive: an input belongs to exactly the active Turn or the next-Turn FIFO, while `/stop` intentionally discards both
- Messages in one session are processed serially through a bounded queue; different sessions may run concurrently
- Outbound messages are ordered per `(channel, chat_id)`; a slow destination must not block unrelated destinations
- Active Turn delivery and ordinary outbound delivery share the same per-`(channel, chat_id)` write lock; never enqueue token deltas into MessageBus
- Slow Turn consumers may skip intermediate snapshots but must receive an explicit bounded terminal delivery; shutdown must call sink abort so platform cleanup remains possible
- Never hold a Session mutex across model, network, or database I/O unless a documented invariant requires it
- Slow work derived from session state must validate `worker_generation`/`state_version` before committing results
- A same-session `send_message` write may avoid advancing `state_version` only when its task-local Turn ID still owns that session's active Turn; all other writes remain versioned
- Related durable mutations use Storage transaction APIs; persistence failure must not leave silent memory/database divergence
- Long-lived gateway tasks must be owned by `TaskSupervisor`; connection-local tasks must be explicitly joined or aborted by their owner
- Connection, retry sleep, queue wait, and shutdown join paths must observe cancellation and have hard time bounds
@ -137,10 +104,10 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
### Key Constraints
- Gateway **changes working directory** to workspace in `GatewayState::new` (`src/gateway/mod.rs`)
- Session/message persistence uses SQLite via `sqlx`; DB stored in `<config_dir>/data/picobot.db` by default (`config_dir` is `~/.picobot`), independent of the workspace
- Session/message persistence uses SQLite via `sqlx`; DB stored in workspace as `picobot.db` by default
- `ChannelManager` owns the `MessageBus` and all channel instances
- `OutboundDispatcher` routes outbound messages to the correct channel via `ChannelManager`
- Layered config/workspace `.env` loading uses `unsafe { env::set_var(...) }` during single-threaded startup — don't move it after Gateway tasks are spawned or refactor it without understanding process-wide side effects
- Config `.env` loading uses `unsafe { env::set_var(...) }` — don't refactor to safer patterns without understanding side effects
## Change Workflow
@ -159,6 +126,3 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
- `docs/ARCHITECTURE.md` — maintainer-facing runtime design, invariants, lifecycle, and extension guidance
- `AGENTS.md` — concise operational rules for repository agents
- `resources/skills/about-picobot/references/` — runtime knowledge shipped to PicoBot; update it only when the assistant's built-in product knowledge must change
## Version Management
- 在每次功能变化、架构变化后，适当地更新整个产品的版本号。功能变化增加中段数字，ui变化、bug修改增加末端数字。 注意版本号变更是提交时和仓库中的版本比较，不要在长时间的工程中，不断变化版本号。

35
Cargo.toml
View File

@ -1,23 +1,21 @@
[package]
name = "picobot"
version = "1.22.0"
version = "1.1.2"
edition = "2024"
[dependencies]
reqwest = { version = "0.13.4", default-features = false, features = ["json", "rustls", "multipart"] }
reqwest = { version = "0.13.3", default-features = false, features = ["json", "rustls", "multipart"] }
serde = { version = "1.0", features = ["derive"] }
serde_path_to_error = "0.1"
regex = "1.13"
regex = "1.12"
serde_json = "1.0"
serde_yaml = "0.9"
async-trait = "0.1"
thiserror = "2.0.19"
tokio = { version = "1.53", features = ["full"] }
thiserror = "2.0.18"
tokio = { version = "1.52", features = ["full"] }
tokio-util = { version = "0.7", features = ["rt", "io"] }
dashmap = "6.2"
uuid = { version = "1.24", features = ["v4"] }
dashmap = "6.1"
uuid = { version = "1.23", features = ["v4"] }
axum = { version = "0.8", features = ["ws", "multipart"] }
tokio-tungstenite = { version = "0.30.0", features = ["rustls-tls-webpki-roots", "rustls"] }
tokio-tungstenite = { version = "0.29.0", features = ["rustls-tls-webpki-roots", "rustls"] }
futures-util = "0.3"
clap = { version = "4", features = ["derive"] }
dirs = "6.0.0"
@ -25,24 +23,23 @@ prost = "0.14"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json", "local-time"] }
tracing-appender = "0.2"
time = { version = "0.3", features = ["formatting", "local-offset"] }
anyhow = "1.0"
mime_guess = "2.0"
base64 = "0.23"
sha2 = "0.11"
base64 = "0.22"
sha2 = "0.10"
tempfile = "3"
cron = "0.17"
cron = "0.16"
chrono-tz = "0.10"
ratatui = "0.30"
crossterm = { version = "0.29", features = ["event-stream"] }
termimad = "0.35"
termimad = "0.34"
textwrap = "0.16"
unicode-width = "0.2"
chrono = "0.4"
sqlx = { version = "0.9", features = ["sqlite", "macros", "chrono", "runtime-tokio"] }
jieba-rs = "0.10"
sqlx = { version = "0.8", features = ["sqlite", "macros", "chrono", "runtime-tokio"] }
jieba-rs = "0.9"
which = "8"
rmcp = { version = "2.2", default-features = false, features = [
rmcp = { version = "1.7", default-features = false, features = [
"client",
"transport-child-process",
"transport-streamable-http-client-reqwest",
@ -52,12 +49,12 @@ http = "1"
encoding_rs = "0.8"
zstd = "0.13"
tar = "0.4"
fantoccini = { version = "0.22", default-features = false, features = ["rustls-tls"] }
portable-pty = "0.9"
[dev-dependencies]
dotenv = "0.15"
tower = "0.5"
tokio = { version = "1.53", features = ["test-util"] }
[build-dependencies]
zstd = "0.13"

29
Dockerfile
View File

@ -7,7 +7,7 @@
# Build image:
# docker build -t picobot .
#
# Run gateway: docker run -d -v ~/.picobot:/app/.picobot -p 19876:19876 picobot gateway --host 0.0.0.0
# Run gateway: docker run -d -v ~/.picobot:/app/.picobot -p 19876:19876 picobot gateway
# Run chat: docker run -it -v ~/.picobot:/app/.picobot picobot chat
# =============================================================================
@ -51,8 +51,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& pip3 install --no-cache-dir --break-system-packages uv
# Install Node.js and npx. agent-browser's npm package requires Node.js 24+.
RUN curl -fsSL https://deb.nodesource.com/setup_24.x | bash - \
# Install Node.js and npx
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& npm config set registry https://registry.npmmirror.com \
&& npm cache clean --force \
@ -69,14 +69,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
&& ln -sf /usr/bin/fdfind /usr/local/bin/fd \
&& rm -rf /var/lib/apt/lists/*
# Install Chromium plus the validated native agent-browser CLI. PicoBot talks
# to agent-browser over its JSON CLI contract; ChromeDriver/WebDriver is not used.
# Debian's chromium package is real (not a snap shim like Ubuntu 24.04).
# Install Chromium and chromedriver for browser automation
# Debian's chromium package is real (not a snap shim like Ubuntu 24.04)
RUN apt-get update && apt-get install -y --no-install-recommends \
chromium \
chromium-driver \
&& ln -sf /usr/bin/chromium /usr/local/bin/chrome \
&& npm install -g --registry=https://registry.npmjs.org agent-browser@0.33.0 \
&& npm cache clean --force \
&& ln -sf /usr/bin/chromedriver /usr/local/bin/chromedriver \
&& rm -rf /var/lib/apt/lists/*
# Create non-root user
@ -84,15 +83,14 @@ RUN useradd -m -s /bin/bash app
WORKDIR /app
# Install the pre-built binary in the standard executable path.
COPY target/release/picobot /usr/local/bin/picobot
# Copy pre-built binary from host
COPY target/release/picobot /app/picobot
# Copy config template
COPY resources/templates/config.example.json /app/config.json.example
# Create persistent application directories. Transient browser data stays in
# /tmp; optional Chrome profiles live under the persisted .picobot volume.
RUN mkdir -p /app/.picobot/workspace /app/.picobot/media /app/.picobot/browser/profiles && \
# Create required directories
RUN mkdir -p /app/.picobot/workspace /app/.picobot/media /app/.picobot/tmp && \
chown -R app:app /app
USER app
@ -100,10 +98,9 @@ ENV HOME=/app
# Environment variables for Chromium in containers
ENV CHROME_BIN=/usr/bin/chromium
ENV AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium
ENV TMPDIR=/tmp
ENV TMPDIR=/app/.picobot/tmp
ENTRYPOINT ["/usr/local/bin/picobot"]
ENTRYPOINT ["/app/picobot"]
CMD ["gateway"]
EXPOSE 19876

251
README.md
View File

@ -9,15 +9,12 @@ PicoBot 是一个用 Rust 编写的个人 AI 助手运行时。它在本地启
## 适合做什么
- 在终端里和本地 AI 助手持续对话。
- 从脚本或命令行发送一条任务，等待完整的模型/工具循环后只输出最终结果。
- 在 TUI 或浏览器中实时查看正文、思考过程和工具执行状态，并在完成后收敛到持久化历史。
- 在浏览器中查看日志、任务和记忆，修改运行配置与助手档案。
- 在 WebUI 顶栏查看当前会话的累计输入/输出 Token、上下文窗口和占用比例。
- 在浏览器中聊天，并查看日志、任务和记忆，修改运行配置与助手档案。
- 复杂任务可创建 session 级 Todo 计划，把不同子项并行委托给多个子 Agent；聊天页侧栏实时显示进度。
- 将同一套 Agent 能力接入飞书/Lark，并可选用单张卡片实时更新回复。
- 将同一套 Agent 能力接入飞书/Lark。
- 让 Agent 使用本地文件、Shell、搜索、HTTP、浏览器、MCP 工具完成任务。
- 把长期偏好、事实和历史摘要存成可检索记忆。
- 用 Cron 运行隔离的 Root 或命名 Agent，以结构化结果决定始终通知、异常通知或静默记录。
- 用 Cron 定时执行任务，并把结果发回目标渠道。
- 通过 Skills 为 Agent 注入项目知识和专用操作指南。
## 快速开始
@ -61,7 +58,6 @@ Gateway 首次启动时会把模板释放到 `~/.picobot/config.example.json`。
"model_id": "gpt-4o",
"temperature": 0.7,
"max_tokens": 4096,
"token_limit": 128000,
"input_type": ["text", "image"]
}
},
@ -69,20 +65,15 @@ Gateway 首次启动时会把模板释放到 `~/.picobot/config.example.json`。
"default": {
"provider": "openai",
"model": "gpt-4o",
"max_tool_iterations": 99
"max_tool_iterations": 99,
"token_limit": 128000
}
},
"workspace_dir": "~/.picobot/workspace"
}
```
`.env` 会在启动时由 PicoBot 自己解析，不依赖 dotenv。环境变量按以下顺序分层，越靠后优先级越高：
1. `config.json` 所在目录的 `.env`，作为所有 workspace 共用的基础配置。
2. `workspace_dir/.env`，用于当前 workspace 的覆盖值。
3. 启动 PicoBot 时进程中已有的环境变量，例如 Docker Compose 的 `environment`，优先级最高且不会被文件覆盖。
合并后的值既用于替换配置里的 `<OPENAI_API_KEY>` 等占位符，也会写入 PicoBot 进程环境，供 MCP Server 和工具子进程继承。`workspace_dir` 的位置由配置目录层和进程环境决定；workspace 自己的 `.env` 不能反过来修改 `workspace_dir`。
`.env` 会由 PicoBot 自己解析。配置里的 `<OPENAI_API_KEY>` 这类占位符会在 `.env` 和系统环境变量加载后替换。
### 4. 启动 Gateway
@ -90,33 +81,7 @@ Gateway 首次启动时会把模板释放到 `~/.picobot/config.example.json`。
cargo run -- gateway
```
默认监听 `127.0.0.1:19876`。Gateway 启动后会把进程工作目录切到 `workspace_dir`，默认 SQLite 数据库写到配置目录（`~/.picobot`）`data/` 下的 `picobot.db`，与 workspace 相互独立。
监听地址可通过配置文件或命令行覆盖。命令行参数优先于 `config.json`：
```json
{
"gateway": {
"host": "0.0.0.0",
"port": 19876
}
}
```
```bash
picobot gateway --host 0.0.0.0 --port 19876
```
Docker Compose 默认让容器内 Gateway 监听所有 IPv4 接口。监听地址、宿主机发布地址和端口均可通过环境变量调整：
```bash
PICOBOT_GATEWAY_HOST=0.0.0.0 \
PICOBOT_PUBLISH_HOST=192.168.1.10 \
PICOBOT_GATEWAY_PORT=19876 \
docker compose up -d
```
`PICOBOT_GATEWAY_HOST` 是容器内进程的监听地址；`PICOBOT_PUBLISH_HOST` 是 Docker 在宿主机上发布端口的地址。对局域网开放时应保持 `gateway.require_pairing=true`，并由防火墙限制可信网段。
默认监听 `127.0.0.1:19876`。Gateway 启动后会把进程工作目录切到 `workspace_dir`，默认 SQLite 数据库也会写到该 workspace 下的 `picobot.db`。
### 5. 启动 CLI 客户端
@ -126,35 +91,9 @@ docker compose up -d
cargo run -- chat
```
CLI 默认连接 `ws://127.0.0.1:19876/ws`。TUI 首次使用先运行 `picobot pair`，再执行 `picobot chat --pair-code <CODE>`；客户端令牌会以 `0600` 权限保存到 `~/.picobot/tui_auth_token`。如需指定地址，可使用 `--gateway-url`。
CLI 默认连接 `ws://127.0.0.1:19876/ws`。首次使用先运行 `picobot pair`，再执行 `picobot chat --pair-code <CODE>`；客户端令牌会以 `0600` 权限保存到 `~/.picobot/tui_auth_token`。如需指定地址，可使用 `--gateway-url`。
### 5.1 一次性执行
`run` 通过 Gateway 发送一条消息，复用正常的 SessionManager、AgentLoop 和工具调用流程，收到 Turn 终态后打印最终回复并退出：
```bash
picobot run "检查这个项目并总结测试结果"
printf '使用浏览器打开 example.com 并返回页面标题\n' | picobot run
```
默认情况下 stdout 只包含最终回复，便于管道和脚本消费。`--verbose` 把阶段和工具状态写到 stderr；`--json` 输出包含 session、turn、状态、正文、usage 和错误的一行 JSON；`--timeout` 设置最大等待秒数。超时或按下 Ctrl-C 时，客户端会先向当前会话发送 `/stop`。
连接本机回环地址时不需要人工配对：`run` 自动读取 `~/.picobot/web_admin_token`，Gateway 只有在真实 TCP 对端也是回环地址时才允许该凭据访问 `/ws`。每次调用使用独立的临时 chat scope，不会替换正在运行的 TUI 连接。连接远程 Gateway 时仍使用 `~/.picobot/tui_auth_token` 中已有的配对令牌。
### 5.2 健康检查
启动 Gateway 前可检查 PicoBot 核心命令、已启用功能的依赖、stdio MCP 命令和浏览器运行环境：
```bash
picobot health
picobot health --json
```
缺少核心或当前配置要求的依赖时退出码为 `1`；`rg` / `fd` 等有回退实现的加速项只会标记为 `DEGRADED`。运行中的 Gateway 也提供 `/health` 斜杠命令，Agent 可调用同名 `health` 工具，WebUI 的“配置 → 健康检查”可显示相同的结构化结果并手动复查；这些入口共享同一套只读检查逻辑。
Debian/Ubuntu 将同一个 fd 程序安装为 `fdfind`，两者都视为首选文件搜索后端；只有退回传统 `find` 时才提示性能警告。启用浏览器工具后，Health 除了检查 agent-browser 版本和浏览器路径，还会在隔离的临时 socket namespace 中执行完整离线 doctor，分别报告浏览器安装、真实 headless 启动和运行环境，因此可发现“文件存在但 Chrome 无法启动”或缺少 Linux 共享库等问题。Gateway 内按需检查还会报告定时任务的无效 Agent/渠道引用、投递积压、最近失败/超时/unknown、静默 unknown 和执行周期覆盖；Health 不会触发任务或连接 Provider。
### 5.3 使用 WebUI
### 5.1 使用 WebUI
Gateway 启动后直接打开：
@ -170,33 +109,24 @@ picobot pair
在浏览器配对页输入输出的 8 位代码即可。配对码 5 分钟内有效且只能使用一次；浏览器凭据由 HttpOnly Cookie 保存。需要撤销全部浏览器和 CLI 客户端时运行 `picobot pair --revoke-all`，再用新代码重新配对。
Docker 部署必须在 Gateway 容器内签发配对码，使请求来自容器自身回环地址并能读取映射目录中的管理密钥：
WebUI 随二进制嵌入，不需要 Node.js、npm 或单独部署静态文件，提供：
```bash
docker compose exec picobot picobot pair --gateway-url http://127.0.0.1:19876
```
不要从宿主机经发布端口直接调用签发接口；容器会把该连接识别为非回环来源并拒绝。`picobot` 已加入正式镜像的 `PATH`，可在容器 shell 中直接调用。
WebUI 随二进制嵌入，不需要 Node.js、npm 或单独部署静态文件。界面采用本地实现的 Microsoft Fluent 2 视觉系统，提供语义化中性色表面、品牌蓝交互状态、统一组件层级和完整的浅色/深色主题，并提供：
- 在线聊天、会话创建/切换、历史回放、流式 Markdown、独立思考区、实时工具状态、可折叠历史工具调用卡片，以及基于 Gateway 实时命令清单的 `/` 斜杠命令补全。
- 在线聊天、会话创建/切换、历史回放、Markdown 消息、可折叠工具调用卡片，以及基于 Gateway 实时命令清单的 `/` 斜杠命令补全。
- 文件选择、拖放和剪贴板图片上传；消息中的附件可预览或下载。附件按服务端路径引用，原文件移动或删除后历史附件可能不可用。
- “配置 → 外观”提供浅色/深色模式和六套 Fluent 品牌色；选择即时生效并保存在当前浏览器中，首次访问时明暗模式跟随系统偏好。
- 可持久化的浅色/深色主题，首次访问时跟随系统偏好。
- Cron 定时任务、最近运行记录和后台子任务状态。
- 当前聊天 session 的可展开 Todo 侧栏；计划变化时自动展开，其他 session 的变化显示未读提示。
- Knowledge/Timeline 记忆的分类与全文检索。
- 本地滚动日志的尾部查看、过滤和自动刷新。
- 健康检查结果：按核心与已配置功能展示通过、警告、失败及处理建议。
- `config.json`、`~/.picobot/USER.md`、`~/.picobot/AGENTS.md` 编辑。
配置接口会掩码 API Key、secret、password 和 token；保留 `********` 再保存不会覆盖原密钥。Gateway 加载历史配置时会忽略可恢复的未知字段、类型不匹配字段和不再可用的非核心命名条目，并在日志、Health 和 WebUI 配置页显示对应 JSON Pointer；原始文件不会被自动修改。WebUI 的“一键清除”由后端按配置 revision 删除这些已忽略项，文件在展示后发生变化时会拒绝覆盖；普通保存仍严格拒绝包含无效项的新配置。JSON 语法错误、不可用的 `default` Agent 链路以及无法安全构造 Gateway 的错误仍会阻止启动或重载。运行配置采用原子写入，保存或清理后可执行 `picobot reload` 或发送 `/reload` 热重载；`USER.md` 和 `AGENTS.md` 的修改用于后续构建的 Agent 上下文。
配置接口会掩码 API Key、secret、password 和 token；保留 `********` 再保存不会覆盖原密钥。运行配置采用原子写入并在 Gateway 重启后生效，`USER.md` 和 `AGENTS.md` 则会用于后续构建的 Agent 上下文。
WebUI 默认启用设备配对鉴权，管理 API 与 `/ws` 都拒绝未配对客户端；静态配对页、公开健康检查和配对提交接口除外。唯一的 WebSocket 例外是本机 `picobot run`：请求必须同时来自真实回环对端并持有权限为 `0600` 的 `~/.picobot/web_admin_token`，该管理令牌不能绕过任何管理 API 的设备鉴权。配对令牌只以 SHA-256 哈希写入 `~/.picobot/web_auth.json`。鉴权不提供传输加密；如果通过 `--host 0.0.0.0`、反向代理或端口转发暴露 Gateway，仍必须使用 TLS。可通过 `gateway.require_pairing=false` 显式关闭配对，但不建议在非隔离环境使用。
WebUI 默认启用设备配对鉴权，管理 API 与 `/ws` 都拒绝未配对客户端；静态配对页、公开健康检查和配对提交接口除外。令牌只以 SHA-256 哈希写入 `~/.picobot/web_auth.json`，本地配对码管理密钥位于权限为 `0600` 的 `~/.picobot/web_admin_token`。鉴权不提供传输加密；如果通过 `--host 0.0.0.0`、反向代理或端口转发暴露 Gateway，仍必须使用 TLS。可通过 `gateway.require_pairing=false` 显式关闭配对，但不建议在非隔离环境使用。
#### WebUI 开发
WebUI 源码位于 `webui/`，使用 Svelte 5、Vite 和无样式的 Bits UI 可访问组件原语。Fluent 2 外观由 `webui/src/styles.css` 中的本地语义令牌与 Svelte 组件实现，不加载 CDN 或外部 UI 运行库；明暗模式和品牌色分别保存在浏览器 `picobot-theme`、`picobot-accent` 项中，并由 `theme-init.js` 在应用挂载前恢复。运行发布版 PicoBot 不需要 Node.js；从源码编译或修改前端时需要 Node.js 20+：
WebUI 源码位于 `webui/`，使用 Svelte 5、Vite 和无样式的 Bits UI 可访问组件原语。运行发布版 PicoBot 不需要 Node.js；从源码编译或修改前端时需要 Node.js 20+：
```bash
cd webui
@ -225,17 +155,9 @@ picobot service stop
picobot service uninstall
```
修改配置后无需重启 systemd service：
```bash
picobot reload
```
该命令连接正在运行的 Gateway，先解析并校验新配置，再停止接收新工作，等待当前交互 Turn、Scheduler job 和后台子 Agent 到达安全边界后切换运行代。也可在聊天中发送 `/reload`，或让根交互 Agent 在用户明确要求时调用 `reload_config` 工具；子 Agent 与定时任务不能触发重载。监听地址、workspace 和数据库路径涉及进程级资源，不能热重载；修改这些字段时命令会保留旧配置并提示使用 `picobot service restart`。重载会主动断开 WebSocket，TUI/WebUI 随后可重新连接并从持久化历史恢复。受认证客户端可通过 `GET /api/config/reload/status` 查询 generation、切换阶段和最近错误。
unit 位于 `~/.config/systemd/user/picobot.service`，以执行 `service install` 时的当前目录作为初始工作目录。服务异常退出时由 systemd 自动重启；`stop` 和 `restart` 会通过 SIGTERM 触发 Gateway 的有界优雅关停。
TUI 会把一个随机客户端标识保存到 `~/.picobot/tui_client_id`，因此关闭并重新打开客户端后会恢复同一组 dialog 和最近使用的会话。界面支持流式正文、独立思考与工具状态、历史回放、会话列表与归档筛选、命令补全、Unicode/中文编辑、括号粘贴、多行输入和文件传输。
TUI 会把一个随机客户端标识保存到 `~/.picobot/tui_client_id`，因此关闭并重新打开客户端后会恢复同一组 dialog 和最近使用的会话。界面支持历史回放、会话列表与归档筛选、命令补全、Unicode/中文编辑、括号粘贴、多行输入和文件传输。
常用快捷键：
@ -255,11 +177,11 @@ WebUI/TUI 上传文件默认保存到 `~/.picobot/media/cli_chat`，单文件上
## 运行时数据流
用户消息进入 PicoBot 后，会被转换为统一的 inbound message，经由 MessageBus 交给 SessionManager。SessionManager 选择当前 dialog、组装上下文并创建活动 Turn；AgentLoop 消费 Provider 原生流、执行工具并发出结构化事件，TurnController 将它们归约成可丢中间帧的完整快照。DeliveryCoordinator 把快照投影给 TUI、WebUI 或 Channel，最终消息在 SQLite 原子提交成功后才进入 `Completed`。
用户消息进入 PicoBot 后，会被转换为统一的 inbound message，经由 MessageBus 交给 SessionManager。SessionManager 选择当前 dialog、组装上下文、调用 AgentLoop；AgentLoop 调用模型和工具，最终响应通过 outbound bus 回到原渠道。
详细时序和失败语义见 [架构文档：消息与控制数据流](docs/ARCHITECTURE.md#4-消息与控制数据流)。
同一 session 始终只运行一个 Turn，不同 session 可以并发。Turn 执行期间新发的普通消息默认 steering 当前工作：系统在完整工具批次后或最终回复边界把它作为真实用户消息加入下一次模型调用；使用 `/queue <message>` 可明确等当前 Turn 完成后再处理，使用 `/stop` 可中断当前 Turn 并清空等待输入。Steering mailbox 和 session 队列都有界且带可靠回退。活动 Turn 使用 latest-wins 快照，慢展示端只跳过中间状态，不反压模型。普通出站消息按 `(channel, chat_id)` 分 lane 保序，两条投递路径共享目标写锁。
同一 session 的普通消息由专属有界队列串行处理，不同 session 可以并发；出站消息按 `(channel, chat_id)` 分 lane 保序，慢渠道不会阻塞其他目标。Gateway 的长生命周期任务统一由 `TaskSupervisor` 取消和限时回收。
核心边界：
@ -269,11 +191,10 @@ WebUI/TUI 上传文件默认保存到 `~/.picobot/media/cli_chat`，单文件上
| `bus` | 异步消息队列，承载 inbound、outbound、control 三类消息 |
| `session` | 管理会话生命周期、dialog 操作、上下文、记忆召回、压缩和持久化 |
| `agent` | 执行无状态 LLM/tool 循环，处理模型响应和工具调用 |
| `providers` | OpenAI 兼容接口和 Anthropic Messages API 的原生流解析与回放 |
| `delivery` | 活动 Turn 的展示过滤、latest-wins 节流、终态投递与 TurnSink 生命周期 |
| `providers` | OpenAI 兼容接口和 Anthropic Messages API 客户端 |
| `tools` | Agent 可调用工具集合 |
| `storage` | SQLite schema、CRUD、消息和任务持久化 |
| `scheduler` | 原子领取 occurrence，运行隔离的 Scheduled Agent，并通过持久化 outbox 按策略投递结构化结果 |
| `scheduler` | 领取定时任务，执行普通/巡检 Agent，并按投递策略记录或发送结果 |
| `work` | 管理 session 级单 active plan、并行子项状态和 WebSocket 变更事件 |
| `skills` | 加载 Skill，并把 Skill 指南注入系统提示 |
| `mcp` | 连接 MCP Server，将远端工具包装成普通 Tool |
@ -288,8 +209,6 @@ WebUI/TUI 上传文件默认保存到 `~/.picobot/media/cli_chat`，单文件上
| `cli_chat` | Ratatui 终端客户端，通过 WebSocket 连接 Gateway |
| `feishu` | 飞书/Lark 消息、反应、文件上传下载和媒体引用 |
飞书默认只接受 `allow_from` 中的用户，且群聊消息必须明确 @ 机器人（可通过 `channels.feishu.require_mention=false` 关闭）。回复会使用飞书原生引用/话题语义保持在原消息位置。默认只发送终态结果；设置 `channels.feishu.live_updates=true` 后会创建一张卡片并持续编辑，`live_update_interval_ms` 默认 500ms，运行时限制在 250–5000ms。外部渠道始终不会收到模型 reasoning。
### 会话
Session ID 使用三段式：
@ -309,15 +228,12 @@ Session ID 使用三段式：
| `/switch <dialog_id>` | 切换 dialog |
| `/rename <title>` | 重命名当前 dialog |
| `/delete` | 删除当前 dialog 并创建新 dialog |
| `/compact` | 强制把可压缩的旧完整 Turn 汇总为活动 checkpoint；不改写原始历史 |
| `/info [--json]` | 查看当前 dialog、累计 Token 与上下文窗口信息；可选 JSON 输出 |
| `/compact` | 手动压缩上下文 |
| `/info` | 查看当前 dialog 信息 |
| `/dump` | 导出当前 dialog 为 Markdown |
| `/mcp` | 查看 MCP 服务器和工具状态 |
| `/health` | 检查 PicoBot 运行依赖 |
| `/queue <message>` | 等当前 Turn 完成后再把消息作为下一 Turn 处理 |
| `/stop` | 停止当前任务并清空队列 |
| `/todo [done\|cancel]` | 查看、完成或取消当前 session 的任务计划 |
| `/reload` | 校验并重新加载 Gateway 配置 |
| `/?`, `/help` | 查看帮助 |
### 记忆
@ -329,9 +245,7 @@ PicoBot 有两类记忆：
| Knowledge | 偏好、事实、项目规则、长期可复用信息 | 长期保留，手动删除 |
| Timeline | 长对话压缩后的历史摘要 | 默认保留 90 天 |
每轮处理用户消息时，MemoryManager 会按用户输入召回 Knowledge，并作为运行时上下文附加到本轮用户消息。自动召回是确定性的关键词检索（jieba 分词 + FTS5），按「词项相关度 + 重要度 + 时效」加权并通过相关性/综合分双门槛过滤，条数受 `memory.recall_limit` 约束，搜索受 `memory.recall_timeout_ms` 硬超时保护，超时或无关时本轮不注入。长会话使用一个活动 checkpoint：累计摘要加 `first_retained_seq` 之后的原始消息尾部构成模型上下文，原始消息、工具调用结果、ID 和 seq 均不会被压缩改写。旧工具结果会保留在原始历史中，但 checkpoint 边界推进后不再永久占用 Provider 上下文。成功的语义摘要还会 best-effort 保存为 Timeline，供 `timeline_recall` 检索；Timeline 不参与会话恢复正确性。Scheduler 默认创建一个每日维护任务，按 `memory.timeline_retention_days` 清理过期 Timeline；结果通过 `complete_scheduled_run` 结构化提交，Knowledge 不会被自动删除。
模型的 `models.<name>.token_limit` 给出上下文窗口上限，未配置时默认为 128,000；Agent 的 `agents.<name>.token_limit` 是可选的收紧上限，两者都有配置时有效窗口取二者最小值，因此 Agent 不能扩大模型窗口。自动压缩使用保留量阈值 `context_tokens > context_window - effective_reserve`，默认 reserve 为 16,384 tokens，并尽量原样保留最近 20,000 tokens。小窗口会自动把 reserve 限制为窗口的一半、把近期保留量限制为有效阈值的一半。摘要请求不使用固定 32K 输入上限，而是按有效窗口扣除摘要输出、提示词和安全余量；超大历史只在摘要请求副本中按“已有 checkpoint + 最新消息优先”生成有界 head/tail 转录，SQLite 原文不变。手动 `/compact` 跳过自动阈值；换成小模型后若发送前预检已发现硬超限，或首次请求返回真实 context overflow，语义摘要不可用时才使用明确标记的确定性降级裁剪，正式请求最多重试一次。若 overflow 发生在工具已经执行之后，AgentLoop 只在当前内存转录上裁掉旧完整 Turn 并重试当前模型步骤一次，不会从数据库历史重跑工具。
每轮处理用户消息时，MemoryManager 会按用户输入召回 Knowledge，并作为运行时上下文附加到本轮用户消息。当前召回上限固定为 5；`memory.recall_limit` 已支持解析但尚未接入 worker。上下文压缩产生的摘要会保存为 Timeline，后续可通过 `timeline_recall` 工具检索。Scheduler 默认创建一个每日维护巡检，按 `memory.timeline_retention_days` 清理过期 Timeline；Knowledge 不会被自动删除。
### 工具
@ -346,19 +260,13 @@ PicoBot 有两类记忆：
| `http_request` / `web_fetch` | HTTP 请求和网页文本抽取 |
| `get_skill` | 列出或读取本地 Skill |
| `memory_store` / `memory_recall` / `timeline_recall` / `memory_forget` | 长期记忆操作 |
| `reload_config` | 在用户明确要求时校验并重新加载 Gateway 配置 |
| `delegate` | 向具名 Agent 委托单个或批量任务；`foreground` 等待结果，`background` 异步执行。批量 foreground 会并发运行并按请求顺序聚合 |
| `agent_task` | 查询/列出/读取结果/取消已持久化的具名 Agent run（仅编排启用时注册） |
| `emit_signal` | 后台 run 向主 Agent 发送结构化内部信号（queue/steer 投递；仅带 signal 契约的 run 注册） |
| `delegate` | 启动 inline、background 或 parallel 子 Agent |
| `todo` | 为复杂、多轮任务创建并更新当前 session 的持久化计划 |
| `send_message` | 向指定渠道或当前会话发送消息，可附带文件/截图；WebUI/TUI 当前 Turn 的附件并入最终回复 |
| `send_message` | 向指定渠道发送消息 |
| `chat_manager` | 查看渠道、会话和历史消息 |
| `cron_add/list/remove/enable/disable/update` | 管理定时任务；`agent_id` 选择 Root/命名 Agent，`delivery_policy` 支持 `always/on_alert/never` |
| `cron_runs` | 查询定时任务的结构化执行结果、诊断和投递状态，包括静默任务 |
| `cron_add/list/remove/enable/disable/update` | 管理定时任务 |
| `routine_maintenance` | 安全清理超过保留期的 Timeline，不删除 Knowledge |
| `health` | 检查核心、配置相关和可选运行依赖 |
| `browser` | 可选 agent-browser 浏览器自动化；默认按 dialog 临时使用，长期任务可用 `persistent_id` 复用个人 Profile |
| `browser_profiles` | 创建、设置语义标签、列出或删除浏览器持久 ID 及其 Profile 目录 |
| `browser` | 可选 WebDriver 浏览器自动化 |
| MCP tools | 从配置的 MCP Server 动态发现并注册 |
### Skills
@ -380,8 +288,6 @@ Skill 是包含 `SKILL.md` 的目录。加载优先级从高到低：
| `providers` | LLM Provider 配置 |
| `models` | 模型参数与输入能力 |
| `agents` | Agent 使用哪个 provider/model |
| `context_compaction` | 上下文自动压缩开关、预留 token 与近期原样保留量 |
| `agent_orchestration` | 具名子 Agent 定义目录与编排上限 |
| `gateway` | HTTP/WebSocket、数据库、调度器、后台任务限制 |
| `client` | CLI 客户端默认 Gateway URL |
| `channels` | 渠道配置，目前主要是飞书/Lark |
@ -400,107 +306,13 @@ Skill 是包含 `SKILL.md` 的目录。加载优先级从高到低：
| `gateway.max_concurrent_background_tasks` | `10` |
| `gateway.scheduler.enabled` | `true` |
| `client.gateway_url` | `ws://127.0.0.1:19876/ws` |
| `context_compaction.enabled` | `true` |
| `context_compaction.reserve_tokens` | `16384` |
| `context_compaction.keep_recent_tokens` | `20000` |
| `memory.recall_limit` | `5` |
| `memory.recall_min_relevance` | `0.25` |
| `memory.recall_min_score` | `0.25` |
| `memory.recall_recency_half_life_days` | `30` |
| `memory.recall_timeout_ms` | `1000` |
| `memory.recall_limit` | `5`（当前运行时固定为 5） |
| `memory.timeline_retention_days` | `90` |
| `mcp.tool_timeout_secs` | `180` |
| `mcp.servers[].tool_settings` | `{}`；可按工具名声明 `read_only` / `exclusive`，并发状态自动推导 |
| `browser.enabled` | `true` |
| `channels.feishu.live_updates` | `false` |
| `channels.feishu.live_update_interval_ms` | `500` |
| `channels.feishu.require_mention` | `true` |
| `channels.feishu.max_image_bytes` | `10485760` |
| `channels.feishu.max_file_bytes` | `26214400` |
| `channels.feishu.media_dir_max_bytes` | `536870912` |
| `channels.feishu.request_timeout_secs` | `30` |
### 具名子 Agent（Phase 1）
PicoBot 在 Gateway 候选运行代构造时从配置目录下的 `definitions_dir` 加载 `*.md`（子 Agent 编排是内在机制，始终启用）。相对路径按 `config.json` 所在目录解析，且 canonical path 不得逃逸该目录；角色文件、Provider profile、工具、Skill 和委托边任一无效都会拒绝启动或热重载。支持具名 `foreground`（单/批量）与 Root 发起的具名 `background`（单任务或 `tasks[]` 批量）：每个 run 独立落库、预留 completion 槽、完成后由主 Agent 的 continuation Turn 单独汇总（空闲时完成即返回），可配合 `emit_signal`（queue/steer）推送内部信号。子 Agent 发起的 background 尚未开放（旧匿名 general 已移除）。
```md
---
id: researcher
description: 搜索、阅读并整理技术资料
llm_profile: research
tools:
- file_read
- file_search
- content_search
delegates:
- reviewer
limits:
timeout_secs: 900
max_iterations: 24
---
# Role
你是一名严谨的研究 Agent，只返回与任务有关的结论和证据。
```
每个具名 Agent 的工具集完全由其 Markdown `tools` 列表决定（管理员显式授权），不再有工具侧的可派发门槛；也可内联 `provider`/`model` 直接指定模型（或沿用 `llm_profile` 引用顶层 `agents` key）。`delegate`/`emit_signal`/`get_skill`/`agent_task` 为运行时注入工具，不能写进 `tools`（分别由 `delegates`/`signal`/`skills` 字段派生），`get_skill` 例外作为启用 scoped skill 的开关。每个定义可用 `enabled: false` 单独禁用（保留在磁盘但不加载）。主 Agent 可委托给任意具名子 Agent；子 Agent 能否继续委托由 `delegates` 决定——不写该字段时默认仅可委托内置 `general-purpose`，写 `[]` 表示不可继续委托，写 `["*"]` 表示可委托任意子代理，写列表则按列表指定（self 与祖先在运行时始终被拒绝）。WebUI「子 Agent」页可直接增删改定义、启停并选择工具/Skill/Provider/Model 与委托范围。
| `browser.enabled` | `false` |
更完整的配置字段说明见 [resources/skills/about-picobot/references/config.md](resources/skills/about-picobot/references/config.md)。
## agent-browser 安装与使用
PicoBot 不再使用 Fantoccini、ChromeDriver 或 WebDriver。上层仍暴露一个稳定的 `browser` 工具，底层通过 agent-browser `0.33.0` 的 JSON CLI 驱动原生 Rust daemon 和 Chrome CDP。先安装 CLI 与浏览器：
```bash
# 推荐；npm 只负责安装预编译 CLI
npm install -g agent-browser@0.33.0
agent-browser install
# Linux 需要同时补齐系统库时
agent-browser install --with-deps
# 或直接通过 Rust 工具链安装
cargo install agent-browser --version 0.33.0 --locked
agent-browser install
# macOS 也可使用 Homebrew
brew install agent-browser
agent-browser install
```
已有 Chrome/Chromium 时可在配置中设置 `browser_executable_path`，或通过 `AGENT_BROWSER_EXECUTABLE_PATH` 指定。Docker 镜像已固定安装 agent-browser `0.33.0` 与 Debian Chromium，不包含 ChromeDriver。启用示例：
```json
{
"browser": {
"enabled": true,
"command": "agent-browser",
"headless": true,
"browser_executable_path": null,
"max_sessions": 4,
"idle_timeout_secs": 3600,
"command_timeout_secs": 120,
"max_output_chars": 50000,
"content_boundaries": true,
"allowed_domains": [],
"allow_private_hosts": false,
"artifact_dir": "~/.picobot/media/browser",
"persistence": {
"profile_dir": "~/.picobot/browser/profiles"
}
}
}
```
浏览器没有全局“持久模式”开关，而是按每次调用分流：不传 `persistent_id` 时使用当前 dialog 的普通临时浏览器，连续一小时没有操作后默认自动关闭；涉及长期工作、需要保持浏览器进程或保留登录和站点状态时，Agent 可以自主调用 `browser_profiles(create,label=...)` 生成 `picobot-profile-<uuid>`，并在该工作的后续每个 `browser` action 中持续传入同一个 ID。持久浏览器禁用 daemon 空闲自动关闭，只会在显式 `browser(close)` 或 Profile 删除时关闭。也可以用 `set_label` 随时修改语义化标签。PicoBot 不设置默认 ID，也不会按 dialog 自动选择持久 Profile。同一 ID 跨 dialog 共享 agent-browser session 和串行锁，不同 ID 使用各自的 session、锁与 Chrome Profile，因而可以并发操作。Cookie、localStorage、IndexedDB、Service Worker、缓存和标签随各自目录持久化。
`browser_profiles` 支持 `create`、`set_label`、`list`、`delete`；`list` 返回 ID、标签、目录和 active 状态，浏览器仍始终用不可变 ID 选择，重命名标签不会破坏现有调用。Profile 根目录位于 `~/.picobot` 内，现有 Docker `picobot_data` 卷会一并持久化。agent-browser 无法同时保证 Profile 复用与 `allowed_domains` 域名隔离；配置非空白名单后，普通临时浏览器仍可用，持久身份的创建和使用会被拒绝，health 会给出可选能力警告。
缺少 agent-browser 或 Chrome 不阻止 Gateway 启动，但实际调用会失败并给出安装提示，`picobot health` 也会提前报告。修改后建议先运行 health，再启动或重载 Gateway。旧的 `webdriver_url`、`chrome_path` 配置已删除，出现这两个字段时配置校验会明确失败。实际使用仍由 Agent 调用 `browser`：`open` → `snapshot` 获取 `@e1` 等引用 → `click` / `fill` / `type` → 页面变化后重新 `snapshot`。截图保存到受控产物目录，通过统一工具输出管线交给多模态模型，并默认附到本轮最终回复给用户查看，不再生成 Base64 工具文本；仅需模型内部检查时可显式设置 `present_to_user=false`。
详细开发分层、进程协议、并发/安全边界和故障语义见 [agent-browser 集成设计](docs/AGENT_BROWSER_INTEGRATION.md)。
## WebSocket API
Gateway 暴露：
@ -529,7 +341,7 @@ Inbound 消息类型：
| `get_slash_commands` | 无 |
| `ping` | 无 |
Outbound 消息类型包括活动 Turn 使用的 `turn_updated`，以及 `assistant_response`、`error`、`session_established`、`session_created`、`session_list`、`session_loaded`、`session_history`、`session_renamed`、`session_archived`、`session_deleted`、`history_cleared`、`slash_commands_list`、`pong`、`command_executed` 和 `system_notification`。`turn_updated` 每次携带完整快照和单调 revision，客户端只替换当前 session 的活动 Turn；`assistant_response` 保留给独立完整消息。历史消息可包含 reasoning、turn/iteration、completion status 和结构化工具元数据，但不会暴露 Provider 私有回放状态。
Outbound 消息类型包括 `assistant_response`、`error`、`session_established`、`session_created`、`session_list`、`session_loaded`、`session_history`、`session_renamed`、`session_archived`、`session_deleted`、`history_cleared`、`slash_commands_list`、`pong`、`command_executed` 和 `system_notification`。其中异步 `assistant_response` / `system_notification` 可携带 `session_id`，客户端应避免把迟到结果显示到其他 dialog。
## 测试
@ -558,7 +370,6 @@ src/
channels/ CLI chat 和飞书/Lark 集成
client/ Ratatui 终端 UI
config/ 配置加载、环境变量替换、路径展开
delivery/ 活动 Turn 快照投影、节流与 TurnSink 生命周期
gateway/ Axum HTTP/WebSocket server 和 GatewayState 装配
mcp/ MCP 客户端连接和工具包装
memory/ 记忆管理和记忆类型
@ -587,6 +398,7 @@ docs/ 面向维护者和 Agent 的架构与开发文档
| `reqwest` | LLM 和 HTTP 客户端 |
| `ratatui`, `crossterm`, `termimad` | 终端 UI |
| `rmcp` | MCP 客户端 |
| `fantoccini` | 可选浏览器自动化 |
| `cron`, `chrono-tz` | 定时任务 |
| `jieba-rs` | 中文记忆检索分词 |
| `zstd`, `tar` | 内置 Skill 打包和释放 |
@ -594,7 +406,6 @@ docs/ 面向维护者和 Agent 的架构与开发文档
## 进一步阅读
- [维护者架构文档](docs/ARCHITECTURE.md)
- [配置热重载设计与实现](docs/CONFIG_HOT_RELOAD_DESIGN.md)
- [WebUI 与 TUI 文件收发设计](docs/FILE_TRANSFER_DESIGN.md)
- [内置 Skill：架构机制](resources/skills/about-picobot/references/architecture.md)
- [配置说明](resources/skills/about-picobot/references/config.md)

42
build.rs
View File

@ -13,24 +13,6 @@ fn main() {
let skills_out_dir = Path::new(&out_dir).join("skills");
fs::create_dir_all(&skills_out_dir).unwrap();
println!("cargo:rerun-if-changed=resources/agents");
let agents_dir = Path::new("resources/agents");
let agents_out_dir = Path::new(&out_dir).join("agents");
fs::create_dir_all(&agents_out_dir).unwrap();
let mut agents = Vec::new();
if let Ok(entries) = fs::read_dir(agents_dir) {
for entry in entries.flatten() {
let path = entry.path();
if path.extension().and_then(|e| e.to_str()) != Some("md") {
continue;
}
let agent_name = path.file_stem().unwrap().to_str().unwrap().to_string();
fs::copy(&path, agents_out_dir.join(format!("{agent_name}.md"))).unwrap();
agents.push(agent_name);
}
}
agents.sort();
let mut skills = Vec::new();
if let Ok(entries) = fs::read_dir(skills_dir) {
@ -76,29 +58,6 @@ pub static EMBEDDED_SKILLS: &[EmbeddedSkill] = &[
let generated_path = Path::new(&out_dir).join("embedded_skills.rs");
let mut f = fs::File::create(&generated_path).unwrap();
f.write_all(code.as_bytes()).unwrap();
let mut agent_code = String::from(
r#"pub struct EmbeddedAgent {
pub name: &'static str,
pub content: &'static str,
}
pub static EMBEDDED_AGENTS: &[EmbeddedAgent] = &[
"#,
);
for name in &agents {
let file_path = agents_out_dir
.join(format!("{name}.md"))
.to_string_lossy()
.to_string();
agent_code.push_str(&format!(
" EmbeddedAgent {{ name: \"{name}\", content: include_str!(\"{file_path}\") }},\n",
));
}
agent_code.push_str("];\n");
let agent_path = Path::new(&out_dir).join("embedded_agents.rs");
let mut f = fs::File::create(&agent_path).unwrap();
f.write_all(agent_code.as_bytes()).unwrap();
}
fn build_webui(out_dir: &Path) {
@ -110,7 +69,6 @@ fn build_webui(out_dir: &Path) {
"webui/package-lock.json",
"webui/svelte.config.js",
"webui/vite.config.js",
"webui/public",
] {
println!("cargo:rerun-if-changed={path}");
}

49
config.json
View File

@ -1,49 +0,0 @@
{
"providers": {
"aliyun": {
"type": "openai",
"base_url": "https://example.invalid/v1",
"api_key": "test-only-not-a-real-key",
"extra_headers": {}
}
},
"models": {
"qwen-plus": {
"model_id": "qwen-plus",
"temperature": 0.0,
"max_tokens": 100,
"token_limit": 128000,
"input_type": ["text"]
}
},
"agents": {
"default": {
"provider": "aliyun",
"model": "qwen-plus",
"max_tool_iterations": 20
}
},
"agent_orchestration": {
"definitions_dir": "agents",
"max_tree_depth": 4,
"max_runs_per_tree": 16,
"max_concurrent_runs": 6,
"max_concurrent_runs_per_session": 4,
"max_concurrent_provider_steps": 8,
"max_concurrent_provider_steps_per_session": 4,
"max_concurrent_tool_steps": 16,
"max_concurrent_tool_steps_per_session": 8,
"max_pending_inbox_events_per_session": 128,
"inbox_event_ttl_hours": 168,
"max_inbox_delivery_attempts": 8,
"max_user_turn_burst_before_inbox": 4,
"max_inbox_wait_secs": 30
},
"gateway": {
"host": "127.0.0.1",
"port": 19877,
"require_pairing": true
},
"channels": {},
"workspace_dir": "/tmp/picobot-test-workspace"
}

21
docker-compose.test.yml
View File

@ -1,21 +0,0 @@
services:
picobot:
build:
context: .
dockerfile: Dockerfile
image: picobot:1.4.0
container_name: picobot-test
restart: unless-stopped
ports:
- "${PICOBOT_PUBLISH_HOST:-127.0.0.1}:${PICOBOT_GATEWAY_PORT:-19876}:${PICOBOT_GATEWAY_PORT:-19876}"
volumes:
- "${HOME}/.picobot:/app/.picobot"
environment:
RUST_LOG: "${RUST_LOG:-info}"
TZ: "${TZ:-Asia/Shanghai}"
command:
- gateway
- --host
- "${PICOBOT_GATEWAY_HOST:-0.0.0.0}"
- --port
- "${PICOBOT_GATEWAY_PORT:-19876}"

9
docker-compose.yml
View File

@ -4,19 +4,14 @@ services:
container_name: picobot
restart: unless-stopped
ports:
- "${PICOBOT_PUBLISH_HOST:-0.0.0.0}:${PICOBOT_GATEWAY_PORT:-19876}:${PICOBOT_GATEWAY_PORT:-19876}"
- "19876:19876"
volumes:
- ~/.picobot/config.json:/app/.picobot/config.json:ro
- picobot_data:/app/.picobot
environment:
- RUST_LOG=info
- TZ=Asia/Shanghai
command:
- gateway
- --host
- ${PICOBOT_GATEWAY_HOST:-0.0.0.0}
- --port
- ${PICOBOT_GATEWAY_PORT:-19876}
command: gateway
volumes:
picobot_data:

210
docs/AGENT_BROWSER_INTEGRATION.md
View File

@ -1,210 +0,0 @@
# agent-browser 集成设计
本文描述 PicoBot 1.4.0 的浏览器工具实现。目标是在保持模型侧稳定 `browser` 协议的同时，用 agent-browser 完全替代 Fantoccini、ChromeDriver 和 WebDriver，并让临时会话、单用户多持久 Profile、管理操作、产物和健康检查拥有明确边界。
## 1. 选择与边界
PicoBot 使用 agent-browser CLI 的 `--json` 协议，不直接链接其内部 crate，也不把 agent-browser MCP Server 原样暴露给模型。
原因：
- agent-browser 是原生 Rust CLI + daemon，daemon 通过 Chrome CDP 驱动浏览器；CLI 进程很短，浏览器状态跨命令保存在 daemon 中。
- CLI 是项目的稳定公开边界，PicoBot 不需要依赖 agent-browser 的内部 Rust 模块布局。
- PicoBot 包装层可统一管理个人浏览器身份、限制并发和输出、校验 URL、控制 Profile/截图目录，并把图片接入统一 `ToolOutput` 后处理管线。
- 直接暴露 MCP 会让 session ID、文件路径、输出规模和安全策略落到模型参数中，也难以自动绑定当前 PicoBot dialog。
这不是把浏览器逻辑重新实现一遍。元素定位、accessibility snapshot、页面交互、Chrome 启动、CDP 通信和 daemon 生命周期均由 agent-browser 负责；PicoBot 只负责编排和边界控制。
## 2. 分层
```text
AgentLoop
│ ToolExecutionContext(session_id, turn_id)
▼
BrowserTool / BrowserProfilesTool 浏览与持久 Profile 管理 schema
▼
BrowserManager 临时 dialog session / 共享持久 Profile
├─ security URL、DNS、私网与 allowlist 前置校验
├─ action browser action → CLI argv
└─ AgentBrowserRunner timeout、env、--json、错误与输出解析
▼
agent-browser CLI → Rust daemon → Chrome/Chromium CDP
```
源文件：
- `src/tools/browser/mod.rs`：`browser`、`browser_profiles` schema 与入口。
- `src/tools/browser/action.rs`：严格参数解析和 argv 映射。
- `src/tools/browser/manager.rs`：临时会话表、按 ID 管理的持久 Profile、并发、回收和截图媒体。
- `src/tools/browser/runner.rs`：无 Shell 的子进程调用、硬超时、JSON/错误解析。
- `src/tools/browser/security.rs`：导航策略。
- `src/tools/traits.rs`：向有状态工具提供 `ToolExecutionContext`；其他工具沿用默认实现。
## 3. 会话、持久 ID 与并发
BrowserManager 按每次调用是否携带 `persistent_id` 分流，两种浏览器可在同一个 Gateway 中同时使用：
- 省略 `persistent_id` 时使用普通临时浏览器。`SessionManager` 传入完整 PicoBot session ID，BrowserManager 第一次看到该 ID 时生成随机、不透明的 `picobot-<uuid>` agent-browser session。同一 dialog 串行、不同 dialog 可并发；临时 daemon 连续 `idle_timeout_secs`（默认 3600 秒）没有操作后自动退出，Manager 在容量检查时惰性回收空闲条目，数量受 `max_sessions` 限制。
- 需要长期保留登录或站点状态时，Agent 可以自主调用 `browser_profiles(action=create,label=...)` 创建持久身份，并在后续相关的每个 `browser` action 中显式传入返回的 `persistent_id`。
- 用户可以拥有多个 `picobot-profile-<32 hex>` ID。`browser_profiles(action=create,label=...)` 创建 `persistence.profile_dir/<id>` 专用目录和可选语义标签；标签可通过 `set_label` 重命名，ID 保持不变。
- PicoBot 不保存默认 ID，也不按 dialog 自动选择或绑定持久 Profile。没有 ID 的调用始终回到该 dialog 的临时浏览器，不会隐式选中任何持久身份。
- 每个 ID 分别映射 agent-browser `--session`、Chrome `--profile` 路径和 mutex。同一 ID 可从不同 dialog、子 Agent 或 Scheduler 使用并保持串行；不同 ID 的浏览器状态和锁相互独立，可以并发。
- 持久 daemon 禁用空闲自动退出，适合需要长时间等待的浏览器作业。ID 与 Profile 目录跨 Gateway 重启、配置重载和显式浏览器关闭保持不变；Cookie、localStorage、IndexedDB、Service Worker 和缓存由各 Chrome Profile 自身保存。
- `close` 关闭显式选择的浏览器进程但保留 ID、标签和 Profile；下一次使用该 ID 时从相同目录重新打开。
- `browser_profiles(action=list)` 返回每个合法 ID 的标签、完整目录和当前 Manager 是否 active。
- `browser_profiles(action=set_label,id=...,label=...)` 写入语义标签并同步当前活动实例；标签去除首尾空白，限制为 1–80 个非控制字符。
- `browser_profiles(action=delete,id=...)` 只接受 `create/list` 返回的完整格式 ID。删除时持有管理锁、等待该 ID 的活动 action、尝试关闭其浏览器，再递归删除对应目录。
Profile 根目录和每个生成目录在 Unix 上收敛为 `0700`，目录内 `.picobot-label` 标签文件为 `0600`。列表忽略格式非法的目录和符号链接，选择、改标签和删除拒绝路径穿越、符号链接及非目录目标。Gateway 配置重载会构造新的 ToolRegistry/BrowserManager；旧运行代按现有 drain 规则退出。临时 agent-browser daemon 的空闲退出时间通过 `AGENT_BROWSER_IDLE_TIMEOUT_MS` 设置为配置值，持久 daemon 则固定设置为 `0`（禁用）。
## 4. Action 映射
| PicoBot action | agent-browser 命令 |
|---|---|
| `open` | `open <url>` |
| `snapshot` | `snapshot --interactive --compact [--depth N]` |
| `click` / `fill` / `type` | 同名命令；无 selector 的 type 使用 `keyboard type` |
| `get_text` / `get_title` / `get_url` | `get text/title/url` |
| `focus` / `wait` / `press` / `hover` / `scroll` | 对应原生命令 |
| `click_at` | `mouse move` + `mouse down left` + `mouse up left` |
| `screenshot` | `screenshot <controlled-path> [--full] [--annotate]` |
| `close` | `close` |
每次调用都使用 argv 数组直接启动进程，不经过 Shell。`fill` / `type` 的内容不会写入 PicoBot 日志；日志只记录 action 名、是否绑定 session 和所选持久 ID。
Runner 固定传入 `--session`、`--json` 和明确的 headed 状态；携带持久 ID 的调用额外传入由 Manager 生成的 `--profile <controlled-path>`。Runner 还设置：
- `AGENT_BROWSER_EXECUTABLE_PATH`（配置后）
- `AGENT_BROWSER_CONTENT_BOUNDARIES`
- `AGENT_BROWSER_MAX_OUTPUT`
- `AGENT_BROWSER_ALLOWED_DOMAINS`（非空时）
- `AGENT_BROWSER_IDLE_TIMEOUT_MS`（临时会话使用配置值；持久会话固定为 `0`）
非零退出码、JSON 中 `success=false`、无效 JSON和超时都转换为工具失败。stdout/stderr 在返回模型前有长度上限；页面类结果保留 agent-browser `_boundary` 元数据。
## 5. 截图与媒体
截图绝不返回 Base64。调用方可省略 `path` 自动生成文件名，也可提供单个 `.png` 文件名；绝对路径、目录分隔、`.` 和 `..` 均拒绝。实际文件始终位于 `browser.artifact_dir`。
命令成功后 PicoBot 再验证文件存在、是普通文件且非空，然后返回：
```text
ToolOutput {
result: ToolResult { output: "Screenshot saved: ..." },
artifacts: [ToolArtifact {
media_ref: MediaRef { media_type: "image", path: "..." },
audience: ModelAndUser
}]
}
```
统一处理器会把截图同时交给下一轮多模态 Provider，并附到本 Turn 最终 assistant 回复供用户查看。`present_to_user` 默认为 `true`；显式设为 `false` 时 audience 改为 `Model`，适用于无需展示的内部视觉检查。历史中仍只保存短路径清单，不产生 Base64 上下文膨胀。
## 6. 安全模型
默认策略：
- 只允许 `http://` 和 `https://`，拒绝 URL userinfo。
- `allow_private_hosts=false` 时拒绝 localhost、`.local`、回环、私网、link-local、未指定和组播地址；域名会先解析 DNS，任一结果为私网即拒绝。
- `allowed_domains` 非空时 PicoBot 先校验首个 URL，agent-browser 再对导航、重定向、子资源、WebSocket、EventSource、sendBeacon 和受支持 Chromium 的 WebRTC 实施域名边界。
- 默认开启 content boundaries，并把页面文本限制为 50,000 字符。
- 包装层没有 `eval`、上传、下载、cookie/storage 写入或任意 agent-browser 命令透传，模型只能使用 allowlisted action。
- 截图有单独的产物目录，不能用来覆盖任意文件。
- `browser_profiles` 只允许创建受控 ID，或按格式严格的 ID 设置受限标签、列出和删除 `persistence.profile_dir` 的直接子目录；模型不能提交任意 Profile 路径。
`allowed_domains=[]` 表示不启用 agent-browser 域名过滤，适合通用浏览；这不是 OS 网络沙箱。需要强隔离时，应同时设置明确域名表和容器/主机 egress 策略。允许私网浏览是显式配置，适合本地应用测试，但会扩大 SSRF 风险。
agent-browser 0.33.0 明确拒绝在 `allowed_domains` 启用时复用 Chrome Profile，因为无法保证页面脚本执行前完整安装同等域名约束。PicoBot 因此允许受域名限制的普通临时浏览器继续工作，但会拒绝创建或使用持久 Profile，并在 health 中给出可选能力警告；列表、改标签和删除仍可用于管理已有目录。Profile 包含可直接代表用户身份的登录信息，目录必须视作敏感凭据，不得提交版本控制或跨用户共享。
## 7. 安装
验证版本为 `0.33.0`：
```bash
npm install -g agent-browser@0.33.0
agent-browser install
```
Linux 自动补系统依赖：
```bash
agent-browser install --with-deps
```
不使用 npm 时：
```bash
cargo install agent-browser --version 0.33.0 --locked
agent-browser install
```
macOS 也可执行：
```bash
brew install agent-browser
agent-browser install
```
`agent-browser install` 下载 Chrome for Testing。已有浏览器时设置：
```json
"browser_executable_path": "/usr/bin/chromium"
```
或设置环境变量 `AGENT_BROWSER_EXECUTABLE_PATH`。agent-browser 的 daemon 与 CDP 路径不需要 Node.js；npm 安装方式只需要 npm 用来放置预编译 CLI。Dockerfile 安装 Debian Chromium、`agent-browser@0.33.0` 并设置 executable path。
## 8. 使用
1. 浏览器工具默认启用；从旧配置删除 `webdriver_url`、`chrome_path`，仅在需要改变持久目录位置时配置 `persistence.profile_dir`。没有持久化模式开关。缺少依赖不会阻止 Gateway 启动，只会让 health 和实际浏览器调用失败。
2. 运行 `picobot health`；应看到 agent-browser CLI 版本、浏览器安装、隔离的 offline headless 启动和环境检查通过。
3. 启动或重载 Gateway。
4. 对 Agent 说“使用浏览器打开 …”。模型的推荐动作序列是：
```text
browser(open, url)
browser(snapshot, interactive_only=true, compact=true)
browser(click/fill/type, selector=@eN)
browser(snapshot) # 页面改变后刷新 refs
browser(screenshot, annotate=true) # 需要视觉上下文时
browser(close)
```
agent-browser 的 `@e` 引用属于当前页面快照。导航、弹窗或 DOM 大幅变化后必须重新 snapshot，不能长期缓存旧引用。
持久 Profile 管理：
```text
browser_profiles(create, label="工作账号") # 返回 persistent ID
browser_profiles(list)
browser_profiles(set_label, id=picobot-profile-..., label="个人账号")
browser(open, url, persistent_id=picobot-profile-...)
browser(snapshot, persistent_id=picobot-profile-...)
browser_profiles(delete, id=picobot-profile-...)
```
同一个持久操作链必须持续传入同一 `persistent_id`；一旦省略，调用会明确转到当前 dialog 的普通临时浏览器。不同对话可以同时操作不同 ID。标签用于识别，不能代替 ID 选择浏览器；删除会清除该 ID 的全部浏览器数据、标签和登录态，调用前必须有用户要求并使用 `create/list` 返回的精确 ID。
## 9. Health 三入口
`src/health.rs` 的 `HealthService` 是唯一检查实现：
- `picobot health [--json]`：本机运维入口；核心/配置必需项失败时退出 `1`。
- `/health`：当前 Gateway 配置的聊天入口。
- `health` Tool：Agent 可调用的只读入口，支持 `json=true`。
检查项包括 workspace、Bash、内容/文件搜索后端、可选 systemctl、配置中的 stdio MCP 命令，以及浏览器启用时的持久 Profile 可用性、agent-browser 版本、显式浏览器路径和完整的 offline doctor。Health 为 doctor 创建临时 socket 目录并使用专用 namespace，不读取或清理活动 daemon socket；不使用会跳过真实启动测试的 `--quick`。结构化结果分别展示浏览器安装、headless launch 和其余环境诊断。检查不创建或删除 PicoBot 持久 Profile、不安装软件、不执行 `doctor --fix`、不访问 Provider API，也不输出配置密钥。
## 10. 迁移和故障处理
- 配置使用 `deny_unknown_fields`；遗留 WebDriver 字段会在加载时失败，而不是被静默忽略。
- `failed to start 'agent-browser'`：CLI 不在 PATH，或 `browser.command` 错误；运行 health。
- doctor 失败：运行 `agent-browser doctor` 查看完整诊断，再安装浏览器/系统库。
- session limit：仅临时模式适用；关闭不再使用的 dialog 浏览器，或调整 `max_sessions`。
- persistence policy：非空 `allowed_domains` 下普通临时浏览器仍可用，但创建或使用持久 Profile 会失败；根据需求选择个人登录态复用或严格域名隔离。
- Profile 删除失败：先确认没有外部 Chrome 使用该目录，再用 `browser_profiles(list)` 核对精确 ID 后重试；不要手工扩大删除路径。
- domain blocked：补充站点和必要 CDN 域名；不要用空白 allowlist 绕过生产隔离策略。
- command timeout：确认页面/浏览器未卡死，再按部署风险调整 `command_timeout_secs`。
- 截图不存在：视为工具失败，不构造失效 MediaRef。
Fantoccini crate、旧 `src/tools/browser.rs` WebDriver 实现、ChromeDriver Docker 包和相关配置已全部删除。Cargo 不链接 agent-browser；它是由 health 管理的外部运行依赖。

146
docs/ARCHITECTURE.md
View File

@ -2,8 +2,6 @@
本文档描述 PicoBot 当前实现的运行时边界、数据流、并发模型和演进约束。它面向维护者和后续参与改进的 Agent，是代码架构的主入口；行为细节仍以代码和测试为最终依据。
流式模型输出、reasoning 展示、活动 Turn 快照和 Channel 实时投递的详细设计与取舍见 [STREAMING_TURN_DESIGN.md](STREAMING_TURN_DESIGN.md)。用户输入路由、Session 执行拆分、终态投递确认和历史增量校准的重构方案见 [MESSAGE_FLOW_REFACTOR_DESIGN.md](MESSAGE_FLOW_REFACTOR_DESIGN.md)。已实施的 checkpoint 上下文压缩、pi 风格 reserve 阈值、统一编排和 overflow 失败语义见 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md)。配置运行代、重载边界和失败语义见 [CONFIG_HOT_RELOAD_DESIGN.md](CONFIG_HOT_RELOAD_DESIGN.md)。具名子 Agent、委托图、后台收件箱、结果传递机制与 `queue`/`steer` 信号的设计见 [SUB_AGENT_DESIGN.md](SUB_AGENT_DESIGN.md)。已实施的统一 Scheduled Run、结构化终结协议、中央投递和 v11 数据库迁移见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。
## 1. 设计目标
PicoBot 是一个单进程、异步、可扩展的个人 AI 助手运行时。核心目标是：
@ -18,26 +16,18 @@ PicoBot 是一个单进程、异步、可扩展的个人 AI 助手运行时。
## 2. 运行模式与进程边界
PicoBot 只有一个二进制，提供四种运行模式：
PicoBot 只有一个二进制，提供两种模式：
| 模式 | 入口 | 职责 |
|------|------|------|
| Gateway | `cargo run -- gateway` | 组装服务、监听 HTTP/WebSocket、提供嵌入式 WebUI，运行渠道、会话、调度器和后台任务 |
| CLI client | `cargo run -- chat` | 运行 Ratatui UI，通过 WebSocket 使用 Gateway，不持有业务状态 |
| One-shot client | `cargo run -- run "prompt"` | 使用独立临时 chat scope 通过 WebSocket 提交一条消息，等待 Turn 终态，输出结果后退出 |
| Health diagnostic | `cargo run -- health [--json]` | 只读检查核心、配置相关和可选外部依赖，不启动 Gateway 或连接 Provider |
Linux 上可通过 `picobot service install/start/stop/status/restart/uninstall` 管理 systemd 用户服务。unit 固定为 `picobot.service`，其主进程仍是普通 Gateway 模式，不引入额外 daemon/fork 层；异常退出由 systemd 按 `Restart=on-failure` 拉起。
`picobot reload`、`/reload` 和根交互 Agent 的 `reload_config` 工具共享 Gateway 内部的有界重载控制通道。重载先用启动前捕获的进程环境重新解析配置和 `.env`，构造完整的下一运行代；候选构造失败时旧运行代不变。成功后关闭旧代 admission，等待已进入的消息、交互 Turn、Scheduler job 和后台子 Agent 到达持久化/投递边界，再停止旧渠道和受监督任务、主动关闭旧 WebSocket，并在保留的监听 socket 上启动新运行代。MCP 只在新代激活时连接。每次重载有 generation ID，可通过 `GET /api/config/reload/status` 查询相位。监听 host/port、workspace 和 SQLite 有效路径属于进程级不变量，变更时拒绝热重载并要求完整重启。
原生 Gateway 默认绑定 `127.0.0.1:19876`，`gateway.host`/`gateway.port` 可由命令行 `--host`/`--port` 覆盖。Docker Compose 为保证端口映射可达，默认向容器传入 `0.0.0.0:19876`；`PICOBOT_GATEWAY_HOST` 控制容器内监听地址，`PICOBOT_PUBLISH_HOST` 控制宿主机发布地址，`PICOBOT_GATEWAY_PORT` 同时控制监听与映射端口。
CLI TUI 在 `~/.picobot/tui_client_id` 保存非敏感客户端标识，并通过 WebSocket 查询参数 `client_id` 传给 Gateway。`cli_chat` 以该标识作为稳定 chat scope；重连时恢复内存中的当前 dialog，Gateway 重启后则恢复该 scope 最近活跃的未归档 dialog。无效或缺失的标识会退化为连接级随机 scope。
One-shot client 不绕过 Gateway 直接调用 Provider。每次 `run` 生成独立的 `run-<uuid>` scope，通过相同的 `cli_chat`、MessageBus、SessionManager、AgentLoop 和 Turn delivery 路径执行；它不复用 TUI scope，因而不会替换同一 scope 的活动 WebSocket。默认 stdout 只投影终态 Assistant blocks，进度写到 stderr；超时或 Ctrl-C 会先在当前 scope 发送 `/stop`。
Gateway 启动时先从配置目录 `.env`、workspace `.env` 和既有进程环境合并启动变量，再初始化日志并切换进程工作目录到 `workspace_dir`。优先级为进程环境 > workspace `.env` > 配置目录 `.env`；配置目录层先用于定位 workspace，workspace 层不得重定向自身位置。环境文件只在单线程启动阶段写入进程环境，不能移到后台任务启动之后。重载使用启动前保存的进程环境快照解析各层，但不再修改进程环境，避免多线程运行期调用 `set_var`；新 Provider、MCP 和渠道使用解析后配置中的值。切换完成后所有相对路径都应按 workspace 解释，不能假设仍位于源码仓库。
Gateway 启动时会切换进程工作目录到 `workspace_dir`。因此所有相对路径都应按 workspace 解释，不能假设仍位于源码仓库。
## 3. 组件关系
@ -45,14 +35,12 @@ Gateway 启动时先从配置目录 `.env`、workspace `.env` 和既有进程环
flowchart LR
External[CLI / Feishu] --> Channels[channels]
Channels -->|InboundMessage| Bus[MessageBus]
Bus --> Processor[Gateway inbound/control routers]
Bus --> Processor[Gateway message processor]
Processor --> Sessions[SessionManager]
Sessions --> Agent[AgentLoop]
Agent --> Providers[LLM providers]
Agent --> Tools[ToolRegistry / MCP]
Sessions <--> Storage[(SQLite)]
Sessions -->|TurnSnapshot| Delivery[DeliveryCoordinator]
Delivery -->|TurnSink| Channels
Sessions -->|OutboundMessage| Bus
Scheduler[Scheduler] --> Sessions
Bus --> Dispatcher[OutboundDispatcher]
@ -71,14 +59,12 @@ flowchart LR
| `channels` | 外部协议适配、权限检查、媒体收发 | 会话选择、LLM 调用 |
| `bus` | 三条有界异步队列与出站投递协调 | 会话路由、业务状态 |
| `session` | dialog 路由、会话状态、串行工作队列、上下文和持久化协调 | 外部渠道协议 |
| `agent` | 单次无状态模型/工具循环、上下文压缩、子 Agent、Turn 语义事件 | 持有 dialog 生命周期 |
| `providers` | 把统一请求映射为原生模型流，并归一化正文、reasoning、工具和 usage | Session、Bus 或 Channel 感知 |
| `delivery` | 活动 Turn 快照投影、latest-wins 节流、终态重试和 TurnSink 生命周期 | Provider 协议、会话历史、平台 API 细节 |
| `tools` / `mcp` | 工具定义、注册和执行适配；MCP 工具的本地执行属性声明 | 隐式修改会话路由 |
| `health` | 聚合只读依赖检查，供 CLI、Tool 与 slash command 复用 | 安装、修复或连接 Provider |
| `agent` | 单次无状态模型/工具循环、上下文压缩、子 Agent | 持有 dialog 生命周期 |
| `providers` | 把统一请求映射到模型 API | Session、Bus 或 Channel 感知 |
| `tools` / `mcp` | 工具定义、注册和执行适配 | 隐式修改会话路由 |
| `storage` | SQLite schema、迁移、原子 CRUD | 运行时调度策略 |
| `memory` | Knowledge/Timeline 的存取与召回 | 直接驱动消息发送 |
| `scheduler` | 原子领取 occurrence、运行隔离的 Root/命名 Agent、提交结构化结果并 drain 持久化投递 outbox | 解析模型自然语言、绕过 Bus 直接调用 Channel |
| `scheduler` | 领取到期任务、执行普通/巡检 Agent、应用投递策略、原子记录结果 | 复用聊天会话历史、直接感知 Channel |
| `work` | session 级单 active plan、并行子项状态机、版本和变更事件 | 执行模型调用、持有 Channel/WebSocket |
| `task_supervisor` | 后台任务注册、取消、限时回收 | 业务级重试和结果语义 |
@ -86,9 +72,9 @@ flowchart LR
`MessageBus` 包含三条容量相同的 Tokio MPSC 队列：
- `inbound`：Channel → Gateway inbound router。
- `inbound`：Channel → Gateway message processor。
- `outbound`：Session/Tool → `OutboundDispatcher`。
- `control`：WebSocket/Channel → Gateway control router，用于 dialog 操作。
- `control`：WebSocket/Channel → Gateway message processor，用于 dialog 操作。
### 普通消息
@ -96,68 +82,30 @@ flowchart LR
sequenceDiagram
participant C as Channel
participant B as MessageBus
participant G as Inbound router
participant G as Message processor
participant S as SessionManager
participant W as Per-session worker
participant A as AgentLoop / Provider
participant T as TurnController
participant L as DeliveryCoordinator
participant A as AgentLoop
participant D as OutboundDispatcher
C->>B: publish InboundMessage
B->>G: consume inbound
G->>S: handle_message
S->>W: try_send AgentTask (idle or /queue)
S->>W: try_send AgentTask
S-->>G: AgentProcessing
W->>T: start Turn
W->>L: subscribe latest snapshots
W->>A: process_streaming(history)
C->>B: ordinary input during active Turn
B->>G: consume inbound
G->>S: handle_message
S->>A: bounded steering mailbox
A->>A: drain after tool batch / before final
A-->>T: reasoning/text/tool events
T-->>L: complete TurnSnapshot
L->>C: TurnSink update (best effort)
A-->>W: emitted messages
W->>W: atomic persistence
W->>T: Completed
T-->>L: terminal snapshot
L->>C: TurnSink finish (bounded retry)
W->>B: independent messages/fallback only
W->>A: process(history)
A-->>W: final response
W->>B: publish OutboundMessage
B->>D: consume outbound
D->>C: Channel::send
```
关键语义：
- Gateway inbound router 按 `(channel, chat_id)` 使用容量 32 的短生命周期 lane 保持入口顺序，不同聊天可并发路由；没有活动 Turn 时，普通消息进入对应 session worker 后立即返回 `AgentProcessing`。
- `/stop` 绕过同聊天的 inbound lane，直接使正在运行的 worker/Turn 失效；其他 slash command 仍在聊天 lane 内有序执行。
- 活动 Turn 存在时，普通消息默认作为 steering 进入本 Turn 的有界 mailbox；`/queue <message>` 明确进入下一 Turn。AgentLoop 只在完整工具批次结束后、或准备接受无工具最终回复时排空 mailbox，并把输入作为真实、可持久化的 `role=user` 消息加入下一次模型请求。
- Steering mailbox 最多容纳 32 条、合计 64 KiB 文本与元数据。mailbox 已关闭或满时，输入可靠回退到 session 队列；若 session 队列也满则明确拒绝。Session 在入站时分配单调序号，Turn 结束时未消费的 steering 由 worker 本地恢复队列接管，并与 `/queue` 输入按该序号合并选择，不能丢失或互相超越。
- 每个 session 有一条容量为 32 的普通队列，同一 session 仍只运行一个 Turn，不同 session 的 worker 可并发执行。
- Gateway 的主消息处理循环不等待模型完成；普通消息进入对应 session worker 后立即返回 `AgentProcessing`。
- 每个 session 有一条容量为 32 的队列，同一 session 串行处理，不同 session 的 worker 可并发执行。
- 队列满时明确拒绝新消息，不允许无界积压。
- Slash command 通常不进入 Agent 队列，由 `SessionManager` 直接执行；`/queue` 是显式排队输入，`/stop` 是显式中断并清空当前 mailbox 与队列。
- WebSocket `user_input.client_message_id` 只用于让 `turn_committed` 以同一消息 ID 替换 WebUI 的乐观用户气泡；它不改变入站顺序或 steering/queue 决策。
- `InboundMessage` 只保存规范化输入：`sender_id`、`received_at`、媒体和一个 `ChannelContext`。核心只解释其中的 `reply_to`，其语义是本轮出站应回复的当前入站消息；被用户引用的父消息只用于补充模型上下文。reaction/message ID、话题 root/thread 等平台字段作为 `private` 不透明传到对应 Turn/普通回复，不能散落为核心层 magic key。持久化的用户消息保留真实接收时间和 `UserInput` 来源，客户端历史投影不暴露来源中的平台用户 ID。
- Session 为每个 Agent 请求创建一个 `TurnController`。Provider 向 AgentLoop 发 delta，AgentLoop 发结构化 TurnEvent，只有 TurnController 能把事件归约为有序 block 和单调 revision 的完整快照。
- Turn 快照经 Tokio `watch` 发布，语义为 latest-wins；慢客户端或慢渠道跳过中间状态，不反压 Provider。终态明确编码在快照中，不依赖 sender 关闭。
- Agent 本轮消息原子持久化成功后才发布 `Completed`。取消或失败若已有可见正文，则保存为 `cancelled`/`interrupted` partial；只有 reasoning 时不创建 assistant 历史。
### 活动 Turn 投递
`DeliveryCoordinator` 与普通出站投递并列：
- `TurnDeliveryService` 根据 Channel 创建本轮独占的 `TurnSink`；sink 私有保存远端消息 ID 和清理资源。
- `PresentationPolicy` 在快照离开 Gateway 核心前过滤内容。TUI/WebUI 展示独立 reasoning 和详细工具状态；外部 Channel 不接收 reasoning，只接收紧凑工具状态；无人值守投递只保留正文。
- `LivePolicy::Snapshot` 按渠道间隔发送最新运行态；`FinalOnly` 忽略运行态，只处理终态。终态绕过节流并只对明确的瞬态错误重试。
- `TurnDeliveryService` 返回可等待的终态句柄；sink 生命周期启动不等于终态已送达。Session 在终态重试最终失败时通过普通出站路径兜底一次。
- `cli_chat` 将同一 `turn_updated` 快照发给 TUI 和 WebUI。客户端只保留当前 session 中 revision 更新的 `active_turn`，终态随后由持久化历史校准。
- 飞书对每个 DATA 帧先在 2 秒硬期限内 ACK，再进行有界分片重组，并把完整事件交给容量 32 的连接内处理队列；媒体下载和引用查询不占用正常的 WebSocket 读循环。队列饱和时当前事件在连接任务中同步处理而不丢弃。连接异常采用有上限的指数退避持续重连，不因累计故障永久停止。
- 飞书在协议解析阶段按 `allow_from` 拒绝未授权用户；群聊默认必须明确 @ 运行时解析出的机器人身份，身份解析失败时安全地忽略群消息。飞书把当前消息 ID 作为 `reply_to`，并在私有 metadata 中携带 root/thread 信息；Sink 使用原生 reply API 及 `reply_in_thread` 保持客户端引用和话题位置。飞书默认 `FinalOnly`；开启 `live_updates` 后，第一个可见快照创建卡片，后续编辑同一卡片，终态编辑失败则发送完整结果兜底。reaction 清理在 finish、abort 和 Gateway shutdown 中幂等执行。
- 飞书入站响应体按类型流式执行字节上限和超时检查，写盘前校验媒体目录总容量，客户端文件名先收敛为安全 basename；出站上传也先检查本地文件大小。消息发送和资源下载对网络错误、429、5xx 和 401 进行有界重试，401 或飞书失效 token 业务码会使租户 token 缓存失效后重新获取。
- DeliveryCoordinator 与 OutboundDispatcher 共享 `(channel, chat_id)` 写锁，避免活动 Turn 终态与独立消息并发写入同一目标。
- Slash command 不进入 Agent 队列，由 `SessionManager` 直接执行，因此 `/stop` 等控制操作不会排在长模型调用之后。
### 出站投递
@ -171,21 +119,9 @@ sequenceDiagram
不要把“已进入 Bus”误认为“外部渠道已收到”。需要确认语义时必须使用 `deliver_outbound`。
### Scheduled Run
Scheduler 不复用聊天历史，也不根据模型正文猜测是否通知。一次到期状态在同一 SQLite 事务中取得 Job 租约、插入 `job_runs(status=claimed)`、快照 Agent/投递目标/策略，并提前推进 recurring `next_run_at`；`At` 在 claim 时立即禁用。事件循环以独立有界 JoinSet 执行 Run 和 drain delivery，长任务不阻塞其他领取或通知。
每次 Run 通过 `AgentCoordinator` 建立顶层 `agent_runs` 审计记录并执行隔离的 Root 或命名 Agent。Scheduled origin 贯穿所有后代，但只有顶层获得 exactly-once `complete_scheduled_run` sink；后代不继承 sink。Scheduled Agent 不获得 `send_message`、cron/config 管理或 `emit_signal`，不创建 Inbox completion slot，任何 background 委托都收敛为 foreground。普通最终文本不代表成功；没有提交 `ok/alert/failed/refused` 之一即 fail-closed。
顶层 AgentRun 终态、JobRun 的 lifecycle/outcome/message/diagnostic、Job 最近摘要、初始 delivery status 和租约释放在一个事务中提交。`always` 投递所有 outcome，`on_alert` 只抑制结构化 `ok`，`never` 始终只记录。`job_runs` 同时作为轻量 outbox：`pending → delivering → delivered/failed`，瞬态错误最多进行三次持久化尝试；OutboundDispatcher 返回清洗后的类型化回执，Scheduler 不解析错误字符串。
首次投递把目标 dialog 固定到 `target_session_id`，并先用稳定消息 ID `scheduled:<job_run_id>` 幂等写入本地历史，再调用 `MessageBus::deliver_outbound`。发送成功但 ack 提交前崩溃允许带稳定 metadata 的重复通知，不能为避免重复而丢失告警。启动恢复把遗留 claimed/running JobRun 原子收敛为 `unknown+unknown`、关联非终态 AgentRun 收敛为 interrupted，并按 claim-time 策略决定是否进入 outbox；已推进的 occurrence 不自动重跑。完整状态矩阵、v11 schema 和迁移规则见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。
按需 Health 检查只读查询任务引用、投递积压、最近失败/超时/unknown、`never+unknown`、Every 周期被执行时长覆盖以及不可计算的 next run；它不执行任务、不连接 Provider，也不修改数据。
### Control 消息
WebSocket dialog 操作通过 `ControlMessage` 携带一次性回复通道。Gateway control router 以最多 64 个并发受监督任务调用 `SessionManager`，再将 `SessionEvent` 回传给发起者；慢 control 不阻塞其他聊天的 inbound 路由。Bus 只承载消息，不解释操作。
WebSocket dialog 操作通过 `ControlMessage` 携带一次性回复通道。Gateway 在统一 message processor 中调用 `SessionManager`，再将 `SessionEvent` 回传给发起者。Bus 只承载消息，不解释操作。
TUI 的历史回放同样走 control 队列：`get_session_history` 先校验 session 属于当前客户端 scope，再由 SessionManager 从 Storage 读取最近消息。单次查询限制为 1–2000 条，TUI 默认请求最近 1000 条；迟到的历史响应只有在目标仍是当前 dialog 时才允许更新界面。
@ -212,26 +148,21 @@ Session ID 格式为：
4. 慢操作开始前记录 `state_version`，提交前重新验证，防止旧快照覆盖 `/clear`、`/delete` 等并发修改。
5. 持久化写入由 `persistence_lock` 串行化；多条相关记录应使用 Storage 的原子接口。
6. 内存先变更但持久化失败时，必须回滚精确匹配的消息后缀，不能删除无关的新状态。
7. Steering 的接收、最终边界关闭和 `/stop` 必须通过同一个 mailbox 状态串行化；每条输入只能落入当前 Turn 或下一 Turn 之一。
当前 WebUI/TUI Turn 通过 `send_message(files=...)` 向自身 session 投递文件时，文件先进入 task-local Turn delivery 暂存区，成功结束后附加到最终 assistant 消息，与工具链一起原子提交；因此持久化和刷新后的顺序都是工具调用/结果在前、携带附件的最终回复在后，也不会生成带 `[message from ...]` 的自投递气泡。其他同 Turn 自投递仍是受控例外：只有 task-local Turn ID 仍匹配该 session 的 active Turn，写入才允许不递增 `state_version`。跨 Turn、跨 session 以及无法证明所有权的写入仍必须递增版本。Provider 回放历史附件时，只有 user 输入和当前工具结果可生成模型原生媒体块；assistant/system 附件只回放文本清单，避免把图片放到供应商不接受的角色。
SessionManager 负责组装会话上下文：系统提示、Skills、召回的 Knowledge、压缩后的 Timeline、可选的 active plan 摘要和当前消息历史。普通闲聊 session 没有 plan 摘要；计划状态由 `WorkManager` 从 SQLite 读取，在历史压缩之后追加，因此不以自然语言摘要作为权威来源。`AgentLoop` 接收完整输入执行一次模型/工具循环，本身不拥有会话状态。
SessionManager 负责组装会话上下文：系统提示、Skills、召回的 Knowledge、可选的 active plan 摘要，以及由活动 `ContextCheckpoint` 投影出的会话历史。`messages` 始终是 append-only 原始日志；每个 Session 最多有一个活动 checkpoint，模型历史确定为“一条累计摘要 + `seq >= first_retained_seq` 的原始尾部”，Timeline 只是 checkpoint 提交后的 best-effort 检索副本，恢复流程不读取 Timeline、也不调用 Provider。所选 Model 的可选 `token_limit` 定义上下文硬上限，缺失时固定回退 128000；Agent 的可选 `token_limit` 只允许收紧该上限，有效窗口为 `min(agent_token_limit, model_token_limit_or_128K)`，Agent 不能扩大模型窗口。主 Agent Profile、具名 Agent 的内联 Provider/Model 和 `llm_profile` 引用必须使用同一规则，definition 上的显式值也只能收紧已解析 Profile。`session::turn_input` 只在 Session 锁外并行读取 Knowledge 和 active plan；完整请求草稿随后使用同一个 reserve 预算评估，自动触发公式唯一为 `context_tokens > context_window - effective_reserve`。手动、自动和首次 context-overflow 共用唯一的 `compact_session_context` 编排与 checkpoint CAS 提交路径；candidate 自带快照 generation，提交成功后必须从当前 raw log 重新投影，不能返回摘要前的旧尾部向量。摘要请求输入预算由有效 `token_limit` 扣除动态摘要输出、固定提示词和安全余量得到；超大压缩源只在 request-local 副本中保留已有 checkpoint 和最新材料、对单条内容做确定性 head/tail 截取，不能用固定 32K 上限拒绝压缩，也不能改写 durable raw log。每个 Turn 最多进行一次语义摘要；换模后的发送前预检若已硬超限，可在摘要失败时直接生成明确标记的确定性降级 checkpoint，避免先发送必然失败的普通请求；首次 Provider 请求 overflow 最多正式重试一次。AgentLoop 把 Provider overflow 转成类型化错误：工具尚未执行时交回 Session；工具已执行后只在同一个 AgentLoop 中删除旧完整 Turn 的请求副本并重试当前 Provider step 一次，保留本 Turn tool call/result，绝不从 durable history 重启并重复副作用工具。首次请求与重试必须复用同一个 runtime assembly，不能复制系统提示、丢失 mailbox steering 或把压缩投影写回原始历史。Provider prompt usage 仅在 provider/model/checkpoint generation/raw seq 和完整请求摘要都匹配时复用，否则完整保守估算；不再按消息数外推。普通闲聊 session 没有 plan 摘要；计划状态由 `WorkManager` 从 SQLite 读取，因此不以自然语言摘要作为权威来源。`AgentLoop` 接收完整输入执行一次模型/工具循环，本身不拥有会话状态，但通过本 Turn 的 mailbox 在安全边界接收追加用户输入。执行工具时额外传递只包含 session/turn 身份的 `ToolExecutionContext`；无状态工具使用默认实现忽略它，有状态外部适配器用它路由资源，但可按明确的单用户配置跨 dialog 共享，且不能自行反向查询 SessionManager。完整压缩设计见 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md)。
当前 Turn 的工具进度只从 `AgentLoop` 的结构化 `TurnEvent` 进入 `TurnController`，不能另建字符串 notification 通道重复投递。具名子 Agent 基础已接入：候选运行代从受信任配置目录严格加载不可变 `AgentCatalog`，Definition 固定 Provider/Model（内联或 `llm_profile`）、工具/Skill allowlist、委托边和执行限制，工具集完全由定义文件决定；单个定义校验失败（坏 YAML、未知 provider/profile/model/tool/skill、或显式委托到缺失目标）仅停用该定义并记入 `load_errors`（`GET /api/agents` 返回），不会阻塞启动或热重载，配置与目录信任级错误仍然致命；`delegate` 使用 `foreground/background` 两个 canonical 生命周期词，批量并发与生命周期正交。具名 foreground 支持批量并发、显式 `AgentExecutionContext`、祖先环路和委托边校验；Root 对具名 Agent 的 background（单任务或批量，批量并发、每个 run 独立 completion 事件）走 durable run/inbox + continuation 投递，空闲时完成即返回。内置 general-purpose 定义随二进制释放，WebUI「子 Agent」页可增删改与启停定义。旧匿名 general 兼容路径已移除。自动标题属于非关键派生工作：Turn 持久化完成后由 `TaskSupervisor` 调度，Session worker 不等待模型生成；同一 Session 同时最多有一个标题任务，提交时仍校验标题保持默认值，避免覆盖用户改名。
每个 session 最多有一个 active plan，但 plan 内多个 item 可以分别绑定不同子 Agent 并行执行。主 Agent 通过 `todo` 创建和维护计划，通过 `delegate.plan_item_id` 委托；子 Agent 的工具集由其定义文件决定，能否继续委托由其 `delegates` 白名单决定。领取 item 使用条件更新防止重复执行，迟到结果只有在 plan 仍 active 且 execution ID 匹配时才允许提交。
每个 session 最多有一个 active plan，但 plan 内多个 item 可以分别绑定不同子 Agent 并行执行。主 Agent 通过 `todo` 创建和维护计划，通过 `delegate.plan_item_id` 委托；子 Agent 不获得 `todo` 或 `delegate`。领取 item 使用条件更新防止重复执行，迟到结果只有在 plan 仍 active 且 execution ID 匹配时才允许提交。
## 6. 持久化
`Storage` 使用 SQLx + SQLite，默认数据库为 `{config_dir}/data/picobot.db`（`config_dir` 默认 `~/.picobot`），与 workspace 相互独立。连接启用：
`Storage` 使用 SQLx + SQLite，默认数据库为 `{workspace_dir}/picobot.db`。连接启用：
- WAL journal mode。
- foreign keys。
- 5 秒 busy timeout。
- schema version 迁移。
持久化范围包括 sessions、messages、context checkpoints、session turn usage、memories、task plans/items、scheduled jobs、job runs、agent run/inbox/session state。checkpoint 插入、活动指针切换和 `context_generation` 递增在同一事务中完成；`/clear` 在删除消息的事务内使活动 checkpoint 失效，旧 checkpoint 行仅作为审计记录保留。消息保存可展示 `reasoning_content`、Provider 私有回放状态、`turn_id`、iteration 和 completion status；私有 Provider 状态不进入 WebSocket/Channel，且只允许回放给同一 Provider。成功 Turn 的 Provider usage 与消息批次在同一事务中写入 `session_turn_usage`，以 `turn_id` 幂等累计会话输入、输出、缓存输入和请求数；升级前的历史没有可归属 usage，统计起点必须显式呈现。成功持久化一个 Turn 后，交互 Channel 收到只包含公开字段的 `CommittedTurnDelta`，其中 `history_revision` 是本批次最高 durable sequence；客户端按 revision 幂等合并，正常完成不重新加载整段历史，断线重连和失败/取消仍使用 `SessionHistory` 校准。Provider 诊断和失败审计只记录模型、消息数、工具数等请求摘要，不保存正文、reasoning 或签名 payload。修改 schema 时应：
持久化范围包括 sessions、messages、memories、task plans/items、scheduled jobs、job runs 和 background tasks。修改 schema 时应：
1. 更新集中式 schema/迁移逻辑。
2. 保留已有数据库的升级路径。
@ -240,7 +171,7 @@ SessionManager 负责组装会话上下文：系统提示、Skills、召回的 K
### 安全边界
- API Key 和渠道凭据只来自配置占位符、配置目录/workspace 的 `.env` 或进程环境，不得写入仓库。既有进程环境优先级最高，workspace `.env` 可覆盖配置目录 `.env`；日志只能记录所加载的文件路径，不能记录变量值。
- API Key 和渠道凭据只来自配置占位符、`.env` 或进程环境，不得写入仓库。
- 日志不得输出 token、secret、Authorization header，或包含临时凭据的完整 URL；应记录脱敏后的 host/path 和必要诊断字段。
- Gateway 把 cwd 切到 workspace，因此相对文件路径和 Shell 默认从 workspace 开始；这不是硬沙箱。当前内置文件工具接受绝对路径，Bash 也可访问进程权限允许的位置。若某场景需要硬边界，必须显式配置/实现 allowed directory 和进程隔离。
- `http_request` 和 `web_fetch` 的私网/回环地址校验属于 SSRF 防线，重构网络层时不能绕过。
@ -248,7 +179,7 @@ SessionManager 负责组装会话上下文：系统提示、Skills、召回的 K
## 7. 后台任务与生命周期
`TaskSupervisor` 是 Gateway 内部后台任务的统一所有者。inbound/control routers、inbound lanes、outbound dispatcher、scheduler、session workers、Turn delivery、outbound lanes、自动标题和子 Agent 后台任务都应通过它注册。
`TaskSupervisor` 是 Gateway 内部后台任务的统一所有者。message processor、outbound dispatcher、scheduler、session workers、outbound lanes、通知消费者和子 Agent 后台任务都应通过它注册。
两种注册方式：
@ -264,28 +195,19 @@ SessionManager 负责组装会话上下文：系统提示、Skills、召回的 K
WebSocket 每个客户端的 writer task 是连接局部任务，由连接 handler 自己限时回收；它不跨越连接生命周期。
Turn delivery 使用 `spawn_graceful`。全局取消发生时先停止读取快照，再在共享目标写锁下有界调用 `TurnSink::abort`，使平台 reaction 等私有资源能在 Supervisor 宽限期内清理。
### WebUI 与管理 API
Gateway 在 `/` 提供随二进制编译的 HTML/CSS/JavaScript，不依赖外部 CDN 或前端运行服务。前端源码位于 `webui/`，由 Svelte 5 + Vite 构建，Bits UI 提供无样式的可访问组件原语。视觉层通过 `webui/src/styles.css` 中的本地 Fluent 2 语义令牌实现浅色/深色表面、六套品牌色、状态色、层级和控件状态；页面组件必须复用语义别名，不能把独立硬编码调色板或外部 Fluent 运行库引入发布产物。明暗模式和品牌色只保存在浏览器 `localStorage`，`theme-init.js` 必须在 Svelte 挂载前恢复 `data-theme` 与 `data-accent`，防止首屏颜色闪烁；这些外观选项不属于 Gateway 配置，也不跨设备同步。`build.rs` 监听前端源码、锁文件和构建配置，增量地将生产资源生成到 Cargo `OUT_DIR`，Rust 再从该目录编译嵌入；前端产物不进入仓库，最终用户使用发布二进制时不需要 Node.js。浏览器聊天继续使用 `/ws` 和 `cli_chat` 渠道，因此复用现有 dialog scope、每会话串行 worker、历史持久化和出站 lane；会话历史帧保留工具调用 ID、名称、参数和工具结果角色，WebUI 在正常完成时合并 `turn_committed` 增量并将工具信息渲染为默认折叠的工具卡片；聊天页的 Todo 侧栏默认隐藏，按 session 保存快照和未读状态，并通过结构化 `session_plan`/`plan_updated` 帧刷新，计划变化不会写入聊天历史；活动状态栏通过结构化 `session_stats` 展示当前 session 的已提交 Turn 用量与上下文占用，累计量来自 Provider usage，窗口占用明确区分精确匹配的 Provider 实测与完整字符估算；斜杠命令补全通过 `get_slash_commands` 获取 Gateway 的实时命令与别名清单，不在前端重复定义；WebUI 不直接调用 Provider 或 SessionManager。
Gateway 在 `/` 提供随二进制编译的 HTML/CSS/JavaScript，不依赖外部 CDN 或前端运行服务。前端源码位于 `webui/`，由 Svelte 5 + Vite 构建，Bits UI 提供无样式的可访问组件原语。`build.rs` 监听前端源码、锁文件和构建配置，增量地将生产资源生成到 Cargo `OUT_DIR`，Rust 再从该目录编译嵌入；前端产物不进入仓库，最终用户使用发布二进制时不需要 Node.js。浏览器聊天继续使用 `/ws` 和 `cli_chat` 渠道，因此复用现有 dialog scope、每会话串行 worker、历史持久化和出站 lane；会话历史帧保留工具调用 ID、名称、参数和工具结果角色，WebUI 在本轮完成后刷新历史并将其渲染为默认折叠的工具卡片；聊天页的 Todo 侧栏默认隐藏，按 session 保存快照和未读状态，并通过结构化 `session_plan`/`plan_updated` 帧刷新，计划变化不会写入聊天历史；斜杠命令补全通过 `get_slash_commands` 获取 Gateway 的实时命令与别名清单，不在前端重复定义；WebUI 不直接调用 Provider 或 SessionManager。
WebUI/TUI 文件字节通过受鉴权的 HTTP 接口流式传输，WebSocket 只携带短期 `upload_id` 和结构化附件描述。`UploadRegistry` 在内存中按 `cli_chat` chat scope 校验并消费待发送上传；消息继续以 `media_refs` 保存 Gateway 本地路径，不建立永久附件资产。下载接口必须通过 client、session、message 和附件序号反查路径，不能接受客户端路径。历史附件路径失效属于正常状态，不得影响历史文本读取。待发送但未进入消息的上传由 `TaskSupervisor` 所有的限时清理任务回收。Agent 上下文会为所有媒体注入内部路径清单，客户端响应不得暴露该路径。
所有工具调用统一归一化为 `ToolOutput`，并由 `AgentLoop` 中唯一的 `ToolOutputProcessor` 后处理。普通文本工具仍实现 `ToolResult`，默认转换会将其包装为无产物的 `ToolOutput`；产物工具返回带 `ToolArtifact` 的输出，并用 `Model`、`User` 或 `ModelAndUser` 声明受众。处理器只发布成功工具的产物，去重后分别形成下一轮模型媒体和最终用户回复附件。工具只负责经过自身路径策略校验后声明产物与意图，不感知当前模型、Provider、Session 或 Channel。`AgentLoop` 仅将最新连续工具结果批次的模型媒体交给 `MediaHandlerRegistry`，紧随工具结果的 steering 不会使该批次媒体失去可见性，而旧工具媒体只回放文本和路径，避免历史 Base64 膨胀；用户媒体累积到本 Turn 最终 assistant 消息，随工具链原子持久化，并由 committed-history 或普通出站路径呈现。OpenAI-compatible Provider 保持 `tool` 结果为文本，并在完整工具批次后构造仅存在于请求内的临时多模态 `user` 消息，同时保持后续 user steering 的顺序；Anthropic Provider 将同批媒体放入对应 `tool_result.content`，并将紧随的 user steering 合并进 API 所需的同一 `role=user` 内容数组，持久化消息仍彼此独立。媒体加载、格式或能力检查失败必须降级成文本，不得使历史记录不可读取。
工具默认通过 `ToolResult` 返回文本；需要把图片等产物交给模型时，通过 `Tool::execute_with_media` 返回文本和结构化 `MediaRef`。工具只负责经过自身路径策略校验后声明媒体，不感知当前模型或 Provider。`AgentLoop` 仅将最新连续工具结果批次的媒体交给 `MediaHandlerRegistry`，旧工具媒体只回放文本和路径，避免历史 Base64 膨胀。OpenAI-compatible Provider 保持 `tool` 结果为文本，并在完整工具批次后构造仅存在于请求内的临时多模态 `user` 消息；Anthropic Provider 将媒体放入对应 `tool_result.content`。媒体加载、格式或能力检查失败必须降级成文本，不得使历史记录不可读取。
MCP 发现的工具在 `ToolRegistry` 中使用 `mcp_<server-name>_<tool-name>` 命名空间，避免与内置工具混淆；`tool_settings` 仍按 MCP 原始 `<tool-name>` 键入。MCP 协议不提供 PicoBot 可依赖的副作用或并发契约，因此每个 `mcp.servers[].tool_settings.<tool-name>` 可在受信任本地配置中声明 `read_only` 与 `exclusive`。未声明的 MCP 工具保守地按“可能有副作用、顺序执行”处理。`concurrency_safe` 不保存为独立状态，而是严格由 `read_only && !exclusive` 推导；工具批次只有全部工具满足该条件才允许并发执行。WebUI 的 MCP 工具展开项提供这两个声明的复选框，并将推导结果显示为“可并发”；属性编辑先保留在页面草稿中，只有选择“保存并应用”才原子写入配置并触发一次热重载，离开 MCP 标签或刷新页面会丢弃草稿。
`browser` 是有状态工具适配器：`BrowserTool` 保持模型侧 action schema，`BrowserManager` 按每次调用是否带 `persistent_id` 分流。省略 ID 时把 PicoBot dialog 映射到随机临时 agent-browser session，并用每 session mutex 保证同一页面串行、不同 dialog 并发；临时 daemon 按 `browser.idle_timeout_secs`（默认一小时）自动退出，Manager 在容量检查时惰性回收对应空闲条目。长期工作需要保持浏览器进程或保留登录和站点状态时，Agent 可自主创建持久身份并在后续相关 action 中持续传入同一个 ID。Manager 按持久 ID 保存 agent-browser session 和 mutex，持久 daemon 的空闲超时固定为禁用；同一 ID 跨 dialog 共享且串行，不同 ID 相互独立并可并发，Gateway 重启或显式关闭浏览器后仍可继续使用原 Profile；没有全局持久化开关、默认 ID 或按 dialog 隐式选择。`browser_profiles` 在受控根目录下创建、设置语义化标签、列出或删除格式合法的 ID；标签只负责识别，选择仍使用不可变 ID，删除活动 ID 时先等待其 action 并关闭浏览器。`AgentBrowserRunner` 以 argv 和 `--json` 调用外部原生 CLI，设置硬超时、输出/content boundaries/domain allowlist，并在持久调用中传入受控 `--profile` 路径，底层 daemon 通过 Chrome CDP 工作。PicoBot 不链接 agent-browser 内部 crate、不直接暴露其 MCP、不使用 Fantoccini/ChromeDriver/WebDriver。持久 Profile 与 `allowed_domains` 因上游安全边界互斥；设置域名限制时临时浏览器仍可用，持久调用会被拒绝。截图只能写入配置的 artifact directory，并作为 `ModelAndUser` 产物返回，默认附到最终用户回复；仅当调用显式设置 `present_to_user=false` 时才作为模型内部观察。完整边界见 [AGENT_BROWSER_INTEGRATION.md](AGENT_BROWSER_INTEGRATION.md)。
`HealthService` 是依赖检查的唯一实现。CLI `picobot health`、只读 `health` 工具、`/health` 斜杠命令和 WebUI“配置 → 健康检查”必须复用它；受鉴权的 `GET /api/health` 按需返回 `HealthReport`，公开 `GET /health` 仍只承担轻量在线与版本探测，避免页面常驻轮询反复执行外部诊断命令。检查可探测命令、版本和配置路径；`fd` 与 Debian/Ubuntu 的同程序命令名 `fdfind` 都是首选后端，传统 `find` 才是降级回退。浏览器检查使用临时 socket 目录和专用 namespace 运行完整的 agent-browser offline doctor，将浏览器安装、真实 headless 启动和其余环境问题拆分报告，既不接触活动 daemon socket，也不跳过 launch probe。Health 不能安装/修复软件、连接模型 API 或泄漏配置秘密。
`AuthManager` 默认保护所有管理 API 和 `/ws`。静态资源、`/health`、`/api/auth/status` 与 `/api/auth/pair` 保持公开，使未配对浏览器只能加载配对界面。`picobot pair` 使用权限为 `0600` 的本机管理密钥调用仅接受真实回环连接的 `/api/auth/code`；反向代理即使从回环连接也无法在没有该密钥时签发代码。本机 `picobot run` 可用同一个管理密钥直接认证 `/ws`，但中间件必须同时验证请求路径严格等于 `/ws` 且 `ConnectInfo` 中的真实 TCP 对端为回环地址；这一身份不能访问管理 API。远程 `run` 与 TUI 一样使用已配对的 Bearer token。配对码为 8 位、5 分钟有效、单次消费，并按来源实施失败锁定。浏览器收到 HttpOnly、SameSite=Strict Cookie，CLI 使用 Bearer token；服务端仅持久化 SHA-256 哈希。`--revoke-all` 的持久化成功后才清空内存令牌，活动 WebSocket 每 5 秒复核身份并回收已撤销连接。
`AuthManager` 默认保护所有管理 API 和 `/ws`。静态资源、`/health`、`/api/auth/status` 与 `/api/auth/pair` 保持公开，使未配对浏览器只能加载配对界面。`picobot pair` 使用权限为 `0600` 的本机管理密钥调用仅接受真实回环连接的 `/api/auth/code`；反向代理即使从回环连接也无法在没有该密钥时签发代码。配对码为 8 位、5 分钟有效、单次消费，并按来源实施失败锁定。浏览器收到 HttpOnly、SameSite=Strict Cookie，CLI 使用 Bearer token；服务端仅持久化 SHA-256 哈希。`--revoke-all` 的持久化成功后才清空内存令牌，活动 WebSocket 每 5 秒复核身份并回收已撤销连接。
同源 `/api/*` 管理接口只提供显式白名单能力：
- `GET /api/health` 返回当前运行代 `HealthService` 的完整只读报告；它只在用户进入健康检查页或手动刷新时运行，不属于 Gateway 在线探测轮询。
- 配置读取/原子写入；响应中的密钥字段统一掩码，原样提交掩码会恢复现有值。配置加载器在不修改原始文件的前提下，从请求内副本移除可恢复的未知字段、类型/枚举错误和失效的非核心命名条目，生成有效 `Config`、RFC 6901 诊断路径和原始文件 SHA-256 revision；数组元素另外保留原始索引映射，避免恢复过程中索引移动导致清理错位。`models.*` 的 `flatten` Provider 扩展参数以及 MCP 的 `env`/`headers`/`tool_settings` 动态键属于显式扩展面，不作为未知项。JSON 损坏、不可用的 `default` Agent Provider/Model 链路和候选运行代无法安全构造仍是致命错误。普通 `PUT /api/config` 保持严格，不允许写入会被忽略的新配置；`POST /api/config/cleanup-invalid` 只按同一加载器报告的路径清理，并在 revision 不匹配时返回冲突。`POST /api/config/reload` 通过同一重载控制器校验并切换 Gateway 运行代，`GET /api/config/reload/status` 查询 generation、相位与最近错误。
- 配置读取/原子写入；响应中的密钥字段统一掩码，原样提交掩码会恢复现有值。运行配置只在重启后生效，不热替换运行中组件。
- `USER.md`、`AGENTS.md` 只允许固定文件名，不接受任意路径。
- 日志、记忆、任务和运行记录均限制单次返回数量；日志目录固定为 `~/.picobot/logs`。
- 任务与记忆读取复用 Storage API，不允许 WebUI 直接持有 SQLite 连接或拼接任意查询。
@ -298,12 +220,12 @@ MCP 发现的工具在 `ToolRegistry` 中使用 `mcp_<server-name>_<tool-name>`
### 启动
1. 解析配置路径，加载配置目录 `.env`，据此定位 workspace，再加载 workspace `.env`；既有进程环境保持最高优先级，合并后通过统一容错加载器重新解析配置。可恢复项只影响有效运行时投影并产生诊断，原始 `config.json` 保持不变。
2. 初始化日志，创建并切换到 workspace，初始化 WebUI 配对存储与本机管理密钥。
1. 加载配置和 `.env`，初始化 WebUI 配对存储与本机管理密钥。
2. 创建并切换到 workspace。
3. 初始化 SQLite、MemoryManager、MessageBus 和 SessionManager；Scheduler 启用时幂等创建默认日常维护巡检。
4. 注册内置工具、渠道、MCP 工具和 Cron 工具。
5. 启动所有 Channel。
6. 通过 TaskSupervisor 启动 inbound/control routers、dispatcher 和 scheduler。
6. 通过 TaskSupervisor 启动 message processor、dispatcher 和 scheduler。
7. 注册 WebUI 静态资源、管理 API 与聊天 WebSocket 路由。
8. 绑定 Axum listener，开始接收请求。
@ -325,7 +247,6 @@ MCP 发现的工具在 `ToolRegistry` 中使用 `mcp_<server-name>_<tool-name>`
3. 将可重试错误表示为 `ConnectionError`/`SendError`，永久错误使用其他类型。
4. 为 start/stop 幂等性、取消建连、投递失败和媒体边界增加测试。
5. 不要从 Channel 直接调用 SessionManager 或 Provider。
6. 若支持活动 Turn，实现 `live_policy`、`presentation_policy` 和每 Turn 一个实例的 `open_turn`；sink 必须消费完整快照而不是拼接 token，并使 finish/abort 清理幂等。
### 新增 Tool
@ -334,9 +255,6 @@ MCP 发现的工具在 `ToolRegistry` 中使用 `mcp_<server-name>_<tool-name>`
3. 明确工具需要“workspace 默认目录”还是“不可逃逸的硬边界”；后者必须显式校验 canonical path，不能只依赖 cwd。
4. 网络工具必须保留 SSRF/私网地址校验。
5. 长操作应有超时；后台执行应交给 SubAgentManager/TaskSupervisor。
6. 需要跨调用保存外部状态时，实现 `execute_with_context` 并按 session 隔离；不得让模型控制底层全局 session ID。
没有模型可调用的前台 `sleep`/等待工具：Agent 等待异步工作时，应结束当前 Turn 让排队完成/信号开启续接 Turn，或轮询状态工具。Turn 进入 `Cancelled` 时仍必须把 `Running` 的工具块同步归约为 `Cancelled`。需要跨重启的可靠延迟必须使用 Scheduler/后台任务。
### 新增 Provider

390
docs/CONFIG_HOT_RELOAD_DESIGN.md
View File

@ -1,390 +0,0 @@
# PicoBot 配置热重载设计与实现
本文档描述 PicoBot 1.3.0 配置热重载功能的设计目标、运行时模型、实现边界、失败语义和维护要求。代码与测试是最终事实来源；本文用于解释为什么采用当前方案，以及后续修改必须保持哪些不变量。
## 1. 背景
PicoBot 的配置并非只在一个全局对象中读取。Gateway 启动时会把配置拆分并复制到多个长生命周期组件：
- `SessionManager`、现有 `Session`、子 Agent 和 Scheduler 持有 Provider/模型配置。
- `ChannelManager` 按配置创建并启动飞书、CLI Chat 等 Channel。
- MCP 配置在启动时用于连接 Server，并把发现的工具注册到 `ToolRegistry`。
- Browser、文件上传、鉴权、后台任务并发度等配置在各自组件构造时固化。
- Gateway 的监听 socket、进程 cwd 和 SQLite 连接具有进程级生命周期。
因此，简单地重新读取 `config.json` 或替换一个 `Config` 指针并不能可靠生效。这样会造成请求处理组件混用新旧配置，例如新会话使用新模型、旧 Session 仍使用旧 Provider，或者配置显示飞书已禁用但旧连接仍在接收消息。
当前实现采用“运行代（runtime generation）切换”：先在旧运行代仍然服务时解析、校验并构造完整候选运行代；候选可用后排空当前交互工作，再回收旧运行代并激活新运行代。
## 2. 目标与非目标
### 2.1 目标
- 提供统一的 `picobot reload`、`/reload` 和 `reload_config` 工具入口。
- 在停止旧运行代前完成候选配置解析、关键字段校验和依赖构造。
- 配置错误或候选构造失败时继续使用旧运行代，不中断服务。
- 尽量让正在执行及已经排队的交互 Turn 完成，避免热重载直接截断发起重载的 Turn。
- 重新创建所有启动期固化配置的组件，使 Provider、Channel、MCP、Scheduler、Browser、鉴权和上传策略一致地切换。
- 保持监听 socket，不释放端口，避免切换期间被其他进程抢占。
- 保持运行期环境变量操作线程安全：热重载不得调用 `std::env::set_var`。
- 为所有入口提供相同的校验、排队和错误语义。
### 2.2 非目标
- 不支持热变更监听地址、workspace 或 SQLite 路径。
- 不承诺 WebSocket 连接无感迁移；切换会主动关闭旧连接，客户端需要重连。
- 不实现 nginx 式新旧 worker 长时间并行处理连接。PicoBot 是单进程、单 Gateway 运行代模型，采用保留 socket 的顺序切换。
- 不动态修改已经启动进程的环境变量；`.env` 新值只用于重新解析配置占位符和显式组件配置。
- 不把“请求已接受”解释为“切换已经完成”。触发方在候选运行代构造成功后收到响应，实际切换在排空阶段之后发生。
- 不赋予本地 admin token 调用管理 API 的新权限；重载 HTTP API 遵循现有设备鉴权边界。
## 3. 核心设计：Gateway 运行代
### 3.1 生命周期结构
Gateway 进程拥有两层生命周期：
```text
进程生命周期
├── 固定配置路径
├── 启动前进程环境快照
├── 启动 cwd
├── 原始监听 socket
├── ReloadController
└── 当前 Gateway 运行代（可替换）
├── GatewayState
├── SessionManager / Session workers
├── MessageBus / routers / outbound dispatcher
├── ChannelManager / Channel connections
├── Scheduler / MCP / tools
├── AuthManager / UploadRegistry
├── Axum Router / WebSocket connections
└── TaskSupervisor
```
进程级资源在 `gateway::run()` 外层只创建一次；运行代资源由 `GatewayState::from_config()` 重新构造。
### 3.2 为什么保留监听 socket
`gateway::run()` 首次启动时创建一个非阻塞 `std::net::TcpListener`，并在整个进程生命周期内持有它。每个运行代通过 `try_clone()` 获得一个 Tokio listener 交给 Axum。
切换时旧 Axum serve future 停止接受连接并退出，但原始 listener 仍然占有地址。旧运行代清理完成后，新运行代再克隆同一个 listener 开始接受连接。这样可以：
- 避免重新 bind 失败或端口被其他进程抢占。
- 保留内核 listen backlog；短暂切换窗口中的新 TCP 连接可能排队等待新运行代接收。
- 允许 Axum Router、鉴权 middleware 和 WebSocket state 随运行代完整替换。
这不是零停顿切换。旧运行代停止和受监督任务回收期间没有 Axum accept loop；当前回收宽限期上限为 10 秒，通常会更短。
## 4. 重载控制通道
重载协调类型位于 `src/gateway/reload.rs`：
- `ReloadHandle`：可克隆的请求端，注入 SessionManager、工具和 Gateway HTTP state。
- `ReloadController`：由 `gateway::run()` 独占，持有请求 receiver、启动环境快照和启动 cwd。
- `ReloadRequest`：包含 generation ID 与 oneshot response，用于把候选校验/构造结果返回触发方。
- `ReloadStatus`：记录 `preparing`、`draining`、`activating`、`active`、`failed` 相位、时间与最近错误。
控制通道使用容量为 8 的 Tokio MPSC 队列，并用原子 pending 标记保证同一时间最多只有一次重载。`ReloadHandle::request()` 使用 `try_send`：
- 已有重载尚未进入 `active` 或 `failed` 终态时立即返回 `another configuration reload is already pending`。
- Gateway 正在退出、receiver 已关闭时立即返回 `gateway is shutting down`。
- 请求成功入队后等待对应 oneshot 结果。
有界队列避免错误调用或模型重复调用形成无界重载积压；并发请求不会排队形成连续运行代切换，而是收到明确冲突错误。
## 5. 三种触发入口
三个入口只负责鉴权、参数适配和结果展示，最终都调用同一个 `ReloadHandle::request()`。
| 入口 | 实现 | 行为 |
|------|------|------|
| `picobot reload` | `src/main.rs`、`client::reload_gateway()` | 把 WebSocket/HTTP Gateway URL 转为 HTTP base URL，使用已保存的 TUI bearer token 调用 `POST /api/config/reload` |
| `/reload` | `SessionManager::execute_slash_command()` | 通过普通 slash command 路由执行，不进入 Agent 队列；结果作为 command output 返回当前 Channel |
| `reload_config` | `ReloadConfigTool` | 仅注册到根交互 Agent；无参数、独占执行，描述要求仅在用户明确要求重载时调用。子 Agent、Cron 和 managed scheduled Agent 无权获得该工具 |
`POST /api/config/reload` 返回 accepted generation；`GET /api/config/reload/status` 返回当前重载相位和最近错误。并发 `POST` 返回 409，Gateway 退出或候选准备失败返回 503，配置或不可变字段错误返回 400。
HTTP 路由属于现有 protected Router，因此：
- pairing 关闭时沿用 `PairingDisabled` 身份。
- pairing 开启时需要已配对的 bearer/cookie 凭据。
- 本地 `web_admin_token` 仍只允许用于既有的 loopback WebSocket 特例，不能绕过管理 API 鉴权。
pairing 开启但本机尚未保存 TUI token 时，`picobot reload` 会收到 HTTP 401；应先完成现有配对流程，或从一个已认证的聊天/WebUI 连接触发 `/reload`。
`GatewayState::new()` 只构造独立 state，没有运行代主循环，因此其中的 reload handle 明确不可用。正式 Gateway 必须通过 `gateway::run()` 启动，才能执行热重载。
## 6. 端到端时序
```mermaid
sequenceDiagram
participant Caller as CLI / Slash / Tool
participant RC as ReloadController
participant Old as Old GatewayState
participant New as Candidate GatewayState
participant HTTP as Axum / Listener
Caller->>RC: ReloadHandle::request()
RC->>RC: 重新读取 config + .env
RC->>RC: 校验 default agent、飞书凭据、不可变字段
RC->>New: GatewayState::from_config(candidate)
alt 解析、校验或构造失败
RC-->>Caller: Error
Note over Old: 旧运行代继续服务
else 候选构造成功
RC->>Old: 关闭 admission，拒绝新工作
RC-->>Caller: Accepted + generation / 等待当前任务后切换
RC->>Old: 等待 inbound、Session、Scheduler 与后台 Agent 排空（最多 60s）
RC->>Old: cancel WebSocket connections
RC->>HTTP: graceful shutdown 当前 serve future
RC->>Old: stop_all channels
RC->>Old: TaskSupervisor shutdown(10s)
RC->>New: start_all channels + message processing
RC->>HTTP: 从保留 listener 创建新 serve future
end
```
### 6.1 准备阶段
准备阶段在旧运行代继续提供服务时执行：
1. `load_candidate()` 重新读取当前 Gateway 启动时确定的配置文件。
2. 使用启动环境快照和启动 cwd 解析 `.env`、占位符与相对 workspace；统一配置加载器在请求内副本上忽略可恢复的未知字段、类型错误和失效非核心条目，并保留原始 JSON Pointer 诊断，不改写磁盘文件。
3. 校验 default agent 能解析为完整 `LLMProviderConfig`；核心链路不可恢复时仍拒绝候选。
4. 若飞书启用，校验 `app_id` 和 `app_secret` 非空。
5. 比较不可热变更字段。
6. 调用 `GatewayState::from_config()` 构造候选运行代。
候选构造会重新创建 Storage handle、MemoryManager、MessageBus、ChannelManager、SessionManager、工具集、MCP 配置 wrapper、AuthManager 和 UploadRegistry。外部 Channel、MCP 连接、消息 routers、outbound dispatcher 和 Scheduler 在候选成为当前运行代前不会激活。`from_config(..., false)` 也不会再次修改进程 cwd 或释放默认配置文件。
MCP 的真实连接位于激活阶段，避免准备候选时启动双份 stdio 子进程或提前覆盖进程级 `MCP_SERVER_STATUS`。单个 MCP Server 连接失败沿用启动语义：记录错误并跳过其工具，不会让整个运行代激活失败。
SessionManager 构造期注册的通知消费者和周期清理任务已经归候选 `TaskSupervisor` 所有，但在激活前没有候选消息入口；周期清理也会跳过首次 interval tick。
### 6.2 接受响应
候选运行代构造成功并关闭旧代 admission 后，Controller 通过 oneshot 返回 generation 与消息：
```text
配置校验通过；Gateway 将在当前任务结束后切换到新配置。
```
此响应表示候选配置已经通过准备阶段，不表示切换完成。先返回响应有两个原因：
- `/reload` 的 command output 需要通过旧运行代发送给用户。
- `reload_config` 工具需要返回 tool result，让发起它的 Agent Turn 正常完成。
### 6.3 排空阶段
每个运行代有一个 `RuntimeAdmission`。Inbound 在进入会话 lane 前获取 activity guard，因此已经进入 lane 的消息也计入排空；关闭 admission 后新消息不再进入会话处理，并尽量收到“正在重新加载”提示。`/reload` 的 command output 使用 outbound delivery acknowledgement，guard 只有在回复实际投递成功或明确失败后才释放，不再依赖固定 sleep。
`SessionManager::wait_until_idle()` 同时检查所有已加载 Session：
- `current_cancel.is_some()` 表示当前有 Agent Turn 正在执行。
- Session MPSC sender 的剩余容量小于最大容量，表示仍有排队任务。
- 必须连续空闲 100ms 才认为稳定，避免 worker 刚取出任务、尚未设置 `current_cancel` 的竞态窗口。
Scheduler 在领取和执行任务前检查 admission，已执行任务持有 guard 到结果提交与投递完成；后台子 Agent 同样持有 guard。最长统一等待 60 秒。超时不会撤销已经接受的重载，而是记录 warning 并继续回收旧运行代；未完成工作随后会被取消。
Axum 自身会优雅等待已进入 handler 的 HTTP 请求；其 graceful shutdown 另有 10 秒硬上限，超过后 abort serve task。未纳入 admission 的维护型后台任务由 `TaskSupervisor` 的 10 秒有界关停负责。
### 6.4 切换与回收阶段
切换顺序是：
1. 取消旧 `connection_shutdown`，使 WebSocket handler 主动退出。
2. 取消当前 Axum generation shutdown token，停止接受新请求并等待已进入的请求完成。
3. 调用旧 `ChannelManager::stop_all()`，停止外部消息入口。
4. 取消旧 `TaskSupervisor`，最多等待 10 秒，超时任务被 abort。
5. 把候选 state 设为当前 state。
6. 启动候选 Channel、MCP、消息处理循环、outbound dispatcher 和 Scheduler。
7. 从原始 listener 克隆新 Tokio listener，构建并运行新 Axum Router。
旧 WebSocket 不跨运行代迁移。TUI/WebUI 重连后按原有 client scope 从 SQLite 恢复 dialog 和历史；运行中的内存 Session 不直接搬迁到新 SessionManager。
## 7. 配置与环境变量语义
### 7.1 启动加载
正常启动使用以下优先级解析配置：
```text
进程启动环境 > workspace/.env > config目录/.env
```
启动时合并的 `.env` 值会在单线程阶段写入进程环境，供后续 MCP 和工具子进程继承。
### 7.2 热重载加载
Gateway 在首次调用 `Config::load_from()` 前保存：
- 原始进程环境 `startup_process_env`。
- 切换 workspace 前的 `startup_cwd`。
热重载调用 `Config::load_for_reload()`：
- 重新读取 config 目录 `.env` 和 workspace `.env`。
- 继续以原始启动环境作为最高优先级，避免启动时注入进程环境的旧 `.env` 值错误覆盖新文件。
- 相对 `workspace_dir` 始终相对于启动 cwd 解析，不受 Gateway 已经 `chdir(workspace)` 影响。
- 只解析得到候选配置，不调用 `env::set_var`，避免多线程进程中修改全局环境。
因此，`.env` 的修改会影响配置中的 `<VAR_NAME>` 占位符和由配置显式传入的新组件。它不会改变现有进程环境；仅依赖继承环境、但没有通过配置显式传值的 Shell/子进程仍会看到启动时环境。需要变更这类继承环境时应完整重启 Gateway。
## 8. 热重载边界
### 8.1 可通过新运行代生效的配置
以下配置消费者会随 `GatewayState` 重建：
| 配置区域 | 新运行代中的效果 |
|----------|------------------|
| `providers`、`models`、`agents` | 新 SessionManager、Session、主 Agent、子 Agent 和 Scheduler Agent 使用新 Provider/模型参数 |
| `channels` | ChannelManager 重新创建并启动已启用 Channel，allowlist、凭据、媒体和实时投递策略更新 |
| `mcp` | 重新连接 MCP Server，并重新生成工具注册表 |
| `browser` | 根据新配置注册或移除 Browser 工具 |
| `memory` | 重建 MemoryManager；维护任务使用新的 retention 配置 |
| `gateway.scheduler` | 启用、关闭或按新并发/轮询/超时参数创建 Scheduler |
| `gateway.max_concurrent_background_tasks` | 新 SubAgentManager 使用新的并发上限 |
| `gateway.file_transfer` | 新 UploadRegistry 和 WebSocket capability 使用新限制 |
| `gateway.require_pairing` | 新 Router/AuthManager 使用新鉴权要求；已有 WebSocket 在切换时断开 |
配置中尚未被运行时代码消费的字段，在热重载后仍然不会产生功能效果。例如当前 `session_ttl_hours` 和 `cleanup_interval_minutes` 只完成了解析，尚未接入 Session 清理逻辑。
`client.gateway_url` 是 CLI 侧配置：`picobot reload` 在发请求前读取它来确定目标 Gateway，但它不是 Gateway 运行代设置。
### 8.2 必须完整重启的配置
| 字段 | 原因 |
|------|------|
| `gateway.host`、`gateway.port` | 原始监听 socket 在进程生命周期内固定；热重载不会重新 bind |
| `workspace_dir` | Gateway 已修改进程 cwd，工具路径、媒体路径和相对文件语义均依赖它 |
| `gateway.session_db_path` | Storage、SessionManager、Scheduler 和历史恢复必须共享同一个数据库身份 |
| 未显式进入配置组件的继承环境变量 | 热重载禁止运行期修改进程全局环境 |
候选值与当前值不一致时，`load_candidate()` 返回错误，并明确提示重启 Gateway。`workspace_dir` 在比较前按启动 cwd 解析并 canonicalize；通过校验后会被归一化为当前绝对 workspace 路径，避免候选构造时受当前 cwd 影响。
## 9. 原子性与失败语义
这里的“原子”是指请求处理组件的配置可见性：旧运行代不会被逐项改造成半套新配置。它不是数据库事务，也不是两个 worker 的瞬时指针交换；进程级 MCP status 的提前更新是下文记录的已知例外。
| 失败阶段 | 行为 |
|----------|------|
| 已有 pending 重载/控制器关闭 | 分别返回 409/503，不读取配置 |
| JSON、`.env`、占位符或默认 Agent 校验失败 | 返回错误，旧运行代保持不变；可恢复的历史字段问题只产生诊断，不进入此失败分支 |
| 不可热变更字段发生变化 | 返回 restart-required 错误，旧运行代保持不变 |
| `GatewayState::from_config()` 构造失败 | 返回错误；候选被丢弃，其 TaskSupervisor 随对象释放取消；旧请求处理运行代保持不变 |
| 单个 MCP Server 连接或工具发现失败 | 记录 MCP 失败状态，候选继续构造且不注册该 Server 的工具；这不视为整体 reload 失败 |
| 等待交互空闲超过 60 秒 | 记录 warning，继续切换，旧运行中的剩余工作可能被取消 |
| 旧 Channel 停止失败 | 记录 error，继续回收其他组件和切换 |
| 旧受监督任务 10 秒内未退出 | TaskSupervisor abort 剩余任务，继续切换 |
| 候选激活阶段 `start_all()` 失败 | `gateway::run()` 返回错误；若由 systemd 管理，则按 service restart policy 重启 |
准备阶段成功后才向调用者返回 accepted。激活阶段仍可能遇到运行时错误，因此调用者不应把 accepted 当作健康检查；可使用 `GET /api/config/reload/status` 等待相同 generation 进入 `active`，并结合 `/health` 与客户端重连确认。
WebUI `PUT /api/config` 只负责严格校验、原子写文件、恢复被掩码的 secret 并返回 `restart_required: true`；它不会允许新提交内容依赖容错忽略，也不会隐式触发重载。`GET /api/config` 返回原始文件 SHA-256 revision 和可恢复诊断；`POST /api/config/cleanup-invalid` 必须提交相同 revision，后端才会在共享配置写锁下原子删除已诊断路径，revision 变化返回 409。显式的 `POST /api/config/reload` 将文件写入与运行代切换解耦，使用户可以批量编辑或清理后主动决定生效时机。
## 10. 并发与生命周期不变量
维护或扩展热重载时必须保持以下约束：
1. 只有 `gateway::run()` 拥有 reload receiver 和当前运行代，其他组件只能持有 `ReloadHandle`。
2. 候选构造不得修改旧 `GatewayState`，也不得提前连接 MCP 或替换旧 MessageBus、Channel、ToolRegistry、MCP status 或 Router。
3. 不得在热重载路径调用 `env::set_var`；环境文件只允许在单线程首次启动时安装到进程环境。
4. 不得释放原始 listener 后再尝试 bind 同一地址。
5. 旧 WebSocket 必须观察 `connection_shutdown`，不能在新鉴权/配置运行代启动后继续无限存活。
6. 旧长生命周期任务必须由旧 `TaskSupervisor` 回收；候选任务必须由候选 Supervisor 所有。
7. 排空检查不得长时间持有 `SessionManagerInner` 或 Session mutex；当前实现先复制 Session Arc，再逐一短暂检查。
8. reload tool 必须保持 exclusive 且只对根交互 Agent 可见，避免后台或子 Agent 触发进程级切换。
9. admission 必须在会话 lane 入队前获取；Slash 回复、Scheduler job 和后台子 Agent 必须持有 guard 到其持久化/投递边界。
10. 不可热变更字段的比较必须按有效路径归一化，并发生在候选运行代构造之前。
11. 新增启动期配置消费者时，应确认它是否由 `GatewayState::from_config()` 重建，并更新本文的热重载边界表。
## 11. 关键实现位置
| 文件/符号 | 职责 |
|-----------|------|
| `src/gateway/reload.rs` | 重载请求通道、候选配置加载、不可变字段校验 |
| `src/gateway/mod.rs::run` | 持有 listener、当前运行代和切换主循环 |
| `src/gateway/mod.rs::GatewayState::from_config` | 构造一套完整运行代依赖 |
| `src/config/mod.rs::Config::load_for_reload` | 使用启动环境/cwd 安全重新解析配置，不修改进程环境 |
| `src/session/session.rs::wait_until_idle` | 检查活动 Turn、Session 队列和稳定空闲窗口 |
| `src/session/session.rs::execute_slash_command` | `/reload` 入口 |
| `src/tools/reload_config.rs` | Agent 可调用的独占重载工具 |
| `src/gateway/http.rs::reload_config` | 受保护的 `POST /api/config/reload` |
| `src/client/mod.rs::reload_gateway` | CLI HTTP 客户端与 bearer token 注入 |
| `src/main.rs::Command::Reload` | `picobot reload` CLI 定义 |
## 12. 测试策略
当前回归测试覆盖：
- 候选配置允许 Provider/模型等运行时字段变化。
- 可恢复未知字段、类型错误和失效非核心条目不阻止候选，并保留原始数组索引诊断；严格 WebUI 写入拒绝同样内容，清理只删除诊断路径。
- 相对 `workspace_dir` 按启动 cwd 正确解析。
- workspace 变化被拒绝且返回明确错误。
- `None` 与显式指向同一有效数据库路径（默认 `{config_dir}/data/picobot.db`）时允许重载。
- admission 关闭后拒绝新工作，并等待现有 activity guard 释放。
- command output 在 dispatcher 明确确认投递前不会释放处理任务。
- 真实子进程 Gateway 可完成 generation 2 切换；无效候选返回 400 且旧代 `/health` 继续可用。
- `/reload` alias 能解析到规范命令。
- 全量 Rust 单元测试验证 Gateway、Session、Channel、鉴权和 TaskSupervisor 既有行为。
- 离线协议集成测试验证 slash/WebSocket 相关协议没有退化。
- Clippy、Cargo build 和 WebUI check/build 验证完整构建链。
后续适合增加的集成测试：
1. 使用可控假 Provider 验证新 model 被下一 Turn 实际使用。
2. 在长 Turn 中调用 `reload_config`，验证 tool result 和最终消息投递后才断开。
3. Session 队列有积压时验证重载等待队列排空。
4. 排空超时、Channel stop 超时和候选激活失败的故障注入。
5. 重载前后鉴权策略变化以及旧 WebSocket 失效。
## 13. 运维使用
修改配置并保存后执行：
```bash
picobot reload
```
连接非默认 Gateway：
```bash
picobot reload --gateway-url https://gateway.example.com
```
也可以在支持 slash command 的聊天中发送：
```text
/reload
```
若配置修改涉及监听地址、workspace、数据库路径或必须进入进程继承环境的变量，应执行完整重启：
```bash
picobot service restart
```
建议的运维流程是：
1. 原子保存配置文件。
2. 执行 reload 并检查返回是否为候选已接受。
3. 使用已认证请求轮询 `GET /api/config/reload/status`，确认返回的 generation 进入 `active`。
4. 等待客户端重连，检查 `/health` 和关键 Channel。
5. 若激活失败，由 systemd 重启或人工恢复配置后再次启动。
## 14. 已知限制与演进方向
- 旧运行代回收与新运行代激活是顺序执行，存在短暂 accept/Channel intake 空窗。
- WebSocket 需要客户端自行重连，服务端没有连接迁移协议。
- 候选激活失败没有自动回滚到已经回收的旧运行代，依赖 systemd restart 或人工恢复。
- 同一时间只允许一个 pending 重载；并发请求返回 409，不做合并或排队。
- `.env` 不能热修改进程继承环境。
- admission 覆盖消息入口、Scheduler 与后台子 Agent；其他维护型任务仍依赖 TaskSupervisor 的有界关停。
可能的后续演进包括：
- 将 Channel 激活前检查拆成显式 `prepare()`，把更多运行时失败提前到旧运行代仍可回退的阶段。
- 在不破坏 Channel/Session 边界的前提下，引入动态 Router service，实现新旧 HTTP generation 短期重叠。
- 为 TUI/WebUI 增加 reload 完成通知和自动重连状态提示。

369
docs/CONFIG_HOT_RELOAD_REVIEW.md
View File

@ -1,369 +0,0 @@
# 配置热重载功能审核报告
> 状态：审查完成；主要意见已于 2026-07-21 落地。本文前六节保留首次审查快照，行号、测试数和“当前实现”描述可能已过时；以下处置表与代码为最新结论。
>
> 本文是对 `docs/CONFIG_HOT_RELOAD_DESIGN.md` 描述的配置热重载功能及其当前未提交实现的综合审核。审核覆盖架构合理性、实现一致性、关键不变量和改进建议；具体行为以代码和测试为最终依据。相关实现位于 `src/gateway/reload.rs`、`src/gateway/mod.rs::run`、`src/config/mod.rs::load_for_reload`、`src/session/session.rs::wait_until_idle`、`src/tools/reload_config.rs` 与 `src/gateway/http.rs::reload_config`。
## 0. Review 意见处置与实现结果
| 意见 | 处置 | 结论与实现 |
|------|------|------------|
| A1 / E1 MCP 准备期副作用 | 接收 | `connect_all()` 移到运行代激活阶段，候选构造不再启动双份 MCP 或提前覆盖全局 status。 |
| A2 / E2 generation 与状态查询 | 接收 | 每次请求分配 generation；新增 `GET /api/config/reload/status`，状态为 `preparing/draining/activating/active/failed`。 |
| A3 激活失败回滚 | 部分接收 | 接收“需要明确失败状态”，但驳回先启动新 Channel 再停旧 Channel。飞书长连接双开会重复消费事件，风险高于短暂切换空窗；完整 prepare/activate 或可恢复旧代留作后续。 |
| A4 / L4 Agent 工具风险 | 部分接收 | 保留用户要求的 `reload_config`，但只允许根交互 Agent 使用；子 Agent、Cron、managed scheduled Agent 均剔除该工具，并保持 exclusive。驳回直接删除工具。 |
| A5 / E5 后台任务排空 | 接收 | 引入运行代 admission/activity guard；Scheduler 已执行 job 与后台子 Agent 持有 guard，新任务在 drain 后不再进入。 |
| I1 / E4 Slash 回复可能丢失 | 接收 | `/reload` 所在 inbound 在 lane 入队前持有 guard；command output 改为显式等待 outbound delivery acknowledgement，删除固定 500ms sleep。 |
| I2 / F2 DB 路径误判 | 接收 | 比较归一化后的有效路径，`None` 与 `./picobot.db` 可判为同一数据库。 |
| I3 零散预校验 | 部分接收 | 保留 default agent 与 Feishu 的快速错误提示；其余组件统一由候选 `from_config()` 构造验证，不继续扩张 ad-hoc 字段校验。长期采用显式 prepare contract。 |
| I4 HTTP 错误码 | 接收 | pending 返回 409，退出/准备故障返回 503，配置与不可变字段错误返回 400。错误改为 `ReloadError` 类型。 |
| I5 候选双 Storage | 驳回为正确性缺陷 | 同库多连接是 sqlx/SQLite 的正常模式，候选无消息入口；路径身份仍被强制保持一致。该点保留为测试与锁竞争观察项，而非阻止上线。 |
| F1 集成测试 | 部分接收 | 新增真实 Gateway 子进程测试，覆盖成功切换到 generation 2、状态查询、无效候选不影响旧代健康。可控假 Provider、长 Turn 和故障注入仍待补充。 |
| L1 prepare/activate 分离 | 方向接收 | 本轮已把 MCP activation 与候选构造分离；完整组件级接口留作后续架构演进。 |
| L2 动态 Router | 暂缓 | 当前保留 listener 并为 Axum graceful shutdown 增加 10 秒硬上限；不为本功能引入动态 service 复杂度。 |
| L3 客户端完成通知 | 暂缓 | 后端 generation/status 已具备；TUI/WebUI 展示可在后续独立实现。 |
首次 Review 未指出、但本轮一并修复的两个关键问题：候选构造原先直接在 `select!` 分支内 await，会暂停轮询旧 Axum serve 与进程信号；现在 serve 独立受监督运行，准备和排空阶段都继续响应服务退出。其次，原实现排空前没有关闭入口，持续新消息可能让排空永不稳定；现在 admission 先关闭再 drain。
## 1. 审查范围与依据
### 1.1 审查对象
- 设计文档：`docs/CONFIG_HOT_RELOAD_DESIGN.md`
- 实现：当前未提交的 17 个文件变更与 2 个新增文件（`src/gateway/reload.rs`、`src/tools/reload_config.rs`），共 ~304 行净增
- 关联变更：`Cargo.toml`、`webui/package.json` 版本号 1.2.2 → 1.3.0；`README.md`、`AGENTS.md`、`docs/ARCHITECTURE.md`、`resources/skills/about-picobot/references/config.md` 文档同步
### 1.2 审查依据
- 仓库既有架构边界与并发不变量（见 `docs/ARCHITECTURE.md`）
- 现有相似机制（Channel 生命周期、TaskSupervisor、TurnController、OutboundDispatcher）
- 验证命令：`cargo build`、`cargo clippy --all-targets --all-features -- -D warnings`、`cargo test --lib`（330 passed）、`webui && npm run check && npm run build`
### 1.3 验证结果
| 命令 | 结果 |
|------|------|
| `cargo build` | 通过 |
| `cargo clippy --all-targets --all-features -- -D warnings` | 通过 |
| `cargo test --lib` | 330 passed / 0 failed |
| `cd webui && npm run check && npm run build` | 0 errors / 0 warnings |
## 2. 架构审查
### 2.1 整体架构评估
**结论：架构方向正确，运行代（runtime generation）切换模型是 PicoBot 配置散落现状下的唯一可靠方案。**
PicoBot Gateway 启动时把配置拆分复制到 `SessionManager`、`ChannelManager`、MCP、Scheduler、Browser、Auth、Upload 等长生命周期组件；`gateway.host/port`、进程 cwd、SQLite 连接具有进程级生命周期。在这种结构下，任何"原地替换 Config 指针"的方案都会导致请求处理组件混用新旧配置（新会话用新模型、旧 Session 仍用旧 Provider；或飞书配置显示禁用但旧连接仍在接收消息）。
运行代切换通过"先在旧代仍服务时构造完整候选代；候选可用后排空当前交互工作，再回收旧代并激活新代"避免了半套配置暴露。这一选择与 PicoBot 单进程、单 Gateway 模型契合，不引入 daemon/fork 层。
### 2.2 运行代模型合理性
运行代模型的关键设计点均合理：
| 设计点 | 评估 |
|--------|------|
| 保留原始 `std::net::TcpListener`，每代 `try_clone()` | ✅ 避免 bind 失败与端口抢占，保留内核 backlog |
| 候选构造期间不修改旧 `GatewayState` | ✅ 避免半套配置暴露 |
| `Config::load_for_reload` 使用启动环境快照、不调 `set_var` | ✅ 多线程运行期修改进程环境的危险被正确规避 |
| 不可变字段（host/port/workspace/db_path）边界清晰 | ✅ 边界划分正确 |
| 三个入口（CLI、`/reload`、`reload_config` 工具）共享控制通道 | ✅ 单一校验路径，语义一致 |
| 拒绝 nginx 式新旧 worker 长期并行 | ✅ 单进程规模不值得这份复杂度 |
| `ReloadHandle` 用有界 MPSC + `try_send` | ✅ 无界积压被正确拒绝 |
### 2.3 边界划分评估
热重载边界表（设计文档第 8 节）覆盖完整：
- 可热重载：`providers`/`models`/`agents`、`channels`、`mcp`、`browser`、`memory`、`gateway.scheduler`、`gateway.max_concurrent_background_tasks`、`gateway.file_transfer`、`gateway.require_pairing`
- 必须重启：`gateway.host`/`port`、`workspace_dir`、`gateway.session_db_path`、未进入配置组件的继承环境变量
边界划分与实现中 `load_candidate()` 的校验项一一对应。`workspace_dir` 在比较前按启动 cwd 解析并 canonicalize（`reload.rs:80-89`），与 `from_config` 中 `ensure_workspace_dir` 的 canonicalize 行为一致，比较基准正确。
### 2.4 并发不变量评估
设计文档第 10 节列出的 10 条不变量在实现中均得到遵守：
| 不变量 | 实现位置 | 遵守情况 |
|--------|----------|----------|
| 只有 `run()` 拥有 receiver 与当前运行代 | `mod.rs:331` ReloadController 在 `run()` 内创建 | ✅ |
| 候选构造不修改旧 GatewayState | `mod.rs:399` `from_config` 创建独立 state | ✅ |
| 不在热重载路径调 `env::set_var` | `config/mod.rs:611` `apply_to_process=false` | ✅ |
| 不释放原始 listener 后 rebind | `mod.rs:345` listener 在 `run()` 内持有 | ✅ |
| 旧 WebSocket 观察 `connection_shutdown` | `mod.rs:421` 切换前 cancel | ✅ |
| 旧任务由旧 TaskSupervisor 回收 | `mod.rs:432-436` 旧 supervisor shutdown | ✅ |
| 排空检查不长时间持有 Session mutex | `session.rs:2113-2124` 先克隆 Arc 再短锁 | ✅ |
| reload tool 保持 exclusive | `reload_config.rs:49` `exclusive: true` | ✅ |
| 不可变字段比较在候选构造之前 | `reload.rs:80-100` | ✅ |
| 新增启动期配置消费者需更新边界表 | 文档约束 | ⚠️ 维护性约束，无机制强制 |
## 3. 实现审查
### 3.1 与设计文档的一致性
实现与设计文档的关键路径高度一致：
| 设计文档章节 | 实现位置 | 一致性 |
|--------------|----------|--------|
| §6.1 准备阶段：load_candidate → from_config | `mod.rs:386-408` | ✅ |
| §6.2 接受响应：候选构造成功后通过 oneshot 返回 | `mod.rs:410-411` | ✅ |
| §6.3 排空阶段：wait_until_idle 60s + 500ms 投递窗口 | `mod.rs:412-419` | ✅ |
| §6.4 切换顺序：connection_shutdown → generation_shutdown → channel stop → task_supervisor → 新代启动 | `mod.rs:421-422, 429-436, 350-351` | ✅ |
| §4 控制通道：容量 8、try_send、队列满立即返回错误 | `reload.rs:8, 47-54` | ✅ |
| §7.2 环境变量语义：不修改进程环境 | `config/mod.rs:541-553` | ✅ |
| §9 失败语义：候选构造失败时旧代不变 | `mod.rs:394-408` continue 不切换 | ✅ |
### 3.2 关键路径分析
#### 3.2.1 切换时序
`gateway::run` 的主循环（`mod.rs:349-441`）正确实现了运行代切换：
```
外层 loop {
start_all / start_message_processing // 新代激活
内层 loop { select! { serve | process_signal | reload_request } }
serve.await // 等待旧 axum graceful shutdown
channel_manager.stop_all // 停止旧 channel intake
task_supervisor.cancel + shutdown(10s)// 回收旧受监督任务
state = next_state // 切换
}
```
`wait_for_shutdown_signal()` 在外层循环内创建（`mod.rs:365`），每代新建 future，不存在重复 poll 已完成 future 的 UB。
#### 3.2.2 候选构造与旧代隔离
`from_config`（`mod.rs:51-236`）为候选创建独立的 Storage、MessageBus、ChannelManager、SessionManager、ToolRegistry、AuthManager、UploadRegistry。候选的 `channel_manager.init()` 仅构造 Channel 对象，不调用 `start()`，因此不会与旧 channel 并发连接飞书 API。
#### 3.2.3 监听 socket 保留
`std::net::TcpListener`（`mod.rs:345`）在进程生命周期内持有；每代通过 `try_clone()`（`mod.rs:353`）获得新 fd。旧代 `serve` future 退出时仅释放克隆 fd，原始 socket 不释放，新代可重新克隆并 accept。内核 backlog 在切换窗口中暂存新 TCP 连接。
### 3.3 测试覆盖评估
**结论：单元测试覆盖不足，集成测试完全缺失。**
当前测试：
| 测试 | 位置 | 覆盖范围 |
|------|------|----------|
| `candidate_accepts_runtime_changes_and_rejects_workspace_changes` | `reload.rs:127` | load_candidate 的 model_id 变更接受与 workspace 拒绝 |
| `resolve_slash_command("reload")` | `session.rs:3485` | slash 命令解析 |
缺失但设计文档第 12 节明确列出的测试：
1. 启动真实 Gateway、修改 model/channel 配置、HTTP 触发重载、验证新代生效
2. 配置无效时验证旧 WebSocket 与旧 Provider 仍可工作
3. 长 Turn 中调用 `reload_config`、验证 tool result 与最终消息投递后才断开
4. Session 队列积压时验证重载等待排空
5. 排空超时、Channel stop 超时、候选激活失败的故障注入
6. 重载前后鉴权策略变化与旧 WebSocket 失效
当前测试仅覆盖纯函数路径（`load_candidate`、slash 解析），未验证任何运行时切换行为。这是上线前的主要风险点。
### 3.4 代码质量
- **Clippy**：`-D warnings` 通过
- **类型安全**：`ReloadHandle::unavailable()`（`reload.rs:39`）通过 drop receiver 使 `try_send` 返回 `Closed`，正确表达"Gateway 未由 `run()` 启动"的语义
- **错误处理**：候选构造失败时 `request.response.send(Err(...))` 后 `continue`，不切换；MCP 单 server 失败沿用启动语义（记录错误、跳过工具），不阻断候选构造
- **资源管理**：旧 TaskSupervisor 的 10s 有界 shutdown + abort 保证回收有硬时间边界
## 4. 发现的问题
### 4.1 架构层面问题
#### A1. MCP 在准备阶段连接，制造进程级副作用【中】
`from_config`（`mod.rs:171`）调用 `mcp::connect_all()`，立即建立 MCP 客户端连接或启动 stdio 子进程，并更新进程级 `MCP_SERVER_STATUS`（`mcp/mod.rs:42`）。这违反了设计文档第 10 节不变量 #2"候选构造不得修改旧 GatewayState"的精神——MCP status 虽非请求处理状态，但仍是旧代可见的进程级状态。
**具体影响：**
- 候选构造期间，旧代的 `/mcp` 命令显示候选的连接状态而非旧代状态
- stdio MCP 子进程双份运行（旧代 + 候选）可能竞争资源或 stdin/stdout
- 候选在 `connect_all` 之后失败（如 `ensure_default_maintenance_job` 失败，`mod.rs:194`），MCP 连接被 drop 但 `MCP_SERVER_STATUS` 仍显示 `connected: true`，旧代 `/mcp` 显示陈旧数据
设计文档第 6.1 节与第 14 节将此列为"已知例外"。但该例外的收益仅为"提前发现 MCP 失败"——而 MCP 单 server 失败本就被当非致命跳过（`mcp/mod.rs:171`），不需要提前连接来验证。
#### A2. 无 generation ID 与状态查询【中】
调用方只能得到准备阶段结果（"配置校验通过；Gateway 将在当前任务结束后切换到新配置"），无法查询重载最终是否完成。运维需要翻日志或重连客户端确认切换状态。设计文档第 14 节将此列为"演进方向"，但 generation ID（一个 atomic 计数器）+ `/api/config/reload/status` 端点成本极低，收益显著，应在 v1 内置。
#### A3. 候选激活失败无回滚【中-高】
切换顺序为：停旧 channel → 取消旧 TaskSupervisor → 切换 state → 启动新 channel（`mod.rs:429-351`）。若新代 `start_all()` 失败，`run()` 返回错误，依赖 systemd 拉起。但 channel 启动失败常为瞬时问题（飞书 5xx、端口冲突），丢掉本来正常服务的旧代去重启是可用性损失。旧代此时已被回收，无法回滚。
#### A4. `reload_config` Agent 工具的软约束【低-中】
`ReloadConfigTool`（`reload_config.rs`）注册到默认工具集，依赖 description"仅在用户明确要求重新加载配置时调用"约束 LLM。LLM compliance 是软约束，非可靠边界。`exclusive: true` 仅保证不与其他副作用工具并行，不保证调用时机正确。
#### A5. 排空仅覆盖交互 Session，不覆盖 Scheduler/后台 Agent【低】
`wait_until_idle` 仅检查内存中 Session 的 `current_cancel` 与 `agent_tx` 队列。Scheduler job、独立后台子 Agent、HTTP handler 不在排空范围内。Scheduler job 可能正在写 DB 或发送消息，被 TaskSupervisor 10s abort 截断可能留下不一致状态。设计文档第 6.3 节已承认此范围。
### 4.2 实现层面问题
#### I1. `/reload` slash command 绕过 `wait_until_idle`【中】
`session.rs:2631` 显示 slash command 在 `handle_message` 内联处理，返回 `HandleResult::CommandOutput`，**不进入 session worker 队列**。因此 `wait_until_idle` 检查 `current_cancel`/`agent_tx.capacity()` 时看不到 `/reload` Turn 的活动状态，立即返回（仅 100ms 稳定 + 500ms sleep）。
slash command output 经 `process_inbound` → `publish_command_output` → outbound dispatcher → channel API 投递，整条链路必须在 ~600ms（+ serve graceful shutdown + 10s TaskSupervisor shutdown）内完成。对 CLI channel 足够，但对 Feishu 等远端 channel 较紧。设计文档第 6.3 节"为 slash command output 和终态投递留出发送窗口"承认了 500ms 窗口，但 500ms 是固定值，无背压保证。
实际窗口因 TaskSupervisor 的 10s shutdown 较宽，不会丢消息——但若 channel `stop_all()` 关闭了连接，in-flight 的 `send_message` 可能失败。
#### I2. `session_db_path` 比较为原始字符串【低】
`reload.rs:91` 直接比较 `current.gateway.session_db_path != candidate.gateway.session_db_path`。若用户把 `null` 改为 `"./picobot.db"`（解析后同一路径），会被误拒。保守是对的，但产生假阳性。`workspace_dir` 已做 canonicalize 比较，`session_db_path` 应保持一致。
#### I3. `load_candidate` 仅校验 Feishu 凭据【低】
`reload.rs:73-78` 仅校验飞书 `app_id`/`app_secret` 非空。其他 channel 配置（若有）、MCP 配置、Browser 配置等在 `from_config` 期间才验证，可能在那里失败。这与设计文档第 6.1 节一致（"若飞书启用，校验 app_id 和 app_secret 非空"），但将失败发现延后到了候选构造阶段。
#### I4. `reload_config` HTTP handler 错误码语义【低】
`http.rs:355` 对所有错误用 `ApiError::bad_request`（400）。"gateway is shutting down"（队列关闭）更适合 503 Service Unavailable，"another configuration reload is already pending"更适合 409 Conflict。
#### I5. 候选 Storage 与旧 Storage 共享同一 SQLite 文件【低】
`from_config` 为候选创建新 Storage 连接到同一 `picobot.db`。候选的 background notification consumer 与 cleanup task 已归候选 TaskSupervisor，cleanup 跳过首次 tick，无入口触发 notification，故实际不写。理论上有并发写锁竞争可能，实际风险低。设计文档未显式说明此点。
### 4.3 严重度分级
| 问题 | 严重度 | 影响 |
|------|--------|------|
| A3 候选激活失败无回滚 | 中-高 | 瞬时 channel 故障导致整个 Gateway 重启 |
| A1 MCP 准备阶段连接 | 中 | 进程级 status 污染、子进程双份、失败后状态陈旧 |
| A2 无 generation ID | 中 | 运维无法确认切换完成状态 |
| I1 `/reload` 绕过排空 | 中 | 远端 channel output 投递窗口紧 |
| A4 reload_config 软约束 | 低-中 | LLM 误调用风险 |
| A5 排空不覆盖 Scheduler | 低 | 后台 job 被硬取消可能留下不一致 |
| I2 session_db_path 假阳性 | 低 | 用户需重启而非重载 |
| I3 仅校验 Feishu | 低 | 失败发现延后 |
| I4 HTTP 错误码 | 低 | 语义不准确 |
| I5 双 Storage 连接 | 低 | 理论并发风险 |
## 5. 改进建议
### 5.1 必须修复（上线前）
#### F1. 补充端到端集成测试
至少覆盖设计文档第 12 节列出的前 3 项：
1. 启动真实 Gateway、修改 model_id、HTTP 触发重载、验证新 model 在新 Turn 中生效
2. 配置无效（如 default agent 解析失败）时验证旧 WebSocket 与旧 Provider 仍可工作
3. 长 Turn 中调用 `reload_config`、验证 tool result 与最终消息投递后才断开
这些测试无法用单元测试替代，需启动真实 Gateway 进程。
#### F2. `session_db_path` 比较归一化
`reload.rs:91` 应将 `session_db_path` 相对于 workspace 解析并 canonicalize 后比较，与 `workspace_dir` 处理方式一致。`None` 与 `"picobot.db"`（默认值）应视为等价。
### 5.2 建议增强（近期演进）
#### E1. MCP 连接移出 `from_config`，消除"已知例外"
将 `mcp::connect_all()` 从 `from_config`（`mod.rs:171`）移到 `start_all()` 阶段，与 channel 启动同相位。收益：
- 消除进程级 `MCP_SERVER_STATUS` 在候选构造期间被污染
- 消除 stdio 子进程双份运行
- 候选失败时无 MCP 连接泄漏
- 不再需要设计文档第 10 节不变量 #2 的"已知例外"声明
代价：MCP 连接失败从"候选构造失败"延后到"激活失败"。但 MCP 单 server 失败本就非致命（跳过该 server 工具），整体激活失败语义不变。
#### E2. 引入 generation ID 与 status 查询
在 `ReloadController` 增加 `Arc<AtomicU64>` generation 计数器与 `ReloadState` enum（`Idle`/`Preparing`/`Draining`/`Activating`/`Active`/`Failed`）。提供 `GET /api/config/reload/status` 端点。调用方在收到 "accepted" 后可轮询确认切换完成。成本极低（一个 atomic + 一个路由 + 一个 enum），运维收益显著。
#### E3. 旧代保留至新代 `start_all()` 成功
调整切换顺序为：先启动新 channel（新代 channel_manager.start_all），成功后再停止旧 channel。短暂双 channel 并存对飞书 webhook 幂等消息可容忍。代价是需处理两代 channel 并存的资源冲突（如媒体目录、飞书事件去重），但避免瞬时 channel 故障导致 Gateway 整体重启。
若实现成本过高，至少应在 `start_all()` 失败时尝试重启旧代 channel（旧 TaskSupervisor 已 cancel，可能无法恢复；需评估可行性）。
#### E4. `/reload` slash command 排空路径
两种方案：
- **方案 A**：让 reload controller 记录触发源（channel, chat_id），等待该 inbound lane 的当前消息处理完成后再切换，而非泛化等待所有 session idle
- **方案 B**：为 outbound dispatcher 增加显式 drain contract，在 `stop_all` 前等待 outbound 队列排空或超时
方案 A 更精确，方案 B 更通用。两者都比固定 500ms sleep 可靠。
#### E5. Scheduler 与后台 job 协作式 drain
在 Scheduler job 的协作取消边界检查 reload token，允许 job 在写 DB 前/后选择继续完成或退出。避免 TaskSupervisor 10s abort 截断 DB 写一半的 job。设计文档第 14 节已列出此项。
### 5.3 演进方向（中长期）
#### L1. `prepare()` / `activate()` 显式分离
将 `from_config` 拆为：
- `construct()`：纯内存，无 I/O，可反复调用
- `prepare()`：可失败的 I/O（channel health check、MCP 连接、Storage ping），旧代仍服务
- `activate()`：开始接收消息
使更多失败前移到旧代仍可回退的阶段，而非等到 activate 才暴露。设计文档第 14 节已列出此项。
#### L2. 动态 Router service
引入新旧 HTTP generation 短期重叠，消除 accept/Channel intake 空窗。需不破坏 Channel/Session 边界。设计文档第 14 节已列出此项。
#### L3. WebUI/TUI reload 完成通知
客户端在 WebSocket 重连后显示 reload 完成状态与自动重连提示。依赖 E2 的 generation ID。
#### L4. 重新评估 `reload_config` Agent 工具
考虑移除该工具，仅保留 CLI 与 `/reload`。Agent 触发进程级状态切换的风险（A4）可能不抵边际收益。若保留，应在 SessionManager 层做调用方校验（如要求参数带确认 token），而非靠 description。
## 6. 总体结论
### 6.1 设计评估
设计文档质量高，边界清晰，不变量明确，失败语义完整。运行代切换模型是 PicoBot 当前架构下的正确选择。设计文档诚实地列出了已知限制（第 14 节），未掩饰缺陷。
主要设计层面的不足是把几件本应 v1 内置的能力（generation ID、MCP 相位对齐、旧代保留至新代激活成功）推迟到"演进方向"，导致可用性与可观测性打了折扣。MCP 的"已知例外"（A1）是设计妥协被文档化的典型，留着会让后续维护者也认为"再来一个例外无所谓"，应尽早消除而非长期承担。
### 6.2 实现评估
实现忠实遵循设计，关键不变量均得到遵守。代码通过 `cargo build`、`cargo clippy -D warnings`、`cargo test --lib`（330 passed）与 WebUI `check/build`。版本号、文档、AGENTS.md 同步更新。
主要实现层面的不足是测试覆盖：仅 `load_candidate` 与 slash 解析有单元测试，无任何运行时切换行为的集成验证（I1-I5 中多数问题需要集成测试才能暴露）。设计文档第 12 节明确列出但未实现的 6 项集成测试是上线前的主要风险。
### 6.3 上线建议
| 项 | 判定 |
|----|------|
| 架构方向 | ✅ 可接受 |
| 实现一致性 | ✅ 可接受 |
| 代码质量 | ✅ 可接受（Clippy/tests/build 全通过） |
| 测试覆盖 | ⚠️ 不足，需补集成测试（F1） |
| 已知限制 | ⚠️ 文档已承认，但 A1/A3 应优先修复 |
**建议：在完成 F1（集成测试）与 F2（session_db_path 归一化）后可提交。A1（MCP 相位）、A3（无回滚）应列为后续优先修复项，不应长期承担。**
## 附录：关键文件与符号索引
| 文件/符号 | 职责 | 行号 |
|-----------|------|------|
| `src/gateway/reload.rs::ReloadController` | 重载控制通道、候选配置加载、不可变字段校验 | 19-36 |
| `src/gateway/reload.rs::ReloadHandle` | 可克隆的请求端，注入 SessionManager/工具/HTTP state | 14-59 |
| `src/gateway/reload.rs::load_candidate` | 候选配置解析与不可变字段校验 | 61-102 |
| `src/gateway/mod.rs::run` | 持有 listener、当前运行代与切换主循环 | 318-441 |
| `src/gateway/mod.rs::GatewayState::from_config` | 构造一套完整运行代依赖 | 51-236 |
| `src/gateway/mod.rs::build_router` | 为每代构建 Axum Router | 448-489 |
| `src/config/mod.rs::Config::load_for_reload` | 使用启动环境/cwd 安全重新解析配置 | 541-553 |
| `src/config/mod.rs::Config::load_from_with_process_env` | 共享的配置加载实现，支持不写入进程环境 | 547-625 |
| `src/session/session.rs::wait_until_idle` | 检查活动 Turn、Session 队列与稳定空闲窗口 | 2109-2144 |
| `src/session/session.rs::execute_slash_command` | `/reload` slash 入口 | 2093-2099 |
| `src/tools/reload_config.rs::ReloadConfigTool` | Agent 可调用的独占重载工具 | 6-52 |
| `src/gateway/http.rs::reload_config` | 受保护的 `POST /api/config/reload` | 347-360 |
| `src/client/mod.rs::reload_gateway` | CLI HTTP 客户端与 bearer token 注入 | 101-124 |
| `src/main.rs::Command::Reload` | `picobot reload` CLI 定义 | 59-64, 132-138 |
| `src/mcp/mod.rs::connect_all` | MCP 连接（当前在 from_config 期间调用，见 A1） | 126-181 |
| `src/mcp/mod.rs::MCP_SERVER_STATUS` | 进程级 MCP 状态（A1 的副作用源） | 36-45 |

1008
docs/CONTEXT_COMPACTION_DESIGN.md
View File

File diff suppressed because it is too large Load Diff

553
docs/MEMORY_SYSTEM_DESIGN.md
View File

@ -1,553 +0,0 @@
# PicoBot 记忆系统设计与迁移方案
编写日期：2026-06-17
> 历史说明：本文的 `ContextCompressor`、Timeline 回填和时间戳边界描述是 1.19.0 之前的基线。当前上下文恢复由单一活动 checkpoint 和 durable `seq` 边界驱动，Timeline 只是提交后的派生检索记录；以 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md) 和 [ARCHITECTURE.md](ARCHITECTURE.md) 为准。
## 背景
PicoBot 当前已经具备最基础的记忆能力：
- `Knowledge` 记忆：长期事实、偏好、项目知识。
- `Timeline` 记忆：由上下文压缩产生的会话摘要。
- `memory_recall` / `timeline_recall`：模型可主动检索记忆。
- 上下文压缩时会把摘要写入 `Timeline`，并在恢复会话时回填最近摘要。
这套实现已经能工作，但它更像“存储 + 召回”的初版，而不是完整的记忆系统。主要短板是：
- 记忆写入依赖显式工具调用，缺少自动抽取与整理闭环。
- 检索主要是关键词/FTS，缺少排序、衰减和冲突消解。
- `Timeline`、`Knowledge`、运行时上下文之间的边界还不够严格。
- 记忆治理能力不足，缺少过期、失效、来源追踪、置信度等元数据。
本方案目标是把 PicoBot 的记忆从“能记住”升级为“记得准、找得到、能更新、会遗忘、可观测”。
## 现状基线
当前代码中的记忆链路大致如下：
1. `SessionManager` 在处理用户消息时，先召回 `Knowledge` 记忆并拼进运行时上下文。
2. `ContextCompressor` 在上下文过大时压缩消息历史，并把压缩结果写为 `Timeline` 记忆。
3. `timeline_recall` 工具允许模型主动检索历史摘要。
4. `memory_store` / `memory_recall` / `memory_forget` 允许模型手动管理 `Knowledge` 记忆。
现有实现的特点：
- 存储是 SQLite。
- 检索是 FTS5 + LIKE 回退。
- 记忆条目只有 `key/content/category/importance/session_id/timestamps`。
- 配置里已经预留了 `recall_limit`、`timeline_retention_days`、`idle_consolidation_minutes` 等参数，但整体闭环还没有完全落地。
## 设计目标
### 必须达到
1. 自动化
- 从对话中自动抽取稳定事实、偏好、项目约束、关键决定。
- 不依赖模型每次都显式调用 `memory_store`。
2. 可控
- 记忆写入要有明确来源、置信度和类别。
- 支持更新、失效、覆盖、删除。
3. 可检索
- 检索不能只依赖关键词匹配。
- 需要结合相关性、重要性、时效性、会话范围进行排序。
4. 可回填
- 会话恢复时，要能回填“最近摘要 + 相关历史 + 相关知识”，但不能把噪声无限回灌。
5. 可治理
- 需要定期清理过期 timeline。
- 低质量或冲突记忆要可降权、可 supersede、可追溯。
### 暂不做
- 不在第一阶段引入复杂的分布式记忆服务。
- 不强制接入外部向量数据库。
- 不把记忆系统做成一个独立的产品边界；它仍然属于 PicoBot runtime。
## 目标架构
建议把记忆系统拆成四层。
### 1. 运行时上下文层
用途：
- 当前轮的系统提示词。
- 运行时间、会话 ID、临时提醒、技能提示。
- 不应被长期记忆污染。
规则：
- 只属于当前 turn。
- 不入库，或仅作为可追踪的审计记录入库，不参与长期 recall。
### 2. Timeline 层
用途：
- 会话摘要。
- 上下文压缩结果。
- 历史状态回放。
规则：
- 按 session 归属。
- 可被 `timeline_recall` 查询。
- 默认保留有限时间，过期可清理。
- 适合作为“发生过什么”的记录，而不是“世界上长期成立的事实”。
### 3. Knowledge 层
用途：
- 用户稳定偏好。
- 项目事实。
- 长期决策。
- 可复用的经验和约束。
规则：
- 需要来源追踪和更新时间。
- 可以被时间衰减、冲突消解、覆盖和删除。
- 适合被 `memory_recall` 检索并注入 system/runtime context。
### 4. Archive 层
用途：
- 低价值但不该直接丢弃的历史。
- 被 supersede 的旧知识。
- 过期 timeline 的冷存档。
规则：
- 不参与默认 recall。
- 仅在排障、导出、审计或手工恢复时查看。
## 统一数据模型
建议将 `memories` 表从“扁平文本”升级为“可治理条目”。
### 推荐字段
```text
id 唯一 ID
key 语义 key，稳定标识一条知识或摘要
content 正文
category knowledge / timeline / archive / scratch
session_id 归属会话
source_session_id 来源会话
source_message_id 来源消息或 turn 标识
source_type explicit_tool / auto_consolidation / context_compression / manual
importance 重要性 0.0-1.0
confidence 置信度 0.0-1.0
created_at 创建时间
updated_at 更新时间
last_accessed_at 最近召回时间
expires_at 过期时间，可空
superseded_by 被哪条记忆覆盖
status active / superseded / archived / deleted
tags 便于过滤和检索
embedding_ref 未来可选的向量引用
```
### 字段意义
- `importance`：这条记忆值不值得保留。
- `confidence`：这条记忆有多可靠。
- `source_*`：这条记忆从哪里来，方便审计和冲突处理。
- `expires_at`：是否该被自动遗忘。
- `superseded_by`：是否已经被更可信的新版本覆盖。
## 写入策略
### 1. 显式写入
仍保留 `memory_store` 工具。
适用场景：
- 用户明确要求记住。
- 模型确认了稳定事实。
- 人工/外部流程显式提供知识条目。
要求：
- 必须带稳定 `key`。
- 建议附带 `importance` 和 `confidence`。
- 允许覆盖同 key 的旧值，但要保留变更痕迹。
### 2. 自动知识抽取
新增一条 consolidation 流程，从 turn 中抽取结构化记忆：
- `facts`
- `preferences`
- `decisions`
- `constraints`
- `open_loops`
抽取结果应满足：
- 只写“稳定可复用”的信息。
- 不写临时情绪、不写会话噪声、不写大段原文。
- 默认先进入“候选记忆区”，通过规则或 LLM 二次确认后再升格为 active knowledge。
### 3. 会话压缩写入 Timeline
当前 `ContextCompressor` 继续承担“会话摘要”的职责，但建议改成两步：
1. 压缩当前上下文，生成可注入的短摘要。
2. 同时生成结构化 timeline entry，写入 timeline store。
这样 timeline 摘要和给模型看的压缩摘要可以一致，但不必完全相同。
## 读取策略
### 默认读取顺序
每轮 user turn 建议按以下顺序构建上下文：
1. 运行时上下文
2. 当前会话最近消息
3. 当前会话最近 timeline 摘要
4. 与当前 query 相关的 Knowledge 记忆
5. 必要时再补充更旧的 Timeline 召回
### Knowledge 召回排序
建议使用混合评分：
```text
final_score =
relevance_score
+ importance_weight
+ recency_weight
- redundancy_penalty
- superseded_penalty
```
建议排序规则：
- 先按相关性筛选候选。
- 再按重要性和更新时间重排。
- 被 supersede 的条目默认不参与主召回。
- 低置信度条目只在结果不足时补位。
### Timeline 召回策略
Timeline 更适合按 session 和时间窗口召回：
- 恢复会话时优先加载最近几条摘要。
- 只有当模型显式需要回顾历史，或者当前话题明显切换，才主动拉更多 timeline。
- 跨会话回顾时先查同主题摘要，再查同 session 摘要。
## 冲突与失效
### 冲突类型
1. 同 key 冲突
- 新记忆与旧记忆 key 相同。
- 处理方式：upsert，旧版本保留更新历史。
2. 语义冲突
- 内容不同，但描述的是同一事实。
- 处理方式：标记旧条目 superseded，保留新条目为 active。
3. 时效冲突
- 旧事实已经过期。
- 处理方式：按 `expires_at` 或规则自动归档。
### 推荐处理流程
1. 新记忆先入候选队列。
2. 对候选记忆做相似度检查。
3. 如果与已有 active knowledge 冲突：
- 保留新条目。
- 给旧条目标记 `superseded_by`。
4. 如果置信度过低：
- 降权但不立即删除。
5. 如果确定失效：
- 移入 archive 或直接删除。
## 过期与清理
### Timeline 清理
Timeline 默认保留 90 天是合理起点，但建议把它变成真正的后台任务：
- 每日或定时运行。
- 清理 `category = timeline` 且过期的条目。
- 清理前先做统计和日志记录。
### Knowledge 清理
Knowledge 不建议简单按天数删。
更合理的是：
- 低重要度 + 低置信度 + 长时间未访问的记忆，先降权。
- 明确过期的条目进入 archive。
- 被 superseded 的条目保留一段审计窗口后再清理。
### 噪声过滤
禁止写入或默认不回灌的内容：
- 自动压缩摘要的中间副本。
- 运行时元信息。
- 工具回显噪声。
- 模板/框架泄漏。
- 明显的无意义重复片段。
## 与现有代码的映射
建议的模块职责演进如下：
### `src/memory/`
保留高层 API，但增加：
- 条目元数据
- 记忆状态机
- 统一评分接口
- 冲突处理接口
### `src/storage/memory.rs`
负责：
- SQLite CRUD
- FTS/LIKE 检索
- Timeline 清理
- 批量更新 superseded 状态
后续可扩展：
- `last_accessed_at` 更新
- 记忆状态批处理
- 按 session / namespace 的索引优化
### `src/agent/context_compressor.rs`
继续负责上下文压缩，但建议拆成两步：
- 压缩历史
- 产出 timeline 记录
并把“是否生成 timeline / 是否写入 memory”做成明确开关。
### `src/session/session.rs`
负责：
- 选择哪些记忆进入当前 turn
- 会话恢复时注入最近 timeline
- 持久化 session 级别的压缩/归档状态
不应承担记忆抽取和冲突消解的重逻辑。
### `src/tools/memory.rs`
保留工具接口，但建议：
- 增加 `confidence`、`expires_at`、`tags` 等参数。
- `memory_recall` 支持更清晰的过滤条件。
- `memory_forget` 支持软删和 hard delete 两种模式。
## 迁移方案
建议分 5 个阶段推进。
### Phase 0: 文档和协议对齐
目标：
- 先把目标讲清楚，避免边改边跑偏。
产物：
- 本文档。
- 更新 `README` 中的记忆入口。
- 补齐记忆数据模型和流程图。
验收：
- 团队能明确区分 runtime / timeline / knowledge / archive 四层。
### Phase 1: 只扩 schema，不改行为
目标：
- 给后续治理能力留好数据位。
改动建议：
- `memories` 表增加 `confidence`、`source_type`、`source_message_id`、`source_session_id`、`last_accessed_at`、`expires_at`、`superseded_by`、`status`、`tags` 等字段。
- 兼容老数据：老字段缺省时回退到旧逻辑。
- `SessionMeta` 可继续保留现有压缩时间戳。
验收：
- 老数据能正常读写。
- 现有记忆工具和压缩流程不需要改调用方。
### Phase 2: 补自动 consolidation
目标：
- 从“显式写记忆”升级为“自动抽取记忆”。
改动建议：
- 在一次 turn 结束后，异步启动 consolidation。
- 从最近 turn 中抽取：
- timeline summary
- knowledge candidates
- 新增去噪逻辑：
- 不写工具噪声
- 不写运行时上下文
- 不写重复摘要
推荐落点：
- `src/session/session.rs`
- `src/agent/context_compressor.rs`
- `src/memory/`
验收：
- 用户不手动调用 `memory_store` 时，也能逐步积累稳定知识。
- timeline 记录与知识条目不再混写。
### Phase 3: 引入混合召回与排序
目标：
- 让 recall 真正“像记忆”而不是“像全文搜索”。
改动建议：
- 召回增加排序层：
- 相关性
- 重要性
- 时效性
- 状态过滤
- `Knowledge` 与 `Timeline` 使用不同召回策略。
- 恢复会话时优先加载最近 timeline，再按 query 召回知识。
可选增强：
- 后续接入 embedding / rerank。
验收：
- 长会话和多主题对话的召回质量明显提升。
- 旧记忆不会总是压过新记忆。
### Phase 4: 冲突消解、失效与清理
目标：
- 让记忆系统可治理。
改动建议：
- 对知识条目做冲突检测。
- 支持 supersede。
- 对 timeline 做定期清理。
- 对低质量或长期未访问的知识降权或归档。
推荐定时任务：
- timeline retention cleanup
- stale memory decay
- archive compaction
验收：
- 过期内容不会无限膨胀。
- 旧事实能被新事实覆盖。
### Phase 5: 兼容层收口
目标：
- 把临时兼容逻辑收敛成稳定接口。
改动建议：
- 清理历史遗留的“摘要直接当知识”路径。
- 统一只从 memory service 读取记忆，不再绕过治理层直接查表。
- 为 debug/export 保留只读视图，但不参与默认注入。
验收：
- 记忆系统对外只有稳定接口。
- 内部实现可继续演进，而不影响 Session / Agent 调用方。
## 推荐实现顺序
如果只做一轮最有性价比的改造，我建议按这个顺序：
1. `schema + metadata`
2. `自动 consolidation`
3. `混合召回与重排`
4. `冲突消解与衰减`
5. `定时清理`
这个顺序的好处是：
- 先把可观察和可治理的数据补齐。
- 再补自动写入，避免“写进去但以后无法管理”。
- 最后再优化召回体验。
## 风险与回退
### 风险
- 自动抽取可能把短期上下文误判成长期知识。
- 召回排序不稳时，模型可能看到过多或过少记忆。
- 迁移 schema 时如果没有兼容逻辑，老数据会丢。
### 回退策略
- 保留现有 `memory_store` / `memory_recall` 作为兼容入口。
- 所有新字段都应可空。
- consolidation 可以通过配置关闭。
- 召回排序可退回 FTS5 + importance 的简化模式。
## 验收标准
记忆系统完成迁移后，应满足：
1. 用户不手动存记忆时，系统仍能逐步积累稳定知识。
2. `Knowledge` 的 recall 结果更准，噪声更少。
3. `Timeline` 不会无限膨胀，且能按 session 回放。
4. 旧事实可被新事实覆盖，冲突状态可追踪。
5. `SessionManager` 不再承担记忆抽取的核心业务逻辑。
6. 任意记忆条目都能回答三个问题：
- 它从哪里来？
- 它为什么还活着？
- 它什么时候该被忘掉？
## 对 PicoBot 当前实现的落地建议
最直接的落点是：
- 先在 `src/storage/memory.rs` 和 `src/memory/types.rs` 扩字段。
- 再在 `src/agent/context_compressor.rs` 增加结构化摘要输出。
- 然后在 `src/session/session.rs` 增加 consolidation hook。
- 最后把 `src/tools/memory.rs` 升级为带治理字段的工具接口。
如果只做最小闭环，至少要先实现：
- `Timeline` 的定时清理
- `Knowledge` 的自动抽取
- `Knowledge` 的冲突消解
- 召回结果的时间衰减和重排
这样 PicoBot 的记忆系统就会从“可用”变成“可信、可演进”。

363
docs/MESSAGE_FLOW_REFACTOR_DESIGN.md
View File

@ -1,363 +0,0 @@
# 用户消息到 LLM 回复链路重构设计
> 状态：实施中（2026-07）。
>
> 本文定义用户消息入口、Session 执行、Turn 提交和客户端校准链路的重构方案。运行时总览见 `docs/ARCHITECTURE.md`，流式状态模型见 `docs/STREAMING_TURN_DESIGN.md`；代码和测试始终是最终事实来源。
## 1. 背景
现有链路已经具备 Channel/Provider 隔离、每 Session 串行、Turn latest-wins 快照和持久化后才发布 `Completed` 等正确基础，但演进过程中留下了以下问题：
1. `TurnDeliveryService::start` 只报告 sink task 已启动，Session 丢弃 task 的最终结果；sink 终态失败后可能没有普通消息兜底。
2. Gateway 用一个循环同步等待所有 inbound 和 control 操作，慢命令或数据库查询会阻塞无关会话。
3. Session worker 同时负责上下文准备、Agent 执行、overflow 恢复、提交、投递降级、标题生成和清理。
4. 普通工具通知与结构化 `TurnEvent::ToolStarted/ToolFinished` 重复。
5. `InboundMessage` 声明了 sender、接收时间和 metadata，但进入 Session 后部分字段被丢弃；平台字段依赖字符串约定透传。
6. TUI/WebUI 每次收到终态都重新请求最多 1000 条历史，重复传输刚刚已经通过 TurnSnapshot 下发的结果。
7. 自动标题生成在 Turn 完成后仍占用 Session worker，阻塞下一条排队消息。
## 2. 设计目标
1. sink 终态失败必须可观测，并至多触发一次普通消息兜底。
2. 一个会话的慢 control/command 不得阻塞其他会话的输入和 `/stop`。
3. Session worker 只负责队列和生命周期编排，慢步骤由职责明确的 helper/service 承担。
4. 上下文首次准备与 overflow 恢复复用同一构建路径。
5. 工具进度只有一个权威来源：TurnEvent。
6. 用户消息的发送者和接收时间要么被持久化，要么从公共数据契约中删除，不能静默丢失。
7. 平台私有上下文以不透明值传递，核心层不解释平台 key。
8. 终态提交向客户端提供历史增量；全量历史只用于初次加载、重连和 revision 缺口恢复。
9. 标题生成不属于 Turn 完成关键路径，并且迟到结果必须条件提交。
10. 所有新增等待、队列、重试和后台任务都必须受 `TaskSupervisor` 管理并有硬边界。
## 3. 非目标
- 不合并 `TurnSnapshot` 与持久化 `ChatMessage`；两者分别是暂态展示和耐久事实。
- 不把 token delta 放入 MessageBus。
- 不取消每 Session 串行语义。
- 不让 Channel、Provider 或客户端直接访问 Session 内部状态。
- 不在本次重构中改变 SQLite schema 版本；新增消息来源信息复用现有 `source` JSON。
- 不保证运行中 Turn 在 Gateway 重启后恢复逐帧状态。
## 4. 目标数据流
```text
Channel
│ normalize + authorize
▼
InboundEnvelope
│
▼
IngressRouter ───────────────► scoped command task
│ │
▼ ▼
per-session AgentTask queue Session command API
│
▼
ConversationExecutor
├── persist user message
├── TurnInputBuilder.prepare/recover
├── TurnRunner (AgentLoop + cancel)
├── TurnCommitter (generation check + atomic persistence)
├── DeliveryHandle.await_terminal/fallback
└── schedule TitleService
│
├── TurnSnapshot ─► TurnSink
└── TurnCommitted ─► client history delta
```
## 5. 可观测的 Turn 投递
### 5.1 接口
`TurnDeliveryService::start` 返回一个必须消费的 handle：
```rust
pub struct TurnDeliveryHandle {
completion: oneshot::Receiver<Result<(), DeliveryError>>,
}
impl TurnDeliveryHandle {
pub async fn wait(self) -> Result<(), DeliveryError>;
}
```
启动失败与异步失败语义分开：
- `start(...) -> Err`：Channel 不存在、`open_turn` 失败或 supervisor 已停止，Session 从一开始使用普通终态投递。
- `start(...) -> Ok(handle)`：sink 生命周期已启动，但不代表终态已经到达外部平台。
- `handle.wait() -> Err`：终态重试耗尽或 shutdown abort 失败，Session 执行普通消息兜底。
### 5.2 兜底规则
1. Agent 结果必须先持久化，之后 Turn 才能 `Completed`。
2. Session 发布终态后等待 delivery handle；等待本身由 coordinator 的 sink timeout/retry 限制。
3. delivery 成功：不发送普通消息。
4. delivery 失败：通过 `MessageBus::deliver_outbound` 发送最终正文，并记录明确错误。
5. `Cancelled`/`Failed` 终态不重复发送正文；只有存在可展示 partial 且 sink 失败时才发送 partial/failure 摘要。
6. Channel sink 内部可以做平台特定编辑降级，但不得把“未找到目标/未发送”报告为成功。
### 5.3 测试
- open 失败时发送一次普通终态。
- open 成功、finish 永久失败时发送一次普通终态。
- finish 瞬态失败后成功时不发送普通终态。
- 持久化失败时不得把失败前正文作为 completed fallback 发送。
- shutdown/cancel 不造成双重终态。
## 6. Gateway 入口并发
### 6.1 现状问题
单个 `message-processor` 在 `tokio::select!` 分支内等待 `handle_message` 和 control I/O。`/compact` 的 LLM 调用、大历史查询或慢 SQLite 操作会造成跨 Session 队头阻塞。
### 6.2 目标
拆成两个只负责消费和派发的 supervisor task：
- `inbound-router`：消费 `InboundMessage`，为每条输入启动受监督的短派发任务；普通消息最终进入 Session 队列。
- `control-router`：消费 `ControlMessage`，为每个请求启动受监督任务并通过一次性回复通道返回。
Session 内部继续用 mutex、`persistence_lock`、`worker_generation` 和 `state_version` 保证同一对话的一致性。Gateway 不再通过全局串行获得隐式正确性。
### 6.3 有界性
- MessageBus 仍是全局 admission queue。
- 普通 AgentTask 仍受每 Session 容量 32 限制。
- router 通过 `TaskSupervisor::spawn` 管理请求任务；spawn 失败必须向调用者/Channel 返回错误。
- control reply 使用 `oneshot`，每个请求只有一个结果。
- `/stop` 直接修改目标 Session cancellation/generation，不进入 AgentTask 队列。
### 6.4 测试
- Session A 的慢 command 不阻塞 Session B 的普通输入。
- Session A 的慢 history query 不阻塞 Session B `/stop`。
- router shutdown 后新请求收到明确失败。
- 同一 Session 的 AgentTask 顺序保持不变。
## 7. Session 执行拆分
### 7.1 `TurnInputBuilder`
输入：稳定的 `SessionTurnSnapshot`、用户输入、skills、MemoryManager、WorkManager。
输出：
```rust
struct PreparedTurnInput {
messages: Vec<ChatMessage>,
base_state_version: u64,
compression_update: Option<CompressionUpdate>,
}
```
职责：
- 并发读取 Knowledge memory 和 active plan；
- 运行 ContextCompressor；
- 统一插入 system prompt；
- 统一向最后一条用户消息追加 runtime context；
- 返回需要条件提交的 compression metadata，不直接持有 Session 锁做慢 I/O。
`recover_after_overflow` 复用同一 assembly 函数，只替换 context window 和压缩结果，不能复制 prompt/runtime context 拼装逻辑。
### 7.2 `TurnRunner`
职责：
- 创建 `TurnController`、`AgentTurnContext` 和 delivery handle；
- 在 `AgentLoop` 与 cancel receiver 之间 select；
- 最多执行一次 context-overflow recovery；
- 返回类型化 `TurnRunOutcome`，不直接构造 OutboundMessage。
```rust
enum TurnRunOutcome {
Completed(AgentProcessResult),
Cancelled(Option<ChatMessage>),
Failed { error: AgentError, partial: Option<ChatMessage> },
Stale,
}
```
### 7.3 `TurnCommitter`
职责：
- 提交前验证 generation/state version；
- 原子持久化 `emitted_messages`；
- 成功后发布 `Completed`；
- 失败/取消时按 partial 规则持久化并发布对应终态；
- 产生 `CommittedTurnDelta`。
### 7.4 Session worker 保留职责
- 从队列接收 AgentTask；
- 持久化原始用户消息；
- 捕获稳定快照；
- 顺序调用 builder/runner/committer；
- 清理 active turn 和 cancel handle；
- 调度非关键后台工作。
## 8. 上下文策略收口
上下文策略分两级，但 owner 明确：
- `TurnInputBuilder`：跨轮历史压缩、Timeline/Memory/Plan、overflow recovery。
- `AgentLoop`：单次工具循环中临时裁剪过大的旧 tool result，不修改 Session 历史。
二者不能重复构建 system/runtime prompt。`AgentLoop` 的“缺 system 时自动注入”仅保留给明确的 stateless API；交互 Session 调用使用要求首条必须为 system 的入口或 debug assertion。
Memory recall 和 active plan 查询互不依赖，应使用 `tokio::join!` 并发执行。任何结果提交前都验证 `base_state_version`。
## 9. 工具进度唯一来源
交互 Turn 删除 `AgentLoop.notify_tx: UnboundedSender<String>` 和每消息 notification publisher。工具进度仅由：
```text
ToolStarted → TurnController → TurnSnapshot
ToolFinished → TurnController → TurnSnapshot
```
后台子 Agent 的 `TaskNotification` 是另一种领域事件，继续保留，因为它表达跨 Turn 的任务完成，而不是当前 Turn 的工具进度。
## 10. Inbound 与 ChannelContext
### 10.1 规范化输入
```rust
struct InboundMessage {
channel: String,
chat_id: String,
sender_id: String,
content: String,
received_at: i64,
media: Vec<MediaItem>,
channel_context: ChannelContext,
}
```
`metadata` 与 `forwarded_metadata` 合并为语义明确的不透明 `ChannelContext`。核心只允许：
- 原样传给本轮 `TurnTarget` 或普通错误回复；
- 从通用 typed 字段读取 `reply_to`；
- 不解析 `feishu.*` 等平台 key。
平台 message/reaction ID 最终由具体 sink 持有。`feishu.parent_id` 要么映射为 typed `reply_to`，要么删除，不能继续作为无消费者字段。
### 10.2 持久化用户来源
用户 `ChatMessage` 使用原始 `received_at`，并设置：
```rust
MessageSource {
kind: UserInput,
from_channel: Some(channel),
from_user_id: Some(sender_id),
...
}
```
客户端历史投影可以隐藏内部 sender ID；LLM 上下文是否展示发言者由独立策略决定，不能直接泄露平台标识。
## 11. 终态历史增量
### 11.1 协议
持久化成功后发布：
```rust
WsOutbound::TurnCommitted {
session_id: String,
history_revision: u64,
messages: Vec<HistoryMessage>,
}
```
规则：
- `TurnUpdated(Completed)` 仍负责结束 active turn 展示。
- `TurnCommitted` 只包含本轮新持久化消息，负责把 transcript 校准到数据库事实。
- Session 维护单调 `history_revision`；客户端只接受连续 revision。
- revision 缺口、重连或显式切换 Session 时才请求全量历史。
- 全量 `SessionHistory` 返回当前 revision。
第一阶段可使用最终 assistant/tool message IDs 去重；如果不修改数据库 schema，revision 使用 Session 内 `state_version`/最新 message sequence 投影，重启后从 Storage 最大 sequence 恢复。
### 11.2 测试
- 正常 Turn 完成不触发全量 history 请求。
- 增量包含 assistant tool call、tool result 和最终 assistant。
- 重复增量按 message ID 幂等。
- revision 缺口触发一次全量校准。
- 其他 Session 的增量只更新对应缓存/未读状态。
## 12. 标题后台化
Turn 提交后，Session worker 调用 `TitleService::schedule` 并立即处理下一条任务。
后台任务：
1. 在锁内捕获 title prompt、session ID 和 `state_version`。
2. 在锁外调用 Provider。
3. 获取 `persistence_lock`。
4. 只有标题仍为默认值且 generation/state 条件允许时提交。
5. 任务由 `TaskSupervisor` 管理；shutdown 时取消并限时回收。
标题失败只记录 warning，不改变 Turn 状态，不向用户发送错误消息。
## 13. 错误模型
Session worker 不再散落构造英文字符串 OutboundMessage，而是返回类型化错误：
```rust
enum TurnFailureKind {
InputPersistence,
AgentCreation,
ContextPreparation,
Provider,
TurnPersistence,
Delivery,
}
```
统一 `TurnFailurePresenter` 根据 Channel/PresentationPolicy 生成用户可见内容。原始 provider/storage 错误只进入安全日志和 Turn internal error，不直接暴露 secrets。
Gateway 的 `handle_message` 错误不能只写日志；必须通过输入携带的 ChannelContext 返回一个关联到原消息的错误结果。
## 14. 迁移与提交顺序
1. 文档：落地本设计和架构链接。
2. Delivery：返回 completion handle，Session 消费结果并做一次兜底。
3. Gateway：拆分 inbound/control router，消除全局慢操作串行。
4. Session：提取输入构建和 overflow recovery，再提取 run/commit helper。
5. Progress/title：删除旧工具通知，标题移入 supervisor。
6. Contract：规范 InboundMessage、用户来源和 ChannelContext。
7. Protocol：增加 TurnCommitted/history revision，客户端改为增量校准。
8. 最终清理：删除不可达 `HandleResult::AgentResponse` 交互分支和重复 helper。
每一步都保持可编译、可测试、可单独回滚，不允许一个提交同时更改全部并发和协议语义。
## 15. 回归测试矩阵
| 风险 | 必需测试 |
|---|---|
| sink 异步终态失败 | fallback 恰好一次，成功时零次 |
| Gateway 队头阻塞 | 慢 A 不阻塞 B 输入/control |
| stale worker | generation/state 改变后不得提交 |
| overflow recovery | prompt/runtime context 只附加一次 |
| duplicate tool progress | 交互 Turn 不产生普通工具通知 |
| inbound fidelity | sender、received_at、reply_to 正确保留 |
| title race | 用户重命名后迟到标题不得覆盖 |
| history delta | 连续、重复、缺口、跨 Session |
| shutdown | router、delivery、title task 均被监督和有界回收 |
## 16. 完成条件
- `cargo test --lib`
- `cargo test --test test_scheduler --test test_request_format`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo build`
- `webui/npm run check`
- `webui/npm run build`
- 新增的失败、取消、并发、revision 和 fallback 测试全部通过。
- `docs/ARCHITECTURE.md`、AGENTS.md 中的运行时不变量与实现一致。

1142
docs/SCHEDULED_RUN_DESIGN.md
View File

File diff suppressed because it is too large Load Diff

789
docs/STREAMING_TURN_DESIGN.md
View File

@ -1,789 +0,0 @@
# 流式 Turn、Reasoning 展示与 Channel 投递设计
> 状态：已实现（2026-07）。
>
> 本文记录 PicoBot 流式模型输出、reasoning 展示、工具过程展示和 Channel 实时投递的设计依据与架构决策。当前运行时总览见 `docs/ARCHITECTURE.md`，具体行为以代码和测试为准。
## 1. 背景
改造前 Provider 只提供一次性 `chat()` 调用。AgentLoop 等待完整响应，将 `content`、`reasoning_content` 和 tool calls 组装为 `ChatMessage`，Session 在 AgentLoop 完成后原子持久化本轮消息，再通过 `OutboundMessage` 发送最终正文。
这个模型具有清晰的持久化语义，但无法表达：
- 正文和 reasoning 的实时增量；
- reasoning、正文、工具调用在一个 Turn 中的自然交错；
- TUI 和 WebUI 对同一个运行中 Turn 的一致展示；
- 飞书等 Channel 通过编辑消息呈现流式效果；
- 不支持编辑的 Channel 自动降级为只发送最终结果；
- 取消、失败、慢消费者和投递失败时的确定行为。
旧 `Channel::send_delta(chat_id, delta)` 没有 Turn 身份、消息身份、reasoning/text 分类、工具边界、终态和取消语义，也绕过出站排序机制，因此已被 `TurnSink` 取代。
## 2. 参考实现结论
本设计综合了 `reference/ryvos`、`reference/zeroclaw`、`reference/hermes-agent` 和 PicoBot 当前实现。
### 2.1 Ryvos
Ryvos 把 thinking 作为正式内容块，并区分 `TextDelta` 与 `ThinkingDelta`。它说明 Provider 层必须结构化解析 reasoning、正文和工具调用，不能依赖最终字符串中的 `<think>` 后处理。
可吸收：
- Provider 流的类型化增量；
- reasoning 与正文分离；
- 工具参数增量组装；
- thinking-only 响应的明确处理；
- reasoning effort/thinking budget 的统一配置概念。
### 2.2 ZeroClaw
ZeroClaw 把 reasoning 作为不透明 Provider 数据保留，用于要求历史回放的模型，同时处理 `reasoning_content`、`reasoning`、内联 `<think>` 和不同 Provider 的回放限制。
可吸收：
- 可展示 reasoning 与 Provider 回放状态分离；
- reasoning 字段别名归一化；
- Provider 专用历史状态不能跨 Provider 发送；
- 内联 think block 必须在 Provider 归一化边界处理；
- reasoning 是否展示与是否回放是两个独立策略。
### 2.3 Hermes
Hermes 新增了 Agent 到 Gateway 的结构化展示事件，并明确规定流事件属于 presentation，而不是 conversation history。它还通过 message segment boundary 处理“工具前正文 → 工具 → 工具后正文”。
可吸收：
- 流事件描述发生的事实，不携带平台发送策略；
- 展示流与持久化历史严格分离；
- 工具边界必须结束当前正文 segment；
- 高频更新需要合并，关键事件发送前需要 flush；
- TUI 的活动 Turn 与已完成 transcript 分离；
- Channel/平台决定如何呈现统一状态。
不直接复制：
- typed events、旧 callbacks 和 TUI 字符串事件并存；
- reasoning 使用独立 callback，没有进入新事件模型；
- 客户端用大型 TurnController 重建服务端状态；
- 一个 GatewayStreamConsumer 同时承担聚合、限流、平台编辑、think 清理、overflow 和 fallback。
### 2.4 PicoBot
PicoBot 已有以下适合保留的不变量：
- 同一 Session 由单 worker 串行处理；
- 不同 Session 并发；
- `worker_generation` 和 `state_version` 防止迟到结果提交；
- 完整 Agent Turn 通过原子持久化接口提交；
- 普通出站消息按 `(channel, chat_id)` 有序投递；
- Channel、Session、Agent、Provider 和 Storage 边界明确。
流式设计不能破坏这些不变量。
## 3. 设计目标
1. OpenAI-compatible 和 Anthropic Provider 支持流式正文、reasoning、tool calls 和 usage。
2. TUI 与 WebUI 使用同一运行态模型显示正文、reasoning、工具进度和取消/失败状态。
3. Channel 可以选择实时更新或只发送最终结果。
4. 支持消息编辑的 Channel 能以同一远端消息呈现流式效果。
5. 慢客户端或慢 Channel 不得反压 Provider 和 AgentLoop，也不得积压大量过时 token。
6. 丢失任意中间更新后，下一次更新必须自动收敛到正确状态。
7. `Completed` 必须表示本轮数据库提交已经成功。
8. reasoning 展示文本与 Provider 回放状态必须隔离。
9. Channel 展示差异不能改变模型上下文和数据库历史。
10. 保持模块数量、事件词汇和状态 owner 尽可能少。
## 4. 非目标
- 不保留旧 WebSocket、TUI、WebUI 或 Channel 流式协议兼容性。
- 不要求每个 Provider 都能返回可展示 reasoning。
- 不把 Provider 的加密/签名 reasoning payload 展示给用户。
- 不逐 token 持久化数据库。
- 不保证重连后恢复尚未完成 Turn 的每一个历史帧。
- 不让所有外部 Channel 默认以多条追加消息模拟流式效果。
- 不把流式展示事件作为可重放的事件溯源日志。
## 5. 核心决策
### 5.1 只有两个权威模型
系统只维护两个跨层权威模型：
- `ConversationMessage`：最终持久化事实，用于会话历史和下轮模型上下文；
- `TurnState`：单次运行的临时展示状态。
Provider 的 SSE chunk 是 Provider 内部输入；Channel 的远端消息 ID 是单个 TurnSink 的私有投递状态。两者都不是全局领域模型。
### 5.2 服务端拥有唯一运行态
Session 侧 `TurnController` 是运行中 Turn 的唯一状态 owner。TUI、WebUI 和 Channel 不根据一串增量自行重建 reasoning、正文、工具和 segment 关系，只渲染服务端发布的 `TurnSnapshot`。
### 5.3 下发幂等快照，不下发可靠 token 流
Provider 到 AgentLoop 使用 delta；TurnController 到展示端使用包含完整当前状态的快照。
每个快照带单调递增 `revision`。消费者只接受 revision 更大的快照，并以新快照整体替换旧状态。因此：
- 丢失中间更新不会损坏内容；
- 慢消费者可以跳过过时状态；
- 更新重试不会重复拼接正文；
- 最终快照能修复暂态渲染；
- 外部 Channel 编辑消息天然获得完整累计内容。
### 5.4 latest-wins，而不是 token 队列
每个活动 Turn 使用 Tokio `watch` 或等价 latest-value primitive 发布 `Arc<TurnSnapshot>`。生产者覆盖旧值，消费者读取最新值。终态显式编码在快照中，不能仅依赖 sender 关闭表达完成。
### 5.5 每个 Channel Turn 使用独立 TurnSink
Channel 为每次 Turn 创建一个 sink。Sink 独占远端消息 ID、编辑状态和平台私有资源，终态后销毁。通用协调器不保存平台消息映射，也不理解飞书卡片 API。
### 5.6 完整消息投递与活动 Turn 展示分离
- `MessageBus` / `OutboundDispatcher`：完整消息、通知、命令结果和需要可靠确认的独立投递；
- `DeliveryCoordinator`：活动 Turn 快照、展示策略、节流和 TurnSink 生命周期。
不把 token 或快照塞入现有 outbound MPSC。
## 6. 数据模型
### 6.1 Provider 私有流
```rust
pub enum ProviderChunk {
Text(String),
Reasoning(String),
ToolCallStart {
index: usize,
id: Option<String>,
name: Option<String>,
},
ToolCallArguments {
index: usize,
delta: String,
},
ProviderState(ProviderReasoningState),
Usage(Usage),
Done(FinishReason),
}
```
`ProviderChunk` 只允许在 `providers` 与 `agent` 模块间使用，不进入 Bus、Session 协议或 Channel。
### 6.2 可展示 reasoning 与回放状态
```rust
pub struct ProviderReasoningState {
pub provider: String,
pub payload: serde_json::Value,
}
pub struct AssistantMessageData {
pub content: String,
pub reasoning: Option<String>,
pub provider_state: Option<ProviderReasoningState>,
pub tool_calls: Vec<ToolCall>,
}
```
约束：
- `reasoning` 可以按展示策略下发；
- `provider_state` 永远不下发给客户端或 Channel；
- `provider_state.provider` 与当前 Provider 不一致时禁止回放；
- 压缩历史时默认不把原始 reasoning 写入 Timeline；
- 日志不得记录完整 reasoning 或 provider payload。
### 6.3 Turn 标识
```rust
pub struct TurnId(pub uuid::Uuid);
pub struct BlockId(pub uuid::Uuid);
```
一次用户输入对应一个 Turn。Turn 开始时预分配最终 assistant `message_id`，使运行态和最终历史能稳定关联。
### 6.4 TurnState
```rust
pub struct TurnState {
pub id: TurnId,
pub session_id: String,
pub message_id: String,
pub revision: u64,
pub status: TurnStatus,
pub phase: TurnPhase,
pub blocks: Vec<TurnBlock>,
pub usage: Option<Usage>,
pub error: Option<String>,
}
pub enum TurnStatus {
Running,
Completed,
Cancelled,
Failed,
}
pub enum TurnPhase {
Queued,
Reasoning,
Responding,
Acting,
Finalizing,
}
```
`TurnPhase` 是展示状态，不等于模型 reasoning：
- 等待首个模型 chunk 时可显示 `Queued`；
- 收到 reasoning delta 时进入 `Reasoning`；
- 收到正文 delta 时进入 `Responding`；
- 执行工具时进入 `Acting`；
- 模型结束、等待持久化时进入 `Finalizing`。
### 6.5 有序 TurnBlock
```rust
pub enum TurnBlock {
Reasoning {
id: BlockId,
iteration: u32,
text: String,
},
Assistant {
id: BlockId,
iteration: u32,
text: String,
},
Tool {
id: String,
iteration: u32,
name: String,
arguments: serde_json::Value,
status: ToolStatus,
preview: Option<String>,
},
}
pub enum ToolStatus {
Running,
Completed,
Failed,
}
```
有序 block 直接表达 reasoning、正文和工具的交错，不再维护多个平行字符串或让客户端猜测工具边界。
### 6.6 Agent 语义事件
```rust
pub enum TurnEvent {
ReasoningDelta {
iteration: u32,
delta: String,
},
TextDelta {
iteration: u32,
delta: String,
},
TextSegmentFinished {
iteration: u32,
},
ToolStarted {
iteration: u32,
call: ToolCall,
},
ToolFinished {
iteration: u32,
call_id: String,
success: bool,
preview: Option<String>,
},
}
```
AgentLoop 只发过程事实。Turn 的 start、finalize、complete、cancel 和 fail 由 Session worker 调用 TurnController，因为 Session 才拥有生命周期、持久化和 stale-state 判断。
## 7. Provider 层
### 7.1 流式优先接口
```rust
#[async_trait]
pub trait LLMProvider: Send + Sync {
async fn stream(
&self,
request: ChatCompletionRequest,
) -> Result<ProviderStream, ProviderError>;
}
```
标题生成、压缩等需要完整响应的代码通过 `collect_provider_stream()` 收集同一实现，避免分别维护 stream 和 non-stream HTTP 路径。
### 7.2 OpenAI-compatible
至少处理：
- `delta.content`；
- `delta.reasoning_content`；
- `delta.reasoning`；
- 同一 payload 同时出现 content 与 reasoning；
- tool call id/name/arguments 分片；
- usage-only final chunk；
- reasoning-only 响应；
- 内联 think tag 被任意 SSE chunk 切分。
`<think>`、`<reasoning>` 等内联标签使用有状态 parser 在 Provider 归一化边界转换为 `ProviderChunk::Reasoning`，不能在 Channel 或客户端重复清理。
### 7.3 Anthropic
至少处理：
- text content block；
- thinking/redacted thinking block；
- signature 或其他回放元数据；
- tool_use block 和 input JSON delta；
- content block start/delta/stop；
- message usage 和 stop reason。
thinking 文本进入 `reasoning`，签名和原始 block 进入 `provider_state`。历史回放必须保持 Provider 要求的块顺序和签名完整性。
### 7.4 reasoning-only
Provider 不擅自把 reasoning 提升为正文。最终轮只有 reasoning、没有正文且没有工具调用时，AgentLoop 保存空正文 assistant 消息及其 reasoning，Turn 正常进入 Completed；交互 UI 仍可显示 reasoning，但不会把它冒充最终答案。
## 8. AgentLoop 与 TurnController
### 8.1 AgentLoop
AgentLoop 消费 `ProviderChunk` 并：
- 累积本轮完整 AssistantMessageData；
- 将展示事实发给 `TurnEmitter`；
- 组装 tool calls；
- Provider 完成当前迭代后执行工具；
- 在工具开始前发出 `TextSegmentFinished`；
- 将完整 assistant/tool messages 加入内部历史；
- 返回最终 `AgentProcessResult`。
AgentLoop 不访问 MessageBus、DeliveryCoordinator 或 Channel。
### 8.2 TurnController
TurnController：
- 是 TurnState 的唯一写入者；
- 把 TurnEvent reduce 为有序 blocks；
- 维护 revision、status 和 phase；
- 发布最新 TurnSnapshot；
- 不执行 Provider、工具、数据库或 Channel I/O。
推荐接口：
```rust
impl TurnController {
pub fn start(... ) -> (Self, TurnEmitter, watch::Receiver<Arc<TurnSnapshot>>);
pub fn begin_finalizing(&mut self);
pub fn complete(&mut self, usage: Option<Usage>);
pub fn cancel(&mut self, reason: Option<String>);
pub fn fail(&mut self, error: String);
}
```
`TurnEmitter` 应轻量、无 Channel 感知，并在 Turn 终止后拒绝新事件。
### 8.3 stale-state
Session worker 在以下位置校验 `worker_generation` 和必要的 `state_version`：
- 创建 Turn 后、调用 Provider 前；
- 发布会产生用户可见变化的快照前；
- 工具批次完成后；
- 最终数据库提交前；
- 发布 Completed 前。
旧 generation 的 Turn 必须进入 Cancelled 或静默终止，禁止继续编辑 Channel 远端消息。
## 9. DeliveryCoordinator
DeliveryCoordinator 是活动 Turn 的唯一展示协调器，职责包括：
1. 订阅 `watch::Receiver<TurnSnapshot>`；
2. 解析当前目标的 PresentationPolicy；
3. 在下发前移除隐藏的 reasoning/tool blocks；
4. 根据 Channel LivePolicy 节流；
5. 为 Turn 创建并持有 TurnSink；
6. 对 Running 快照进行 best-effort 更新；
7. 对终态快照立即 flush；
8. 有界等待 sink 结束；
9. 报告最终投递结果。
### 9.1 合并和背压
- `watch` 自动覆盖过时快照；
- WebSocket/cli_chat 建议最多约 30 FPS；
- 飞书建议从 500ms 更新间隔开始；
- 正在滚动或渲染压力高时，客户端无需向服务端反馈节流，丢弃中间快照即可；
- 终态不等待节流 timer，必须立即发送；
- Running 更新失败不重试旧快照，等待下一最新快照；
- Completed 最终投递使用正常可靠重试语义。
### 9.2 与 OutboundDispatcher 的关系
DeliveryCoordinator 不取代 OutboundDispatcher。
| 组件 | 负责 |
|------|------|
| OutboundDispatcher | 完整独立消息、通知、命令结果、定时投递、可靠重试 |
| DeliveryCoordinator | 活动 Turn 的运行快照、节流、展示过滤和 sink 生命周期 |
两者对同一 `(channel, chat_id)` 的最终写操作必须有统一排序边界。实现时可以复用 per-conversation lane owner，但不能把所有快照排进 lane 的普通 MPSC；lane 应只持有 TurnSink 任务或 latest snapshot receiver。
## 10. Channel 与 TurnSink
### 10.1 接口
删除 `Channel::send_delta`，保留普通 `send`，增加：
```rust
pub enum LivePolicy {
FinalOnly,
Snapshot {
min_interval: Duration,
},
}
#[async_trait]
pub trait Channel: Send + Sync + 'static {
fn live_policy(&self) -> LivePolicy;
async fn open_turn(
&self,
target: TurnTarget,
) -> Result<Box<dyn TurnSink>, ChannelError>;
async fn send(&self, msg: OutboundMessage) -> Result<(), ChannelError>;
}
#[async_trait]
pub trait TurnSink: Send {
async fn update(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError>;
async fn finish(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError>;
async fn abort(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError>;
}
```
`TurnSink` 的每个调用都接收完整、过滤后的快照。Sink 不拼接 token。终态方法保留 `&mut self`，使协调器可以在瞬态错误或超时后重试同一个、仍持有远端消息 ID 的 sink；终态成功或重试耗尽后由协调器销毁 sink。
### 10.2 cli_chat
- `LivePolicy::Snapshot`，默认约 33ms；
- update/finish/abort 都发送统一 `turn_updated` frame；
- WebUI 与 TUI 使用完全相同的 TurnSnapshot；
- Session 不匹配时客户端忽略渲染，但可标记未读；
- 终态后客户端可请求历史作最终校准。
### 10.3 飞书
- 配置关闭实时展示时使用 FinalOnly sink；
- 开启时使用 Snapshot sink；
- 第一个有可见内容的 Running 快照创建卡片；
- 后续快照编辑同一卡片；
- sink 内持有远端 message ID；
- reasoning/tool block 由 PresentationPolicy 决定是否进入卡片；
- 中间编辑失败不影响 Agent；
- finish 做最终编辑，必要时退化为发送一条完整最终消息；
- 卡片长度限制、拆分和平台限流属于 FeishuTurnSink 私有实现。
### 10.4 不支持编辑的 Channel
实现 FinalOnlyTurnSink：忽略 Running 快照，只在 finish 时发送最终投影。默认不通过多条追加消息模拟流式，以免产生无法收回的碎片消息。
## 11. 展示策略
```rust
pub struct PresentationPolicy {
pub live: bool,
pub reasoning: ReasoningVisibility,
pub tools: ToolVisibility,
}
pub enum ReasoningVisibility {
Hidden,
Collapsed,
Expanded,
}
pub enum ToolVisibility {
Hidden,
Compact,
Detailed,
}
```
建议默认值：
- TUI/WebUI：live=true，reasoning=Collapsed，tools=Detailed；
- 外部 Channel：live 由 Channel 配置决定，reasoning=Hidden，tools=Compact；
- Scheduler/无人值守投递：FinalOnly，reasoning=Hidden。
策略由 DeliveryCoordinator 在数据离开 Gateway 核心前应用。Channel 和客户端不能只靠“隐藏 UI”实现 reasoning 保密。
## 12. TUI 与 WebUI
客户端状态简化为：
```text
history 已持久化消息
active_turn 当前 TurnSnapshot（每个 session 最多一个）
```
收到快照时：
```text
if snapshot.revision > active_turn.revision:
active_turn = snapshot
```
渲染规则：
- Reasoning block 显示为折叠或展开区域；
- Assistant block 显示为正文 segment；
- Tool block 显示运行中/成功/失败状态；
- phase 控制 spinner 文案；
- Completed/Cancelled/Failed 显示明确终态；
- 当前 session 之外的快照不进入当前消息列表；
- Completed 后以历史响应替换 active turn，避免展示态与数据库态长期并存。
WebUI 对 Running Markdown 可以按动画帧或快照频率渲染，Completed 时进行最终 sanitize。TUI 原地重绘 active turn，不把每个更新追加为新 transcript 行，也不在用户向上滚动时强制跳到底部。
## 13. WebSocket 协议
Agent 主 Turn 不再通过 `assistant_response` 发送最终正文，活动与终态都使用统一 frame。`assistant_response` 仅保留给不属于活动 Turn 的完整独立消息：
```json
{
"type": "turn_updated",
"snapshot": {
"id": "...",
"session_id": "...",
"message_id": "...",
"revision": 12,
"status": "running",
"phase": "responding",
"blocks": []
}
}
```
Completed、Cancelled 和 Failed 仍使用 `turn_updated`，只改变完整快照的 status。避免为每个生命周期阶段增加一组容易漂移的 frame 类型。
历史协议应返回持久化后的 reasoning、completion_status、turn_id 和 iteration，但不返回 provider_state。
## 14. 持久化和提交顺序
流式过程中不逐 token 写 SQLite。完成顺序固定为：
```text
Provider 完成
→ AgentLoop 组装 emitted_messages
→ Session 校验 generation/state_version
→ 原子写入本轮全部消息
→ TurnController.complete()
→ 发布 Completed 快照
→ TurnSink.finish()
```
因此 `TurnStatus::Completed` 的含义是：数据库已经提交成功，最终展示可以安全收敛到历史。
### 14.1 取消
采用以下语义：
- 没有 Assistant 正文：不持久化 assistant 消息，Turn 标记 Cancelled；
- 已向用户展示部分正文：持久化部分正文并标记 `completion_status=cancelled`；
- 已完成的 assistant/tool/tool-result 链必须保持 Provider 可接受的结构；
- reasoning 可随取消消息保存，但展示仍受 policy 控制；
- 取消后 TurnEmitter 关闭，迟到 delta 被丢弃。
### 14.2 失败
- Provider 在任何可见正文前失败：Turn Failed，错误作为结构化 error 展示，不创建 assistant 历史；
- 已产生部分正文后失败：按 interrupted partial 保存，标记 `completion_status=interrupted`；
- 持久化失败：不得发布 Completed，Turn Failed，并明确告知用户流式预览未保存；
- Running Channel 更新失败不改变 Turn 结果；最终 finish 失败按现有投递错误处理。
## 15. SQLite 迁移
schema v4 为 `messages` 增加：
```text
turn_id TEXT NULL
iteration INTEGER NULL
completion_status TEXT NOT NULL DEFAULT 'completed'
reasoning_content TEXT NULL -- 已存在，语义调整为可展示 reasoning
provider_state TEXT NULL -- JSON，Provider 私有回放状态
```
要求：
- 新库 schema 测试；
- 旧库迁移测试；
- 已有 `reasoning_content` 数据原样保留；
- `provider_state` 解析失败时降级为不回放，不能使历史不可读；
- Session 加载继续修复 tool-call chains；
- 原子提交覆盖完整 Turn 的所有 emitted messages。
当前不新增 `turns` 表。运行中 Turn 只存在内存，历史可通过 messages.turn_id 分组。如果未来要跨 Gateway 重启恢复运行态，再单独设计 durable turn lease/state。
## 16. 生命周期与并发不变量
1. 每个 Session 最多一个活动主 Turn。
2. TurnController 是 TurnState 的唯一写入者。
3. AgentLoop、DeliveryCoordinator 和 TurnSink 不持有 Session mutex 执行慢 I/O。
4. `worker_generation` 变化后，旧 Turn 不得发布新快照或提交消息。
5. Running 快照是 best-effort；终态快照必须显式、完整且有界投递。
6. 慢 sink 只能跳过中间状态，不能阻塞 Provider 或 AgentLoop。
7. Completed 必须晚于数据库成功提交。
8. TurnSink 的生命周期由 DeliveryCoordinator 所有，并通过 TaskSupervisor 回收。
9. Provider stream、节流 timer、Channel 编辑、取消和 shutdown 都必须有硬时间界限。
10. presentation 过滤不能修改 ConversationMessage 或 Agent 历史。
## 17. 模块布局
```text
src/providers/stream.rs
ProviderChunk、ProviderStream、FinishReason、collect helper
src/agent/turn_event.rs
TurnEvent、TurnEmitter
src/session/turn.rs
TurnState、TurnBlock、TurnController、TurnSnapshot
src/delivery/mod.rs
src/delivery/coordinator.rs
src/delivery/policy.rs
watch 订阅、展示过滤、节流、sink 生命周期
src/channels/base.rs
Channel、LivePolicy、TurnSink
src/channels/cli_chat.rs
WebSocketTurnSink
src/channels/feishu.rs
FeishuTurnSink
src/protocol.rs
TurnSnapshot 序列化
```
`observability` 继续记录 Agent/tool 遥测，不承担 UI stream。`MessageBus` 不新增 token/turn 队列。
## 18. 明确拒绝的替代方案
### 18.1 每 token 一个 OutboundMessage
拒绝原因：填满 bounded bus/lane、重试乱序、慢 Channel 反压 Agent、最终消息和暂态更新语义混淆。
### 18.2 端到端 delta 协议
拒绝原因：客户端和每个 Channel 都必须实现累积、去重、segment、取消和丢帧恢复状态机，最终产生多个事实 owner。
### 18.3 客户端自行组合 reasoning/tool/text
拒绝原因：TUI、WebUI 和 Channel 行为会漂移；重连和切 session 时难以恢复；服务端已经拥有全部事实。
### 18.4 把流式事件写入数据库
拒绝原因：消息历史膨胀，事务语义复杂，压缩和 Provider 回放被展示细节污染。
### 18.5 在 Channel 全局保存 turn_id 映射
拒绝原因：owner 和清理边界不清晰。每 Turn 一个 sink 可以让远端消息状态自然随生命周期释放。
### 18.6 一个巨型跨平台 StreamConsumer
拒绝原因：通用合并/策略与平台 API 细节耦合。DeliveryCoordinator 只做统一调度，具体远端编辑由各 TurnSink 自己实现。
## 19. 验证策略
### 19.1 Provider
- SSE 任意字节和 UTF-8 边界切分；
- reasoning/content 同时出现；
- reasoning 与 content 字段别名；
- think tag 跨 chunk；
- 多 tool calls 交错参数 delta；
- usage-only chunk；
- Anthropic thinking signature round-trip；
- 中途断线、超时和取消。
### 19.2 TurnController
- reasoning → text → tool → reasoning → text 的 block 顺序；
- segment boundary；
- parallel tool status；
- revision 严格递增；
- 终态后拒绝新事件；
- reasoning-only、empty response、取消和失败。
建议用 property tests 验证：任意合法 TurnEvent 序列 reduce 后不产生相邻可合并同类 block、重复 tool id 或终态后变更。
### 19.3 DeliveryCoordinator
- 慢 sink 只收到最新快照；
- 终态绕过节流；
- hidden reasoning 在到达 sink 前已经移除；
- Running 更新失败后能由下一快照恢复；
- finish 可靠重试；
- shutdown 有界；
- stale generation 停止更新。
### 19.4 客户端和 Channel
- WebUI/TUI revision 去重；
- session 切换不显示迟到 Turn；
- Completed 后历史校准；
- Markdown 未完成块与最终块；
- 飞书 create/edit/final fallback；
- FinalOnly sink 不发送中间内容；
- 远端长度限制与节流。
### 19.5 Storage
- 新库 schema；
- 旧 schema 迁移；
- provider_state 损坏降级；
- cancelled/interrupted 消息恢复；
- 完整 Turn 原子提交失败不产生部分历史。
## 20. 实施记录
实现按可独立验证的里程碑完成：SQLite 消息语义、Turn 状态机、OpenAI 原生流、Agent/Session 生命周期、DeliveryCoordinator、WebSocket/TUI/WebUI、Anthropic 签名回放、FeishuTurnSink，最后删除过渡适配并同步运行时文档。每个里程碑均保持非流式最终回复可用，且没有为旧增量协议保留双栈。
## 21. 已采用的产品策略
这些选择不改变架构，但决定默认产品行为：
1. `/stop` 后持久化已展示的部分正文并标记 `cancelled`；只有 reasoning 时不创建 assistant 历史。
2. TUI/WebUI 在独立区域展示 reasoning；WebUI 默认折叠，TUI 直接显示。
3. 外部 Channel 默认隐藏 reasoning，工具只显示紧凑状态。
4. reasoning-only 不提升为正文。
5. `provider_state` 随 assistant 消息保留，用于同 Provider 精确回放；不下发客户端或 Channel，损坏时安全忽略。
## 22. 架构验收标准
设计完成实现后，应能用以下陈述准确描述系统：
- Provider 只负责模型协议，AgentLoop 只负责模型/工具语义。
- Session 拥有 Turn 生命周期和最终持久化。
- TurnController 是运行态的唯一事实来源。
- DeliveryCoordinator 只投影展示，不修改历史。
- 每个 Channel Turn 的远端状态只存在于一个 TurnSink。
- 客户端只渲染服务端快照，不重建领域状态。
- 中间快照可以丢，最终状态一定可收敛。
- reasoning 展示、reasoning 回放和“系统正在工作”是三个不同概念。
- Completed 永远意味着数据库已经提交成功。

279
docs/SUB_AGENT_DESIGN.md
View File

@ -1,279 +0,0 @@
# 子 Agent 设计
本文说明 PicoBot 具名子 Agent（named Agent）的运行时设计，重点是**结果如何从子 Agent 传回主 Agent**，以及实现过程中踩过的坑。文中描述以当前代码为准（`src/agent/`、`src/storage/agent_run.rs`、`src/storage/agent_inbox.rs`、`src/tools/delegate.rs` 等）。
## 1. 概述与设计目标
子 Agent 让主 Agent 把一个独立、可验收的子任务交给一个**具名角色**去执行，角色有独立的 Provider/模型、工具集、系统提示词和执行预算。核心目标：
- **一切可审计**：每次委托（run）先落库再执行，终态、结果、工具调用数、迭代数都持久化，WebUI 可查。
- **结果不丢**：后台任务的完成结果即使进程崩溃、收件箱打满、唤醒丢失也能最终送达主 Agent。
- **fail-closed**：定义文件、工具集、委托边、信号契约任一处非法都拒绝加载或拒绝执行，绝不悄悄放宽权限。
- **有界**：树深度、树内 run 数、并发、信号频率、结果长度全部有上限，模型只能收窄不能扩张。
## 2. 概念模型
### 2.1 Agent 定义（Markdown）
每个子 Agent 是 `definitions_dir`（默认 `agents/`）下的一个 `*.md` 文件，文件名必须等于 `id`。前面是 YAML frontmatter，后面是角色正文（role body）：
```md
---
id: researcher
description: Research primary sources
llm_profile: research # 引用 config.json 顶层 agents 的 key
# 或者内联指定（WebUI 首选）：
# provider: openai
# model: gpt-4.1
tools: [file_read, file_search, web_fetch]
delegates: [coder] # 下一级委托目标；缺省=general-purpose，[]=不可，["*"]=任意
skills: [summarize] # get_skill 的作用域
limits:
timeout_secs: 900
max_iterations: 24
max_result_chars: 16000
signal: # 可选：启用 emit_signal
delivery: queue # queue | steer
---
# Role
只返回有证据支撑的结论。
```
关键校验（`src/agent/definition.rs`）：
- `deny_unknown_fields`：frontmatter 出现未知字段直接拒绝。
- `enabled`（默认 `true`）：单个定义的开关；`enabled: false` 的定义保留在磁盘供管理 UI 查看，但不进入活动 catalog。
- `id` 必须匹配文件名、小写字母开头、长度 ≤64，且保留 `root/main/default/general`。
- `llm_profile` 或内联 `provider`+`model` 二选一必填；内联的 provider 和 model 必须成对出现。
- 文件必须是非符号链接的普通文件，≤256KB；role body 非空且 ≤64K 字符。
- 计算 `definition_hash`（canonical frontmatter + role body 的 SHA256），随 run 持久化，用于识别运行代内定义是否变更。
内置 `general-purpose` 定义在首次启动释放到 `~/.picobot/agents/`，作为 `delegates` 缺省时的默认委托目标。
### 2.2 Catalog 与委托图
`AgentCatalog` 每个运行代不可变。加载时把定义解析成 `AgentDefinition`（含解析后的 provider config），并校验：
- Provider profile 存在、工具名/Skill 名在注册表里、`delegates` 列表里显式列出的目标存在（`*` 与缺省不校验）。
- 任一无效 → 整代拒绝启动/热重载。
委托规则：
- **主 Agent（ROOT）**：可委托给任意具名子 Agent（`root_can_delegate` 只判断目标是否在 catalog 里）。
- **子 Agent 的下一级**：由定义里的 `delegates` 决定，语义如下：
| `delegates` | 含义 |
|-------------|------|
| 缺省（不写该字段） | 仅可委托内置 `general-purpose` |
| `[]` | 不可继续委托 |
| `["*"]` | 可委托任意子代理（除自己） |
| `["a", "b"]` | 按列表指定 |
self 与祖先链上的 Agent 在 `resolve_agent` 时永远被拒绝（循环检测）。`can_delegate(caller, target)` / `root_can_delegate(target)` / `delegate_targets(caller)` 是这套语义的唯一实现点。
### 2.3 Run 与执行上下文
一次委托 = 一个 run，持久化在 `agent_runs`。执行上下文 `AgentExecutionContext`（`src/agent/run.rs`）携带：
| 字段 | 含义 |
|------|------|
| `root_session_id` | 整棵委托树所属的会话 |
| `run_id` / `execution_id` | run ID 与「执行尝试」ID；首次两者相同，`execution_id` 用于条件状态转换，迟到的旧执行写不进状态 |
| `parent_run_id` / `ancestry` | 父 run 与祖先链（用于循环检测、授权） |
| `depth` | 委托深度（≥1） |
| `budget` | 剩余 run 数与剩余深度 |
| `tree_runs` | 整棵树的共享原子计数，强制 `max_runs_per_tree` |
| `signal_contract` | 信号契约（`None` 表示该 run 不能发信号） |
| `cancellation` | CancellationToken（父取消会向子级联） |
`child()` 构造子上下文：深度 +1、预算 -1、`parent_run_id` 设为父 run、`ancestry` 追加目标，并**共享** `tree_runs`。
## 3. 执行模型
### 3.1 foreground（同步等待）
`delegate` 工具 `mode=foreground` 时，调用方（主 Agent 或某个子 Agent）阻塞等待结果：
1. 先解析所有 target（任何非法请求在写库之前失败，不留孤儿行）。
2. 一次性持久化所有 run（`accept_agent_runs`，status=queued）。
3. 若调用方是具名 Agent，把父 run 置为 `waiting_children`（等待期间不占 step permit）。
4. 并发执行（`join_all`），结果**保持请求顺序**返回。
5. 每个 run 各自 commit terminal；父 run 恢复 `running`。
结果直接作为工具返回值回到模型，同时完整结果持久化到 `agent_runs.result`。
### 3.2 background（异步 + 收件箱）
`mode=background` 时，`delegate` 只做「接纳」就立即返回 run ID；真正执行在后台 runner 里，结果通过 durable inbox 送达。这是结果传递机制最复杂的部分，见第 4 节。
只有 ROOT 能发起 background；子 Agent 发起的 background、以及 background 里再 background 都不开放。
## 4. 结果传递机制（重点）
foreground 的结果是「调用即返回」，没有跨 Turn 的传递问题。**真正需要设计的是 background 的结果如何可靠地回到主 Agent**——因为 background runner 跑在后台，主 Agent 可能正在忙别的 Turn，甚至已经结束上一个 Turn。
核心思路：**结果不是直接通知 Channel，而是落进一个持久化收件箱，由主 Agent 的「续接 Turn」（continuation Turn）读取并汇入会话**。
```
background runner
└─ terminal commit（原子事务）
├─ agent_runs → 终态（execution_id + generation 条件）
├─ plan item 完成（若有）
└─ 预留槽 → agent_inbox_events 完成事件（completion）
│
▼
session 收件箱 worker（queue lane）
│ claim(pending→leased)
▼
continuation Turn（hidden 触发 + 只读工具集）
│ commit_continuation_turn（原子）
▼
可见的 assistant 结果 + 事件 consumed
```
### 4.1 完成槽预留（保证不丢）
接纳 background 批次时，先对每个 run 预留一个完成槽：
```sql
UPDATE agent_session_state
SET reserved_completion_slots = reserved_completion_slots + ?,
revision = revision + 1, updated_at = ?
WHERE root_session_id = ?
AND pending_event_count + reserved_completion_slots + ? <= ?
RETURNING revision;
```
- 这是**条件更新**：只有 `pending + reserved + 新增 ≤ 上限` 时才成功，避免并发 `COUNT(*)` 漂移。
- 预留成功后才持久化 run；预留失败则整批拒绝。
- 意义：background 的完成事件**永远占得住位置**，不会因为收件箱被 signal 打满而丢失。signal 只能在「未预留」的容量里插入（见 4.4）。
### 4.2 终态提交（单写者）
`commit_agent_terminal`（`src/storage/agent_run.rs`）在一个事务里完成：
1. `UPDATE agent_runs SET status=终态, result=?, error=?, usage...`，条件是 `WHERE id=? AND execution_id=? AND runtime_generation=? AND status IN ('queued','running','waiting_children')`。**命中 0 行 = 迟到的旧结果，直接丢弃（返回 `None`）**。
2. 若 run 绑定了 plan item，用同一个 `execution_id` 条件完成该子项。
3. 若 `completion_slot_reserved`（background），把预留槽**转换成**一条 completion 事件写入 `agent_inbox_events`（status=completed/failed/timed_out/cancelled/interrupted，携带 result/error/signal_ids）。
三步同一事务提交：要么全部生效，要么全部回滚，**内存与数据库永不分叉**。
### 4.3 收件箱事件状态机
`agent_inbox_events` 里每条事件（signal 或 completion）走：
```
pending ──claim──▶ leased ──admit(steer)──▶ admitted ──▶ consumed
▲ │ │
└──release(backoff)◀──────────────────────────┘
pending/leased/admitted ──supersede──▶ superseded（显式取消）
pending/leased/admitted ──dead_letter──▶ dead_letter（归档/删除/超限）
```
- **claim**：`pending → leased`，带 `lease_token` + `lease_until` + `attempt_count+1`。claim 条件 `status='pending'`，天然防双租。
- **admit**（仅 steer）：`leased → admitted`，绑定 `admitted_turn_id`。
- **release**：`leased/admitted → pending`，带重试 `next_attempt_at`。lease token 防止别的 worker 已消费后又被释放。
- **consume**：在续接 Turn 提交事务里原子完成。
- **supersede**：显式取消 run 时，把其未消费 signal 置为 superseded（completion 永不 supersede）。
- **dead_letter**：会话归档/删除、或投递超过 `max_inbox_delivery_attempts` 时；最多发一次有界 system fallback 提示。
### 4.4 两条投递 lane：queue 与 steer
事件按 `delivery` 分两种语义（`SignalDelivery`，定义在 `signal.delivery`）：
| lane | 语义 | 到达方式 |
|------|------|----------|
| `queue` | 排队到**下一个** Turn | 收件箱 worker 在调度边界把事件变成续接 Turn |
| `steer` | 注入**当前活动** Turn 的安全边界 | 两阶段准入：claim → 预留 mailbox 槽 → DB admit(turn_id) |
**steer 两阶段准入**（任何一步失败都必须无损回退）：
1. claim（pending→leased，拿到 lease token）。
2. 在 TurnMailbox 的 agent lane 预留一个槽（容量独立于 user lane）。
3. `admit_inbox_event`（leased→admitted，绑 turn_id）。
4. 同一 Turn/代激活。
失败路径：claim 失败 → 释放 lease 并 wake queue lane；mailbox 满 → 释放 lease 回 pending，等 queue lane 以 continuation 送达。**steer 可靠退化为 queue**：当活动 Turn 关闭时，已 admit 的 steer 事件按 lease token 释放回 pending，绝不静默丢弃。
### 4.5 续接 Turn（continuation Turn）
queue lane 的 worker claim 一批事件后，把它们合成为一条**隐藏的触发消息**（`build_continuation_trigger`）：completion 事件渲染为「后台任务完成（Agent、状态、Run ID、任务、结果/错误）」，signal 渲染为「后台信号（级别、摘要）」。
续接 Turn 的特殊性：
- 触发消息 `client_visibility=hidden`、`turn_origin=agent_continuation`——**不进客户端历史、不进 Channel 投递、只供模型回放**。
- 工具集受限为只读：`file_read/file_search/content_search/web_fetch/calculator/agent_task`。续接 Turn 不能写文件、发消息、再委托、调度。
- `commit_continuation_turn` 在**一个事务**里写 hidden trigger + 可见 assistant/tool 消息 + usage + 事件 consume + session 计数，客户端永远不会看到「半成品续接」。
`requires_continuation=false` 的完成事件（如 `/stop` 产生的 cancel 完成）直接写成 consumed，不触发续接。
### 4.6 唤醒与公平调度
事件 commit 成功后，Coordinator 通过 `AgentInboxNotifier` 做一次**尽力而为**的 wake（弱引用、late-bound，避免与 SessionManager 形成强引用环）。**wake 丢失不是错误**：durable inbox 是唯一事实源，worker 有周期性重新 claim 的兜底。
公平调度：空闲（无用户积压）时 due 事件立即 claim（完成即返回）；忙碌时，连续处理 `max_user_turn_burst_before_inbox` 个用户 Turn 后，或最老 pending 事件等待超过 `max_inbox_wait_secs`，下一个调度项必须是一批 inbox 事件。当前活动 Turn 从不被 queue 事件抢占。
### 4.7 emit_signal（信号）
只在定义声明 `signal` 块时，run 才会被注入 `emit_signal` 工具。契约字段（总量、单条字节、最小间隔、burst、severity allowlist、dedupe 冷却窗、JSON 深度）全部由工具与 Coordinator 强制，模型只提供 key/severity/summary/details/dedupe_key。
- 结构校验（severity 是否在 allowlist、summary/key 长度、payload 大小与深度）在工具内做，不依赖模型自觉。
- 频率限制是每 run 内存态（工具实例为单个 run 的 registry 创建）。
- Coordinator `emit_signal` 再校验：run 存在、`execution_id` 匹配、非终态；`insert_agent_signal` 在 `pending + reserved + 1 ≤ 上限` 下条件插入，并做冷却窗 dedupe（`run_id + event_type + event_key` 唯一）。
- 信号 ID 记入 `emitted_signals`，最终写进该 run 的 completion 事件 payload，供主 Agent 交叉核对。
## 5. 持久化模型
三张 agent 表（schema v8）：
- **`agent_runs`**：每次委托一行。含 run id、root session、父子、caller 身份（`caller_agent_id`/`caller_scope_id`）、agent/definition 快照（`definition_hash`）、provider/model、mode、depth、task/context、budget、signal 契约快照、status（queued/running/waiting_children/终态）、result/error、usage、`execution_id`、`completion_slot_reserved`、时间线、revision。`execution_id` 唯一索引。
- **`agent_inbox_events`**：收件箱。`run_id`（NOT NULL，FK）、event_type（signal/completion）、event_key（去重键）、delivery（queue/steer）、requires_continuation、severity、payload、status、attempt/lease、`UNIQUE(run_id, event_type, event_key)`。
- **`agent_session_state`**：每根会话一行，权威容量计数（`pending_event_count` + `reserved_completion_slots` + 单调 `revision`）。所有容量增减都是条件 UPDATE。
结果不复制大文本：完整结果在 `agent_runs.result`，inbox payload 只放有界摘要/元数据。
## 6. 取消与恢复
- **取消 run**（`cancel_run`）：先按树位置授权、确认非终态，然后 `cancel_agent_run_with_completion`（写终态 + 若预留槽则转换 completion 事件），取消 CancellationToken，并 supersede 未消费 signal。`suppress_continuation=true` 时 completion 写成 consumed（`/stop`/归档后不再续接）。
- **取消会话**（`cancel_session`）：取消该会话所有非终态 run，完成事件写 consumed。
- **启动恢复**（`recover_agent_state`）：旧运行代的 queued/running/waiting_children → interrupted（background 转换 failure completion）；过期 lease → pending 带 backoff、超限 → dead_letter；按行重算容量计数，差异修复并告警。
## 7. 授权
run ID 不是凭证。ROOT 可访问本会话所有 run；具名 Agent 只能访问自己的 run 及其**后代**（沿 `parent_run_id` 向上走到自己）。其他会话一律拒绝读取/取消。
## 8. 易出错点总结
实现过程中反复踩坑的地方，按重要程度排序：
1. **execution_id 条件更新**。终态、running、信号写入都必须带 `execution_id`（和 generation）条件，命中 0 行 = 迟到旧结果，静默丢弃。否则一个超时后被重试的旧 runner 可能覆盖新终态。
2. **收件箱容量 = pending + reserved，且必须条件 UPDATE**。signal 不能挤掉 background 的完成预留；用无锁 `COUNT(*)` 推断会并发漂移，必须在同一写事务里 `UPDATE ... WHERE pending+reserved+n ≤ limit RETURNING`。
3. **完成槽预留 → 完成事件转换必须在终态提交的同一事务里**。一旦分开，崩溃就会留下「已预留但永远不产出 completion」的槽。
4. **wake 是尽力而为，不是正确性来源**。任何依赖「wake 一定到达」的逻辑都会在丢 wake 时漏投。事实源是 durable inbox，wake 只加速，周期性重新 claim 兜底。
5. **steer 两阶段准入的无损性**。claim → mailbox 预留 → DB admit 任何一步失败都要释放 lease 并 wake queue lane；Turn 关闭时已 admit 的 steer 要按 lease token 放回 pending（退化为 queue）。绝不静默丢弃。
6. **输入归属互斥**（steer / 下一 Turn FIFO / `/stop`）。一条输入要么属于当前活动 Turn，要么进下一 Turn FIFO，`/stop` 两者都丢弃；三者必须无损且互斥。
7. **委托循环、预算、深度、树 run 上限要在解析阶段就拦下**。`ancestry` 判环、`budget` 判耗尽、`reserve_tree_run` 用共享原子计数强制 `max_runs_per_tree`。
8. **fail-closed 顺序：先解析所有 target 再写库**。否则批量里一个非法 target 会留下前几个 run 的孤儿行。
9. **spawn 失败的补偿**。TaskSupervisor 拒绝 spawn（如关机）时，要取消该 run 并释放其完成槽，否则槽永远占着。
10. **子 Agent 管理器对 Coordinator 用 `Weak`**。Coordinator 拥有 manager，manager 若强引用 coordinator 会成环。
11. **结果两段式：模型看到截断、库存全量**。`max_result_chars` 截断返回给模型的内容并提示「用 agent_task get_result 查全量」；`full_content` 原样持久化。截断要按 `floor_char_boundary`，否则 UTF-8 边界 panic。
12. **MIN 聚合无行时返回 NULL 被解成 0**。曾导致 worker 空转；`oldest_pending_due`/`next_pending_due_at` 用 `Option<Option<i64>>` 显式区分「无行」与「值为 0」。
13. **claim 的确定性与防双租**。claim 按 `created_at, id` 排序保证确定性；lease token 让 release 只作用于本 worker 租下的事件，防止把别人已消费的又放回 pending。
14. **idempotency_key 的部分唯一索引**。`UNIQUE(root_session_id, caller_scope_id, idempotency_key) WHERE idempotency_key IS NOT NULL`——`NULL` 不参与去重，否则 SQLite 里所有 NULL 会互相冲突。
15. **signal 冷却窗去重键要含时间窗**（`event_key = signal:{key}:{now/cooldown}`）。否则「窗口内去重、窗口外再发」无法表达。
16. **父 run 的 `waiting_children` 状态必须对称恢复**。父等待时释放 step permit，子结束后要 `restore_agent_run_running`，否则父 run 卡在 waiting 状态。
## 9. 配置项
`agent_orchestration`（`src/config/mod.rs`）：
| 键 | 默认 | 含义 |
|----|------|------|
| `definitions_dir` | `agents` | 定义目录（相对 config.json 所在目录） |
| `max_tree_depth` | `4` | 委托树最大深度 |
| `max_runs_per_tree` | `16` | 一棵树内最大 run 数 |
| `max_concurrent_runs` / `max_concurrent_runs_per_session` | `6` / `4` | 全局/每会话并发 run 上限 |
| `max_pending_inbox_events_per_session` | `128` | 每会话 pending + reserved 上限 |
| `max_inbox_delivery_attempts` | `8` | 投递尝试上限（超过进 dead-letter） |
| `max_user_turn_burst_before_inbox` | `4` | 公平调度：连续处理多少个用户 Turn 后必须清 inbox |
| `max_inbox_wait_secs` | `30` | 最老事件等待上限（秒） |

BIN
docs/avatar.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 692 KiB

BIN
docs/logo.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 732 KiB

616
docs/superpowers/plans/2026-07-23-p0-webui-foundation.md
View File

@ -1,616 +0,0 @@
# P0 地基 Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 建立 Signal Deck 设计系统（双主题 tokens + 内嵌字体）、应用外壳（扁平导航 + 全局聊天 WS + 活动脊 + 主题/鉴权），并按新设计重构聊天页，产出可工作的聊天优先控制台地基。
**Architecture:** 用 CSS 自定义属性表达 Signal Deck tokens（`:root` 暗色 / `:root[data-theme="light"]` 亮色），整体重写 `styles.css`。将聊天 WebSocket 连接从 ChatPage 提升为模块级单例 `lib/chat.svelte.js`，由 App 外壳统一持有，使活动脊在所有页面可用；ChatPage 订阅帧并保留全部现有逻辑（会话/消息/turn 快照/计划/上传/斜杠命令）。两个拉丁字体经 vite `publicDir` 以固定名输出，`http.rs` 用 `include_bytes!` 内嵌并提供同源路由，维持单二进制与现有 CSP。
**Tech Stack:** Svelte 5（runes）、Bits UI、Vite、CSS custom properties；Rust/Axum（字体路由）、build.rs + vite（嵌入管线）。
**验证约定（重要）：** 本仓库前端**没有单元测试框架**。前端任务以 `npm run check`（svelte-check）+ `npm run build` + 浏览器目检为验证手段（见 AGENTS.md）；涉及 Rust 的任务以 `cargo build` + `cargo test --lib` + `cargo clippy --all-targets --all-features -- -D warnings` 验证。不要虚构前端测试。
**参考文档：** 设计规格 `docs/superpowers/specs/2026-07-23-webui-refactor-design.md`（§4 设计系统、§5 信息架构、§6.1 聊天页、§8 前端架构）。配色/组件 mockup 见 `.superpowers/brainstorm/111044-1784795642/`（design-system.html、page-chat.html）。
---
## File Structure
**Create:**
- `webui/public/fonts/space-grotesk-500.woff2`、`space-grotesk-700.woff2`、`jetbrains-mono-400.woff2`、`jetbrains-mono-700.woff2` — 内嵌拉丁字体（vite publicDir 原样复制到产物根）
- `webui/public/theme-init.js` — 首屏防闪烁主题初始化脚本（CSP 安全，经 `/theme-init.js` 路由提供）
- `webui/src/lib/theme.js` — 主题检测/应用/持久化
- `webui/src/lib/chat.svelte.js` — 全局聊天 WS 单例（连接/重连/订阅/发送/最新 turn 快照）
- `webui/src/lib/components/ActivitySpine.svelte` — 全局活动脊
**Modify:**
- `webui/src/styles.css` — 全面重写为 Signal Deck tokens + @font-face + 组件样式
- `webui/src/App.svelte` — 外壳：扁平导航、全局 WS、活动脊、主题切换、鉴权
- `webui/src/pages/ChatPage.svelte` — 改用全局 chat client + Signal Deck 三栏布局（保留全部逻辑）
- `webui/src/pages/PairingPage.svelte` — 套用新 tokens（结构不变）
- `webui/index.html` — theme-color 更新为 `#0b1017` + `<head>` 引入 `/theme-init.js`
- `webui/src/lib/ToolCallCard.svelte`、`TurnView.svelte`、`Markdown.svelte`、`Toast.svelte` — 套用新 tokens/类名（StatusBadge 无独立样式，随 styles.css 的 `.badge.*` 更新）
- `src/gateway/http.rs` — 字体路由（include_bytes! + font/woff2）+ `/theme-init.js` handler（include_str!）
- `src/gateway/mod.rs` — 公开静态路由组追加 `/fonts/{name}` 与 `/theme-init.js`
- `build.rs` — `rerun-if-changed` 增加 `webui/public`
**不动：** 后端聊天/配置/记忆等现有端点（P0 纯前端 + 字体路由）。
---
## Chunk 1: 设计 tokens 与字体内嵌管线
### Task 1.1: 内嵌字体（publicDir + http.rs 路由）
**Files:**
- Create: `webui/public/fonts/{space-grotesk-500,space-grotesk-700,jetbrains-mono-400,jetbrains-mono-700}.woff2`
- Modify: `src/gateway/http.rs`（新增字体 handler 与路由）
- Modify: `src/gateway/mod.rs`（注册 `/fonts/{name}` 路由，公开静态资源层）
- Modify: `build.rs`（`rerun-if-changed=webui/public`）
- [ ] **Step 1: 获取并提交字体文件**
从 @fontsource 取 latin 子集 woff2（版本锁定、可复现）：
```bash
cd webui
npm i -D @fontsource/space-grotesk @fontsource/jetbrains-mono
mkdir -p public/fonts
cp node_modules/@fontsource/space-grotesk/files/space-grotesk-latin-500-normal.woff2 public/fonts/space-grotesk-500.woff2
cp node_modules/@fontsource/space-grotesk/files/space-grotesk-latin-700-normal.woff2 public/fonts/space-grotesk-700.woff2
cp node_modules/@fontsource/jetbrains-mono/files/jetbrains-mono-latin-400-normal.woff2 public/fonts/jetbrains-mono-400.woff2
cp node_modules/@fontsource/jetbrains-mono/files/jetbrains-mono-latin-700-normal.woff2 public/fonts/jetbrains-mono-700.woff2
```
若 @fontsource 文件路径/命名随版本不同，用 `ls node_modules/@fontsource/*/files/ | grep latin` 找到对应 latin 500/700/400 的 normal woff2。确认 4 个文件均为非空 woff2。@fontsource 仅为取字体的 devDependency，运行时不依赖。
- [ ] **Step 2: build.rs 监听 public 目录**
在 `build.rs` 的 `build_webui` 的监听列表（约 64-73 行）追加：
```rust
"webui/public",
```
- [ ] **Step 3: http.rs 增加字体 handler**
在 `src/gateway/http.rs`（`webui_styles` 之后）新增：
```rust
const EMBEDDED_FONTS: &[(&str, &[u8])] = &[
(
"space-grotesk-500.woff2",
include_bytes!(concat!(env!("OUT_DIR"), "/webui/fonts/space-grotesk-500.woff2")),
),
(
"space-grotesk-700.woff2",
include_bytes!(concat!(env!("OUT_DIR"), "/webui/fonts/space-grotesk-700.woff2")),
),
(
"jetbrains-mono-400.woff2",
include_bytes!(concat!(env!("OUT_DIR"), "/webui/fonts/jetbrains-mono-400.woff2")),
),
(
"jetbrains-mono-700.woff2",
include_bytes!(concat!(env!("OUT_DIR"), "/webui/fonts/jetbrains-mono-700.woff2")),
),
];
pub async fn webui_font(Path(name): Path<String>) -> Response {
let bytes = EMBEDDED_FONTS
.iter()
.find(|(font_name, _)| *font_name == name)
.map(|(_, bytes)| *bytes);
let Some(bytes) = bytes else {
return StatusCode::NOT_FOUND.into_response();
};
Response::builder()
.header(header::CONTENT_TYPE, "font/woff2")
.header(header::CACHE_CONTROL, "public, max-age=31536000, immutable")
.header("X-Content-Type-Options", "nosniff")
.body(Body::from(bytes))
.expect("valid font response")
}
```
（`Path` 已在文件顶部 `axum::extract` 导入。）
- [ ] **Step 4: mod.rs 注册字体路由（公开层，随静态资源）**
在 `src/gateway/mod.rs` 的公开静态路由组（约 592-596 行，`/`、`/app.js`、`/styles.css` 处）追加：
```rust
.route("/fonts/{name}", routing::get(http::webui_font))
```
字体属静态资源层，不进设备鉴权（与 app.js/styles.css 同级；CSP `default-src 'self'` 已允许同源 font）。
- [ ] **Step 5: 构建验证**
Run: `cargo build`（会自动触发 vite 构建，public/fonts 复制到 OUT_DIR/webui/fonts）
Expected: 编译成功，无 clippy 级错误。
Run: `cargo clippy --all-targets --all-features -- -D warnings`
Expected: 无警告。
- [ ] **Step 6: Commit**
```bash
git add webui/public/fonts build.rs src/gateway/http.rs src/gateway/mod.rs webui/package.json webui/package-lock.json
git commit -m "feat(webui): embed latin fonts and serve via /fonts route"
```
### Task 1.2: 重写 styles.css 为 Signal Deck tokens
**Files:**
- Modify: `webui/src/styles.css`（整体重写）
- [ ] **Step 1: 写入 @font-face 与 tokens**
将 `styles.css` 顶部的 `:root` / `:root[data-theme="light"]` 块整体替换为（保留文件其余组件类，随后在 Step 2 调整）：
```css
@font-face {
font-family: "Space Grotesk";
src: url("/fonts/space-grotesk-500.woff2") format("woff2");
font-weight: 500; font-style: normal; font-display: swap;
}
@font-face {
font-family: "Space Grotesk";
src: url("/fonts/space-grotesk-700.woff2") format("woff2");
font-weight: 700; font-style: normal; font-display: swap;
}
@font-face {
font-family: "JetBrains Mono";
src: url("/fonts/jetbrains-mono-400.woff2") format("woff2");
font-weight: 400; font-style: normal; font-display: swap;
}
@font-face {
font-family: "JetBrains Mono";
src: url("/fonts/jetbrains-mono-700.woff2") format("woff2");
font-weight: 700; font-style: normal; font-display: swap;
}
:root {
--font-ui: "Space Grotesk", ui-sans-serif, system-ui, "PingFang SC", "Microsoft YaHei", "Noto Sans SC", sans-serif;
--font-mono: "JetBrains Mono", ui-monospace, SFMono-Regular, Consolas, monospace;
color-scheme: dark;
font-family: var(--font-ui);
color: #e7ecf3;
background: #0b1017;
--bg: #0b1017;
--panel: #0e1520;
--panel-2: #131c29;
--sidebar: #0d131c;
--header: rgb(11 16 23 / 84%);
--line: #1d2733;
--line-strong: #2c3a4c;
--muted: #8fa3b8;
--faint: #5b6b7e;
--text: #e7ecf3;
--text-soft: #b8c4d4;
--accent: #ffb454; /* amber = 活动 */
--accent-hover: #ffc370;
--accent-contrast: #1a1206;
--accent-soft: rgb(255 180 84 / 12%);
--accent-border: rgb(255 180 84 / 35%);
--signal: #2dd4bf; /* teal = 健康 */
--signal-soft: rgb(45 212 191 / 12%);
--signal-border: rgb(45 212 191 / 35%);
--info: #6aa6ff;
--info-soft: rgb(106 166 255 / 12%);
--danger: #ff7b86;
--danger-soft: rgb(255 123 134 / 12%);
--danger-border: rgb(255 123 134 / 35%);
--warning: #ffb454;
--warning-soft: rgb(255 180 84 / 10%);
--success-soft: rgb(45 212 191 / 12%);
--overlay: #101826;
--code-bg: #080c12;
--user-bubble: #221d38;
--spine-bg: #0e1520; /* 活动脊：亮色下也保持深色 */
--shadow: 0 16px 45px rgb(0 0 0 / 35%);
--radius: 11px;
}
:root[data-theme="light"] {
color-scheme: light;
color: #1a2230;
background: #eef1f5;
--bg: #eef1f5;
--panel: #ffffff;
--panel-2: #f4f6f9;
--sidebar: #f7f9fc;
--header: rgb(238 241 245 / 86%);
--line: #d8dee8;
--line-strong: #c2ccd9;
--muted: #5b6b7e;
--faint: #8494a8;
--text: #1a2230;
--text-soft: #3d4b5e;
--accent: #c47400;
--accent-hover: #a86300;
--accent-contrast: #ffffff;
--accent-soft: rgb(196 116 0 / 10%);
--accent-border: rgb(196 116 0 / 35%);
--signal: #0d9488;
--signal-soft: rgb(13 148 136 / 10%);
--signal-border: rgb(13 148 136 / 35%);
--info: #2f6fd0;
--info-soft: rgb(47 111 208 / 10%);
--danger: #d94354;
--danger-soft: rgb(217 67 84 / 10%);
--danger-border: rgb(217 67 84 / 35%);
--warning: #c47400;
--warning-soft: rgb(196 116 0 / 8%);
--success-soft: rgb(13 148 136 / 10%);
--overlay: #ffffff;
--code-bg: #f7f9fc;
--user-bubble: #ece7fb;
--spine-bg: #0e1520; /* 亮色下活动脊仍是深色 LED 条 */
--shadow: 0 16px 45px rgb(31 41 55 / 12%);
}
```
- [ ] **Step 2: 调整组件类以适配新 tokens**
逐个检查并更新其余组件类（原文件 60 行起）：
- 所有 `font-family` 硬编码处改用 `var(--font-ui)`；数据/日志/时间戳/`code`/`.mono` 类用 `var(--font-mono)`。
- 原紫色相关（`--accent` 旧值、`--user-bubble`）已由 tokens 替换，确认无残留硬编码 hex。
- `.primary` 按钮：`background: var(--accent); color: var(--accent-contrast);`（暗色下琥珀底深字，亮色下深琥珀底白字）。
- 状态点/在线指示：健康用 `var(--signal)`，活动/进行中用 `var(--accent)`，错误用 `var(--danger)`。
- **两处硬编码绿色必须手动改为 `var(--signal)`**（否则不随 tokens 更新）：`.gateway-status i.online { color: #48b985 }`（约 93 行）与 `.badge.ok { color: #38a877 }`（约 267 行，StatusBadge 的颜色实际来自这里）。
- 新增工具类（供组件使用）：
```css
.mono { font-family: var(--font-mono); }
.label-caps { font-family: var(--font-mono); font-size: 9px; letter-spacing: .16em; color: var(--faint); }
.panel { background: var(--panel); border: 1px solid var(--line); border-radius: var(--radius); }
.cap { display: inline-flex; align-items: center; gap: 4px; font-size: 9.5px; font-weight: 600; border-radius: 6px; padding: 2.5px 8px; }
.cap.signal { color: var(--signal); background: var(--signal-soft); border: 1px solid var(--signal-border); }
.cap.accent { color: var(--accent); background: var(--accent-soft); border: 1px solid var(--accent-border); }
.cap.danger { color: var(--danger); background: var(--danger-soft); border: 1px solid var(--danger-border); }
.cap.info { color: var(--info); background: var(--info-soft); border: 1px solid var(--line); }
@keyframes spine-pulse { 0%,100% { opacity: 1; } 50% { opacity: .35; } }
.pulse-dot { width: 7px; height: 7px; border-radius: 50%; display: inline-block; animation: spine-pulse 1.6s ease-in-out infinite; }
@media (prefers-reduced-motion: reduce) { .pulse-dot { animation: none; } }
```
- [ ] **Step 3: 验证**
Run: `cd webui && npm run check && npm run build`
Expected: svelte-check 无错误；构建成功。
- [ ] **Step 4: Commit**
```bash
git add webui/src/styles.css
git commit -m "feat(webui): Signal Deck design tokens and base styles"
```
---
## Chunk 2: 核心 lib 与应用外壳
### Task 2.1: theme.js 主题管理
**Files:**
- Create: `webui/src/lib/theme.js`
- [ ] **Step 1: 实现**
```js
const STORAGE_KEY = "picobot-theme";
export function preferredTheme() {
const saved = localStorage.getItem(STORAGE_KEY);
if (saved === "light" || saved === "dark") return saved;
return matchMedia("(prefers-color-scheme: dark)").matches ? "dark" : "light";
}
export function applyTheme(theme) {
document.documentElement.dataset.theme = theme;
document.documentElement.style.colorScheme = theme;
document
.querySelector('meta[name="theme-color"]')
?.setAttribute("content", theme === "dark" ? "#0b1017" : "#eef1f5");
localStorage.setItem(STORAGE_KEY, theme);
}
```
- [ ] **Step 2: 验证** — `cd webui && npm run check`（无错误）
- [ ] **Step 3: Commit** — `git add webui/src/lib/theme.js && git commit -m "feat(webui): theme helpers"`
### Task 2.2: chat.svelte.js 全局聊天客户端
**Files:**
- Create: `webui/src/lib/chat.svelte.js`
将 ChatPage 的连接/重连生命周期提取为模块级单例。帧分发保留给订阅者（ChatPage 搬入其 `handleFrame` 逻辑）；客户端额外暴露最新 turn 快照供活动脊使用。
- [ ] **Step 1: 实现**
```js
import { clientId } from "./api.js";
class ChatClient {
connected = $state(false);
turn = $state(null); // 最新 turn 快照（任意 session），供活动脊
#socket = null;
#handlers = new Set();
#reconnectTimer = null;
#stopped = false;
connect() {
if (this.#socket) return;
this.#stopped = false;
const scheme = location.protocol === "https:" ? "wss" : "ws";
const ws = new WebSocket(`${scheme}://${location.host}/ws?client_id=${encodeURIComponent(clientId())}`);
this.#socket = ws;
ws.onopen = () => {
this.connected = true;
this.#dispatch({ type: "_open" });
};
ws.onerror = () => ws.close();
ws.onclose = () => {
this.connected = false;
this.#socket = null;
this.#dispatch({ type: "_close" });
if (!this.#stopped) this.#reconnectTimer = setTimeout(() => this.connect(), 1800);
};
ws.onmessage = (event) => {
const frame = JSON.parse(event.data);
if (frame.type === "turn_updated" && frame.snapshot) this.turn = frame.snapshot;
this.#dispatch(frame);
};
}
disconnect() {
this.#stopped = true;
clearTimeout(this.#reconnectTimer);
this.#socket?.close();
this.#socket = null;
}
send(frame) {
if (this.#socket?.readyState === WebSocket.OPEN) {
this.#socket.send(JSON.stringify(frame));
return true;
}
return false;
}
subscribe(handler) {
this.#handlers.add(handler);
return () => this.#handlers.delete(handler);
}
#dispatch(frame) {
for (const handler of this.#handlers) handler(frame);
}
}
export const chat = new ChatClient();
```
- [ ] **Step 2: 验证** — `cd webui && npm run check`
- [ ] **Step 3: Commit** — `git add webui/src/lib/chat.svelte.js && git commit -m "feat(webui): global chat websocket client"`
### Task 2.3: ActivitySpine.svelte 活动脊
**Files:**
- Create: `webui/src/lib/components/ActivitySpine.svelte`
- [ ] **Step 1: 实现**
活动脊显示：Turn 实时状态（来自 `chat.turn` 快照）+ 吞吐（前端对相邻帧 `usage.completion_tokens` 差值求导）+ 连接状态。gen/uptime/metrics 等字段在 P1 由 `/api/status` 补充，P0 先显示版本与连接态。
```svelte
<script>
import { chat } from "../chat.svelte.js";
let { version = "" } = $props();
let lastTokens = $state(null); // { at, completion }
let rate = $state(null);
$effect(() => {
const turn = chat.turn;
if (!turn || turn.status !== "running") { rate = null; return; }
const completion = turn.usage?.completion_tokens;
const now = Date.now();
if (completion != null && lastTokens && now > lastTokens.at) {
const delta = completion - lastTokens.completion;
const secs = (now - lastTokens.at) / 1000;
if (delta >= 0 && secs > 0) rate = Math.round(delta / secs);
}
if (completion != null) lastTokens = { at: now, completion };
});
const running = $derived(chat.turn?.status === "running");
const turnLabel = $derived(chat.turn ? `TURN ${String(chat.turn.id ?? "").slice(0, 6).toUpperCase()}` : "");
const ctx = $derived(chat.turn?.usage?.prompt_tokens != null
? `${(chat.turn.usage.prompt_tokens / 1000).toFixed(1)}k` : null);
</script>
<div class="spine mono">
{#if running}
<span class="spine-turn active"><i class="pulse-dot" style="background:var(--accent);box-shadow:0 0 10px var(--accent)"></i>{turnLabel} · STREAMING</span>
{#if rate != null}<span class="spine-rate">▲ {rate} tok/s</span>{/if}
{#if ctx}<span>ctx {ctx}</span>{/if}
{:else if chat.turn}
<span class="spine-turn idle"><i class="pulse-dot" style="background:var(--signal);animation:none"></i>IDLE</span>
<span>最近 {turnLabel}</span>
{:else}
<span class="spine-turn idle"><i class="pulse-dot" style="background:var(--signal);animation:none"></i>READY</span>
{/if}
<span class="spine-right">
<span class:spine-ok={chat.connected} class:spine-down={!chat.connected}>{chat.connected ? "已连接" : "重连中"}</span>
{#if version}<span>{version}</span>{/if}
</span>
</div>
<style>
.spine { display: flex; align-items: center; gap: 14px; font-size: 10.5px; color: var(--muted);
background: var(--spine-bg); border-bottom: 1px solid var(--line); padding: 8px 16px; }
.spine-turn { display: inline-flex; align-items: center; gap: 7px; font-weight: 700; }
.spine-turn.active { color: var(--accent); }
.spine-turn.idle { color: var(--signal); }
.spine-rate { color: var(--signal); }
.spine-right { margin-left: auto; display: inline-flex; gap: 14px; color: var(--faint); }
.spine-ok { color: var(--signal); }
.spine-down { color: var(--warning); }
</style>
```
（`.mono`、`.pulse-dot` 来自 styles.css 工具类。）
- [ ] **Step 2: 验证** — `cd webui && npm run check`
- [ ] **Step 3: Commit** — `git add webui/src/lib/components/ActivitySpine.svelte && git commit -m "feat(webui): global activity spine"`
### Task 2.4: App.svelte 外壳重构
**Files:**
- Modify: `webui/src/App.svelte`
- [ ] **Step 1: 重构**
要点（保留现有鉴权/配对/health 逻辑，替换导航与布局）：
- `onMount` 中：`applyTheme(preferredTheme())`；health 轮询保留（取 version 传给 ActivitySpine）。
- **WS 生命周期跟随"已鉴权外壳"而非根 onMount**：用 `$effect` 监听 `authenticated`——`authenticated` 为真时 `chat.connect()`，为假（如凭据被撤销、外壳卸载回配对页）时 `chat.disconnect()`。避免鉴权失效后客户端仍在后台每 1.8s 静默重连。
```js
$effect(() => {
if (authenticated) { chat.connect(); } else { chat.disconnect(); }
});
```
- 页面数组改为扁平导航（图标 + 名称）：`["chat","◫","聊天"]`、`["overview","◉","概览"]`、`["tools","🧰","工具&Skills"]`、`["logs","≋","日志"]`、`["memory","◇","记忆"]`、`["tasks","⌁","任务"]`、`["settings","⚙","配置"]`。P0 中 overview/tools 页面尚未实现，先渲染占位 `<div class="empty-card">即将上线</div>`（P1/P2 补齐）；logs/memory/tasks/settings 复用现有页面组件。
- 结构：`<aside class="sidebar">`（品牌 + 扁平 nav + 底部网关状态/主题切换）+ `<main>` 内 `<ActivitySpine {version} />` 置顶 + 页面区。
- 主题切换按钮调用 `applyTheme(theme === "dark" ? "light" : "dark")` 并更新 `theme` 状态。
- 需要的新 import：`import { chat } from "./lib/chat.svelte.js"`、`import { applyTheme, preferredTheme } from "./lib/theme.js"`、`import ActivitySpine from "./lib/components/ActivitySpine.svelte"`。
- WS 断开由上面的 `$effect` 负责（`authenticated=false` 时 disconnect）；如需双保险，`onMount` 清理函数 `return () => chat.disconnect()` 亦可，两者不冲突。
- [ ] **Step 2: index.html 防主题闪烁（CSP 安全方案）**
现有 CSP 为 `script-src 'self'`（无 `'unsafe-inline'`），**不能**写内联 `<script>`。改为独立同源脚本文件：
1. Create `webui/public/theme-init.js`（vite publicDir 会原样复制到 `OUT_DIR/webui/theme-init.js`）：
```js
try {
var t = localStorage.getItem("picobot-theme");
if (!t) t = matchMedia("(prefers-color-scheme: dark)").matches ? "dark" : "light";
document.documentElement.dataset.theme = t;
} catch (e) {}
```
2. `src/gateway/http.rs` 新增 handler（与 `webui_script` 同构）：
```rust
pub async fn webui_theme_init() -> Response {
static_response(
"text/javascript; charset=utf-8",
include_str!(concat!(env!("OUT_DIR"), "/webui/theme-init.js")),
)
}
```
3. `src/gateway/mod.rs` 公开静态路由组追加 `.route("/theme-init.js", routing::get(http::webui_theme_init))`。
4. `webui/index.html`：`<meta name="theme-color">` 的 `#0d1117` 改为 `#0b1017`；`<head>` 内加解析阻塞引用 `<script src="/theme-init.js"></script>`（同源，被 `script-src 'self'` 允许）。
5. File Structure 与 Task 1.1 的 build.rs 监听已含 `webui/public`（theme-init.js 随之复制）。
- [ ] **Step 3: 验证** — `cd webui && npm run check && npm run build`
- [ ] **Step 4: 目检** — `cargo run -- gateway` 后打开 http://127.0.0.1:19876/，确认：暗/亮主题切换生效且持久化、无首屏闪烁；活动脊显示"已连接/READY"；导航 7 项齐全；未实现页面显示占位。
- [ ] **Step 5: Commit** — `git add webui/public/theme-init.js src/gateway/http.rs src/gateway/mod.rs webui/src/App.svelte webui/index.html && git commit -m "feat(webui): app shell with flat nav, activity spine, and theme init"`
### Task 2.5: PairingPage 套用新 tokens
**Files:**
- Modify: `webui/src/pages/PairingPage.svelte`
- [ ] **Step 1: 将硬编码颜色替换为新 tokens**（结构与逻辑不变，仅样式对齐 Signal Deck）。
- [ ] **Step 2: 验证** — `npm run check`；未配对状态下目检配对页。
- [ ] **Step 3: Commit** — `git add webui/src/pages/PairingPage.svelte && git commit -m "style(webui): pairing page Signal Deck tokens"`
---
## Chunk 3: 聊天页重构
### Task 3.1: ChatPage 接入全局客户端 + 三栏布局
**Files:**
- Modify: `webui/src/pages/ChatPage.svelte`
这是 P0 最大的改动。**原则：全部现有业务逻辑（handleFrame 各分支、上传、斜杠补全、计划侧栏、历史校准）原样保留**，只做两件事：(a) 连接生命周期改用 `chat` 单例；(b) 套用 Signal Deck 类名/三栏布局。
- [ ] **Step 1: 连接改造**
- 删除组件内 `connect()`/`socket`/`reconnectTimer`/`stopped` 与 `onMount` 中的连接代码。
- `onMount` 中改为（**注意：重连时必须重置计划相关状态，与重构前 `connect()` 的 `onopen` 行为完全一致**）：
```js
const unsubscribe = chat.subscribe(handleFrame);
const onOpen = (frame) => {
if (frame.type !== "_open") return;
// 与重构前一致：每次（重）连接都重置计划状态再拉取
plansBySession = {};
unseenPlanSessions = {};
todoOpen = false;
chat.send({ type: "list_sessions", include_archived: false });
chat.send({ type: "get_slash_commands" });
};
const unsubOpen = chat.subscribe(onOpen);
if (chat.connected) onOpen({ type: "_open" }); // 已连接时首次挂载也走同一逻辑
return () => { unsubscribe(); unsubOpen(); clearPendingUploads(); };
```
- 所有 `send(...)` 调用改为 `chat.send(...)`；`connected` 改读 `chat.connected`。
- `handleFrame` 中原 `session_established`/`session_list`/... 分支逻辑**不变**。
- [ ] **Step 2: 布局与样式改造**
- 顶层 `<section class="page chat-layout">` 三栏：`sessions-panel`（左）｜ `chat-panel`（中）｜ `todo-panel`（右，`{#if todoOpen && currentPlan}`）。
- 消息气泡：用户用 `var(--user-bubble)` + 右下小圆角；助手无气泡底色、正文 `var(--text-soft)`。
- reasoning `<details>` 用 `.cap.info` 风格摘要；工具调用沿用 ToolCallCard（Task 3.2 重制）。
- 流式 turn（TurnView）下方显示 `▲ tok/s`（可复用 ActivitySpine 的速率逻辑，或简单显示 `status`）。
- 输入区（composer）：容器 `var(--panel)` + `var(--line-strong)` 边框；发送按钮 `.primary`（琥珀）；连接状态点用 `--signal`/`--warning`。
- 会话项激活态：左边框 `var(--accent)` + `var(--panel-2)` 底。
- 头部操作、Todo 侧栏沿用现有结构，仅换 tokens。
- [ ] **Step 3: 验证** — `cd webui && npm run check && npm run build`
- [ ] **Step 4: 端到端目检** — `cargo run -- gateway` + 浏览器：新建对话、发消息、收到流式回复（活动脊出现 STREAMING）、工具卡片折叠展开、/ 命令补全、附件上传、Todo 侧栏随 plan_updated 弹出、切换亮/暗主题聊天页正常。
- [ ] **Step 5: Commit** — `git add webui/src/pages/ChatPage.svelte && git commit -m "feat(webui): refactor chat page onto global client and Signal Deck"`
### Task 3.2: 重制共享组件（ToolCallCard / TurnView / Toast / Markdown）
**Files:**
- Modify: `webui/src/lib/ToolCallCard.svelte`、`TurnView.svelte`、`Toast.svelte`、`Markdown.svelte`
（`StatusBadge.svelte` 本身无 `<style>`，其颜色来自 `styles.css` 的 `.badge.*`，已在 Task 1.2 处理，不在此列。）
- [ ] **Step 1: ToolCallCard** — 默认折叠卡片：左边框运行中=`var(--accent)`（脉冲点）、完成=`var(--signal)`、失败=`var(--danger)`；名称/耗时用 `.mono`；展开显示参数与结果（`<details>`）。
- [ ] **Step 2: TurnView** — 流式渲染 reasoning（折叠）+ 正文 + 工具卡片 + 光标（`.pulse-dot` 或方块闪烁）+ `▲ tok/s`。
- [ ] **Step 3: Toast / Markdown** — 套用 tokens：Toast 用 `var(--overlay)` + 对应语义色边框；Markdown 的 code/pre 用 `var(--code-bg)` + `var(--font-mono)`，链接用 `var(--info)`。
- [ ] **Step 4: 验证** — `npm run check && npm run build`；目检聊天流中的卡片/Toast/代码块。
- [ ] **Step 5: Commit** — `git add webui/src/lib && git commit -m "style(webui): shared components Signal Deck"`
### Task 3.3: P0 收尾验证
- [ ] **Step 1: 全量构建与测试**
Run: `cd webui && npm run check && npm run build`
Run: `cargo build`
Run: `cargo test --lib`
Run: `cargo clippy --all-targets --all-features -- -D warnings`
Expected: 全部通过。
- [ ] **Step 2: 回归目检清单** — 配对流程、主题切换持久化、活动脊实时性、聊天全链路（含附件/命令/计划）、既有 logs/memory/tasks/settings 页面在新 tokens 下无样式崩坏。
- [ ] **Step 3: 版本号** — 按 AGENTS.md「功能变化后更新版本号」，在 `Cargo.toml` 与 `webui/package.json` bump minor（如 1.3.0 → 1.4.0），并同步 README 中对 WebUI 的描述（如有）。
- [ ] **Step 4: Commit** — `git add -A && git commit -m "chore(release): P0 webui foundation"`
---
## P0 完成标志
- 单二进制 `cargo build` 成功，字体经 `/fonts/*` 同源提供，无 CDN。
- 亮/暗双主题覆盖外壳与聊天页，活动脊全局可见且随 turn 实时变化。
- 聊天页功能与重构前完全一致（会话/消息/流式/工具/附件/命令/计划），仅视觉与连接归属变化。
- `npm run check`、`npm run build`、`cargo build`、`cargo test --lib`、`cargo clippy -- -D warnings` 全绿。
后续 P1（观测：Metrics + /api/status + 概览页 + 工具&Skills 页）、P2（日志流式 + 记忆可写 + 任务页）、P3（配置编辑器）将各自编写独立计划。

600
docs/superpowers/plans/2026-07-24-p1-observability.md
View File

@ -1,600 +0,0 @@
# P1 观测 Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 为 PicoBot 增加运行状况观测能力：进程级 `Metrics` 采集（token/费用/工具调用/turn/延迟）、`GET /api/status` 聚合快照、`GET /api/tools` 与 `GET /api/skills` 只读端点，以及前端概览页（运行仪表盘）与工具&Skills 页。
**Architecture:** 新增进程级 `Metrics`（`OnceLock<Arc<Metrics>>` 全局访问器，结构体可直接单测；选择全局而非穿线，避免改动 5 个 AgentLoop 构造点，且天然跨配置重载存活——precedented by `mcp::MCP_SERVER_STATUS`）。AgentLoop 在工具执行/模型调用/turn 完成处记录到全局 Metrics。`/api/status` handler 聚合 Metrics + 各服务只读内省（需为 MessageBus/TaskSupervisor/SessionManager/OutboundDispatcher 补轻量内省方法 + ws 连接计数 + 进程 uptime 静态）。前端概览页每 2s 轮询 `/api/status`，工具页拉取 `/api/tools`+`/api/skills`。
**Tech Stack:** Rust（Axum handler、tokio、原子计数）、Svelte 5（runes）、手写 SVG sparkline。
**关键设计决策（务必遵守）：**
1. **Metrics 进程级全局**：`src/observability/metrics.rs` 定义 `Metrics` 结构体 + `pub fn global_metrics() -> Arc<Metrics>`（`OnceLock` 首次调用初始化）。AgentLoop 与 `/api/status` 都通过 `global_metrics()` 访问，不穿线、不给 GatewayState 加字段。
2. **cost 可选**：在 `LLMProviderConfig` 增加可选 `price_input_per_million: Option<f64>` / `price_output_per_million: Option<f64>`（serde default None）。Metrics 记录 token；cost 仅在 AgentLoop 知道单价时累加（从 provider config 取），缺省为 0。不引入硬编码价格表。
3. **uptime**：`src/gateway/mod.rs` 增加进程级 `static STARTED: OnceLock<std::time::Instant>` + `fn process_uptime_secs() -> u64`，跨重载存活。
4. **active_lanes**：`OutboundDispatcher` 增加 `active_lanes: Arc<AtomicUsize>` 字段（构造时注入），lane 生成时 +1、lane 任务结束时 -1；暴露 `active_lane_count()`。Gateway 侧保留一个 clone 供 `/api/status` 读取。
5. **failed_7d 近似**：`/api/status` 的调度器失败数用「`last_status` 为失败态（error/timeout/delivery_error）的任务数」近似，避免 2s 轮询时逐任务查运行记录。在响应字段命名为 `failed_jobs`（语义=最近一次运行失败的任务数），UI 标注清楚。
6. **provider status 派生**：Metrics 维护 per-provider 最近调用错误率；`status` = 最近窗口错误率 > 阈值（如最近 10 次中 ≥3 次失败）→ `"degraded"`，否则 `"ok"`。
**参考：** 规格 `docs/superpowers/specs/2026-07-23-webui-refactor-design.md` §6.2/§6.3/§7.2/§7.3/§7.6；P0 计划 `docs/superpowers/plans/2026-07-23-p0-webui-foundation.md`（前端组件/页面模式）。
**验证约定：** Rust 改动 → 定向测试 + `cargo test --lib` + `cargo clippy --all-targets --all-features -- -D warnings` + `cargo build`；前端 → `cd webui && npm run check && npm run build`。前端无单测框架，勿虚构。
## Implementation Progress (2026-07-26)
**Branch:** `feat/webui-p1`
**Completed and reviewed:**
- Task 1.1 Metrics collector — `11474c0`
- Task 1.2 optional provider pricing — `115e77f`
- Task 1.3 AgentLoop instrumentation — `5d0cf5b`
- Task 2.1 runtime introspection — `aa989cb`
- Task 2.2 protected `GET /api/status` — `b14edc4`, with review fixes in `87cf500`
- Task 2.3 protected `GET /api/tools` — `119df57`
- Task 2.4 protected `GET /api/skills` + SkillsLoader accessor — `5e2771c`
- Task 3.1 Sparkline/CapacityMeter/MetricTile components — `6e719e7`
- Task 3.2 OverviewPage runtime dashboard — `0b02fd5`
- Task 3.3 ToolsPage tools & skills browser — `266fb8c`
- Task 3.4 App.svelte wiring — `4621210`
- Task 4.1 P1 release — `7676502`
**Final verification baseline:** `cargo test --lib` 350 passed; `cargo clippy --all-targets --all-features -- -D warnings` clean; `cargo build` passed; `npm run check` 0 errors/warnings; `npm run build` passed.
**Status:** P1 COMPLETE. Version bumped to 1.5.0.
---
## Chunk 1: Metrics 后端
### Task 1.1: Metrics 模块（结构体 + 记录/快照 + 全局访问器 + 单测）
**Files:**
- Create: `src/observability/metrics.rs`
- Modify: `src/observability/mod.rs`（`pub mod metrics;`，若 observability 模块存在；否则按代码库实际模块组织放置——先读 `src/observability/` 确认）
- [ ] **Step 1: 写失败测试**（在 `metrics.rs` 底部 `#[cfg(test)]`）
覆盖：record_turn 累加 tokens/turns/延迟窗口；record_tool_call 累加总量与 per-tool；record_provider 累加 per-provider token/cost/延迟/错误；snapshot 返回正确聚合；p95 计算；provider status 派生（ok vs degraded）。示例：
```rust
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn records_turns_tokens_and_p95() {
let m = Metrics::new();
for i in 1..=100 {
m.record_turn(Some(&Usage { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30, ..Default::default() }), i);
}
let s = m.snapshot();
assert_eq!(s.turns, 100);
assert_eq!(s.tokens_in, 1000);
assert_eq!(s.tokens_out, 2000);
assert!(s.turn_latency_p95_ms >= 95); // p95 of 1..=100
}
#[test]
fn records_per_tool_counts() {
let m = Metrics::new();
m.record_tool_call("bash", true);
m.record_tool_call("bash", true);
m.record_tool_call("read_file", false);
let s = m.snapshot();
assert_eq!(s.tool_calls, 3);
assert_eq!(s.per_tool.get("bash"), Some(&2));
assert_eq!(s.per_tool.get("read_file"), Some(&1));
}
#[test]
fn derives_provider_status() {
let m = Metrics::new();
for _ in 0..7 { m.record_provider("openai", "gpt-4o", None, 100, false); }
for _ in 0..3 { m.record_provider("openai", "gpt-4o", None, 100, true); }
let s = m.snapshot();
let p = s.providers.iter().find(|p| p.name == "openai").unwrap();
assert_eq!(p.status, "degraded"); // 3/10 recent errors
}
}
```
- [ ] **Step 2: 运行测试确认失败** — `cargo test --lib metrics` → FAIL（模块不存在）
- [ ] **Step 3: 实现 Metrics**
```rust
use std::collections::{HashMap, VecDeque};
use std::sync::{Arc, Mutex, OnceLock};
use std::time::Instant;
use crate::providers::Usage;
const WINDOW: usize = 100; // 延迟/错误滚动窗口大小
const DEGRADE_THRESHOLD: usize = 3; // 最近 10 次中失败 ≥3 → degraded
const DEGRADE_WINDOW: usize = 10;
#[derive(Default)]
struct ProviderStat {
model: String,
tokens_in: u64,
tokens_out: u64,
cost: f64,
calls: u64,
last_latency_ms: u64,
latencies: VecDeque<u64>,
recent_results: VecDeque<bool>, // true = error
}
pub struct Metrics {
tokens_in: std::sync::atomic::AtomicU64,
tokens_out: std::sync::atomic::AtomicU64,
turns: std::sync::atomic::AtomicU64,
tool_calls: std::sync::atomic::AtomicU64,
per_tool: Mutex<HashMap<String, u64>>,
turn_latencies: Mutex<VecDeque<u64>>,
providers: Mutex<HashMap<String, ProviderStat>>, // key = provider name
}
#[derive(serde::Serialize)]
pub struct MetricsSnapshot {
pub tokens_in: u64,
pub tokens_out: u64,
pub cost: f64,
pub turns: u64,
pub tool_calls: u64,
pub turn_latency_p95_ms: u64,
pub per_tool: HashMap<String, u64>,
pub providers: Vec<ProviderSnapshot>,
}
#[derive(serde::Serialize)]
pub struct ProviderSnapshot {
pub name: String,
pub model: String,
pub status: String, // "ok" | "degraded"
pub latency_ms: u64,
pub latencies: Vec<u64>, // sparkline
pub tokens_in: u64,
pub tokens_out: u64,
pub cost: f64,
}
impl Default for Metrics {
fn default() -> Self { Self::new() }
}
impl Metrics {
pub fn new() -> Self {
Self {
tokens_in: Default::default(),
tokens_out: Default::default(),
turns: Default::default(),
tool_calls: Default::default(),
per_tool: Mutex::new(HashMap::new()),
turn_latencies: Mutex::new(VecDeque::new()),
providers: Mutex::new(HashMap::new()),
}
}
pub fn record_turn(&self, usage: Option<&Usage>, latency_ms: u64) {
use std::sync::atomic::Ordering::Relaxed;
self.turns.fetch_add(1, Relaxed);
if let Some(u) = usage {
self.tokens_in.fetch_add(u.prompt_tokens as u64, Relaxed);
self.tokens_out.fetch_add(u.completion_tokens as u64, Relaxed);
}
let mut q = self.turn_latencies.lock().unwrap_or_else(|e| e.into_inner());
q.push_back(latency_ms);
while q.len() > WINDOW { q.pop_front(); }
}
pub fn record_tool_call(&self, name: &str, _success: bool) {
use std::sync::atomic::Ordering::Relaxed;
self.tool_calls.fetch_add(1, Relaxed);
*self.per_tool.lock().unwrap_or_else(|e| e.into_inner()).entry(name.to_string()).or_insert(0) += 1;
}
pub fn record_provider(&self, name: &str, model: &str, cost: Option<f64>, latency_ms: u64, is_error: bool) {
let mut map = self.providers.lock().unwrap_or_else(|e| e.into_inner());
let stat = map.entry(name.to_string()).or_insert_with(|| ProviderStat { model: model.to_string(), ..Default::default() });
stat.model = model.to_string();
stat.calls += 1;
stat.last_latency_ms = latency_ms;
stat.latencies.push_back(latency_ms);
while stat.latencies.len() > WINDOW { stat.latencies.pop_front(); }
stat.recent_results.push_back(is_error);
while stat.recent_results.len() > DEGRADE_WINDOW { stat.recent_results.pop_front(); }
if let Some(c) = cost { stat.cost += c; }
}
pub fn record_provider_tokens(&self, name: &str, usage: &Usage) {
let mut map = self.providers.lock().unwrap_or_else(|e| e.into_inner());
let stat = map.entry(name.to_string()).or_default();
stat.tokens_in += usage.prompt_tokens as u64;
stat.tokens_out += usage.completion_tokens as u64;
}
pub fn tool_call_count(&self, name: &str) -> u64 {
*self.per_tool.lock().unwrap_or_else(|e| e.into_inner()).get(name).unwrap_or(&0)
}
pub fn snapshot(&self) -> MetricsSnapshot {
use std::sync::atomic::Ordering::Relaxed;
let latencies = self.turn_latencies.lock().unwrap_or_else(|e| e.into_inner());
let p95 = percentile_95(&latencies);
let providers = self.providers.lock().unwrap_or_else(|e| e.into_inner());
let mut cost = 0.0;
let provider_snaps = providers.iter().map(|(name, s)| {
cost += s.cost;
let errors = s.recent_results.iter().filter(|e| **e).count();
ProviderSnapshot {
name: name.clone(),
model: s.model.clone(),
status: if s.recent_results.len() >= DEGRADE_WINDOW && errors >= DEGRADE_THRESHOLD { "degraded".into() } else { "ok".into() },
latency_ms: s.last_latency_ms,
latencies: s.latencies.iter().copied().collect(),
tokens_in: s.tokens_in,
tokens_out: s.tokens_out,
cost: s.cost,
}
}).collect();
MetricsSnapshot {
tokens_in: self.tokens_in.load(Relaxed),
tokens_out: self.tokens_out.load(Relaxed),
cost,
turns: self.turns.load(Relaxed),
tool_calls: self.tool_calls.load(Relaxed),
turn_latency_p95_ms: p95,
per_tool: self.per_tool.lock().unwrap_or_else(|e| e.into_inner()).clone(),
providers: provider_snaps,
}
}
}
fn percentile_95(values: &VecDeque<u64>) -> u64 {
if values.is_empty() { return 0; }
let mut sorted: Vec<u64> = values.iter().copied().collect();
sorted.sort_unstable();
let idx = ((sorted.len() as f64 * 0.95).ceil() as usize).saturating_sub(1).min(sorted.len() - 1);
sorted[idx]
}
static GLOBAL: OnceLock<Arc<Metrics>> = OnceLock::new();
/// Process-global metrics, shared across config reloads (same process).
pub fn global_metrics() -> Arc<Metrics> {
GLOBAL.get_or_init(|| Arc::new(Metrics::new())).clone()
}
```
（`Usage` 字段：`prompt_tokens`/`completion_tokens`/`total_tokens` 等，见 `src/providers/traits.rs:118-129`，`#[derive(Default)]`。`record_provider` 的 degraded 判定按「最近 DEGRADE_WINDOW 次中错误 ≥ DEGRADE_THRESHOLD」；测试需相应构造样本量。若测试断言与实现阈值不符，以实现为准调整测试样本。）
- [ ] **Step 4: 运行测试确认通过** — `cargo test --lib metrics` → PASS
- [ ] **Step 5: clippy** — `cargo clippy --all-targets --all-features -- -D warnings`
- [ ] **Step 6: Commit** — `git add src/observability && git commit -m "feat(observability): process-global Metrics collector"`
### Task 1.2: 配置可选价格字段
**Files:**
- Modify: `src/config/mod.rs`（`LLMProviderConfig` 结构体）
- [ ] **Step 1: 加可选字段** — 在 `LLMProviderConfig` 增加（serde 可选，缺省 None，不影响现有配置解析）：
```rust
#[serde(default)]
pub price_input_per_million: Option<f64>,
#[serde(default)]
pub price_output_per_million: Option<f64>,
```
- [ ] **Step 2: 加 cost 辅助方法**（在 LLMProviderConfig impl）：
```rust
pub fn cost_of(&self, prompt_tokens: u32, completion_tokens: u32) -> Option<f64> {
match (self.price_input_per_million, self.price_output_per_million) {
(Some(pi), Some(po)) => Some(prompt_tokens as f64 / 1e6 * pi + completion_tokens as f64 / 1e6 * po),
_ => None,
}
}
```
- [ ] **Step 3: 验证** — `cargo test --lib config` + `cargo clippy -- -D warnings`（确认现有配置测试仍过，新字段不破坏反序列化）。
- [ ] **Step 4: Commit** — `git add src/config/mod.rs && git commit -m "feat(config): optional provider pricing for cost metrics"`
### Task 1.3: AgentLoop 埋点（记录到全局 Metrics）
**Files:**
- Modify: `src/agent/agent_loop.rs`
埋点位置（来自探查）：
- 工具执行：`execute_one_tool`（约 970-1021 行），`tool_name` 已知（约 977 行），成功与否在 `result.success`（约 1004/1015 行）。在已有 observer `ObserverEvent::ToolCall` 附近加 `crate::observability::metrics::global_metrics().record_tool_call(&tool_name, success);`
- 模型调用延迟 + per-provider：`stream_completion`（约 488-525 行）是唯一模型调用入口。用 `Instant`（已 import）包裹 `self.provider.stream(request).await` 累加循环，得到 `latency_ms`；调用 `global_metrics().record_provider(self.provider.name(), self.provider.model_id(), cost, latency_ms, is_error)` 与 `record_provider_tokens(name, &response.usage)`。cost 从 provider config 取——但 AgentLoop 当前不持有 provider config 单价；**简化**：cost 传 None（除非能拿到 config）。若 AgentLoop 无法拿到单价，cost 由 record_provider 传 None（保持 0）。**注**：若需 cost，可在构造 AgentLoop 时把 `(price_in, price_out)` 一并传入；为降低侵入，P1 先传 None，cost 留待有 config 接入时补（在计划风险项注明）。
- turn 完成：在 `process_inner` 的三个返回点（约 706、847、879 行）已知 `accumulated_usage`；turn 延迟用进入 `process_inner` 时的 `Instant` 到返回的 elapsed。加 `global_metrics().record_turn(accumulated_usage_opt, latency_ms)`。
- [ ] **Step 1: 工具埋点** — 在 `execute_one_tool` 的成功/失败分支（已有 observer ToolCall 事件处）加 `record_tool_call(&tool_name, success)`。
- [ ] **Step 2: 模型调用埋点** — 在 `stream_completion` 用 Instant 测延迟，结束后 `record_provider(name, model, None, latency_ms, is_error)` + `record_provider_tokens(name, &usage)`。provider 名用 `self.provider.name()`，model 用 `self.provider.model_id()`（见 `src/providers/traits.rs:145-149`）。**注意：`stream_completion` 的错误通过 `?` 提前返回（约 494-497、500-503 行）——必须在这些错误路径也 `record_provider(..., is_error=true)`，否则 provider status 的 degraded 判定在生产中永远不触发（单测因直接调 record_provider 仍能过，会掩盖此 bug）。** 可用一个在成功/失败都执行的收尾闭包或在小函数返回前统一记录。
- [ ] **Step 3: turn 埋点** — 在 `process_inner` 入口记 `let turn_start = Instant::now();`，三个返回点前 `global_metrics().record_turn(usage_opt, turn_start.elapsed().as_millis() as u64)`。
- [ ] **Step 4: 验证** — `cargo test --lib`（现有 agent 测试仍过）+ `cargo clippy -- -D warnings` + `cargo build`。
- [ ] **Step 5: Commit** — `git add src/agent/agent_loop.rs && git commit -m "feat(agent): record tool/turn/provider metrics"`
---
## Chunk 2: 内省接口 + 端点
### Task 2.1: 服务内省方法
**Files:**
- Modify: `src/bus/mod.rs`（MessageBus 队列深度）
- Modify: `src/bus/dispatcher.rs`（active lane 计数）
- Modify: `src/task_supervisor.rs`（运行任务数）
- Modify: `src/session/session.rs`（会话数 + 活动 turn 数）
- Modify: `src/gateway/ws.rs`（ws 连接计数）
- Modify: `src/gateway/mod.rs`（uptime 静态 + ws_connections 字段 + dispatcher lane 计数注入）
- [ ] **Step 1: MessageBus 队列深度** — 在 `src/bus/mod.rs` 加：
```rust
pub fn queue_depths(&self) -> QueueDepths {
QueueDepths {
inbound_depth: (self.inbound_tx.max_capacity() - self.inbound_tx.capacity()) as u64,
inbound_cap: self.inbound_tx.max_capacity() as u64,
outbound_depth: (self.outbound_tx.max_capacity() - self.outbound_tx.capacity()) as u64,
outbound_cap: self.outbound_tx.max_capacity() as u64,
control_depth: (self.control_tx.max_capacity() - self.control_tx.capacity()) as u64,
control_cap: self.control_tx.max_capacity() as u64,
}
}
```
并定义 `#[derive(serde::Serialize)] pub struct QueueDepths { pub inbound_depth: u64, pub inbound_cap: u64, pub outbound_depth: u64, pub outbound_cap: u64, pub control_depth: u64, pub control_cap: u64 }`。（模式参考 `session.rs:2139` 已用 `capacity()`/`max_capacity()`。）
- [ ] **Step 2: OutboundDispatcher active lane 计数** — 给 `OutboundDispatcher` 加字段 `active_lanes: Arc<std::sync::atomic::AtomicUsize>`，构造时注入（在 gateway 创建 dispatcher 处 `Arc::new(AtomicUsize::new(0))`）。**计数增减位置（避免 spawn 失败泄漏）**：把该 `Arc` clone 进 lane 任务，在**已生成的 lane 任务体起始处** `fetch_add(1, Relaxed)`，在 lane 循环的**唯一退出口**（约 131-146 行 break 之后、任务结束前）`fetch_sub(1, Relaxed)`——不要在调用 `spawn_lane` 之前 +1（spawn 可能失败）。在 gateway 侧把同一 `Arc` clone 存入 GatewayState 新字段 `pub outbound_lanes: Arc<std::sync::atomic::AtomicUsize>`（`from_config` 字面量初始化），供 `/api/status` 直接 `state.outbound_lanes.load(Relaxed)` 读取（无需额外方法）。
- [ ] **Step 3: TaskSupervisor 运行任务数** — 在 `src/task_supervisor.rs` 加：
```rust
pub fn running_count(&self) -> usize {
let mut state = self.inner.state.lock().unwrap_or_else(|e| e.into_inner());
state.tasks.retain(|task| !task.handle.is_finished());
state.tasks.len()
}
```
- [ ] **Step 4: SessionManager 会话数 + 活动 turn 数** — 在 `src/session/session.rs` 加（镜像 `wait_until_idle` 约 2125-2158 的判定）：
```rust
pub async fn session_count(&self) -> usize {
self.inner.lock().await.sessions.len()
}
pub async fn active_turn_count(&self) -> usize {
let sessions: Vec<_> = self.inner.lock().await.sessions.values().cloned().collect();
let mut count = 0;
for session in sessions {
let s = session.lock().await;
if s.current_cancel.is_some() { count += 1; }
}
count
}
```
（`inner` 是 `Arc<Mutex<SessionManagerInner>>`，`sessions: HashMap<String, Arc<Mutex<Session>>>`；`Session.current_cancel: Option<oneshot::Sender>` 表示 turn 执行中。锁均为 tokio Mutex；纯内存计数，不跨 I/O 持锁，符合不变量。）
- [ ] **Step 5: ws 连接计数 + uptime 静态** — 在 `src/gateway/mod.rs`：
- 加 `static STARTED: OnceLock<std::time::Instant> = OnceLock::new();` 与 `pub fn process_uptime_secs() -> u64 { STARTED.get_or_init(Instant::now).elapsed().as_secs() }`（在 `run()` 起始调用一次 `STARTED.get_or_init(Instant::now)`）。
- GatewayState 加 `pub ws_connections: Arc<std::sync::atomic::AtomicUsize>`（在 `from_config` 结构体字面量初始化 `Default::default()`）。
- 在 `src/gateway/ws.rs` 的 `handle_socket`（约 39-132 行）：进入时 `state.ws_connections.fetch_add(1, Relaxed)`，在函数尾部清理处（约 120-131 行）`fetch_sub(1, Relaxed)`。为保证多 break 路径都减，用一个 RAII guard 或在唯一尾部清理段减（确认 handle_socket 是否有单一出口；若多出口，用 guard struct `Drop` 减）。
- [ ] **Step 6: 验证** — `cargo build` + `cargo test --lib` + `cargo clippy -- -D warnings`。
- [ ] **Step 7: Commit** — `git add src/bus src/task_supervisor.rs src/session/session.rs src/gateway && git commit -m "feat(gateway): runtime introspection for status endpoint"`
### Task 2.2: GET /api/status 聚合 handler
**Files:**
- Modify: `src/gateway/http.rs`（handler）
- Modify: `src/gateway/mod.rs`（注册路由）
- [ ] **Step 1: handler** — 在 `src/gateway/http.rs` 加（聚合所有来源；用 `serde_json::json!`）：
```rust
pub async fn get_status(State(state): State<Arc<GatewayState>>) -> Result<Json<Value>, ApiError> {
let reload = state.reload.status();
let metrics = crate::observability::metrics::global_metrics().snapshot();
let depths = state.bus().queue_depths();
let active_lanes = state.outbound_lanes.load(std::sync::atomic::Ordering::Relaxed);
let ws_connections = state.ws_connections.load(std::sync::atomic::Ordering::Relaxed);
let background_tasks = state.task_supervisor.running_count();
let sessions_total = state.session_manager.session_count().await;
let active_turns = state.session_manager.active_turn_count().await;
// 渠道状态
let mut channels = Vec::new();
for name in state.channel_manager.list_channel_names().await {
let running = state.channel_manager.get_channel(&name).await.map(|c| c.is_running()).unwrap_or(false);
channels.push(json!({ "name": name, "status": if running { "connected" } else { "stopped" } }));
}
// 调度器（失败数用 last_status 近似）
let jobs = state.storage.list_scheduled_jobs().await.map_err(ApiError::internal)?;
let failed_jobs = jobs.iter().filter(|j| matches!(j.last_status.as_deref(), Some("error") | Some("timeout") | Some("delivery_error"))).count();
let enabled = jobs.iter().filter(|j| j.enabled).count();
let next_run = jobs.iter().filter(|j| j.enabled).map(|j| j.next_run_at).min();
Ok(Json(json!({
"generation": reload.generation,
"version": env!("CARGO_PKG_VERSION"),
"uptime_secs": crate::gateway::process_uptime_secs(),
"phase": reload.phase,
"ws_connections": ws_connections,
"background_tasks": background_tasks,
"sessions": { "total": sessions_total, "active_turns": active_turns },
"metrics": {
"tokens_in": metrics.tokens_in,
"tokens_out": metrics.tokens_out,
"cost": metrics.cost,
"tool_calls": metrics.tool_calls,
"turns": metrics.turns,
"turn_latency_p95_ms": metrics.turn_latency_p95_ms,
},
"bus": {
"inbound": { "depth": depths.inbound_depth, "cap": depths.inbound_cap },
"outbound": { "depth": depths.outbound_depth, "cap": depths.outbound_cap },
"control": { "depth": depths.control_depth, "cap": depths.control_cap },
"active_lanes": active_lanes,
},
"providers": metrics.providers,
"channels": channels,
"scheduler": { "jobs": jobs.len(), "enabled": enabled, "failed_jobs": failed_jobs, "next_run_at": next_run },
"mcp": crate::mcp::get_mcp_status().iter().map(|s| json!({ "name": s.name, "connected": s.connected, "tools": s.tools.len() })).collect::<Vec<_>>(),
})))
}
```
（`ReloadPhase` 已 `Serialize`（snake_case）；`McpServerStatus` 字段见 `src/mcp/mod.rs:27-34`。`state.bus()`/`channel_manager`/`task_supervisor`/`session_manager`/`storage`/`reload` 均为 GatewayState 字段；`outbound_lane_count()` 按 Task 2.1 实际暴露方式调用。）
- [ ] **Step 2: 注册路由** — 在 `src/gateway/mod.rs` 的 protected router（约 574-575 行 `/api/tasks`/`/api/jobs` 附近）加 `.route("/api/status", routing::get(http::get_status))`。确认在 `route_layer(require_auth)` 之内（设备鉴权）。
- [ ] **Step 3: 验证** — `cargo build` + `cargo clippy -- -D warnings`。（可选：`cargo run -- gateway` 后 `curl` 需鉴权，目检 JSON 结构；或写一个轻量集成断言。无鉴权 token 时跳过 curl，靠编译 + 结构审查。）
- [ ] **Step 4: Commit** — `git add src/gateway && git commit -m "feat(gateway): GET /api/status runtime snapshot"`
### Task 2.3: GET /api/tools handler
**Files:**
- Modify: `src/gateway/http.rs`
- Modify: `src/gateway/mod.rs`（路由）
- [ ] **Step 1: handler** — 遍历 `state.session_manager.tools().iter()`（返回 `Vec<(String, Arc<dyn Tool>)>`，见 `src/tools/registry.rs:69-76`）。source 由命名约定派生：名字含 `__` → `"mcp"`（MCP 工具名为 `{server}__{tool}`，见 `src/mcp/tool_wrapper.rs:26`），否则 `"builtin"`。call_count 取 `global_metrics().tool_call_count(&name)`。
```rust
pub async fn get_tools(State(state): State<Arc<GatewayState>>) -> Result<Json<Value>, ApiError> {
let metrics = crate::observability::metrics::global_metrics();
let tools: Vec<Value> = state.session_manager.tools().iter().iter().map(|(name, tool)| {
let source = if name.contains("__") { "mcp" } else { "builtin" };
json!({
"name": name,
"description": tool.description(),
"parameters_schema": tool.parameters_schema(),
"source": source,
"read_only": tool.read_only(),
"exclusive": tool.exclusive(),
"concurrency_safe": tool.concurrency_safe(),
"call_count": metrics.tool_call_count(name),
})
}).collect();
Ok(Json(json!({ "tools": tools })))
}
```
- [ ] **Step 2: 注册路由** — protected router 加 `.route("/api/tools", routing::get(http::get_tools))`。
- [ ] **Step 3: 验证** — `cargo build` + `cargo clippy -- -D warnings`。
- [ ] **Step 4: Commit** — `git add src/gateway && git commit -m "feat(gateway): GET /api/tools with capability metadata"`
### Task 2.4: GET /api/skills handler（+ SkillsLoader 访问器）
**Files:**
- Modify: `src/session/session.rs`（暴露 SkillsLoader 访问器）
- Modify: `src/gateway/http.rs`
- Modify: `src/gateway/mod.rs`（路由）
- [ ] **Step 1: 暴露 SkillsLoader** — `SessionManager` 私有字段 `skills_loader: Arc<SkillsLoader>`（`session.rs:1423`）无公开访问器。加：
```rust
pub fn skills_loader(&self) -> Arc<crate::skills::SkillsLoader> {
self.skills_loader.clone()
}
```
- [ ] **Step 2: source 派生辅助** — `Skill.path` 是技能目录；source 由路径前缀派生（`~/.agents/skills`→`agent`，`~/.picobot/skills`→`picobot`，`{workspace}/skills`→`workspace`）。由于三个目录字段是 SkillsLoader 私有，**简化**：在 handler 里用 `Skill.path` 的字符串包含关系粗略派生（包含 `/.agents/skills`→agent，`/.picobot/skills`→picobot，否则 workspace/other）。或在 SkillsLoader 加 `pub fn source_of(&self, path) -> &str` 辅助（更干净，推荐）。选择在 SkillsLoader 加辅助方法。
- [ ] **Step 3: handler** — 用 `get_loaded_skills()`（`src/skills/mod.rs:252`，返回 `Vec<Skill>`，**不要**用 `list_skills()`）。默认不返回 `content`。
```rust
pub async fn get_skills(State(state): State<Arc<GatewayState>>) -> Result<Json<Value>, ApiError> {
let loader = state.session_manager.skills_loader();
let skills: Vec<Value> = loader.get_loaded_skills().iter().map(|s| json!({
"name": s.name,
"description": s.description,
"always": s.always,
"source": loader.source_of(s.path.as_deref()),
})).collect();
Ok(Json(json!({ "skills": skills })))
}
```
- [ ] **Step 4: 注册路由** — protected router 加 `.route("/api/skills", routing::get(http::get_skills))`。
- [ ] **Step 5: 验证** — `cargo build` + `cargo test --lib` + `cargo clippy -- -D warnings`。
- [ ] **Step 6: Commit** — `git add src/session/session.rs src/skills/mod.rs src/gateway && git commit -m "feat(gateway): GET /api/skills"`
---
## Chunk 3: 前端
### Task 3.1: 可视化组件（Sparkline / CapacityMeter / MetricTile）
**Files:**
- Create: `webui/src/lib/components/Sparkline.svelte`、`CapacityMeter.svelte`、`MetricTile.svelte`
- [ ] **Step 1: Sparkline.svelte** — props `values: number[]`、`color: string`（默认 `var(--signal)`）。手写 SVG：归一化 values 到 viewBox，画折线/柱。无第三方库。
- [ ] **Step 2: CapacityMeter.svelte** — props `depth: number`、`cap: number`、`segments?: number`（默认 8）。按 depth/cap 比例点亮分段块；接近满（>90%）用 `var(--accent)`/`var(--danger)`，否则 `var(--signal)`。
- [ ] **Step 3: MetricTile.svelte** — props `label`、`value`、`sub`（可选小字）、`sparkValues`/`sparkColor`（可选）。大等宽数字（`var(--font-mono)`）+ label-caps + 可选 sparkline。用 `.panel` 容器。
- [ ] **Step 4: 验证** — `cd webui && npm run check && npm run build`。
- [ ] **Step 5: Commit** — `git add webui/src/lib/components && git commit -m "feat(webui): sparkline, capacity meter, metric tile components"`
### Task 3.2: 概览页（轮询 /api/status）
**Files:**
- Create: `webui/src/pages/OverviewPage.svelte`
- Modify: `webui/src/lib/api.js`（如需 status 辅助；现有 `api()` 通用函数已够，可复用）
布局（规格 §6.2，参考 `.superpowers/brainstorm/111044-1784795642/pages-runtime.html` mockup）：
- 主状态条：`RUNNING`（phase=active 时青绿脉冲）、运行代、uptime（格式化为 Xd Xh）、版本、WS 连接、后台任务、上次重载相位。
- 指标块行（MetricTile）：会话数、今日 Token（in+out，+cost sub）、工具调用、今日 Turns（+p95 sub）。
- Provider 表：名称、模型、状态徽标（ok=signal/degraded=accent）、延迟 Sparkline、token、cost。
- 消息总线：inbound/outbound/control 三个 CapacityMeter + 活跃 lane 数 + 调度器状态 + MCP 连接数。
- 渠道状态列表：name + 状态徽标。
- 调度器：jobs/enabled/failed_jobs/next_run（格式化为倒计时或时间）。
- 数据：`onMount` 起 `setInterval` 每 2s `api("/api/status")`；卸载清除。用 `$state` 持有 status，`$derived` 派生展示值。错误时显示离线态。
- [ ] **Step 1: 实现 OverviewPage.svelte**（轮询 + 上述布局，套用 Signal Deck tokens 与 P0 组件）。
- [ ] **Step 2: 验证** — `npm run check && npm run build`。
- [ ] **Step 3: Commit** — `git add webui/src/pages/OverviewPage.svelte && git commit -m "feat(webui): overview runtime dashboard"`
### Task 3.3: 工具 & Skills 页
**Files:**
- Create: `webui/src/pages/ToolsPage.svelte`
布局（规格 §6.3，参考 `.superpowers/brainstorm/111044-1784795642/page-tools-v2.html` mockup）：
- 三标签：工具 / Skills / MCP。
- 工具标签：搜索框 + 能力筛选 chips（全部/只读/可并发/有副作用/独占）+ 图例；工具卡片网格：名称、来源徽标（builtin/mcp）、能力徽标（◇只读=signal / ⇉可并发=info / △有副作用=accent / ■独占=danger）、调用次数、描述、可展开 `<details>` 参数 schema（`<pre>` mono）。
- Skills 标签：名称、描述、always 徽标、来源。
- MCP 标签：服务器名 + 连接状态徽标 + 工具数。
- 数据：`onMount` 拉 `api("/api/tools")` + `api("/api/skills")`；MCP 从 `api("/api/status")` 的 `mcp` 字段取（一次即可，MCP 状态为连接时快照）。能力筛选与搜索为前端纯派生（`$derived` 过滤）。
- [ ] **Step 1: 实现 ToolsPage.svelte**（三标签 + 搜索 + 能力筛选 + 卡片）。
- [ ] **Step 2: 验证** — `npm run check && npm run build`。
- [ ] **Step 3: Commit** — `git add webui/src/pages/ToolsPage.svelte && git commit -m "feat(webui): tools and skills browser"`
### Task 3.4: App.svelte 接线（替换占位）
**Files:**
- Modify: `webui/src/App.svelte`
- [ ] **Step 1: 接线** — import `OverviewPage` 与 `ToolsPage`；把 `overview`/`tools` 的 `{:else}` 占位（`即将上线`）替换为对应页面渲染（`{:else if current === "overview"}<OverviewPage />` `{:else if current === "tools"}<ToolsPage />`）。保留其余页面分支。
- [ ] **Step 2: 验证** — `npm run check && npm run build`。
- [ ] **Step 3: 目检**（可选，需 gateway）— 概览页显示实时指标、工具页列出工具/Skills/MCP、亮暗主题正常。
- [ ] **Step 4: Commit** — `git add webui/src/App.svelte && git commit -m "feat(webui): wire overview and tools pages"`
---
## Chunk 4: 收尾
### Task 4.1: P1 收尾验证 + 版本号
- [ ] **Step 1: 全量验证**
- `cd webui && npm run check && npm run build`
- `cargo build`
- `cargo test --lib`
- `cargo clippy --all-targets --all-features -- -D warnings`
Expected: 全部通过。
- [ ] **Step 2: 版本号** — `Cargo.toml` 与 `webui/package.json` minor bump（1.4.0 → 1.5.0）。检查 README 是否有需更新的观测能力描述（如有则最小化更新）。
- [ ] **Step 3: Commit** — `git add -A && git commit -m "chore(release): P1 observability"`（仅暂存版本号文件 + 可能的 README；确认无构建产物）。
---
## P1 完成标志
- `GET /api/status` 返回完整运行快照（generation/uptime/sessions/metrics/bus/providers/channels/scheduler/mcp/ws_connections/background_tasks），设备鉴权保护。
- `GET /api/tools` 返回工具 + 能力字段（read_only/exclusive/concurrency_safe）+ call_count + source。
- `GET /api/skills` 返回 name/description/always/source（不含 content）。
- 概览页每 2s 轮询并渲染仪表盘；工具页三标签 + 搜索 + 能力筛选。
- Metrics 进程级、跨重载存活、纯内存。
- `npm run check`、`npm run build`、`cargo build`、`cargo test --lib`、`cargo clippy -- -D warnings` 全绿。
## 风险与开放项
- **cost 暂未接入单价**：AgentLoop 埋点 cost 传 None（拿不到 provider config 单价），故 `/api/status` 的 cost 恒为 0，除非后续把 `(price_in, price_out)` 传入 AgentLoop。可选价格字段已加（Task 1.2），接线留待需要时补。若评审认为 P1 必须出 cost，则在 Task 1.3 把单价随 AgentLoop 构造传入（需从 Session 持有的 provider config 取）。
- **failed_jobs 为近似**：用 last_status 而非严格 7 天窗口（避免 2s 轮询逐任务查运行记录）。UI 需标注语义。
- **active_lanes 计数**：依赖 dispatcher lane 生成/结束正确 +1/-1；多 break 路径需保证减计数（用 guard 或单一出口）。
- **Metrics 全局态**：选择进程级全局（precedented by MCP），非穿线；若评审偏好显式依赖注入，可改为经 SessionManagerServices 穿线（成本：5 个 AgentLoop 构造点 + 多处结构体字段）。
- **provider status 派生阈值**：最近 10 次 ≥3 失败 → degraded；阈值可调。

413
docs/superpowers/plans/2026-07-26-p2-logs-data.md
View File

@ -1,413 +0,0 @@
# P2 日志与数据 Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 为 PicoBot WebUI 增加实时日志流（tracing 广播层 + `/ws/logs`）、记忆写入端点（PUT/DELETE）、以及日志页/记忆页/任务页的前端重构。
**Architecture:** 新增 tracing 广播层（自定义 `Layer` impl，格式化后发送到 `tokio::sync::broadcast`，容量 1024，慢客户端丢旧不反压）。全局 `OnceLock<broadcast::Sender<LogEvent>>` 在 `init_logging()` 时初始化。`/ws/logs` 独立 WebSocket 端点复用现有 `require_auth` 中间件，连接后订阅广播、按 level/search 过滤推送。记忆写入复用已有 `Storage::upsert_memory`/`delete_memory`。前端三页重构为 Svelte 5 runes 组件。
**Tech Stack:** Rust（tracing-subscriber Layer trait、tokio broadcast、Axum WebSocket）、Svelte 5（runes）、bits-ui。
**关键设计决策（务必遵守）：**
1. **广播层全局**：`src/logging.rs` 增加 `static LOG_TX: OnceLock<broadcast::Sender<LogEvent>>` + `pub fn log_sender() -> Option<broadcast::Sender<LogEvent>>`。`init_logging()` 初始化 broadcast channel 并挂载自定义 Layer。无订阅者时 send 为 no-op。
2. **LogEvent 结构**：`#[derive(Clone)] pub struct LogEvent { pub ts: String, pub level: String, pub target: String, pub message: String }`。ts 为 RFC 3339。level 为 "TRACE"/"DEBUG"/"INFO"/"WARN"/"ERROR"。
3. **`/ws/logs` 独立于 `/ws`**：不与聊天 WS 复用连接。路由注册在 protected router 内（`require_auth`），handler 签名与 `ws_handler` 类似但更简单（无 session 注册）。查询参数 `level`（可选，最低级别过滤）和 `search`（可选，关键字）。
4. **记忆 PUT 语义**：`PUT /api/memories/{key}` body `{content: string, importance?: f64}`。若 key 已存在则更新 content/importance/updated_at；若不存在则 404（不创建新条目——创建由 agent 内部完成）。需先查询 key 是否存在。
5. **记忆 DELETE 语义**：`DELETE /api/memories/{key}` 删除后返回 `{"deleted": true}`。key 不存在也返回 200（幂等）。
6. **GET /api/logs 保留**：文件尾读取不变，用于进入页面时拉历史与重连对齐。
7. **前端日志页**：进入时先 `GET /api/logs` 拉历史尾 → 建立 `/ws/logs` 接管实时；断线重连重新拉尾对齐。level 过滤（全部/INF/WRN/ERR）、关键字搜索、暂停滚动、下载。
8. **前端记忆页**：行内编辑（textarea + importance 滑块 + 保存/取消）、删除确认分级（Knowledge 普通 / Timeline 强警告）、搜索 + 分类筛选 + 分页加载。
9. **前端任务页**：定时任务增加 cron 表达式显示、下次运行倒计时、最近运行状态点（绿/琥珀/红）；后台任务增加脉冲动画。
**参考：** 规格 `docs/superpowers/specs/2026-07-23-webui-refactor-design.md` §6.4/§6.5/§6.6/§7.4/§7.5；P1 计划 `docs/superpowers/plans/2026-07-24-p1-observability.md`（前端组件/页面模式）。
**验证约定：** Rust 改动 → 定向测试 + `cargo test --lib` + `cargo clippy --all-targets --all-features -- -D warnings` + `cargo build`；前端 → `cd webui && npm run check && npm run build`。前端无单测框架，勿虚构。
---
## Chunk 1: 后端 — tracing 广播 + /ws/logs
### Task 1.1: LogEvent + 广播 Layer + 全局访问器
**Files:**
- Modify: `src/logging.rs`
- [ ] **Step 1: 定义 LogEvent 与全局 Sender**
在 `src/logging.rs` 顶部增加：
```rust
use tokio::sync::broadcast;
use std::sync::OnceLock;
use tracing::field::Visit;
use tracing_subscriber::layer::Context;
use tracing_subscriber::Layer;
const LOG_BROADCAST_CAP: usize = 1024;
#[derive(Clone, Debug)]
pub struct LogEvent {
pub ts: String,
pub level: String,
pub target: String,
pub message: String,
}
static LOG_TX: OnceLock<broadcast::Sender<LogEvent>> = OnceLock::new();
pub fn log_sender() -> Option<broadcast::Sender<LogEvent>> {
LOG_TX.get().cloned()
}
```
- [ ] **Step 2: 实现自定义 Layer**
```rust
struct BroadcastLayer;
struct MessageVisitor(String);
impl Visit for MessageVisitor {
fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) {
if field.name() == "message" {
self.0 = format!("{:?}", value);
}
}
fn record_str(&mut self, field: &tracing::field::Field, value: &str) {
if field.name() == "message" {
self.0 = value.to_string();
}
}
}
impl<S: tracing::Subscriber> Layer<S> for BroadcastLayer {
fn on_event(&self, event: &tracing::Event<'_>, _ctx: Context<'_, S>) {
let Some(tx) = LOG_TX.get() else { return };
if tx.receiver_count() == 0 { return; }
let mut visitor = MessageVisitor(String::new());
event.record(&mut visitor);
let metadata = event.metadata();
let _ = tx.send(LogEvent {
ts: chrono::Local::now().to_rfc3339(),
level: metadata.level().to_string(),
target: metadata.target().to_string(),
message: visitor.0,
});
}
}
```
注意：需确认 `chrono` 是否已在依赖中。若无需添加 `chrono = "0.4"` 到 Cargo.toml。或者用 `time` crate（tracing-subscriber 的 `local-time` feature 已引入 `time`）。优先用 `time::OffsetDateTime::now_local()` 格式化为 RFC 3339，避免新增依赖。
- [ ] **Step 3: 修改 init_logging() 挂载广播层**
在 `init_logging()` 中，`tracing_subscriber::registry()` 链增加 `.with(BroadcastLayer)`，并在函数开头初始化 broadcast：
```rust
pub fn init_logging() {
let (tx, _rx) = broadcast::channel(LOG_BROADCAST_CAP);
let _ = LOG_TX.set(tx);
// ... 现有代码 ...
tracing_subscriber::registry()
.with(env_filter)
.with(console_layer)
.with(file_layer)
.with(BroadcastLayer)
.init();
// ...
}
```
- [ ] **Step 4: 验证** — `cargo build` + `cargo clippy --all-targets --all-features -- -D warnings`
- [ ] **Step 5: Commit** — `git add src/logging.rs Cargo.toml && git commit -m "feat(logging): tracing broadcast layer for real-time log streaming"`
### Task 1.2: /ws/logs WebSocket 端点
**Files:**
- Modify: `src/gateway/ws.rs`（或新建 `src/gateway/ws_logs.rs`）
- Modify: `src/gateway/mod.rs`（路由注册）
- [ ] **Step 1: 实现 ws_logs handler**
在 `src/gateway/ws.rs` 底部（或新文件）增加：
```rust
#[derive(Debug, Default, Deserialize)]
pub struct WsLogsQuery {
level: Option<String>,
search: Option<String>,
}
pub async fn ws_logs_handler(
ws: WebSocketUpgrade,
Query(query): Query<WsLogsQuery>,
Extension(_identity): Extension<super::auth::AuthIdentity>,
) -> Response {
ws.on_upgrade(|socket| async move {
handle_logs_socket(socket, query).await;
})
}
async fn handle_logs_socket(ws: WebSocket, query: WsLogsQuery) {
let Some(tx) = crate::logging::log_sender() else { return };
let mut rx = tx.subscribe();
let (mut ws_sender, mut ws_receiver) = ws.split();
let min_level = query.level.as_deref().map(parse_min_level).unwrap_or(0);
let search = query.search.filter(|s| !s.is_empty()).map(|s| s.to_ascii_lowercase());
loop {
tokio::select! {
result = rx.recv() => {
match result {
Ok(event) => {
if level_rank(&event.level) < min_level { continue; }
if let Some(needle) = &search {
if !event.message.to_ascii_lowercase().contains(needle)
&& !event.target.to_ascii_lowercase().contains(needle) {
continue;
}
}
let json = serde_json::json!({
"ts": event.ts,
"level": event.level,
"target": event.target,
"message": event.message,
});
if ws_sender.send(WsMessage::Text(json.to_string().into())).await.is_err() {
break;
}
}
Err(broadcast::error::RecvError::Lagged(_)) => continue,
Err(broadcast::error::RecvError::Closed) => break,
}
}
msg = ws_receiver.next() => {
match msg {
Some(Ok(WsMessage::Close(_))) | None => break,
_ => {}
}
}
}
}
}
fn parse_min_level(level: &str) -> u8 {
match level.to_ascii_uppercase().as_str() {
"DEBUG" => 1,
"INFO" => 2,
"WARN" => 3,
"ERROR" => 4,
_ => 0,
}
}
fn level_rank(level: &str) -> u8 {
match level.as_str() {
"TRACE" => 0,
"DEBUG" => 1,
"INFO" => 2,
"WARN" => 3,
"ERROR" => 4,
_ => 2,
}
}
```
- [ ] **Step 2: 注册路由** — 在 `src/gateway/mod.rs` 的 protected router（`route_layer(require_auth)` 之内）加：
```rust
.route("/ws/logs", routing::get(ws::ws_logs_handler))
```
放在 `/ws` 路由附近。
- [ ] **Step 3: 验证** — `cargo build` + `cargo clippy --all-targets --all-features -- -D warnings`
- [ ] **Step 4: Commit** — `git add src/gateway && git commit -m "feat(gateway): /ws/logs real-time log streaming endpoint"`
---
## Chunk 2: 后端 — 记忆写入端点
### Task 2.1: PUT /api/memories/{key} + DELETE /api/memories/{key}
**Files:**
- Modify: `src/gateway/http.rs`
- Modify: `src/gateway/mod.rs`（路由）
- [ ] **Step 1: PUT handler**
```rust
#[derive(Deserialize)]
pub struct PutMemoryBody {
content: String,
importance: Option<f64>,
}
pub async fn put_memory(
State(state): State<Arc<GatewayState>>,
Path(key): Path<String>,
Json(body): Json<PutMemoryBody>,
) -> Result<Json<Value>, ApiError> {
let existing = state.storage
.list_memories(None, None, 1)
.await
.map_err(ApiError::internal)?;
// 需要按 key 查询——检查是否有 get_memory_by_key 方法
// 若无，用 search 或直接 SQL 查询
// 简化：直接用 upsert，但需先确认 key 存在
// 实际实现：查询 SELECT * FROM memories WHERE key = ?
// 若不存在返回 404
// 若存在，更新 content/importance/updated_at
...
}
```
**注意**：当前 Storage 无 `get_memory_by_key` 方法。需在 `src/storage/memory.rs` 增加：
```rust
pub async fn get_memory_by_key(&self, key: &str) -> Result<Option<MemoryEntry>, StorageError> {
let row = sqlx::query(
"SELECT id, key, content, category, importance, session_id, created_at, updated_at FROM memories WHERE key = ?"
)
.bind(key)
.fetch_optional(self.pool())
.await?;
match row {
Some(row) => Ok(Some(parse_memory_row(&row)?)),
None => Ok(None),
}
}
```
PUT handler 完整逻辑：
1. `get_memory_by_key(&key)` → None → 404
2. 存在 → 构造更新后的 MemoryEntry（保留 id/category/session_id/created_at，更新 content/importance/updated_at）
3. `upsert_memory(&entry)` → 200 `{"updated": true, "key": key}`
- [ ] **Step 2: DELETE handler**
```rust
pub async fn delete_memory(
State(state): State<Arc<GatewayState>>,
Path(key): Path<String>,
) -> Result<Json<Value>, ApiError> {
state.storage.delete_memory(&key).await.map_err(ApiError::internal)?;
Ok(Json(json!({ "deleted": true })))
}
```
- [ ] **Step 3: 注册路由** — protected router 加：
```rust
.route("/api/memories/{key}", routing::put(http::put_memory).delete(http::delete_memory))
```
注意：现有 `GET /api/memories`（无 path param）不受影响。
- [ ] **Step 4: 验证** — `cargo build` + `cargo test --lib` + `cargo clippy --all-targets --all-features -- -D warnings`
- [ ] **Step 5: Commit** — `git add src/storage/memory.rs src/gateway && git commit -m "feat(gateway): PUT/DELETE /api/memories/{key} write endpoints"`
---
## Chunk 3: 前端 — 日志页重构
### Task 3.1: LogsPage 实时流式重构
**Files:**
- Modify: `webui/src/pages/LogsPage.svelte`
布局（规格 §6.4）：
- 工具栏：level 过滤 chips（全部/INF/WRN/ERR）、关键字搜索、暂停滚动按钮、下载按钮。
- 连接状态条：「实时推送中 · N 行/分」或「已断开 — 重连中」。
- 日志行：时间戳 + level 着色（INF=signal, WRN=accent, ERR=danger, DBG=muted）+ target + 消息。
- 自动跟随尾部（除非暂停）。
- 进入页面：`GET /api/logs?lines=200` 拉历史尾 → 建立 `/ws/logs` WebSocket 接管实时。
- 断线重连：重新拉尾对齐。
- 下载：将当前 lines 导出为 .log 文件（Blob + URL.createObjectURL）。
数据流：
- `onMount`：先 `api("/api/logs?lines=200")` 填充历史 → 建立 WS `new WebSocket(\`${wsBase}/ws/logs?level=...\`)`。
- WS `onmessage`：解析 JSON `{ts, level, target, message}`，追加到 `lines` 数组（上限 2000 行，超出 shift）。
- WS `onclose`：设 disconnected 状态，3s 后重连（重新拉尾 + 重建 WS）。
- level/search 过滤：前端 `$derived` 过滤已存储行（WS 查询参数在连接时固定；切换 filter 需重建 WS 或纯前端过滤——**选择纯前端过滤**，WS 不带 filter 参数，简化重连逻辑）。
- 暂停：`paused` 状态，暂停时不自动滚动到底部，新行仍追加。
- [ ] **Step 1: 重写 LogsPage.svelte**（上述布局与数据流）。
- [ ] **Step 2: 验证** — `cd webui && npm run check && npm run build`。
- [ ] **Step 3: Commit** — `git add webui/src/pages/LogsPage.svelte && git commit -m "feat(webui): real-time streaming logs page"`
---
## Chunk 4: 前端 — 记忆页重构
### Task 4.1: MemoryPage 可编辑/可删除重构
**Files:**
- Modify: `webui/src/pages/MemoryPage.svelte`
布局（规格 §6.5）：
- 统计条：总量 / Knowledge / Timeline。
- 搜索框 + 分类筛选（全部/Knowledge/Timeline）+ 排序（最近更新）。
- 记忆卡片列表：key（mono accent）、content、category badge、importance、updated_at、session_id。
- 行内编辑：点击「编辑」→ content 变 textarea + importance range input + 保存/取消按钮。保存调 `PUT /api/memories/{key}`。
- 删除：点击「删除」→ 确认对话框。Knowledge 普通确认；Timeline 强警告（红色面板 + 说明文字 + 按钮「我了解，确认删除」）。调 `DELETE /api/memories/{key}`。
- 分页：limit=100，底部「加载更多」按钮（offset 通过增大 limit 实现——现有 API 不支持 offset，用 limit 递增近似）。
- [ ] **Step 1: 重写 MemoryPage.svelte**（上述布局）。
- [ ] **Step 2: 验证** — `cd webui && npm run check && npm run build`。
- [ ] **Step 3: Commit** — `git add webui/src/pages/MemoryPage.svelte && git commit -m "feat(webui): editable memory page with delete confirmation"`
---
## Chunk 5: 前端 — 任务页重构
### Task 5.1: TasksPage 增强
**Files:**
- Modify: `webui/src/pages/TasksPage.svelte`
布局（规格 §6.6）：
- 定时任务卡片增强：
- 显示 cron 表达式（mono）。
- 下次运行倒计时（「3 分钟后」格式，每 30s 刷新）。
- 最近运行状态点（最多 10 个圆点：绿=completed / 琥珀=timeout / 红=error）。
- 可展开运行记录（已有，保留）。
- 后台任务增强：
- 运行中状态用脉冲动画（`.pulse` class）。
- 显示耗时（created_at 到现在的差值，或 duration 字段）。
- [ ] **Step 1: 增强 TasksPage.svelte**（上述改动，保留现有结构）。
- [ ] **Step 2: 验证** — `cd webui && npm run check && npm run build`。
- [ ] **Step 3: Commit** — `git add webui/src/pages/TasksPage.svelte && git commit -m "feat(webui): enhanced tasks page with countdown and status dots"`
---
## Chunk 6: 收尾
### Task 6.1: P2 收尾验证 + 版本号
- [ ] **Step 1: 全量验证**
- `cd webui && npm run check && npm run build`
- `cargo build`
- `cargo test --lib`
- `cargo clippy --all-targets --all-features -- -D warnings`
Expected: 全部通过。
- [ ] **Step 2: 版本号** — `Cargo.toml` 与 `webui/package.json` minor bump（1.5.0 → 1.6.0）。
- [ ] **Step 3: Commit** — `git add -A && git commit -m "chore(release): P2 logs and data"`（仅暂存版本号文件 + 可能的文档；确认无构建产物）。
---
## P2 完成标志
- `/ws/logs` 实时推送日志帧（ts/level/target/message），设备鉴权保护，慢客户端丢旧不阻塞。
- `GET /api/logs` 保留，用于历史拉取与重连对齐。
- `PUT /api/memories/{key}` 更新 content/importance，key 不存在返回 404。
- `DELETE /api/memories/{key}` 幂等删除。
- 日志页：实时流 + level 过滤 + 搜索 + 暂停 + 下载 + 断线重连。
- 记忆页：行内编辑 + 分级删除确认 + 搜索 + 分类筛选。
- 任务页：cron 显示 + 倒计时 + 运行状态点 + 脉冲动画。
- `npm run check`、`npm run build`、`cargo build`、`cargo test --lib`、`cargo clippy -- -D warnings` 全绿。
## 风险与开放项
- **chrono vs time**：tracing-subscriber 的 `local-time` feature 已引入 `time` crate。优先用 `time::OffsetDateTime::now_local()` 避免新增 chrono 依赖。若 `now_local()` 在多线程环境报错（`time` crate 的已知限制），回退到 `std::time::SystemTime` + 手动格式化或用 `chrono`。
- **广播层性能**：`receiver_count() == 0` 短路确保无订阅者时零开销。有订阅者时每条日志一次 format + send，对高频 DEBUG 日志可能有微量开销，但 EnvFilter 默认 INFO 已过滤大部分。
- **WS 重连对齐**：前端断线重连时重新拉 `GET /api/logs` 尾部，可能丢失断线期间的少量日志（broadcast 不持久化）。可接受——日志页为运维辅助，非审计。
- **记忆分页**：现有 API 无 offset/cursor，用 limit 递增近似。大量记忆（>1000）时性能可接受（SQLite 单表 LIMIT 查询）。
- **记忆 PUT 不创建**：设计决策——新记忆只由 agent 内部创建，WebUI 仅编辑已有条目。若需创建能力，后续 P3 可加 POST。

67
docs/superpowers/plans/2026-07-26-p3-configuration.md
View File

@ -1,67 +0,0 @@
# P3 配置 Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan.
**Goal:** 重构 WebUI 配置页（SettingsPage），增加实时 JSON 校验、配置大纲、重载状态卡、保存并热重载、放弃修改、未保存提示。
**Architecture:** 纯前端重构。所有后端端点已存在（GET/PUT /api/config、GET/PUT /api/profiles/{name}、POST /api/config/reload、GET /api/config/reload/status）。SettingsPage.svelte 重写为带配置大纲侧栏 + 重载状态卡 + 三操作按钮的布局。
**Tech Stack:** Svelte 5（runes）、bits-ui Tabs。
**关键设计决策：**
1. **布局**：左侧编辑区（现有 editor-card）+ 右侧大纲/状态侧栏（仅 config.json 标签显示）。
2. **JSON 校验**：`$derived` 实时 parse content，显示错误信息或有效状态。
3. **配置大纲**：从 parsed config 提取顶层 key 列表，标注含密钥的 section（providers）和需重启的字段（gateway.host/port/workspace）。
4. **重载状态卡**：onMount 起每 3s 轮询 `GET /api/config/reload/status`，显示 generation、phase badge、last_error。
5. **操作按钮**：保存（PUT /api/config）、保存并热重载（PUT 后 POST /api/config/reload）、放弃修改（重新 load）。
6. **未保存提示**：比较当前 content 与上次加载的 content，不同则显示修改标记。
7. **USER.md / AGENTS.md 标签**：保持简单 textarea + 保存，无大纲/重载。
**验证约定：** `cd webui && npm run check && npm run build` + `cargo build`（验证 OUT_DIR 嵌入）。
---
## Task 1: SettingsPage 重构
**Files:**
- Modify: `webui/src/pages/SettingsPage.svelte`
- [ ] **Step 1: 重写 SettingsPage.svelte**
布局：
```
┌─────────────────────────────────────────────────────┐
│ [config.json] [USER.md] [AGENTS.md] [重载状态卡] │
├───────────────────────────────────┬─────────────────┤
│ editor-head: title + path │ 配置大纲 │
│ + 未保存标记 │ (仅 config) │
│ ───────────────────────────── │ │
│ textarea (JSON/Markdown) │ gateway ⚠重启 │
│ │ providers 🔑 │
│ │ agent │
│ │ channels │
│ │ memory │
│ │ scheduler │
│ ───────────────────────────── │ │
│ [保存] [保存并热重载] [放弃修改] │ │
│ notice / validation error │ │
└───────────────────────────────────┴─────────────────┘
```
功能：
- JSON 实时校验（config 标签）：parse 失败显示红色错误行
- 未保存修改：dirty 标记（对比 original content）
- 保存：PUT /api/config 或 PUT /api/profiles/{name}
- 保存并热重载：保存成功后 POST /api/config/reload，显示结果
- 放弃修改：恢复为上次加载的 content
- 重载状态卡：generation、phase（badge 着色）、last_error
- 配置大纲：顶层 key 列表 + 标注
- [ ] **Step 2: 验证** — `cd webui && npm run check && npm run build`
- [ ] **Step 3: Commit** — `git add webui/src/pages/SettingsPage.svelte && git commit -m "feat(webui): enhanced settings page with outline, reload status, hot reload"`
## Task 2: 收尾
- [ ] **Step 1: 全量验证** — `cargo build` + `cd webui && npm run check && npm run build`
- [ ] **Step 2: 版本号** — 1.6.0 → 1.7.0
- [ ] **Step 3: Commit** — `git add Cargo.toml webui/package.json && git commit -m "chore(release): P3 configuration"`

201
docs/superpowers/plans/2026-07-28-sleep-tool.md
View File

@ -1,201 +0,0 @@
# Sleep Tool Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add a model-callable `sleep` tool that asynchronously waits for 0～86400 whole seconds and terminates cleanly when its Turn is cancelled.
**Architecture:** A focused, stateless `SleepTool` validates its single argument and waits on one bounded Tokio timer. The existing default registry exposes it to agents, dropping the surrounding execution future cancels the timer, and terminal Turn reduction marks active tool blocks cancelled.
**Tech Stack:** Rust 2024, Tokio timers and paused-time tests, existing `Tool`/`ToolResult` interfaces, Serde JSON.
---
## Chunk 1: Tool And Registration
### Task 1: Implement And Register `SleepTool`
**Files:**
- Create: `src/tools/sleep.rs`
- Modify: `src/tools/mod.rs:1-52`
- Modify: `src/tools/mod.rs:74-90`
- Test: `src/tools/sleep.rs`
- [ ] **Step 1: Declare the module and write failing metadata tests**
Add `pub mod sleep;` and `pub use sleep::SleepTool;` to `src/tools/mod.rs`. Create `src/tools/sleep.rs` with a test module that imports `super::*`, `crate::tools::Tool`, `serde_json::json`, and `std::time::Duration`. Add a synchronous test asserting the name is `sleep`, the schema requires `seconds`, and its property type is `integer` with minimum `0`.
- [ ] **Step 2: Write failing validation tests**
Add a zero-second async test asserting exact successful output `Slept for 0 second(s).`. Add a table-driven async test for `{}`, negative, fractional, string, large float, and `86401`; assert each result is unsuccessful with empty output and a populated error. Add a parser boundary test:
```rust
#[test]
fn accepts_24_hour_boundary() {
assert_eq!(parse_seconds(&json!({"seconds": 86_400})), Ok(86_400));
}
```
- [ ] **Step 3: Write failing timer and cancellation tests**
Add the following paused-clock elapsed test. Yield after every `advance` so expired timers are polled deterministically:
```rust
#[tokio::test(start_paused = true)]
async fn waits_for_requested_seconds() {
let handle = tokio::spawn(async { SleepTool::new().execute(json!({"seconds": 2})).await });
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(1)).await;
tokio::task::yield_now().await;
assert!(!handle.is_finished());
tokio::time::advance(Duration::from_secs(1)).await;
tokio::task::yield_now().await;
assert!(handle.await.unwrap().unwrap().success);
}
```
Add a 24-hour boundary test that advances to one second before the deadline, asserts the handle is unfinished, advances the final second, and asserts success.
Add the cancellation test, which yields before aborting to ensure the timer has been registered:
```rust
#[tokio::test(start_paused = true)]
async fn cancellation_drops_an_active_sleep() {
let handle = tokio::spawn(async {
SleepTool::new()
.execute(json!({"seconds": MAX_SLEEP_SECONDS}))
.await
});
tokio::task::yield_now().await;
assert!(!handle.is_finished());
handle.abort();
assert!(handle.await.unwrap_err().is_cancelled());
}
```
- [ ] **Step 4: Run the focused test target and confirm RED**
Run: `cargo test --lib tools::sleep::tests`
Expected: compilation fails because `SleepTool`, `parse_seconds`, and `MAX_SLEEP_SECONDS` are not defined.
- [ ] **Step 5: Implement the minimal tool**
Implement `src/tools/sleep.rs` with this shape:
```rust
use super::traits::{Tool, ToolResult};
use async_trait::async_trait;
use serde_json::json;
use std::time::Duration;
const MAX_SLEEP_SECONDS: u64 = 86_400;
pub struct SleepTool;
impl SleepTool {
pub fn new() -> Self {
Self
}
}
impl Default for SleepTool {
fn default() -> Self {
Self::new()
}
}
fn parse_seconds(args: &serde_json::Value) -> Result<u64, String> {
let seconds = args.get("seconds")
.and_then(serde_json::Value::as_u64)
.ok_or_else(|| "seconds must be a non-negative integer".to_string())?;
if seconds > MAX_SLEEP_SECONDS {
return Err("seconds must not exceed 86400 (24 hours)".to_string());
}
Ok(seconds)
}
#[async_trait]
impl Tool for SleepTool {
fn name(&self) -> &str {
"sleep"
}
fn description(&self) -> &str {
"Pause the current agent execution for a specified number of whole seconds."
}
fn parameters_schema(&self) -> serde_json::Value {
json!({
"type": "object",
"properties": {
"seconds": {
"type": "integer",
"minimum": 0,
"maximum": MAX_SLEEP_SECONDS,
"description": "Number of whole seconds to wait, up to 24 hours."
}
},
"required": ["seconds"]
})
}
async fn execute(&self, args: serde_json::Value) -> anyhow::Result<ToolResult> {
let seconds = match parse_seconds(&args) {
Ok(seconds) => seconds,
Err(error) => {
return Ok(ToolResult {
success: false,
output: String::new(),
error: Some(error),
});
}
};
tokio::time::sleep(Duration::from_secs(seconds)).await;
Ok(ToolResult {
success: true,
output: format!("Slept for {seconds} second(s)."),
error: None,
})
}
}
```
Keep the default `read_only`, `concurrency_safe`, and `exclusive` methods unchanged so a batch containing `sleep` executes sequentially.
- [ ] **Step 6: Register the tool**
In `create_default_tools`, add:
```rust
registry.register(SleepTool::new());
```
Place it with the other stateless core tools, immediately after `CalculatorTool`.
- [ ] **Step 7: Run focused tests and confirm GREEN**
Run: `cargo test --lib tools::sleep::tests`
Expected: all sleep tests pass, including paused-time and cancellation cases.
- [ ] **Step 8: Verify the complete Rust change**
Run: `cargo test --lib`
Expected: all library tests pass.
Run: `cargo clippy --all-targets --all-features -- -D warnings`
Expected: exits successfully with no warnings.
Run: `cargo build`
Expected: debug build succeeds, including embedded WebUI build handling.
- [ ] **Step 9: Inspect the final diff**
Run: `git diff --check && git status --short && git diff -- src/tools/sleep.rs src/tools/mod.rs docs/superpowers/specs/2026-07-28-sleep-tool-design.md docs/superpowers/plans/2026-07-28-sleep-tool.md`
Expected: no whitespace errors; only the intended sleep implementation, Turn cancellation handling, public documentation, tests, and patch-version files are changed. Do not commit unless the user explicitly requests it.

304
docs/superpowers/specs/2026-07-23-webui-refactor-design.md
View File

@ -1,304 +0,0 @@
# PicoBot WebUI 全面重构设计
- 状态：设计已确认，待实现
- 日期：2026-07-23
- 范围：前端（`webui/`）全面重构 + 必要的后端接口新增/调整（`src/gateway/`）
## 1. 背景与目标
现有 WebUI（Svelte 5 + Bits UI，随二进制嵌入）已具备聊天、配置、记忆、任务、日志、主题切换等基础能力，但视觉与交互体验一般，且缺少运行状况观测、工具/Skill 浏览、实时日志等能力。本次重构目标：
1. 前端可直接与 PicoBot 沟通（聊天，已有，增强体验）
2. 可修改 PicoBot 各项配置（已有，增强）
3. 可观察 PicoBot 运行情况（**新增**：运行状况仪表盘）
4. 可查看工具列表、Skill 列表（**新增**）
5. 可查看实时日志（已有轮询，**升级为流式**）
6. 支持亮色/暗色的美观且易用的 UI（**全面重设计**）
7. 可查看并管理记忆、定时任务等信息（记忆**新增可编辑/可删除**）
## 2. 约束与不变量
- **单二进制发布**：前端构建产物仍打包进二进制，运行时从内存提供（`build.rs` → Cargo `OUT_DIR` → `include_str!`/`include_bytes!`）。最终用户无需 Node.js。
- **无外部 CDN**：生产页面不加载任何 CDN 资源。现有 CSP 为 `default-src 'self'; connect-src 'self' ws: wss:; img-src 'self' data:; style-src 'self'; script-src 'self'; base-uri 'none'; frame-ancestors 'none'`。字体等资产必须同源内嵌。
- **设备鉴权**：所有管理 API 与 `/ws` 受 `AuthManager` 保护；新增端点与 `/ws/logs` 同样走现有设备鉴权。
- **聊天复用现有链路**：浏览器聊天继续使用 `/ws` 与 `cli_chat` 渠道，复用 dialog scope、每会话串行 worker、历史持久化、出站 lane 与 turn 快照。WebUI 不直接调用 Provider 或 SessionManager。
- **密钥安全**：`/api/status` 等任何新响应不得包含密钥；日志脱敏在源头（不在日志中记录 secret）。
- **只读优先**：除配置（现有可写）与记忆写入（新增）外，其余新能力均为只读。
## 3. 总体方案
- **定位**：均衡控制台，但**聊天优先**——打开即聊天，其余功能区通过扁平导航平等直达；全局"活动脊"提供常驻运行感知。
- **技术栈**：继续使用 Svelte 5 + Bits UI + Vite，不引入新框架或状态管理库。
- **推进方式**：一份总体设计 + 分阶段实现（见 §9）。
## 4. 设计系统：Signal Deck
视觉方向为"仪表盘 / 工程仪器"：石墨蓝基底 + 琥珀（活动）/青绿（健康）双信号色，数据全部等宽字体，顶部一条永远在呼吸的"活动脊"作为签名元素。
### 4.1 色彩 Tokens
暗色（石墨蓝基底）：
| Token | Hex | 用途 |
|-------|-----|------|
| bg | `#0B1017` | 页面背景 |
| panel | `#0E1520` | 面板/卡片 |
| panel-2 | `#131C29` | 次级面板/悬停 |
| border | `#1D2733` | 边框 |
| border-strong | `#2C3A4C` | 强调边框/输入框 |
| text | `#E7ECF3` | 主文本 |
| text-soft | `#B8C4D4` | 次级文本 |
| muted | `#8FA3B8` | 辅助文本 |
| faint | `#5B6B7E` | 最弱文本/时间戳 |
| amber | `#FFB454` | 活动/进行中/警告 |
| teal | `#2DD4BF` | 健康/成功/只读 |
| danger | `#FF7B86` | 错误/危险/独占 |
| info | `#6AA6FF` | 信息/Timeline/思考 |
| code-bg | `#080C12` | 代码/日志底 |
亮色（冷纸白，信号色加深保证对比）：
| Token | Hex | Token | Hex |
|-------|-----|-------|-----|
| bg | `#EEF1F5` | text | `#1A2230` |
| panel | `#FFFFFF` | text-soft | `#3D4B5E` |
| panel-2 | `#F4F6F9` | muted | `#5B6B7E` |
| border | `#D8DEE8` | faint | `#8494A8` |
| border-strong | `#C2CCD9` | amber | `#C47400`（填充 `#E08600`） |
| teal | `#0D9488` | danger | `#D94354` |
| info | `#2F6FD0` | code-bg | `#F7F9FC` |
**关键规则**：亮色模式下"活动脊"仍为深色条（`#0E1520`），像物理仪器上的 LED 读数——两种主题下同一个记忆点，不做简单反色。
### 4.2 字体
- 展示 / UI：Space Grotesk（内嵌 woff2，仅拉丁），中文回落系统字体（PingFang SC / Microsoft YaHei / Noto Sans SC）。
- 数据 / 等宽：JetBrains Mono（内嵌 woff2），用于所有指标、日志、时间戳、small-caps 标签。
- 字号阶梯：9px（small-caps 标签，letter-spacing .12–.16em）/ 11px（caption、日志）/ 12.5–14px（正文）/ 16px（小标题）/ 19px（标题）/ 24–32px（指标数字）。
- 生产环境不加载 CDN；字体以内嵌二进制资产提供（见 §8.3）。
### 4.3 签名元素：活动脊（Activity Spine）
全局置于每个页面顶部的等宽状态条，两种状态：
- **有 Turn 在跑**：琥珀脉冲点 + `TURN 042 · STREAMING` + 实时 `▲ tok/s`、`ctx`、`queue`、`ws`，右侧 `gen #N · uptime · version`。
- **空闲**：青绿常亮点 + `IDLE` + 最近 turn 摘要。
Turn 实时状态来自聊天 WS 已有的 `turn_updated` 快照（本就实时推送，`WsOutbound::TurnUpdated`）；gen/uptime/version 等来自 `/api/status` 轮询。各字段来源：`▲ tok/s` 由前端对相邻 `turn_updated` 帧的 `usage.completion_tokens` 差值求导（快照本身不含速率字段）；`ctx` 取自 `usage.prompt_tokens`；`queue`/`ws` 取自 `/api/status`。
### 4.4 核心组件
按钮（primary=amber / secondary / ghost / danger）、状态徽标（正常/活动中/异常/离线）、指标块（大等宽数字 + sparkline + 分段容量条）、日志行（level 着色）、输入框、工具调用卡片（默认折叠，运行中=琥珀脉冲、完成=青绿）、表格行、标签页、Toast。图表统一手写 SVG sparkline / 分段仪表，不引入图表库。
## 5. 信息架构与应用外壳
- **导航**：左侧扁平导航——聊天（落地页）、概览、工具&Skills、日志、记忆、任务、配置；底部网关状态 + 主题切换。
- **应用外壳**：全局持有聊天 WS 连接（使活动脊在每个页面可用）、主题状态（`localStorage` 持久化 + `prefers-color-scheme` 默认）、设备鉴权状态（未配对显示 PairingPage）。
- **页面清单**：聊天 / 概览 / 工具&Skills / 日志 / 记忆 / 任务 / 配置，外加 PairingPage（鉴权）。
## 6. 页面设计
### 6.1 聊天页（落地页）
三栏布局：会话列表（搜索/新建/按日期分组/未读点）｜ 消息流 ｜ Todo 计划侧栏（默认收起，按需展开）。
- reasoning 与工具调用默认折叠为紧凑卡片（运行中=琥珀脉冲，完成=青绿）。
- 流式 turn 显示光标与 `▲ tok/s`，输入区出现"停止"按钮。
- 斜杠命令补全来自后端 `get_slash_commands`（不在前端硬编码命令表）。
- 附件走 HTTP 上传（`POST /api/chat/{client_id}/uploads`），WS 只传 `upload_id`；历史附件经 `GET .../attachments/{index}` 下载，安全 MIME 白名单内联预览。
- 正常完成合并 `turn_committed` 增量校准历史，不整段重载；断线/失败/取消用 `SessionHistory` 校准。
### 6.2 概览页（运行仪表盘）
- 主状态条：`RUNNING`、运行代、uptime、版本、WS 连接数、后台任务数、上次重载。
- 指标块（带 sparkline）：会话数、今日 Token（+费用）、工具调用（+运行中）、今日 Turns（+p95 延迟）。
- Provider 表：名称、模型、状态、延迟 sparkline、今日用量、费用。
- 消息总线：inbound/outbound/control 队列深度分段容量条、活跃 lane 数、调度器状态、MCP 连接。
- 渠道状态：feishu / cli_chat 等连接状态。
- 调度器：任务数、下次运行、7 天失败数。
- 实时活动流：最近 turn/memory/job 事件。
- 数据来自 `/api/status`，默认每 2s 轮询。
### 6.3 工具 & Skills 页
- 三个标签页：工具 / Skills / MCP。
- 工具卡片：名称、来源（builtin/mcp）、描述、调用次数、**能力徽标**、可展开参数 schema。
- **能力标识**（来自 `Tool` trait）：
- `◇ 只读`（teal）= `read_only()`
- `⇉ 可并发`（info）= `read_only() && !exclusive()`（即 `concurrency_safe()`）
- `△ 有副作用`（amber）= `!read_only()`
- `■ 独占`（danger）= `exclusive()`（如 bash）
- 支持搜索 + 按能力筛选（全部/只读/可并发/有副作用/独占）+ 图例。
- Skills 标签：名称、描述、always、来源目录。MCP 标签：服务器名 + 连接状态。
### 6.4 日志页（实时流式）
- 工具栏：level 过滤（全部/INF/WRN/ERR）、关键字搜索、暂停滚动、下载。
- 日志行：时间戳 + level 着色 + target + 消息，自动跟随尾部。
- 进入页面：`GET /api/logs`（保留）拉历史尾 → `/ws/logs` 接管实时；断线重连重新拉尾对齐。
- 顶部显示连接状态（实时推送中 · 行/分）。
### 6.5 记忆页（可编辑 / 可删除）
- **权限**：Knowledge 与 Timeline 均可编辑、可删除。
- **大量条目展示**：统计条（总量/Knowledge/Timeline/覆盖会话）→ 语义搜索优先 → 分类/会话/排序筛选 → 日期分组高密度行（列表/卡片视图可切换）→ 虚拟滚动 + 分页加载（"已显示 100 / 1,284 · 加载更多"）。
- 行内编辑：textarea + importance 调节 + 保存/取消（按 key upsert，`updated_at` 自动刷新）。
- **删除警告分级**：
- Knowledge：普通确认（"删除后不可恢复，影响后续召回"）。
- Timeline：**强警告**——"Timeline 是压缩后的历史上下文，删除后模型将永久失去该时段长期记忆且无法自动重建；原始消息仍保留在聊天历史，但不再进入模型上下文"，按钮文案"我了解，确认删除"。
### 6.6 任务页
- 两个标签页：定时任务 / 后台任务。
- 定时任务表：名称、cron 表达式、下次运行倒计时、上次运行、最近 10 次运行状态点（绿/琥珀/红）、启用状态；可展开运行记录（时间/耗时/摘要）。
- 后台子任务列表：名称、来源 session、运行中（脉冲）/完成、耗时。
- 只读浏览。
### 6.7 配置页（唯一可写页面之一）
- 标签页：config.json / USER.md / AGENTS.md。
- config.json：JSON 编辑器，密钥掩码（`********`，原样提交自动还原）、实时 JSON 校验 + default agent 有效性、未保存修改提示。
- 右侧配置大纲（gateway/providers/agent/channels/memory/scheduler），标注"重启生效""含密钥"。
- 重载状态卡：运行代、相位、上次重载结果。
- 操作：保存 / 保存并热重载 / 放弃修改。
- 复用现有 `GET/PUT /api/config`、`GET/PUT /api/profiles/{name}`、`POST /api/config/reload`、`GET /api/config/reload/status`。
- 提示 host/port/workspace 与存储路径为进程级不变量，修改后热重载被拒绝、需重启。
## 7. 后端接口设计
### 7.1 新增端点总览
| 方法 | 路径 | 用途 | 数据来源 |
|------|------|------|----------|
| GET | `/api/status` | 运行状况快照 | `Metrics` + 各服务只读查询 |
| GET | `/api/tools` | 工具列表（含能力字段） | `ToolRegistry` |
| GET | `/api/skills` | Skill 列表 | `SkillsLoader` |
| PUT | `/api/memories/{key}` | 更新记忆 content/importance | `Storage::upsert_memory` |
| DELETE | `/api/memories/{key}` | 删除记忆 | `Storage::delete_memory` |
| WS | `/ws/logs` | 实时日志流 | tracing 广播层 |
所有新端点走现有设备鉴权，注册在 `src/gateway/mod.rs` 的 protected router。
### 7.2 `GET /api/status`
返回单一 JSON 快照，概览页每 2s 轮询：
```json
{
"generation": 7, "version": "1.3.0", "uptime_secs": 266400, "phase": "steady",
"ws_connections": 2, "background_tasks": 3,
"sessions": { "total": 14, "active_turns": 1 },
"metrics": { "tokens_today": 1204882, "cost_today": 0.84,
"tool_calls_today": 312, "turns_today": 87, "turn_latency_p95_ms": 4200 },
"bus": { "inbound": {"depth":0,"cap":32}, "outbound": {"depth":1,"cap":64},
"control": {"depth":0,"cap":64}, "active_lanes": 4 },
"providers": [ {"name":"openai","model":"gpt-4o","status":"ok",
"latency_ms":820,"tokens":980000,"cost":0.61} ],
"channels": [ {"name":"feishu","status":"connected","detail":"3 群"} ],
"scheduler": { "enabled": true, "jobs": 5, "failed_7d": 0 },
"mcp": [ {"name":"github","status":"connected"} ]
}
```
聚合来源分两类——**已有查询**与**需新增的内省接口**（后者是 P1 的真实后端工作量，不可当作现成只读查询）：
已有 / 低成本可得：
- `reload`：generation、相位（现有 `ReloadStatus`）
- `mcp::get_mcp_status()`：MCP 服务器连接状态（现有全局状态注册表）
- `Scheduler` / Storage：任务数、下次运行、7 天失败数（现有 Storage API）
- `ChannelManager`：各渠道连接状态
- `Metrics`：token/费用/工具调用/turn/延迟（见 §7.3，新增）
**需新增的内省接口**（当前代码无对应查询面）：
- `MessageBus`：三条队列的深度与容量。现状只有 publish/consume，且未保留配置容量（`src/bus/mod.rs`）。实现上让 bus 保留各队列 `mpsc::Sender`/容量，深度由 `max_capacity() - capacity()` 派生（tokio `mpsc::Sender` 提供这两个方法）。
- `OutboundDispatcher`：活跃 lane 数。现状只有 `new`/`run`（`src/bus/dispatcher.rs`），需新增计数查询。
- `TaskSupervisor`：运行中任务数。现状无查询面（`src/task_supervisor.rs`），需新增。
- `SessionManager`：会话总数与活动 Turn 数（确认现有方法是否足够，不足则补只读统计）。
- WebSocket 连接数（`ws_connections`）：当前无连接计数器，需在 `ws_handler` 用一个 `Arc<AtomicUsize>` 在连接建立/断开时增减。
这些内省方法必须轻量、非阻塞（不加锁等待慢操作），以支撑每 2s 轮询。
**不含任何密钥**（provider api_key 等一律不出现）。
### 7.3 指标采集（`Metrics`）
- 新增 `Metrics` 结构（原子计数为主）：tokens in/out、cost、per-tool 调用数、turns、per-provider 延迟与错误滚动窗口。
- 由 `AgentLoop` / Provider 在每次 turn / 工具调用时经 `Arc<Metrics>` 更新。
- 纯内存、不持久化、重启归零。"今日"统计为自进程启动起的滚动窗口（文档与 UI 注明，不暗示自然日）。
- provider 状态（ok/降级）由最近错误率派生。
### 7.4 `WS /ws/logs`
- 给 tracing 增加一个广播层：格式化日志记录后发送到 `tokio::sync::broadcast`（容量约 1024）；慢客户端丢旧（lag），不反压。无订阅者时发送为 no-op，近乎零开销。
- handler 连接后订阅，按查询参数 `level` / `search` 过滤，推送 `{ts, level, target, message}` 帧。
- 修改 `src/logging` 的订阅器初始化以挂载该广播层（保持文件轮转不变）。
- `GET /api/logs`（文件尾）保留，用于进入页面时拉取历史与重连对齐。
### 7.5 记忆写入端点
- `PUT /api/memories/{key}`：body `{content, importance?}`，按 key upsert（复用 `Storage::upsert_memory`），`updated_at` 自动刷新。
- `DELETE /api/memories/{key}`：复用 `Storage::delete_memory`。
- path 中的 key 需 URL 解码；实现时校验 key 存在性，返回 404 若不存在。
- 现有 `GET /api/memories`（list/search，含 category/session/limit/query）保留不变。
### 7.6 工具 / Skills 端点
- `GET /api/tools`：遍历 `ToolRegistry`，每项返回 `name, description, parameters_schema, source(builtin|mcp), read_only, exclusive, concurrency_safe, call_count`。`call_count` 取自 `Metrics` 的 per-tool 计数。
- `GET /api/skills`：数据源为 `SkillsLoader::get_loaded_skills()`（返回完整 `Skill { name, description, content, always, path }`，`src/skills/mod.rs`）；**不要**用 `list_skills()`，它只返回 `(name, description)` 二元组，缺少 `always`/`path`。返回 `name, description, always, source`，其中 `source` 由 `Skill.path` 所在目录派生（无独立来源字段）。默认不返回完整 `content`（可能较大）。
## 8. 前端架构
### 8.1 目录与数据层
- 保持 Svelte 5 runes；将 `src/lib/api.js` 扩展为按域划分的客户端模块（如 `api/status.js`、`api/tools.js`、`api/memories.js`），不引入状态管理库。
- 组件库 `src/lib/`：在现有 `Markdown.svelte`、`ToolCallCard.svelte`、`TurnView.svelte`、`Toast.svelte`、`StatusBadge.svelte` 基础上，新增 Signal Deck 组件（ActivitySpine、MetricTile、Sparkline、CapacityMeter、LogStream、BadgeSet 等）。
- 页面 `src/pages/`：重构 ChatPage、新增 OverviewPage、ToolsPage、重写 LogsPage、重构 MemoryPage、重构 TasksPage、重构 SettingsPage、保留 PairingPage。
### 8.2 主题
- 设计 tokens 以 CSS 自定义属性表达：`:root`（暗色）与 `:root[data-theme="light"]`（亮色），替换现有 `styles.css` 的变量集。
- 主题切换持久化到 `localStorage`，默认跟随 `prefers-color-scheme`。
### 8.3 字体内嵌（构建管线变更）
- 字体文件（latin 子集 woff2，取自 @fontsource）放入 `webui/public/fonts/`。Vite 默认 `publicDir` 会把 `public/` 内容**原样、固定名**复制到产物根（`OUT_DIR/webui/fonts/*.woff2`），无需改 `vite.config.js` 的 `assetFileNames`。
- `http.rs`：新增 `/fonts/{name}` 路由，用 `include_bytes!(concat!(env!("OUT_DIR"), "/webui/fonts/...woff2"))` 嵌入（静态 name→bytes 映射），返回 `Content-Type: font/woff2` 与长期缓存头；属公开静态资源层（与 app.js/styles.css 同级，不进设备鉴权）。
- CSP：现有 `default-src 'self'` 已允许同源字体（font-src 回落到 default-src），无需放宽。
- 二进制体积增量约 100–150KB（Space Grotesk + JetBrains Mono，可考虑子集化）。
- `build.rs` 的 `rerun-if-changed` 需追加 `webui/public`；依赖 stamp 逻辑不变。
### 8.4 全局 WS 与活动脊
- 聊天 WS 连接提升到应用外壳层（App.svelte），使活动脊在所有页面可用。
- 活动脊消费 WS 的 turn 快照得到实时 Turn 状态；其余字段轮询 `/api/status`。
## 9. 分阶段实现
- **P0 地基**：设计系统（tokens/组件库/双主题/字体内嵌）+ 应用外壳（扁平导航 + 全局活动脊 + 主题/鉴权）+ 聊天页重构。
- **P1 观测**：`Metrics` + `GET /api/status` + 概览页 + `GET /api/tools`/`/api/skills` + 工具&Skills 页。
- **P2 日志与数据**：tracing 广播层 + `/ws/logs` + 日志页 + 记忆写入端点 + 记忆页重构 + 任务页重构。
- **P3 配置**：配置编辑器重构 + 配置大纲 + profile + reload 状态可视化。
每阶段独立可验证；前端改动须过 `npm run check` + `npm run build` + `cargo build`（验证 OUT_DIR 嵌入），Rust 改动须过定向测试 + `cargo test --lib` + `cargo clippy --all-targets --all-features -- -D warnings`。
## 10. 风险与开放项
- **字体内嵌**：构建管线需同时支持文本（include_str!）与二进制（include_bytes!）资产；需在实现期验证 vite 固定名输出与 Cargo 嵌入路径。若字体子集化复杂，可退回系统字体栈（牺牲部分排版个性）。
- **`Metrics` 侵入性**：在 AgentLoop/Provider 埋点需避免持锁慢操作，遵循"不在持锁时做网络/模型/DB 慢操作"的不变量；计数用原子操作。
- **`/api/status` 聚合成本**：每 2s 轮询，聚合多个服务的只读查询；需确保各查询轻量、不加锁阻塞。必要时缓存短 TTL 快照。
- **tracing 广播层**：需保证无订阅者时零开销、有订阅者时不阻塞日志写入；广播满时丢旧而非阻塞。
- **记忆 key 路由**：key 可能含特殊字符，URL 编解码与 404 语义需在实现期明确。
- **Timeline 删除语义**：UI 已用强警告；后端不做额外保护（用户拥有自己的 Agent），但删除为幂等硬删除。
## 11. 验收标准
- 单二进制 `cargo build` 成功，WebUI 从内存提供，无外部 CDN 依赖。
- 亮/暗双主题完整覆盖所有页面与组件。
- 聊天页保留现有全部能力（dialog scope、历史持久化、turn 快照、斜杠补全、附件、Todo 侧栏）。
- 概览页实时反映运行状况；活动脊在所有页面可见且实时。
- 工具页正确展示 read_only/exclusive/concurrency_safe 能力标识。
- 日志页实时流式推送，支持 level/搜索过滤与暂停。
- 记忆页支持 Knowledge/Timeline 编辑与删除，删除警告分级，大量条目下虚拟滚动流畅。
- 配置页可编辑、密钥掩码、热重载状态可视。
- 所有新端点受设备鉴权保护，响应不含密钥。
- `npm run check`、`npm run build`、`cargo build`、`cargo test --lib`、`cargo clippy -- -D warnings` 全部通过。

41
docs/superpowers/specs/2026-07-28-sleep-tool-design.md
View File

@ -1,41 +0,0 @@
# Sleep Tool Design
## Goal
Add a model-callable `sleep` tool that pauses the current agent tool call for a requested number of whole seconds. This first version only waits in process and does not schedule durable or background work.
## Interface
- Tool name: `sleep`
- Arguments: an object with one required `seconds` field
- `seconds` must be an integer from `0` through `86400` inclusive
- The maximum foreground wait is 24 hours
- `0` is valid and completes immediately
- Unknown object fields are ignored consistently with existing native tools
Invalid, missing, negative, fractional, or values above `86400` return an ordinary unsuccessful `ToolResult`. A successful call returns `Slept for N second(s).`, with the requested duration substituted for `N`.
## Implementation
Create a stateless `SleepTool` in `src/tools/sleep.rs`. Its `Tool::execute` implementation validates `seconds`, waits on one bounded Tokio timer, and returns success after the full duration. The asynchronous timer does not block the Gateway runtime.
Export the type from `src/tools/mod.rs` and register it in `create_default_tools`, making it available to the root agent and to constrained tool registries unless those registries explicitly filter it by name.
The tool does not persist state, create a background task, or send messages. It retains the `Tool` trait's default non-concurrency-safe classification, so a model response containing `sleep` and other calls executes that batch sequentially. `/stop`, supervisor shutdown, scheduler timeout, and sub-agent timeout cancel work by dropping the surrounding agent execution future; dropping that future also drops the current Tokio sleep timer. When a Turn is cancelled, every still-running tool block is normalized to `ToolStatus::Cancelled` before publishing the terminal snapshot.
The 24-hour cap bounds foreground resource retention. Longer or restart-durable waits must use Scheduler or background work rather than holding an interactive session worker.
## Testing
Unit tests cover the tool metadata and schema, immediate success for zero seconds, elapsed-time behavior using Tokio's paused clock, rejection of missing, negative, fractional, string, and over-24-hour values, acceptance of the 24-hour boundary, cancellation of an active sleeping task, and Turn cancellation normalization.
Run the targeted tests, `cargo test --lib`, `cargo clippy --all-targets --all-features -- -D warnings`, and `cargo build`.
## Out Of Scope
- Slash commands or direct user invocation
- Durable sleeps that survive process restart
- Delayed or scheduled message delivery
- A configurable duration limit
This change increments only the product patch version.

126
docs/superpowers/specs/2026-08-13-sub-agent-activity-webui-design.md
View File

@ -1,126 +0,0 @@
# Sub-Agent Activity WebUI Design
## Goal
Restructure the WebUI around sub-agent activity while moving sub-agent *definition* management into the settings page. Concretely:
1. Move sub-agent definition management (CRUD, enable/disable, role prompt) out of the `agents` navigation item into a new "子代理" tab on the settings page, and enlarge the role-prompt editing area.
2. Turn the `agents` navigation item into an activity monitor that shows running sub-agents and historical run results.
3. Add a read-only, chat-like detail view for a single sub-agent run, including a full, incrementally streamed message transcript.
4. Reduce the `tasks` page to scheduled-task information only; move the background-task listing into the sub-agent activity page.
## Non-goals
- No changes to sub-agent orchestration semantics (delegation, budgets, signals, run admission).
- No WebUI routing framework; the detail view is an in-page sub-view.
- No persistence of the main agent's chat history changes; the transcript work is scoped to sub-agent runs.
## Backend
### Schema (v9)
Add a new `agent_run_messages` table for incrementally streamed sub-agent transcripts:
```sql
CREATE TABLE IF NOT EXISTS agent_run_messages (
id TEXT PRIMARY KEY,
run_id TEXT NOT NULL,
seq INTEGER NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
reasoning_content TEXT,
tool_call_id TEXT,
tool_name TEXT,
tool_calls_json TEXT,
created_at INTEGER NOT NULL,
FOREIGN KEY (run_id) REFERENCES agent_runs(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_agent_run_messages_run_seq ON agent_run_messages(run_id, seq);
```
- Bump `SCHEMA_VERSION` from `8` to `9` in `src/storage/mod.rs`.
- Add the table DDL to `src/storage/agent_run.rs::AGENT_SCHEMA_STATEMENTS`.
- **Version-gate the existing agent-table drops.** `migrate_schema` currently drops `agent_runs`/`agent_inbox_events`/`agent_run_groups` unconditionally whenever `current < SCHEMA_VERSION`; on a v8→v9 upgrade this would destroy run history. Gate those drops (and the `background_tasks` drop) on `current < 8` — the batch-group removal is a pre-v8 concern — so a v8→v9 upgrade only adds the new table and preserves existing runs. When the drops do run (pre-v8), drop `agent_run_messages` before `agent_runs` so foreign-key enforcement never blocks the implicit row delete.
- Update the v8 schema test that hardcodes `assert_eq!(version, 8)` (`fresh_database_creates_schema_v8_agent_tables`) to v9 and rename it accordingly.
We deliberately do **not** add a transcript column to `agent_runs`. Incremental append rows in `agent_run_messages` are the single source of truth for transcripts.
### Incremental capture
- Add an optional sink to `AgentLoop`:
- field `transcript_sink: Option<tokio::sync::mpsc::UnboundedSender<ChatMessage>>`
- builder `with_transcript_sink(sender) -> Self`
- at every site where a message is appended to `emitted_messages` (assistant messages and tool result messages), forward a clone to the sink. Convert `append_steering_messages` from an associated function to a method so it can forward too (sub-agents never use steering, but the sink must be consistent for future use).
- `src/agent/sub_agent.rs::execute_resolved`:
- create an `mpsc::unbounded_channel`
- pass the sender via `build_sub_agent_with_provider` → `with_transcript_sink`
- spawn a writer task that owns the receiver and a `seq` counter; for each message it strips `provider_state` (set to `None`) and does `INSERT INTO agent_run_messages`
- restructure so the sender is dropped and the writer task is awaited in **every** exit path, including the `tokio::select!` cancellation arm that currently returns early without `process_with_context` returning; only after the writer drains does the terminal commit run
- the writer uses `self.storage` (the manager already holds `Option<Arc<Storage>>`); if storage is absent, the writer becomes a no-op
The task prompt is not duplicated into the table; the detail view renders `run.task` as the leading user bubble.
### Storage API
Add to `src/storage/agent_run.rs`:
- `append_agent_run_message(run_id, seq, &message, now) -> Result<()>` — single-row insert.
- `list_agent_run_messages(run_id, limit) -> Result<Vec<AgentRunMessageRecord>>` — ordered by `seq`. The transcript is naturally bounded: one run emits at most a handful of messages per tool iteration and iterations are capped by the definition's `limits.max_iterations` (default 99); default `limit` of `10_000` is a generous ceiling, not a pagination contract. `get_agent_run` uses this same default limit when loading the transcript.
Two distinct types to avoid a storage/protocol collision:
- `AgentRunMessageRecord` (storage, in `src/storage/agent_run.rs`): carries the raw columns — `id, run_id, seq, role, content, reasoning_content, tool_call_id, tool_name, tool_calls_json, created_at`.
- `AgentTranscriptMessage` (protocol, in `src/protocol.rs`): the serialized shape — same fields except `tool_calls_json` is parsed into `Vec<providers::ToolCall>`; a `From<AgentRunMessageRecord>` impl performs the parse.
### HTTP API
- `GET /api/agent-runs/{id}` currently returns `{ "run": AgentRunView }`. Extend the response to `{ "run": ..., "session_id": ..., "transcript": [ ... ] }` where:
- `session_id` is `run.root_session_id` (the `AgentRunView` deliberately omits it, so it is added at this endpoint's response level)
- `transcript` is the ordered list of `AgentTranscriptMessage` rows (exposing `reasoning_content` but never `provider_state`)
- `GET /api/agent-runs/{id}/events` is unchanged (signals/completions).
- `GET /api/tasks` is unchanged and already lists all runs in any status; the activity page consumes it.
## Frontend
### Settings page (`SettingsPage.svelte`)
- Add a "子代理定义" tab to the existing vertical `Tabs` (named to disambiguate from the "子代理" activity nav item). Move the definition list and the editor modal from `AgentsPage.svelte` here verbatim, then:
- enlarge the role-prompt `textarea` (`min-height` ~360px, full-width, monospace)
- widen the editor modal (`min(920px, 94vw)`) and put the role-prompt field on its own row
- keep the existing API calls (`/api/agents`, `/api/agents/options`, POST/DELETE) unchanged
### Sub-agent activity page (`AgentsPage.svelte`, rewritten)
- List view:
- "活动中" section: runs in `queued`/`running`/`waiting_children` status, with a pulse indicator, `agent_id`, prompt excerpt, `mode`/`depth`, and elapsed time.
- "历史活动" section: terminal runs (`completed`/`failed`/`timed_out`/`cancelled`/`interrupted`), newest first, with `StatusBadge`, `agent_id`, prompt excerpt, `tool_calls_count`/`iterations`, timestamps, and a "详情" action.
- Poll `GET /api/tasks?limit=200` every 5s while mounted.
- Detail view (in-page sub-view, back button, read-only):
- metadata header: `agent_id`, `StatusBadge`, `provider/model`, `mode`, `depth`, `tool_calls_count`, `iterations`, `session_id`, `started_at`/`finished_at`, duration, `parent_run_id` if present
- leading "task" bubble from `run.task`
- transcript messages rendered like the chat page: `Markdown` for content, collapsible reasoning block, `ToolCallCard` for tool calls. Pair each assistant message's `tool_calls` with its `tool`-role result by `tool_call_id` (mirroring `ChatPage`'s `toolResult()`), so calls and results remain independently collapsible.
- signal cards from `GET /api/agent-runs/{id}/events` (same rendering as the chat page's agent-event cards)
- final `error` card on failure
- while `status` is non-terminal, poll `GET /api/agent-runs/{id}` + `/events` every 2s to stream the growing transcript; stop polling on terminal status
- No composer, no input, no interactive actions other than navigation/collapse.
### Tasks page (`TasksPage.svelte`)
- Remove the "后台任务" tab and the `Tabs` wrapper; keep only the scheduled-job list (jobs + runs dots), which already comes from `/api/jobs` and `/api/jobs/{id}/runs`.
### Navigation (`App.svelte`)
- Update the `agents` page description to reflect activity monitoring ("查看活动中的子代理与历史运行"); keep the nav label "子代理".
- Update the `tasks` page description to scheduled tasks only ("管理定时任务").
## Error handling
- Missing/empty transcript: detail view renders the task bubble + metadata + error (or a "无转录" placeholder for failed runs).
- Storage write failures in the transcript writer are logged and the run still commits its terminal status; a broken transcript never fails the run.
- API auth/5xx reuse the existing `api()` error handling and `notify` toast path.
## Testing
- Rust: v9 migration from a v8 database **preserves existing `agent_runs` rows** (version-gated drops), and the pre-v8 legacy rebuild path still works; `append_agent_run_message` + `list_agent_run_messages` round-trip with `seq` ordering; `provider_state` is stripped from persisted transcript rows; sink drains on success, timeout, and cancellation before terminal commit (including the early-return cancellation arm); `get_agent_run` returns the transcript and `session_id`; `AgentRunView` list responses remain transcript-free; the v8 hardcoded schema-version assertion is updated to v9.
- Run `cargo test --lib` and `cargo clippy --all-targets --all-features -- -D warnings`.
- Frontend: `cd webui && npm run check && npm run build`, then `cargo build` to verify the `OUT_DIR` embedding path. No external runtime dependency; the browser stays on `/ws` and the existing HTTP endpoints.

33
resources/agents/general-purpose.md
View File

@ -1,33 +0,0 @@
---
id: general-purpose
description: 通用目的子代理，处理主 Agent 委托的独立子任务
llm_profile: default
tools:
- bash
- file_read
- file_search
- content_search
- web_fetch
- calculator
---
# General Purpose Agent
You are the **general-purpose** sub-agent of PicoBot, a general-purpose assistant.
## Role
- Handle any independent subtask delegated by the main agent.
- Work autonomously and report back concrete results.
## Principles
- Follow the task description and context provided by the delegator.
- Use the tools you need; prefer read-only operations unless the task requires changes.
- Be accurate: do not fabricate tool results or guesses; report failures honestly.
- If information is missing, ask or state the gap rather than inventing it.
- You do not hold the main session's conversation history; rely on the context given in your task.
- Do not delegate further to other agents unless explicitly instructed.
## Output
- Deliver the final result in a clear, structured, self-contained format.
- Keep it concise unless the task requires detail.
- If the task cannot be completed, explain why and what is blocking it.

6
resources/skills/about-picobot/SKILL.md
View File

@ -13,11 +13,11 @@ PicoBot 是一个基于 Rust 的个人 AI 助手运行时，包含本地 Gateway
| 文件 | 内容 |
|------|------|
| `references/config.md` | 配置字段详解：providers、models、agents、agent_orchestration、gateway、client、channels、memory、mcp、browser |
| `references/db-schema.md` | 数据库表结构与运行约束：sessions、messages、memories、task plans/items、scheduled_jobs、job_runs、llm_calls、agent run/inbox/state |
| `references/config.md` | 配置字段详解：providers、models、agents、gateway、client、channels、memory、mcp、browser |
| `references/db-schema.md` | 数据库表结构与运行约束：sessions、messages、memories、scheduled_jobs、job_runs、llm_calls、background_tasks |
| `references/architecture.md` | 核心架构：消息并发、会话系统、持久化、生命周期、上下文压缩、记忆、MCP、子 Agent |
| `references/faq.md` | 常见问题：模型切换、渠道添加、Skill 安装、历史查询、定时任务、MCP 等 |
| `references/commands.md` | 常用命令：编译、启动网关、Docker/WebUI 设备配对、启动客户端、运行测试 |
| `references/commands.md` | 常用命令：编译、启动网关、启动客户端、运行测试 |
| `references/tools.md` | 内置工具名称、参数和重要使用约束 |
| `assets/config.example.json` | config.json 完整示例 |

49
resources/skills/about-picobot/assets/config.example.json
View File

@ -23,8 +23,7 @@
"qwen-plus": {
"model_id": "qwen-plus",
"temperature": 0.0,
"max_tokens": 8192,
"token_limit": 128000
"max_tokens": 8192
},
"gpt-4o": {
"model_id": "gpt-4o",
@ -43,24 +42,14 @@
"default": {
"provider": "aliyun",
"model": "qwen-plus",
"max_tool_iterations": 99
"max_tool_iterations": 99,
"token_limit": 128000
}
},
"context_compaction": {
"enabled": true,
"reserve_tokens": 16384,
"keep_recent_tokens": 20000
},
"gateway": {
"host": "127.0.0.1",
"port": 19876,
"require_pairing": true,
"scheduler": {
"enabled": true,
"poll_interval_secs": 60,
"max_concurrent": 1,
"execution_timeout_secs": 900
}
"require_pairing": true
},
"client": {
"gateway_url": "ws://127.0.0.1:19876/ws"
@ -71,26 +60,15 @@
"app_id": "<FEISHU_APP_ID>",
"app_secret": "<FEISHU_APP_SECRET>",
"allow_from": ["*"],
"require_mention": true,
"agent": "default",
"media_dir": "~/.picobot/media/feishu",
"reaction_emoji": "Typing",
"live_updates": false,
"live_update_interval_ms": 500,
"max_image_bytes": 10485760,
"max_file_bytes": 26214400,
"media_dir_max_bytes": 536870912,
"request_timeout_secs": 30
"reaction_emoji": "Typing"
}
},
"memory": {
"consolidation_provider": null,
"consolidation_model": null,
"recall_limit": 5,
"recall_min_relevance": 0.25,
"recall_min_score": 0.25,
"recall_recency_half_life_days": 30,
"recall_timeout_ms": 1000,
"idle_consolidation_minutes": 10,
"timeline_retention_days": 90,
"max_failures_before_degrade": 3
@ -100,21 +78,10 @@
"tool_timeout_secs": 180
},
"browser": {
"enabled": true,
"command": "agent-browser",
"enabled": false,
"webdriver_url": "http://127.0.0.1:9515",
"headless": true,
"browser_executable_path": null,
"max_sessions": 4,
"idle_timeout_secs": 3600,
"command_timeout_secs": 120,
"max_output_chars": 50000,
"content_boundaries": true,
"allowed_domains": [],
"allow_private_hosts": false,
"artifact_dir": "~/.picobot/media/browser",
"persistence": {
"profile_dir": "~/.picobot/browser/profiles"
}
"chrome_path": null
},
"workspace_dir": "~/.picobot/workspace"
}

147
resources/skills/about-picobot/references/architecture.md
View File

@ -7,10 +7,8 @@ Channel → MessageBus.inbound → Gateway processor → SessionManager → per-
↑ │
└── Channel ← OutboundDispatcher ← per-conversation lane ← MessageBus.outbound
AgentLoop → TurnEvent → TurnController → latest TurnSnapshot → DeliveryCoordinator → TurnSink → Channel
WebSocket/Channel → MessageBus.control → Gateway processor → SessionManager (dialog 操作)
Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Scheduled Agent → complete_scheduled_run → JobRun outbox → SessionManager/MessageBus
Scheduler → SessionManager.handle_cron_message → AgentLoop → send_message
```
## 模块职责
@ -21,10 +19,9 @@ Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Schedule
| `client` | TUI 聊天客户端 |
| `channels` | 外部集成（飞书、CLI），仅收发消息 |
| `bus` | 有界 inbound/outbound/control 队列；出站 dispatcher 与分目标 lane |
| `session` | 会话生命周期、dialog 操作、每 session 串行队列、Turn 状态、上下文与持久化协调 |
| `agent` | LLM 调用循环、工具执行、上下文压缩、媒体处理、子 Agent、Turn 语义事件 |
| `providers` | OpenAI/Anthropic 原生流解析，统一正文、reasoning、工具、usage 与私有回放状态 |
| `delivery` | 完整 Turn 快照的展示过滤、latest-wins 节流、终态投递和 TurnSink 生命周期 |
| `session` | 会话生命周期、dialog 操作、每 session 串行队列、上下文与持久化协调 |
| `agent` | LLM 调用循环、工具执行、上下文压缩、媒体处理、子 Agent |
| `providers` | LLM API 客户端（OpenAI 兼容、Anthropic） |
| `tools` | Agent 工具（bash、文件操作、搜索、HTTP、web、browser、memory、delegate 等） |
| `skills` | Skill 加载、管理和 prompt 构建 |
| `storage` | SQLite 持久化 |
@ -32,7 +29,6 @@ Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Schedule
| `observability` | Observer 模式，agent/工具遥测事件 |
| `protocol` | WebSocket 协议消息定义 |
| `config` | 配置加载、环境变量替换、路径解析 |
| `health` | CLI、工具、斜杠命令和 WebUI 共用的只读运行依赖检查 |
| `memory` | 长期记忆存储与检索 |
| `mcp` | MCP（Model Context Protocol）工具集成 |
| `task_supervisor` | Gateway 后台任务注册、取消、限时等待和强制回收 |
@ -40,36 +36,28 @@ Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Schedule
## 功能边界
- Channels 通过 MessageBus 发布入站消息，通过 OutboundDispatcher 或每 Turn 一个的 TurnSink 接收出站写入，不感知 session 或 LLM
- Channels 仅收发消息，不感知 session 或 LLM
- MessageBus 本体持有三条有界队列；出站路由、顺序和重试由 `OutboundDispatcher` 负责
- SessionManager 拥有 session 状态、dialog 路由、上下文构建、每 session worker 和活动 Turn 的 steering mailbox，并通过 worker 创建 AgentLoop
- TurnController 是活动 Turn 状态的唯一 owner；Session 在消息原子提交成功后才发布 Completed
- AgentLoop 跨轮无状态，接收已准备的 history，并在安全模型边界排空本 Turn steering 后调用 LLM、执行工具并返回一次结果
- Providers 是纯 HTTP 流客户端，无 bus/session/channel 感知；签名 reasoning 状态只回放给匹配 Provider，不下发客户端或 Channel
- DeliveryCoordinator 只投影完整快照，不修改会话历史；慢消费者跳过中间 revision，终态显式、有界投递
- 每个活动 Turn 独占一个 TurnSink；平台 message ID 和 reaction 清理状态只存在于 sink 内
- Tools 接收原始参数，通常返回字符串结果；有状态适配器额外接收 session/turn `ToolExecutionContext`
- SessionManager 拥有 session 状态、dialog 路由、上下文构建和每 session worker，并通过 worker 创建 AgentLoop
- AgentLoop 跨轮无状态，接收已准备的 history 调用 LLM、执行工具并返回一次结果
- Providers 是纯 HTTP 客户端，无 bus/session/channel 感知
- Tools 接收原始参数，返回字符串结果
- MCP 工具在 Gateway 初始化时连接服务器、发现工具，并包装成普通 Tool 注册到 ToolRegistry
- 具名子 Agent 从运行代不可变 `AgentCatalog` 加载，Definition 固定 Provider/Model、工具/Skill allowlist、委托边与限制；工具集完全由定义文件的 `tools` 列表决定（管理员显式授权），`delegate`/`emit_signal`/`get_skill`/`agent_task` 为运行时注入不可静态声明。单个定义校验失败（坏 YAML、未知 provider/profile/model/tool/skill、或显式委托到缺失目标）仅停用该定义并记入 `load_errors`（`GET /api/agents` 返回），不会阻塞启动或热重载；配置与目录信任级错误仍然致命。支持单个/批量 foreground 和显式父子授权；Root 对具名 Agent 的 background（单任务或批量）走 durable run/inbox + continuation 投递，每个 run 独立完成、空闲时完成即返回。内置 general-purpose 定义随二进制释放到 `~/.picobot/agents/`，WebUI「子 Agent」页可增删改与启停定义
- 子 Agent 由 `delegate` 工具创建，复用 provider 配置和按需过滤后的工具集；后台任务结果通过 MessageBus 发回原会话
- 复杂任务可使用 `todo` 创建 session 级计划；多个子项可通过 `delegate.plan_item_id` 并行委托，子 Agent 不能修改计划
- WebUI 聊天复用 `/ws` 与 `cli_chat`；同源管理 API 只读取受限的日志、任务、记忆，并对白名单配置文件做原子写入
- WebUI 使用 Svelte 5 + Vite，Bits UI 提供无样式可访问组件；`cargo build` 增量生成前端到 Cargo `OUT_DIR`，再嵌入单二进制，仓库不保存生成产物
- WebUI 通过 `get_session_stats`/`session_stats` 显示当前会话累计输入输出 Token 和上下文窗口占用；`/info [--json]` 读取同一份 SessionStats
## 关键约束
- Gateway 启动时切换到 workspace 目录
- SQLite 数据在 `{config_dir}/data/picobot.db`（`config_dir` 默认 `~/.picobot`），与 workspace 相互独立
- SQLite 数据在 `{workspace}/picobot.db`
- ChannelManager 持有 MessageBus 和所有 channel
- OutboundDispatcher 通过 ChannelManager 路由出站消息
- 配置目录 `.env` 与 workspace `.env` 仅在单线程启动阶段分层加载，并使用 `unsafe { env::set_var(...) }` 写入进程环境；优先级为既有进程环境 > workspace > 配置目录
- `browser` 工具默认启用，只有 `browser.enabled=false` 时不注册；缺少 CLI/Chrome 不阻止 Gateway 启动，但 health 和实际调用会给出安装错误。每次调用按参数分流：不传 `persistent_id` 时按 dialog 使用普通临时浏览器，默认空闲一小时后自动关闭；长期工作时 Agent 可自主创建持久身份，并在后续相关 action 中持续传入同一个 ID，持久 daemon 禁用空闲自动关闭。同一 ID 跨 dialog 共享 session/锁，不同 ID 相互独立并可并发。`browser_profiles` 只在受控根目录中创建、设置语义标签、列出或删除合法 ID；没有全局持久化开关、默认 ID，也不自动按 dialog 建立或选择持久 Profile，不依赖 Fantoccini/ChromeDriver/WebDriver
- 所有工具调用统一包装为 `ToolOutput` 并经过公共处理器；产物按模型/用户受众分流。浏览器截图默认同时供模型查看并附到最终回复，`file_read` 图片默认仅供模型理解
- 同一 session 只运行一个 Turn；活动 Turn 期间普通输入默认 steering，`/queue` 明确等待下一 Turn，不同 session 可并发
- steering mailbox 容量为 32 条/64 KiB，满或关闭时可靠回退到容量 32 的 session 队列；两者都无法接收时明确拒绝
- Config `.env` 加载使用 `unsafe { env::set_var(...) }`
- `browser` 工具只有在 `browser.enabled=true` 时注册，依赖 Chrome/Chromium 与 WebDriver
- 同一 session 的普通消息串行处理，不同 session 可并发；session 队列容量为 32，满时明确拒绝
- 出站消息按 `(channel, chat_id)` 分 lane 保序；lane 容量为 64，慢目标不阻塞其他目标
- 活动 Turn 与普通出站消息共享 `(channel, chat_id)` 写锁；禁止把 token delta 放入 MessageBus
- `cli_chat` 向 TUI/WebUI 发送统一 `turn_updated` 完整快照；飞书默认 FinalOnly，开启 `live_updates` 后编辑同一卡片
- 长生命周期后台任务由 TaskSupervisor 管理；连接局部任务由其 owner 限时 join 或 abort
- 外部建连、重试等待和关停 join 必须可取消且有硬超时
- 不得记录 API Key、Authorization header 或包含临时凭据的完整连接 URL
@ -77,7 +65,11 @@ Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Schedule
## 上下文压缩
`messages` 是 append-only 原始日志，压缩不会改写消息、工具结果、ID 或 seq。每个 session 最多有一个活动 `ContextCheckpoint`；Provider 历史确定为“一条累计摘要 + `seq >= first_retained_seq` 的原始尾部”。Model `token_limit` 是上下文硬上限，缺失时默认 128K；可选 Agent `token_limit` 只能收紧它，有效窗口取二者最小值。自动压缩使用 `context_tokens > context_window - effective_reserve`，默认 reserve 16,384、近期原样保留 20,000 tokens；小窗口会自适应缩小两者。摘要输入预算由有效窗口扣除摘要输出、提示词和安全余量得到；超大历史生成 checkpoint 加最新材料优先的有界 request-local 转录，不受固定 32K 上限约束。`/compact`、自动入口和首次 overflow 复用同一压缩编排和 checkpoint 原子提交路径，每次最多一次摘要调用。真实 Provider overflow 或换模后发送前已检测到的硬超限，能在摘要不可用时生成明确标记的确定性降级 checkpoint，并且正式 Provider 请求只重试一次。工具已经执行后若发生 overflow，只在同一个 AgentLoop 中保留本 Turn 工具链、裁掉旧完整 Turn 的请求副本并重试当前模型步骤一次，不从 durable history 重跑工具。
当上下文接近 token 限制时触发：
1. **快速裁剪**：合并连续同角色消息，截断工具输出
2. **硬截断**：移除过老消息
3. 压缩后保留用户消息确保结构完整
## Skill 系统
@ -136,7 +128,7 @@ create → 存入 Storage → 载入 memory → 设为当前 dialog
### 消息处理与并发
没有活动 Turn 时，普通消息先 `try_send` 到该 session 的有界 worker 队列，Gateway 主 processor 随即返回 `AgentProcessing`。活动 Turn 期间普通消息默认进入有界 steering mailbox，并在完整工具批次结束后或无工具最终回复边界作为真实 `role=user` 消息注入下一次模型调用；`/queue <message>` 绕过 mailbox，明确进入下一 Turn。`/stop` 直接取消当前 Turn 并清空 mailbox 与普通队列，不会排在长模型调用后。
普通消息先 `try_send` 到该 session 的有界 worker 队列，Gateway 主 processor 随即返回 `AgentProcessing`。Slash command 直接执行，不进入此队列，因此 `/stop` 不会排在长模型调用后。
Worker 的处理原则：
@ -144,23 +136,13 @@ Worker 的处理原则：
2. 释放锁后执行消息持久化、记忆召回、上下文压缩、LLM 和工具等慢操作。
3. 提交由旧快照产生的结果前重新验证 generation/version，防止 `/stop`、`/clear` 或 `/delete` 后写回陈旧状态。
4. Session 持久化由独立 `persistence_lock` 串行化；批量消息使用原子写入，失败时精确回滚内存后缀。
5. 首次请求上下文溢出且尚未执行工具时，按 Provider 返回的真实限制提交 checkpoint 并正式重试一次；工具执行后的溢出只能在原 AgentLoop 内保留当前工具链进行一次请求级恢复。
6. mailbox 的接收与关闭原子互斥；所有输入在 Session 锁内取得单调序号，未消费 steering 与普通队列按该序号恢复，不能丢失或互相超越。
WebUI/TUI 的 Active Turn 使用 `send_message(files=...)` 向自身 session 投递附件时，附件暂存到 task-local Turn delivery，成功结束后并入最终 assistant 消息，因此工具链始终排在附件回复之前且不会出现自引用来源前缀。其他自投递要求 task-local Turn ID 与 session 的 active Turn 匹配；历史中的 assistant/system 附件只作为文本清单提供给模型，原生媒体块仅用于 user 输入和当前工具结果。
### 活动 Turn
每个主 Agent 请求会创建一个内存 Turn。Provider delta 经 AgentLoop 转换为 reasoning、正文、工具开始/完成等语义事件，TurnController 归约为有序 block 和单调 revision 的完整快照。TUI/WebUI 使用 `history + active_turn` 渲染，不自行拼接 token；中间帧可丢，下一快照会自动收敛。
展示策略在 Gateway 核心出口应用：交互客户端可显示 reasoning 和详细工具状态；外部渠道隐藏 reasoning、工具仅显示紧凑状态；无人值守投递只保留正文。运行态不逐 token 入库，完成、取消或中断时才原子保存消息及 completion status。
5. 上下文溢出时按 Provider 返回的真实限制重新压缩并重试。
### 会话恢复
从 Storage 恢复 session 时：
- 加载全部原始消息及 Session 的活动 checkpoint
- 有 checkpoint 时确定性投影累计摘要和 `first_retained_seq` 之后的原始尾部；没有 checkpoint 时投影全部原始消息
- Timeline 和 `last_compressed_message_at` 不参与恢复边界判断，恢复过程不调用 Provider
- 若 `last_compressed_message_at` 存在：先加载近 3 条 Timeline 记忆作为 `[Previous Context]`，再加载压缩标记后的原始消息
- 若无压缩记录：正常加载全部消息
- 自动修复断链的工具调用（gateway 崩溃中途重启导致）
---
@ -214,9 +196,13 @@ WebUI/TUI 的 Active Turn 使用 `send_message(files=...)` 向自身 session 投
### 上下文压缩与 Timeline
自动压缩在完整请求占用满足 `context_tokens > context_window - effective_reserve` 时触发。压缩器从尾部按完整 Turn 尽量保留近期历史，用一次 LLM 调用生成累计摘要，并以 `first_retained_seq` 记录精确尾部边界。摘要请求按当前模型窗口动态限制输入；若待压缩源更大，只在请求副本中保留已有 checkpoint、最新消息和确定性 head/tail 摘录，原始内容仍完整持久化。checkpoint 与 Session 活动指针在同一 SQLite 事务中提交；提交失败继续使用旧投影。原始消息与工具结果始终保留，旧内容只是不再进入 Provider context。
LLM 对话上下文接近 token 限制 (默认 128K × 70%) 时自动触发压缩：
语义 checkpoint 提交后会 best-effort 写入一条 **Timeline**（importance 0.3）供主动检索，但 Timeline 不是恢复权威。真实 context overflow，或换模/改配置后在发送前已检测到的硬超限，能使用无语义 breadcrumb 降级，且只重试一次正式 Provider 请求；低于硬窗口的普通自动压缩和 `/compact` 失败时不会裁掉历史。
1. **快速裁剪**：工具输出 ≥ 2000 字符时截断
2. **LLM 摘要**：最多 3 轮，每轮找连续用户消息对，将中间的 assistant/tool 消息压缩为摘要 → 摘要作为 **Timeline 记忆** 持久化（importance 0.3）
3. **硬截断**：若仍超 90%，只保留前 N + 后 N 条消息
压缩后 `last_compressed_message_at` 标记边界，后续恢复时从标记点加载原始消息，以 Timeline 提供更早的上下文。
### 关键集成点
@ -224,11 +210,11 @@ WebUI/TUI 的 Active Turn 使用 `send_message(files=...)` 向自身 session 投
|------|------|
| 每次消息处理 | `memory_manager.recall()` 提取 Knowledge 上下文 |
| 系统提示构建 | `MemorySection` 渲染记忆工具指南；匹配的 Knowledge 附加到本轮 user message |
| 有活动 checkpoint 时 | 累计摘要和精确 raw tail 组成 Provider 历史 |
| 语义 checkpoint 提交后 | 摘要 best-effort 存储为 Timeline 记忆 |
| 会话恢复 | 从 checkpoint 与原始 seq 确定性重建，不读取 Timeline |
| 有压缩历史时 | `HistorySection` 提示 LLM 使用 `timeline_recall` |
| 压缩完成后 | 摘要自动存储为 Timeline 记忆 |
| 会话恢复 | 加载最近 Timeline 和压缩边界后的原始消息 |
`memory.recall_limit` 及新增的 `recall_min_relevance`、`recall_min_score`、`recall_recency_half_life_days`、`recall_timeout_ms` 已生效：每轮自动召回用 jieba 分词 + FTS5 检索候选，按「词项相关度 0.5 + 重要度 0.3 + 时效 0.2」加权并通过相关性/综合分双门槛过滤，搜索受硬超时保护。`idle_consolidation_minutes`、`timeline_retention_days` 和 `max_failures_before_degrade` 仍只是配置解析：自动 idle consolidation、Timeline 清理和失败降级循环尚未接入运行循环。不要把“配置可解析”误认为“行为已生效”。
`memory.recall_limit`、`idle_consolidation_minutes`、`timeline_retention_days` 和 `max_failures_before_degrade` 当前会被配置解析；其中每轮 Knowledge 召回在 worker 中仍固定为 5，其余自动维护策略尚未接入运行循环。不要把“配置可解析”误认为“行为已生效”。
---
@ -251,66 +237,13 @@ Gateway 初始化时读取 `config.mcp.servers`：
| 模式 | 行为 |
|------|------|
| `foreground` | 当前轮等待一个或多个子 Agent；批量任务并发执行并按请求顺序聚合，全部持久化到 `agent_runs` |
| `background` | 异步执行并立即返回 run ID；仅限 Root 对具名 Agent 的单任务，结果经 durable inbox 由主 Agent 的 continuation Turn 汇总 |
| `inline` | 当前轮阻塞等待子 Agent 返回 |
| `background` | 后台运行，完成后通过原 channel/chat 通知 |
| `parallel` | 多个子 Agent 并发执行并聚合结果 |
### 具名 Agent Definition（身份设定）
默认工具集是只读工具：`file_read`、`file_search`、`content_search`、`web_fetch`、`http_request`、`calculator`。调用时可通过 `allowed_tools` 显式放开其他工具。后台任务会写入 `background_tasks` 表，默认 24 小时后清理。
启用 `agent_orchestration` 后，每个具名子 Agent 是一个 Markdown 文件：`<配置目录>/agents/<id>.md`（默认 `~/.picobot/agents/`）。frontmatter 只保存非秘密引用与限制（API key/base URL 仍在 `config.json`/`.env`）：
```md
---
id: researcher
description: 搜索、阅读并整理技术资料
llm_profile: research-sonnet
tools:
- file_read
- file_search
- content_search
- web_fetch
delegates:
- reviewer
skills:
- technical-research
limits:
timeout_secs: 900
max_iterations: 24
max_children: 4
max_depth: 3
signal:
delivery: steer
severity_allowlist: [info, warning, critical]
---
# Role
你是一名严谨的研究 Agent。只返回与任务有关的结论、证据和不确定性。
```
- `id`：`[a-z][a-z0-9_-]{0,63}`，文件名必须与 id 一致；`root`/`main`/`default`/`general` 为保留名。重复 ID、大小写折叠冲突、越界 symlink 或引用错误（未知 Provider profile、未注册/不可委托工具、未知 skill 或 delegate 目标）会拒绝整个候选运行代，绝不静默裁剪。
- `llm_profile`：引用 `config.json` 中 `agents` key，Definition 绑定 Provider 与模型，运行中不热切换。
- `tools`/`skills`：`tools` 直接指定该 Agent 可用的全部普通工具；`skills` 声明要求工具集含 `get_skill`，且只注入该 allowlist。运行时注入工具（`delegate`/`emit_signal`/`agent_task`）不能写进 `tools`。
- `delegates`：出边白名单，运行时还校验 ancestry 重复、`max_tree_depth` 与树级 `max_runs_per_tree` 预算。
- `signal`：可选信号契约。带该块的 run 才获得 `emit_signal` 工具（fail-closed）；`delivery: steer` 使信号在活动 Turn 的安全边界注入主 Agent，`queue` 走 continuation。
- 角色正文（`---` 之后）即 `role_prompt`，与 frontmatter 一起做 SHA-256 `definition_hash` 快照。
### 执行与投递
- 每次具名委托先持久化 run（含 execution_id、budget、Definition 快照），`allowed_tools` 只能收窄、不能扩权；子 Agent 输出视为不可信数据。
- foreground 父 run 等待子 run 时进入 `waiting_children` 且不占 provider/tool step permit；run quota 只约束 background 接纳，嵌套 foreground 并发上限为 1 时不死锁。
- background 接纳时预留 completion slot（容量条件更新），runner 持有 run quota permit 与 activity guard 直到 terminal commit；完成后 completion 事件落 `agent_inbox_events`，Session worker 按 `max_user_turn_burst_before_inbox`/`max_inbox_wait_secs` 公平调度，以 hidden trigger + 只读工具集的 continuation Turn 让主 Agent 汇总结果，不再直接发 Channel 通知。失败按 lease token 释放重试，超 `max_inbox_delivery_attempts` 进 dead-letter，重启经 activation recovery 收敛。
- `agent_task`（get/list/get_result/cancel）查询与控制 run；`agent_task.cancel` 会把该 run 未消费的普通信号标记 superseded。`/stop` 取消活动 run 但保留已存在的 pending 事件；archive/delete 取消 run 并将未消费事件 dead-letter。
- 后台 run 内可调用 `emit_signal`（key/severity/summary/details/dedupe_key），总数、速率、burst、severity allowlist、载荷大小/深度与冷却窗去重均由契约强制；steer 信号经两阶段 admission（claim → mailbox 预留 → admit(turn_id) → 激活）注入当前 Turn，`/stop` 时按 token 条件释放回 pending，绝不静默丢弃。
- 每个 run 的完成事件 payload 携带该 run 已发出的 signal IDs，主 Agent 可识别重复报告。
旧匿名 general 兼容路径已移除：委托必须指定具名 `target`。
后台子 Agent 通过 `TaskSupervisor::spawn_graceful` 注册；Gateway 关停时先收到取消信号，再在总宽限期内清理。
后台子 Agent 通过 `TaskSupervisor::spawn_graceful` 注册，受 `gateway.max_concurrent_background_tasks` 限制；Gateway 关停时先收到取消信号，再在总宽限期内清理。
## Session Todo 计划
@ -343,10 +276,8 @@ Gateway 关停顺序：
| `/rename <title>` | 重命名当前对话 |
| `/delete` | 删除当前对话 |
| `/compact` | 手动触发上下文压缩 |
| `/info [--json]` | 显示当前对话、累计 Token 与上下文窗口信息；可选 JSON 输出 |
| `/info` | 显示当前对话信息 |
| `/dump` | 保存当前对话为 markdown |
| `/?`, `/help` | 显示帮助 |
| `/mcp` | 显示 MCP 状态 |
| `/health` | 检查 PicoBot 运行依赖 |
| `/queue <message>` | 等当前 Turn 完成后作为下一 Turn 处理 |
| `/stop` | 停止当前任务并清空消息队列 |

23
resources/skills/about-picobot/references/commands.md
View File

@ -7,16 +7,6 @@ cargo build
# 启动网关 (默认 127.0.0.1:19876)
cargo run -- gateway
# 检查核心、配置相关和可选运行依赖；结构化输出加 --json
picobot health
picobot health --json
# 覆盖监听地址和端口
cargo run -- gateway --host 0.0.0.0 --port 19876
# Docker Compose 默认监听并发布 0.0.0.0:19876；也可分别覆盖
PICOBOT_GATEWAY_HOST=0.0.0.0 PICOBOT_PUBLISH_HOST=192.168.1.10 PICOBOT_GATEWAY_PORT=19876 docker compose up -d
# WebUI 随 Gateway 提供，浏览器打开
# http://127.0.0.1:19876/
@ -26,12 +16,6 @@ picobot pair
# 撤销全部设备并生成新配对码
picobot pair --revoke-all
# Docker 部署：必须在 Gateway 容器内执行，保证请求来自容器回环地址
docker compose exec picobot picobot pair --gateway-url http://127.0.0.1:19876
# 使用仓库测试 Compose 文件时
docker compose -f docker-compose.test.yml exec picobot picobot pair --gateway-url http://127.0.0.1:19876
# 修改 WebUI 后独立检查（Node.js 20+）
cd webui
npm ci
@ -46,11 +30,6 @@ cargo build
cargo run -- chat --pair-code <CODE>
cargo run -- chat
# 浏览器工具依赖（PicoBot 验证版本）
npm install -g agent-browser@0.33.0
agent-browser install
# Linux 缺少浏览器系统库时改用：agent-browser install --with-deps
# 安装并启动 Linux systemd 用户服务
picobot service install
picobot service start
@ -81,5 +60,3 @@ cargo test --test test_tool_calling -- --ignored
`test_scheduler` 和 `test_request_format` 不需要 API Key，也没有标记 `#[ignore]`。只有会真实调用 Provider 的测试需要从 `tests/test.env.example` 创建 `tests/test.env` 后使用 `-- --ignored`。
最终用户使用 WebUI 不需要单独构建；开发源码采用 Svelte 5、Vite 和 Bits UI，`cargo build` 会增量生成前端到 Cargo `OUT_DIR` 并嵌入二进制，生成产物不提交。WebUI 支持在线聊天、动态斜杠命令补全、日志、任务、记忆以及 `config.json`、`USER.md`、`AGENTS.md` 编辑。新设备默认必须配对，管理 API 与 WebSocket 共用设备鉴权；非回环部署仍需要 TLS。
配对码签发接口同时校验真实回环来源和 `~/.picobot/web_admin_token`。Docker 发布端口上的宿主机请求在容器内不是回环连接，因此应使用 `docker compose exec` 在 Gateway 容器中运行 `picobot pair`；不要手工读取或传递管理密钥。配对码为 8 位、5 分钟有效且只能消费一次。

114
resources/skills/about-picobot/references/config.md
View File

@ -1,9 +1,9 @@
# PicoBot 配置说明
配置文件加载顺序：`~/.picobot/config.json` → 当前目录 `./config.json`。
占位符 `<VAR_NAME>` 从启动环境替换。PicoBot 依次加载 `config.json` 同目录的 `.env`、`workspace_dir/.env`，最后保留启动进程已有环境变量作为最高优先级；workspace 层覆盖配置目录层。合并值也会进入进程环境，供 MCP 和工具子进程继承。workspace `.env` 不能修改用于定位自身的 `workspace_dir`。
占位符 `<VAR_NAME>` 从环境变量替换，环境变量从 `.env` 文件或系统环境读取。
Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取时 API Key、secret、password 和 token 会显示为 `********`，保持掩码不变再保存会保留原值；写入采用同目录临时文件替换。Gateway 会忽略可恢复的历史未知字段、类型不匹配字段和失效的非核心命名条目，在配置页按 JSON Pointer 显示诊断，但不会自动改写原始文件；“一键清除”由后端校验文件 revision 后原子删除已诊断项，普通保存仍严格拒绝无效配置。JSON 损坏、不可用的 `default` Agent 链路和无法安全构造运行代的错误不会被忽略。运行配置保存或清理后可执行 `picobot reload`、发送 `/reload`，或由根交互 Agent 在用户明确要求时调用 `reload_config` 工具。Gateway 会先校验候选配置，停止接收新工作并等待交互 Turn、Scheduler job 和后台子 Agent 到达安全边界后切换；失败时继续使用旧配置。`GET /api/config/reload/status` 可查询 generation、相位与最近错误。`gateway.host`、`gateway.port`、`workspace_dir` 和 `gateway.session_db_path` 的有效路径必须通过完整重启变更。`USER.md` 与 `AGENTS.md` 的修改用于后续构建的 Agent 上下文。
Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取时 API Key、secret、password 和 token 会显示为 `********`，保持掩码不变再保存会保留原值；写入采用同目录临时文件替换。运行配置保存后需要重启 Gateway，`USER.md` 与 `AGENTS.md` 的修改用于后续构建的 Agent 上下文。
## config.json 结构
@ -11,9 +11,7 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
{
"providers": {}, // LLM 提供商配置
"models": {}, // 模型配置
"agents": {}, // Provider/Model profile
"context_compaction": {}, // 上下文 reserve 预算与近期保留量
"agent_orchestration": {}, // 具名子 Agent Definition 与编排上限
"agents": {}, // agent 配置
"gateway": {}, // 网关配置
"client": {}, // 客户端配置
"channels": {}, // 渠道配置
@ -42,7 +40,6 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
| `model_id` | 模型标识名称 |
| `temperature` | 采样温度，可选 |
| `max_tokens` | 最大输出 token 数，可选 |
| `token_limit` | 模型上下文窗口硬上限，可选；未配置时默认为 128000，Agent 只能进一步收紧 |
| `input_type` | 模型支持的输入类型，如 `["text"]` 或 `["text", "image"]`，默认 `["text"]`. 纯内部使用，不会传递给 LLM API |
## agents 字段
@ -52,37 +49,7 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
| `provider` | string | - | 提供商名称（对应 providers key） |
| `model` | string | - | 模型名称（对应 models key） |
| `max_tool_iterations` | int | 99 | 最大工具调用轮数 |
| `token_limit` | int | 使用模型上限 | 可选的 Agent 上限；有效窗口取 Agent 与模型（模型未配置时为 128000）的最小值 |
## context_compaction 字段
自动压缩只使用 reserve 公式 `context_tokens > context_window - effective_reserve`。小窗口下 `effective_reserve = min(reserve_tokens, context_window / 2)`，近期原样保留量最多为有效阈值的一半。
| 字段 | 默认 | 说明 |
|------|------|------|
| `enabled` | true | 只控制 Turn 前自动压缩；不禁用 `/compact` 或 overflow 恢复 |
| `reserve_tokens` | 16384 | 为输出、工具迭代和估算误差预留的输入窗口 |
| `keep_recent_tokens` | 20000 | checkpoint 后尽量原样保留的近期历史 token |
## agent_orchestration 字段
子 Agent 编排是 PicoBot 的内在机制，始终启用、不可关闭；该配置块只控制定义目录与各类上限。`definitions_dir` 相对 `config.json` 所在目录解析，且不得通过绝对路径或 symlink 逃逸该受信任配置目录。Gateway 启动和热重载会严格校验全部 Markdown Definition；任一无效 Provider profile、工具、Skill 或委托目标会拒绝整个候选运行代。
| 字段 | 默认 | 说明 |
|------|------|------|
| `definitions_dir` | agents | 第一层 `*.md` Definition 目录 |
| `max_tree_depth` | 4 | Root-relative 委托深度硬上限 |
| `max_runs_per_tree` | 16 | 单任务树 run 预算（树级原子计数强制） |
| `max_concurrent_runs` / `max_concurrent_runs_per_session` | 6 / 4 | background run 接纳配额（global→session 顺序获取，runner 持有至 terminal commit；foreground 不占） |
| `max_concurrent_provider_steps` / `..._per_session` | 8 / 4 | Provider step 上限（global→session） |
| `max_concurrent_tool_steps` / `..._per_session` | 16 / 8 | 普通工具 step 上限（global→session） |
| `max_pending_inbox_events_per_session` | 128 | durable inbox 容量（条件更新，预留槽不可被信号挤占） |
| `inbox_event_ttl_hours` | 168 | inbox 事件 TTL（已预留配置；TTL 清理尚未实现） |
| `max_inbox_delivery_attempts` | 8 | inbox 最大投递次数，超限 dead-letter |
| `max_user_turn_burst_before_inbox` | 4 | 用户 Turn 公平调度阈值 |
| `max_inbox_wait_secs` | 30 | inbox 最大等待阈值 |
已实现：具名 foreground 与 background（Root 单任务或批量）、内联 `provider`/`model` 或 `llm_profile`、工具集由定义文件 `tools` 决定、批量并发、父子委托边校验、durable inbox continuation（空闲时完成即返回）、`emit_signal`（queue/steer）、run quota 与 step gate、内置 general-purpose 定义与 WebUI「子 Agent」管理页。未开放：子 Agent 发起的 background、`idempotency_key` 工具入口。旧匿名 general 兼容路径已移除，委托必须指定具名 `target`。
| `token_limit` | int | 128000 | 上下文 token 限制 |
## gateway 字段
@ -92,8 +59,9 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
| `port` | int | 19876 | 监听端口 |
| `require_pairing` | bool | true | 是否要求 WebUI 与 CLI 设备先使用一次性代码配对 |
| `session_ttl_hours` | int | - | 兼容/预留字段；当前没有会话 TTL 清理循环 |
| `session_db_path` | string | - | SQLite 数据库路径，默认在配置目录 `data/` 下 |
| `session_db_path` | string | - | SQLite 数据库路径，默认在 workspace 下 |
| `cleanup_interval_minutes` | int | - | 兼容/预留字段；当前没有按此间隔运行的 session 清理任务 |
| `max_concurrent_background_tasks` | int | 10 | delegate 后台子任务最大并发数 |
| `scheduler` | object | - | 调度器配置 |
### gateway.scheduler 字段
@ -102,8 +70,8 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
|------|------|------|------|
| `enabled` | bool | true | 是否启动调度器并注册 cron 工具 |
| `poll_interval_secs` | int | 60 | 检查到期任务的轮询间隔 |
| `max_concurrent` | int | 1 | 同时执行的 Scheduled Run 上限，运行时限制在 1–256；投递使用独立有界并发 |
| `execution_timeout_secs` | int | 900 | 单个定时任务 Agent 执行的硬超时；Job 执行租约额外覆盖关停宽限，投递由持久化 outbox 独立恢复 |
| `max_concurrent` | int | 1 | 每批到期任务的最大并发数，运行时限制在 1–256 |
| `execution_timeout_secs` | int | 900 | 单个定时任务 Agent 执行的硬超时；租约会覆盖执行和托管投递等待 |
## memory 字段
@ -111,16 +79,12 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
|------|------|------|------|
| `consolidation_provider` | string | 主 Agent provider | 当前记录在 MemoryManager 中供后续归并使用；压缩摘要仍使用 Session provider |
| `consolidation_model` | string | 主 Agent model | 当前记录在 MemoryManager 中供后续归并使用；压缩摘要仍使用 Session model |
| `recall_limit` | int | 5 | 每轮自动注入上下文的知识记忆条数上限 |
| `recall_min_relevance` | float | 0.25 | 自动召回的相关性硬门槛（命中词项占比低于此值则丢弃） |
| `recall_min_score` | float | 0.25 | 自动召回的综合分门槛（相关度+重要度+时效加权） |
| `recall_recency_half_life_days` | int | 30 | 自动召回时效衰减的半衰期（天） |
| `recall_timeout_ms` | int | 1000 | 自动召回搜索的硬超时；超时本轮不注入记忆 |
| `recall_limit` | int | 5 | 预期的每轮知识召回上限；当前 worker 固定使用 5 |
| `idle_consolidation_minutes` | int | 10 | 预留的空闲归并阈值；当前无对应循环 |
| `timeline_retention_days` | int | 90 | 默认日常维护巡检删除超过该期限的 Timeline；Knowledge 不受影响 |
| `max_failures_before_degrade` | int | 3 | 预留的归并失败阈值；当前无失败降级循环 |
自动召回每轮用当前用户输入做关键词检索（jieba 分词 + FTS5），按「词项相关度 0.5 + 重要度 0.3 + 时效 0.2」加权，通过相关性/综合分双门槛后才注入；搜索有硬超时保证不拖慢 Turn。Timeline 不自动召回，需显式 `timeline_recall`。idle consolidation 和失败降级循环尚未接入。Timeline 清理由默认启用的 `picobot-routine-maintenance` Scheduled Run 执行；该任务使用 `never` 策略，结构化结果只进入运行审计和 Health。
注意：当前 worker 的 Knowledge 召回数量仍固定为 5；idle consolidation 和失败降级循环尚未接入。Timeline 清理由默认启用的 `picobot-routine-maintenance` 定时巡检执行。
## channels.feishu 字段
@ -130,18 +94,9 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
| `app_id` | string | - | 飞书应用 ID |
| `app_secret` | string | - | 飞书应用密钥 |
| `allow_from` | []string | ["*"] | 允许交互的用户列表 |
| `require_mention` | bool | true | 群聊中是否必须明确 @ 机器人；无法解析机器人身份时安全地忽略群消息 |
| `agent` | string | - | 使用的 agent 名称 |
| `media_dir` | string | ~/.picobot/media/feishu | 配置默认值；Gateway 注册渠道时会覆盖为 `{workspace}/media/feishu` |
| `reaction_emoji` | string | "Typing" | 回复意向表达的表情 |
| `live_updates` | bool | false | 是否用单张卡片实时编辑活动 Turn；关闭时只发送终态 |
| `live_update_interval_ms` | int | 500 | 卡片更新最小间隔，运行时限制在 250–5000ms |
| `max_image_bytes` | int | 10485760 | 单个入站/出站图片的最大字节数 |
| `max_file_bytes` | int | 26214400 | 单个入站/出站文件、音频或视频的最大字节数 |
| `media_dir_max_bytes` | int | 536870912 | 飞书媒体目录容量上限；达到上限后拒绝新下载，不自动删除旧文件 |
| `request_timeout_secs` | int | 30 | 单次飞书 HTTP 请求及响应体读取的硬超时，运行时限制在 5–120 秒 |
飞书属于外部渠道：无论是否开启实时卡片，都不会接收模型 reasoning；工具只显示紧凑状态。渠道配置可通过 Gateway 配置重载生效。
## mcp 字段
@ -155,7 +110,6 @@ MCP 服务器单条配置:
| 字段 | 说明 |
|------|------|
| `name` | 服务器名称 |
| `enabled` | 是否启用，默认 true；关闭后启动/重载时不连接该服务器 |
| `transport` | 传输方式: `stdio`、`sse`、`streamable-http` |
| `command` | 启动命令（stdio 模式） |
| `args` | 命令参数 |
@ -163,54 +117,14 @@ MCP 服务器单条配置:
| `url` | URL（sse / streamable-http 模式） |
| `headers` | HTTP 传输额外请求头 |
| `tool_timeout_secs` | 单独的超时设置 |
| `tool_settings` | 按 MCP 原始工具名索引的本地执行属性声明；每项可设 `read_only` 与 `exclusive`，均默认 false |
`tool_settings` 示例：
```json
{
"tool_settings": {
"read_file": { "read_only": true },
"refresh_index": { "exclusive": true }
}
}
```
`可并发` 不需要也不能单独配置：它始终等于 `read_only && !exclusive`。未声明的 MCP 工具按可能有副作用且顺序执行处理。WebUI「工具 → MCP」展开服务器后可编辑这些复选框；编辑先暂存，可跨多个工具一次“保存并应用”并热重载，离开 MCP 标签或刷新页面会丢弃未保存草稿。
## browser 字段
浏览器工具默认开启并注册 `browser` 与 `browser_profiles` 工具。缺少外部依赖不会阻止 Gateway 启动，但实际调用会返回安装错误，`picobot health` 会提前判定。上层由 PicoBot 管理浏览器生命周期与媒体，底层调用 agent-browser JSON CLI；不再依赖 Fantoccini、ChromeDriver 或 WebDriver。
浏览器工具默认关闭，开启后注册 `browser` 工具。依赖 Chrome/Chromium 与 chromedriver/WebDriver。
| 字段 | 类型 | 默认 | 说明 |
|------|------|------|------|
| `enabled` | bool | true | 是否启用浏览器工具；关闭后不注册 `browser` |
| `command` | string | agent-browser | CLI 名称或绝对路径 |
| `enabled` | bool | false | 是否启用浏览器工具 |
| `webdriver_url` | string | http://127.0.0.1:9515 | WebDriver 服务地址 |
| `headless` | bool | true | 是否无头运行 |
| `browser_executable_path` | string | - | 自定义 Chrome/Chromium 可执行文件路径 |
| `max_sessions` | int | 4 | 同时保留的普通 dialog 临时浏览器会话上限；持久身份不计入 |
| `idle_timeout_secs` | int | 3600 | 大于零；普通 dialog 临时浏览器的空闲退出及 Manager 回收时间；持久浏览器不使用该超时 |
| `command_timeout_secs` | int | 120 | 单次 CLI 调用硬超时 |
| `max_output_chars` | int | 50000 | 页面来源文本输出上限 |
| `content_boundaries` | bool | true | 启用 agent-browser 不可信页面边界元数据 |
| `allowed_domains` | []string | [] | 可选域名白名单；空数组表示不启用域名限制 |
| `allow_private_hosts` | bool | false | 是否允许回环、私网和本地域名 |
| `artifact_dir` | string | ~/.picobot/media/browser | 截图产物目录 |
| `persistence.profile_dir` | string | ~/.picobot/browser/profiles | 持久 ID、语义标签和 Profile 目录的受控根目录 |
持久化不是配置模式。`browser` 调用省略 `persistent_id` 时使用当前 dialog 的普通临时浏览器，并在达到 `idle_timeout_secs` 后自动退出；长期工作需要保持浏览器进程或保留登录和站点状态时，Agent 可自主调用 `browser_profiles(create,label=...)` 生成 `picobot-profile-<uuid>`，对应数据位于 `profile_dir/<id>`，然后在该工作的后续每个 action 中持续传入同一个 ID。持久浏览器禁用 daemon 空闲自动退出，只通过显式 `browser(close)` 或 Profile 删除关闭。没有默认 ID，也不会按 dialog 自动选择持久身份。同一 ID 跨 dialog 共享底层 session 与串行锁，不同 ID 各自独立并可并发。`browser_profiles` 还支持 `set_label`、`list` 和使用精确 ID 的 `delete`；标签可重命名，但选择浏览器始终使用不可变 ID。
agent-browser 0.33.0 不允许持久 Profile 与 `allowed_domains` 同时使用。配置域名限制后普通临时浏览器仍可用，创建或使用持久身份会被拒绝，health 会提示该可选能力受限。Profile 包含登录凭据，应把目录视为敏感数据，不得提交到版本控制、跨用户共享或放在不受信任的网络文件系统中。
旧字段 `webdriver_url`、`chrome_path` 不再接受。推荐安装 `agent-browser@0.33.0` 后运行 `agent-browser install`；Linux 可运行 `agent-browser install --with-deps`。使用前用 `picobot health` 检查 CLI、浏览器安装和真实 headless 启动环境。
### 浏览器依赖故障处置
| health / 工具错误 | 处置 |
|---|---|
| `agent-browser` 未找到 | 运行 `npm install -g agent-browser@0.33.0`，或 `cargo install agent-browser --version 0.33.0 --locked` |
| CLI 已安装但找不到 Chrome/Chromium | 运行 `agent-browser install`；已有浏览器则设置 `browser_executable_path` 或 `AGENT_BROWSER_EXECUTABLE_PATH` |
| Linux 缺少共享库/系统包 | 运行 `agent-browser install --with-deps`，然后再运行 `agent-browser doctor` |
| 安装状态不明确 | 先运行 `picobot health` 获取 PicoBot 视角的结果，再运行 `agent-browser doctor` 查看完整上游诊断 |
不得在 Agent 工具调用中自动安装或执行 `doctor --fix`；安装会修改系统且可能需要管理员权限，应把命令报告给用户，由用户确认后执行。临时不需要浏览器时可设置 `browser.enabled=false`，此时 health 不要求 agent-browser/Chrome。
| `chrome_path` | string | - | 自定义 Chrome/Chromium 路径 |

160
resources/skills/about-picobot/references/db-schema.md
View File

@ -1,8 +1,8 @@
# PicoBot 数据库表结构
数据库为 SQLite，默认位于配置目录（`~/.picobot`）`data/` 下的 `picobot.db`，与 workspace 相互独立。
数据库为 SQLite，默认位于 workspace 下的 `picobot.db`。
连接启用 WAL、`synchronous=NORMAL`、foreign keys、5 秒 busy timeout，连接池最多 8 个连接。当前 `PRAGMA user_version=11`；启动时会在单个事务内迁移旧库，遇到比程序更新的 schema version 会拒绝启动。
连接启用 WAL、`synchronous=NORMAL`、foreign keys、5 秒 busy timeout，连接池最多 8 个连接。当前 `PRAGMA user_version=1`；启动时会在事务内补齐旧库字段和索引，遇到比程序更新的 schema version 会拒绝启动。
## sessions 表
@ -22,13 +22,7 @@
| `archived_at` | INTEGER | 归档时间（Unix 毫秒），NULL 表示未归档 |
| `deleted_at` | INTEGER | 软删除时间戳 |
| `last_consolidated_at` | INTEGER | 上次记忆归并时间 |
| `last_compressed_message_at` | INTEGER | 最近 checkpoint 时间戳（兼容/诊断字段，不作为恢复边界） |
| `active_context_checkpoint_id` | TEXT | 当前 Provider 历史投影使用的 checkpoint ID；NULL 表示使用全部原始消息 |
| `context_generation` | INTEGER | checkpoint CAS 提交代；历史清空/改写时递增并清除活动指针 |
| `delivery_context` | TEXT | 渠道声明的可跨 Turn 复用投递上下文 JSON（如飞书 thread/root 身份）；一次性 reply/reaction ID 永不写入 |
| `delivery_context_updated_at` | INTEGER | delivery_context 最后更新时间 |
`session_turn_usage` 以 `turn_id` 幂等保存已提交 Turn 的 Provider usage，包括累计输入、输出、缓存输入、请求数和最后一次请求的 prompt tokens。它与 Turn 消息批次在同一事务中提交，供 WebUI 状态栏和 `/info` 使用；升级前历史无法可靠回填，因此统计起点以首条 usage 记录为准。
| `last_compressed_message_at` | INTEGER | 上次上下文压缩边界时间戳 |
`(channel, chat_id, dialog_id)` 唯一。普通列表排除 `deleted_at`；是否包含归档记录由查询参数决定。
@ -47,103 +41,30 @@
| `tool_calls` | TEXT | 工具调用参数 JSON |
| `source` | TEXT | 消息来源（跨会话消息时标记来源 session_id） |
| `created_at` | INTEGER | 创建时间（Unix 毫秒） |
| `reasoning_content` | TEXT | 可展示的模型 reasoning（如有） |
| `provider_state` | TEXT | Provider 私有回放状态 JSON；只回放给匹配 Provider，不下发客户端或 Channel |
| `turn_id` | TEXT | 产生该消息的活动 Turn ID |
| `iteration` | INTEGER | Agent 工具循环中的迭代序号 |
| `completion_status` | TEXT | `completed` / `cancelled` / `interrupted`，旧数据默认 completed |
| `client_visibility` | TEXT | `visible` / `hidden`，默认 visible；hidden 只供模型回放（continuation 内部触发），客户端历史/投影/投递一律过滤 |
| `turn_origin` | TEXT | `user` / `agent_continuation` / `scheduled`，默认 user；客户端据此渲染"后台结果处理"标签而不创建用户气泡 |
| `reasoning_content` | TEXT | provider 返回的推理内容（如有） |
`(session_id, seq)` 有唯一索引，防止并发写入重复序号。物理删除 session 会通过外键级联删除 messages；普通对话删除使用 `deleted_at` 软删除，因此保留关联行。索引 `(session_id, client_visibility, seq)` 支撑按可见性分层查询。
`(session_id, seq)` 有唯一索引，防止并发写入重复序号。删除 session 会通过外键级联删除 messages。
## context_checkpoints 表（schema v10）
## background_tasks 表
checkpoint 只保存累计摘要和精确 raw-tail 边界，不复制或删除原始消息。每个 Session 的 `active_context_checkpoint_id` 最多指向其中一行；历史行保留用于审计。
| 字段 | 说明 |
|------|------|
| `id` | checkpoint ID，主键 |
| `session_id` / `generation` | 所属 Session 与单调提交代；组合唯一 |
| `parent_checkpoint_id` | 上一个累计 checkpoint（审计链） |
| `summary` | 不含 Provider 私有 reasoning 的累计摘要 |
| `first_retained_seq` | Provider 原样保留尾部的第一条 durable seq |
| `source_max_seq` | 生成候选时快照的最大 seq |
| `trigger_reason` | manual / auto / overflow |
| `provider_kind` / `model` | 生成摘要的 Provider/model |
| `tokens_before` / `tokens_after` | 候选验证和诊断数据 |
| `degraded` | 是否为 Provider overflow 或发送前硬超限的确定性无语义降级 |
| `created_at` | 创建时间 |
checkpoint 插入、Session 活动指针更新与 `context_generation` 递增在同一事务内完成。`/clear` 原子删除 messages 并使活动 checkpoint 失效；只有物理删除 Session 才会通过外键级联删除全部 checkpoint，普通 `/delete` 软删除会保留 checkpoint 行。
## agent_runs 表（schema v8，Agent 编排）
每次具名委托（foreground 与 background 一致）先落库再执行；`execution_id` 条件更新保证迟到结果丢弃。批量委托只是多个 run 的集合，不再存在组头（schema v7 的 `agent_run_groups` 表已删除）。
delegate 后台子任务表。`session_id` 不使用数据库外键，因为 session 使用软删除，关联关系由应用层维护。
| 字段 | 类型 | 说明 |
|------|------|------|
| `id` | TEXT PK | run ID |
| `root_session_id` | TEXT | 根会话 |
| `parent_run_id` | TEXT FK | 父 run（RESTRICT），NULL 表示 Root 直接委托 |
| `caller_agent_id` / `caller_scope_id` | TEXT | 调用方身份；Root 的 caller_scope_id 固定 `"ROOT"` |
| `agent_id` / `definition_hash` / `provider_profile` | TEXT | Definition 快照（绑定运行代，运行中不热切换） |
| `provider_name` / `model_id` | TEXT | Provider 与模型 |
| `mode` | TEXT | foreground / background |
| `depth` | INTEGER | 委托深度（>=1） |
| `plan_item_id` | TEXT | 绑定计划子项（接纳时原子领取） |
| `execution_id` | TEXT | 执行尝试 ID，唯一索引 |
| `task` / `context_json` | TEXT | 任务与调用方上下文 |
| `budget_json` | TEXT | 树级剩余预算 |
| `signal_contract_json` / `signal_delivery` | TEXT | Definition 信号契约快照与投递 lane（queue/steer） |
| `status` | TEXT | queued / running / waiting_children / completed / failed / timed_out / cancelled / interrupted |
| `result` / `error` | TEXT | 终态完整结果/错误（get_result 与 tool 结果同源） |
| `prompt_tokens` / `completion_tokens` / `cost` | INTEGER/REAL | Provider usage |
| `tool_calls_count` / `iterations` | INTEGER | 执行统计 |
| `runtime_generation` / `attempt` | INTEGER | 运行代与重试次数 |
| `completion_slot_reserved` | INTEGER | background 完成槽预留 |
| `deadline_at` / `started_at` / `finished_at` / `created_at` / `updated_at` | INTEGER | 时间线 |
| `revision` | INTEGER | 客户端投影修订号 |
索引：`execution_id` 唯一、`(root_session_id, caller_scope_id, idempotency_key)` 部分唯一、`(root_session_id, created_at DESC)`、`(parent_run_id, created_at)`、`(runtime_generation, status, deadline_at)`（恢复扫描）。
## agent_session_state 表（schema v8，Agent 编排）
每根会话一行，inbox 容量与客户端 revision 的权威计数：
| 字段 | 说明 |
|------|------|
| `root_session_id` | TEXT PK |
| `revision` | 单调客户端投影修订号 |
| `pending_event_count` | 未消费事件数（容量条件更新） |
| `reserved_completion_slots` | 已接纳 background run 预留的完成槽 |
| `updated_at` | 最后更新时间 |
容量判断在同一写事务内做条件 `UPDATE`（`pending + reserved + 新增 <= 上限`），杜绝并发 `COUNT(*)` 漂移。
## agent_inbox_events 表（schema v8，Agent 编排）
background 完成/信号投递的唯一事实源：`pending → leased → admitted → consumed`，失败按 token 释放回 pending，超限进 dead-letter，崩溃靠 lease 过期恢复。
| 字段 | 说明 |
|------|------|
| `id` | TEXT PK |
| `root_session_id` | TEXT | 根会话 |
| `run_id` | TEXT FK NOT NULL（RESTRICT） |
| `event_type` | signal / completion |
| `event_key` | 去重键（signal 含冷却窗口 id） |
| `delivery` | queue / steer |
| `requires_continuation` | 是否反向启动 continuation Turn（cancel 产物为 false） |
| `severity` / `payload_json` | 信号级别与结构化载荷（completion 含 signal_ids） |
| `status` | pending / leased / admitted / consumed / superseded / dead_letter |
| `attempt_count` / `lease_token` / `lease_until` / `next_attempt_at` | 投递尝试与租约 |
| `admitted_turn_id` | steer 事件接纳的 Turn（/stop 按此条件释放） |
| `last_error` | 最近失败原因 |
| `revision` | 投影修订号 |
| `updated_at` | 最后更新时间 |
| `created_at` / `consumed_at` / `superseded_at` / `dead_lettered_at` / `fallback_notified_at` / `fallback_suppressed_reason` | 状态时间线 |
`(run_id, event_type, event_key)` 唯一（signal 冷却窗去重）。消费/死信会同步递减 `agent_session_state.pending_event_count`。
| `id` | TEXT PK | 后台任务 ID |
| `session_id` | TEXT | 所属会话 |
| `channel` | TEXT | 回传渠道 |
| `chat_id` | TEXT | 回传目标对话 |
| `prompt` | TEXT | 子任务提示 |
| `allowed_tools` | TEXT | 允许工具 JSON |
| `status` | TEXT | pending / running / completed / failed / cancelled |
| `result` | TEXT | 执行结果 |
| `error` | TEXT | 错误信息 |
| `tool_calls_count` | INTEGER | 工具调用次数 |
| `iterations` | INTEGER | Agent 迭代次数 |
| `started_at` | INTEGER | 开始时间 |
| `finished_at` | INTEGER | 结束时间 |
| `created_at` | INTEGER | 创建时间 |
## task_plans / task_items 表
@ -176,21 +97,22 @@ background 完成/信号投递的唯一事实源：`pending → leased → admit
| `name` | TEXT | 任务名称 |
| `schedule` | TEXT | 调度规则 JSON（at/every/cron） |
| `prompt` | TEXT | 任务提示词 |
| `agent_id` | TEXT | 可选命名 Agent；NULL 表示 Root |
| `channel` | TEXT | 目标渠道 |
| `channel` | TEXT | 执行渠道 |
| `chat_id` | TEXT | 目标对话 |
| `delivery_policy` | TEXT | `always` / `on_alert` / `never` |
| `model` | TEXT | 可选模型标记；当前会存储/展示，但 Scheduler 执行仍使用默认 Agent 模型 |
| `enabled` | INTEGER | 是否启用 (1/0) |
| `delete_after_run` | INTEGER | 执行后自动删除 (1/0) |
| `next_run_at` | INTEGER | 下次执行时间 |
| `last_run_at` | INTEGER | 上次执行时间 |
| `last_outcome` | TEXT | 最近结构化结果：ok/alert/failed/refused/unknown |
| `last_status` | TEXT | 上次执行状态 |
| `last_error` | TEXT | 上次错误信息 |
| `locked_at` | INTEGER | 本次领取时间 |
| `lock_owner` | TEXT | 本次 occurrence 的唯一 owner token |
| `lease_until` | INTEGER | 租约到期时间 |
| `lock_owner` | TEXT | 领取任务的 Scheduler owner UUID |
| `lease_until` | INTEGER | 租约到期时间；进程崩溃后允许其他实例重新领取 |
| `created_at` | INTEGER | 创建时间（Unix 毫秒） |
| `updated_at` | INTEGER | 更新时间（Unix 毫秒） |
Scheduler 在领取事务中插入 JobRun、快照执行/投递字段并推进下次时间。`At` 在领取时立即禁用；执行失败或崩溃不重放同一个 occurrence。
Scheduler 使用原子 `UPDATE ... RETURNING` 领取到期任务。任务结果、下次运行时间和租约释放在同一事务中提交，并校验 owner，防止过期 worker 覆盖已恢复的任务。
## job_runs 表
@ -198,26 +120,12 @@ Scheduler 在领取事务中插入 JobRun、快照执行/投递字段并推进
|------|------|------|
| `id` | INTEGER PK | 自增 ID |
| `job_id` | TEXT FK | 关联任务，外键关联 scheduled_jobs(id) |
| `scheduled_for` | INTEGER | 本 occurrence 原计划时间 |
| `agent_run_id` | TEXT FK | 顶层 AgentRun 审计记录 |
| `agent_id` | TEXT | claim-time Agent 快照 |
| `delivery_policy` | TEXT | claim-time 投递策略快照 |
| `target_channel` / `target_chat_id` | TEXT | claim-time 目标快照 |
| `target_session_id` | TEXT | 首次投递时固定的目标 dialog |
| `started_at` | INTEGER | 开始时间 |
| `finished_at` | INTEGER | 结束时间 |
| `status` | TEXT | claimed/running/completed/failed/timed_out/cancelled/interrupted/unknown |
| `outcome` | TEXT | ok/alert/failed/refused/unknown；与 status 有联合约束 |
| `message` | TEXT | 面向用户的结构化结果 |
| `diagnostic` | TEXT | 有界内部诊断 |
| `status` | TEXT | 执行状态 |
| `output` | TEXT | 执行输出 |
| `error` | TEXT | 错误信息 |
| `duration_ms` | INTEGER | 耗时（毫秒） |
| `delivery_status` | TEXT | awaiting_result/not_requested/suppressed/pending/delivering/delivered/failed |
| `delivery_attempts` | INTEGER | 持久化投递尝试次数，最多 3 次 |
| `delivery_next_attempt_at` | INTEGER | 瞬态失败后的退避时间 |
| `delivery_lease_owner` / `delivery_lease_until` | TEXT / INTEGER | outbox 领取租约 |
| `delivery_error` | TEXT | 清洗后的投递失败摘要 |
JobRun 是执行结果和投递状态的唯一权威。顶层 AgentRun 与 JobRun 终态、Job 最近摘要和租约释放原子提交；启动恢复将遗留运行归为 `unknown`，不会自动重跑。
## llm_calls 表
@ -229,9 +137,9 @@ JobRun 是执行结果和投递状态的唯一权威。顶层 AgentRun 与 JobRu
| `created_at` | INTEGER | 调用时间 |
| `provider` | TEXT | 提供商类型 |
| `model` | TEXT | 模型名称 |
| `request_body` | TEXT | 请求摘要 JSON；旧记录可能是完整请求体 |
| `request_body` | TEXT | 请求体 JSON |
| `response_body` | TEXT | 响应体 JSON |
| `error` | TEXT | 错误信息 |
| `duration_ms` | INTEGER | 耗时（毫秒） |
旧数据中的 `request_body`/`response_body` 可能包含用户内容，排障和导出数据库时应按敏感数据处理。新 Provider 请求的 `request_body` 只保存模型、消息数、工具数和 stream 标志等摘要；错误响应仍可能包含服务端回显内容。
`request_body`/`response_body` 可能包含用户内容，排障和导出数据库时应按敏感数据处理。

20
resources/skills/about-picobot/references/faq.md
View File

@ -22,23 +22,9 @@
内置 Skill 只在目标目录不存在时释放，不会覆盖已安装目录。升级 PicoBot 后如需获取新版内置文档，应先备份自己的修改，再删除旧的 `~/.picobot/skills/about-picobot/` 并重启。也可把定制版放在 `{workspace}/skills/about-picobot/`，它的优先级更高。
## Q: Docker 部署如何获取 WebUI 设备配对码？
在 Gateway 容器内执行：
```bash
docker compose exec picobot picobot pair --gateway-url http://127.0.0.1:19876
```
使用 `docker-compose.test.yml` 时增加 `-f docker-compose.test.yml`。签发接口要求请求来自 Gateway 的真实回环地址，并校验 `/app/.picobot/web_admin_token`；因此不要从宿主机经发布端口直接请求，也不要复制或输出管理密钥。代码为 8 位、5 分钟有效且只能使用一次。
## Q: 数据库文件在哪里？
默认 `{config_dir}/data/picobot.db`，`config_dir` 默认 `~/.picobot`，与 workspace 相互独立。
## Q: 如何禁用某个 skill 或 MCP 服务器？
Skill 安装后默认启用，可在 WebUI「工具 → Skills」页用开关禁用；禁用状态记录在 `~/.picobot/skills_state.json`，被禁用的 skill 不再进入提示词、列表和 `get_skill`。MCP 服务器在配置 `mcp.servers[].enabled`（默认 true）中控制，可在 WebUI「工具 → MCP」页开关；关闭后下次启动/重载时不连接该服务器。
默认 `{workspace}/picobot.db`，workspace 默认 `~/.picobot/workspace/`。
## Q: 如何查看历史会话？
@ -50,7 +36,7 @@ Skill 安装后默认启用，可在 WebUI「工具 → Skills」页用开关禁
## Q: 上下文压缩是什么意思？
对话接近模型 token 限制时，PicoBot 用一份累计 checkpoint 摘要替代 Provider 上下文中的旧前缀，并原样保留近期消息尾部。原始消息和工具结果仍永久保存在聊天历史/SQLite 中，只是不再永久占用模型上下文；语义摘要还可通过 `timeline_recall` 检索。Model 的 `token_limit` 是窗口硬上限，未配置时为 128K；Agent 的可选 `token_limit` 只能收紧它，两者都有时取最小值。自动阈值采用窗口减 reserve 的机制，摘要输入按有效窗口动态限制而非固定 32K；换成更小模型后若历史已经超限，会在首次普通模型请求前压缩或明确降级。`/compact` 可在阈值前手动强制执行。
对话历史过长超出模型 token 限制时，系统自动精简历史消息。压缩后旧消息可通过 `timeline_recall` 工具检索。
## Q: 如何修改 gateway 监听端口？
@ -64,7 +50,7 @@ LLM 调用记录存储在 `llm_calls` 表中。可通过 SQLite 客户端直接
## Q: 为什么修改了某些 memory 配置却没有看到行为变化？
当前 `recall_limit` 以及新增的 `recall_min_relevance`、`recall_min_score`、`recall_recency_half_life_days`、`recall_timeout_ms` 均已生效，控制每轮自动知识召回的相关性门槛、加权与超时。`idle_consolidation_minutes`、`timeline_retention_days` 和 `max_failures_before_degrade` 仍只是配置解析：自动 idle consolidation、Timeline 清理（由维护任务执行）和失败降级循环尚未接入。以当前代码行为为准。
当前 `recall_limit`、`idle_consolidation_minutes`、`timeline_retention_days` 和 `max_failures_before_degrade` 都能被配置解析，但每轮 Knowledge 召回仍固定为 5，自动 idle consolidation、Timeline 清理和失败降级循环尚未接入。以当前代码行为为准。
## Q: Gateway 为什么无法立即退出？

99
resources/skills/about-picobot/references/tools.md
View File

@ -15,8 +15,6 @@
`files` 支持绝对路径和 workspace 相对路径，媒体类型由文件扩展名/MIME 自动判断。目前 schema 不支持手工指定 `file_types`。
目标可以是当前会话，例如把浏览器截图或生成文件直接交付给正在聊天的用户。WebUI/TUI 中，同一 active Turn 的附件会并入本轮最终回复：先显示工具调用与结果，再显示自然回复和内联图片，不生成 `[message from ...]` 自引用前缀。后续模型回放只读取 assistant 附件的文本清单，不会把图片放入 assistant 内容块。
### 示例
```json
@ -50,15 +48,14 @@
## Cron 定时任务工具
Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅在 `gateway.scheduler.enabled=true` 时注册。
Cron 不是一个带 `action` 的统一工具，而是六个独立工具；仅在 `gateway.scheduler.enabled=true` 时注册。
| 工具 | 主要参数 | 说明 |
|------|----------|------|
| `cron_add` | `schedule`, `prompt`, `channel`, `chat_id`; 可选 `name`, `agent_id`, `delivery_policy` | 创建任务 |
| `cron_add` | `schedule`, `prompt`, `channel`, `chat_id`; 可选 `name`, `model` | 创建任务 |
| `cron_list` | 可选 `status=all|enabled|disabled` | 列出任务 |
| `cron_runs` | `job_id`; 可选 `run_id`, `limit` | 查询结构化运行和投递记录，包括静默结果 |
| `cron_update` | `job_id`; 可选 `name`, `prompt`, `schedule`, `channel`, `chat_id`, `agent_id`, `delivery_policy` | 更新指定字段；`agent_id:null` 切回 Root |
| `cron_remove` | `job_id` | 无活动 Run 或 pending delivery 时永久删除任务 |
| `cron_update` | `job_id`; 可选 `prompt`, `schedule`, `channel`, `chat_id`, `model` | 更新指定字段 |
| `cron_remove` | `job_id` | 永久删除任务和关联 job runs |
| `cron_enable` | `job_id` | 启用并重新计算下次运行时间 |
| `cron_disable` | `job_id` | 禁用但保留任务 |
@ -70,9 +67,7 @@ Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅
{"type":"cron","expr":"0 0 9 * * *","tz":"Asia/Shanghai"}
```
时间戳和间隔单位为毫秒；Cron 表达式为 6 段（秒、分、时、日、月、周）。过去时间的 At 不能创建、更新或直接重新启用。定时 Agent 不复用聊天历史，`prompt` 必须包含完整上下文；`agent_id` 省略时使用 Root，否则使用当前 AgentCatalog 中的命名 Agent。
每次运行必须恰好一次调用 `complete_scheduled_run(outcome,message)`，outcome 只能是 `ok`、`alert`、`failed`、`refused`。普通最终文本不会被解释为结果，缺少结构化终结会 fail-closed。投递完全由 Scheduler 决定：`always` 投递所有结果，`on_alert` 只抑制 `ok`，`never` 只保留记录。Scheduled Agent 不能自行发送最终通知；子 Agent 委托会同步完成，也不会产生后台 Inbox/Signal。
时间戳和间隔单位为毫秒；Cron 表达式为 6 段（秒、分、时、日、月、周）。定时 Agent 不复用聊天历史，`prompt` 必须包含完整上下文。`kind` 可为 `task` 或 `monitor`；`delivery_policy` 可为 `always`、`on_alert` 或 `never`。托管任务由 Scheduler 投递，巡检返回 `NO_REPLY[INFO]` 时静默，`NO_REPLY[FAIL]`/`NO_REPLY[REFUSE]` 仍视为需关注结果。升级前创建的任务保留 Agent 直接调用 `send_message` 的兼容行为。`model` 当前会持久化和展示，但执行仍使用默认 Agent Provider/Model，不能依赖它实现模型覆盖。
---
@ -140,36 +135,27 @@ Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅
| 参数 | 必填 | 说明 |
|------|------|------|
| `target` | 具名 Agent 必填 | 目标 Agent ID；主 Agent 可委托给任意具名子 Agent，子 Agent 按自身 Definition 的 `delegates` 白名单决定 |
| `task` | 单任务必填 | 明确、独立、可验收的子任务 |
| `context` | 否 | 子 Agent 所需的显式事实；不会继承完整主会话历史 |
| `mode` | 否 | `foreground`（默认）或 `background`；批量并发不是第三种 mode |
| `tasks` | 批量必填 | 子任务数组；foreground 并发执行、结果保持请求顺序 |
| `allowed_tools` | 否 | 只能收窄具名 Definition 的工具集，不能扩权 |
| `plan_item_id` | 否 | 绑定当前计划子项；批量数组中的每项可分别绑定 |
| `action` | 是 | `run`, `check_task`, `cancel_task`, `list_tasks` |
| `prompt` | run 必填 | 子任务描述 |
| `mode` | 否 | `inline`, `background`, `parallel`，默认 `inline` |
| `allowed_tools` | 否 | 子 Agent 可用工具列表；默认只读工具集 |
| `max_iterations` | 否 | 最大迭代次数，默认 99 |
| `timeout_secs` | 否 | 超时秒数，默认 3600 |
| `tasks` | parallel 必填 | 并行子任务数组 |
| `task_id` | 查询/取消必填 | 后台任务 ID |
| `plan_item_id` | 否 | 将 inline/background 子 Agent 绑定到当前计划子项；parallel 数组中的每项也可分别绑定 |
子 Agent 的工具集完全由其定义文件（`~/.picobot/agents/*.md`）的 `tools` 列表决定，可直接内联 `provider`/`model` 指定模型。具名 background（`target` + `mode=background`，单任务或 `tasks[]` 批量）经 durable run/inbox 接纳：每个 run 先落 `agent_runs` 并预留 inbox completion slot，完成后由主 Agent 的 continuation Turn 处理结果，不再直接发 Channel 通知；批量并发执行、每个 run 独立返回（无用户输入积压时完成即返回）。子 Agent 发起的 background 尚未开放。后台运行可以在任务中调用 `emit_signal` 发送结构化内部信号（队列或 steer 投递），Steer 信号会在当前 Turn 的安全边界注入主 Agent；`agent_task.cancel` 会把该 run 未消费的普通信号标记 superseded。
## agent_task — 具名 Agent Run 查询与控制
仅在 `agent_orchestration` 启用时注册。查询或控制已持久化的具名 foreground run；授权来自调用者身份（ROOT 限本 session，具名 Agent 限自身子树），run ID 本身不是凭证。
| 参数 | 必填 | 说明 |
|------|------|------|
| `action` | 是 | `get` 查询单个 run；`list` 列出 session 的 run；`get_result` 读取终态完整结果；`cancel` 取消未终态 run |
| `run_id` | get/get_result/cancel 必填 | 目标 run ID |
| `cursor_created_at` / `cursor_id` | 否 | list 分页游标，必须成对出现 |
| `limit` | 否 | list 上限，默认 20 |
默认只读工具集：`file_read`、`file_search`、`content_search`、`web_fetch`、`http_request`、`calculator`。
## todo — Session 任务计划
仅用于明确的复杂、多轮或并行任务。`create` 创建当前 session 唯一的 active plan；`view` 查看；`append` 增加子项；`update` 修改子项状态；`close` 完成或取消计划。普通闲聊和单步操作不应创建计划。子 Agent 始终不能使用 `todo`；只有 Definition 声明委托边的具名 Agent 会获得运行时注入的 `delegate`，且只能 foreground 委托允许目标。
仅用于明确的复杂、多轮或并行任务。`create` 创建当前 session 唯一的 active plan；`view` 查看；`append` 增加子项；`update` 修改子项状态；`close` 完成或取消计划。普通闲聊和单步操作不应创建计划。子 Agent 始终被过滤掉 `todo` 和 `delegate`，计划结构只由主 Agent 管理。
---
## browser — 浏览器自动化
默认注册；设置 `browser.enabled=false` 后不注册。PicoBot 上层包装统一 action 和结构化媒体，底层逐次调用 agent-browser `--json`。调用时不传 `persistent_id`，每个 dialog 使用各自的普通临时 session，默认空闲一小时后自动关闭；长期工作需要保持浏览器进程或保留登录和站点状态时，Agent 可自主用 `browser_profiles(create,label=...)` 创建身份，并在后续相关 action 中持续传入同一个 ID。持久 session 不会因空闲自动关闭。PicoBot 没有全局持久化开关或默认持久 ID，也不按 dialog 自动选择持久身份；同一 ID 跨 dialog 共享 session 和串行锁，不同 ID 使用独立 session/锁并可并发。CLI daemon 通过 Chrome CDP 工作，不使用 Fantoccini、ChromeDriver 或 WebDriver。
仅在 `browser.enabled=true` 时注册。底层使用 WebDriver/Chrome。
| action | 说明 |
|--------|------|
@ -178,31 +164,15 @@ Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅
| `click`, `click_at` | 点击元素或坐标 |
| `fill`, `type`, `press` | 输入文本或按键 |
| `get_text`, `get_title`, `get_url` | 读取页面信息 |
| `screenshot` | 保存到 `browser.artifact_dir`，交给模型并默认附到最终用户回复；支持 `full_page`、`annotate`，可用 `present_to_user=false` 仅供模型检查 |
| `screenshot` | 截图，可写入文件或返回 base64 |
| `focus`, `hover`, `scroll`, `wait` | 常见交互和等待 |
| `close` | 关闭浏览器会话 |
典型流程：`open` → `snapshot` 获取 `@e` 引用 → 交互 → 页面变化后重新 `snapshot`。`path` 只接受 `.png` 文件名，不能逃逸产物目录。`open` 默认拒绝非 HTTP(S)、userinfo、回环、私网、本地域名及 DNS 解析到私网的地址；配置 `allowed_domains` 后，agent-browser 同时限制导航、子资源、WebSocket、EventSource 与 WebRTC。页面输出是不可信内容，默认开启 content boundary 元数据和 50,000 字符上限。
## browser_profiles — 持久浏览器身份管理
与 `browser` 一同注册。`create` 生成新的持久 ID 和 Profile 目录，并可接受 1–80 字符的语义标签；`set_label` 按精确 ID 重命名标签；`list` 返回每个合法 Profile 的 ID、标签、目录和 active 状态；`delete` 必须传入 `create/list` 返回的精确 ID，并删除该 ID 的完整 Chrome Profile、标签与登录态。删除活动 Profile 时会等待其操作完成并尝试关闭浏览器。标签只用于识别，不能代替 ID 选择；该工具不能接收任意目录。非空 `browser.allowed_domains` 下普通临时浏览器仍可用，但持久身份不能创建或使用。
依赖缺失时必须把错误和处置命令返回给用户，不能声称已浏览，也不能在工具内部静默安装：CLI 不存在时安装 `agent-browser@0.33.0`；Chrome 不存在时运行 `agent-browser install`；Linux 共享库不完整时运行 `agent-browser install --with-deps`。用 `picobot health` 复查，再用 `agent-browser doctor` 获取详细上游诊断。用户明确不需要浏览器时才建议 `browser.enabled=false`。
## health — 依赖检查
无参数时返回可读报告；`json=true` 返回结构化报告。核心必需项、当前配置启用后必需的依赖、可选功能分别标记。`fd` 与 Debian/Ubuntu 的 `fdfind` 是同一个首选文件搜索程序；只有传统 `find` 会产生降级警告。浏览器启用时会检查 CLI、可执行路径，并通过隔离的完整 offline doctor 分别验证浏览器安装、真实 headless 启动和环境。该工具只读，与 CLI `picobot health [--json]`、`/health` 斜杠命令以及 WebUI“配置 → 健康检查”复用同一个 `HealthService`。
## reload_config — 重载配置
重新读取并校验 PicoBot 配置，然后让 Gateway 优雅切换到新配置。无参数，仅 Root 交互 Agent 可用；仅在用户明确要求重新加载配置时调用，会先验证再切换，失败则保留旧运行代。
---
## MCP 工具
如果 `config.mcp.servers` 配置了 MCP 服务器，Gateway 启动时会连接服务器、发现工具，并把 MCP 工具包装后注册到 ToolRegistry。注册名固定为 `mcp_<服务器名>_<工具名>`，例如 `mcp_filesystem_read_file`；配置 `tool_settings` 仍使用 MCP 原始工具名 `read_file`。使用 `/mcp` 查看当前连接状态和工具列表。MCP 协议不声明副作用或并发安全性；可在服务器配置的 `tool_settings` 中为各工具设置本地受信任的 `read_only`、`exclusive` 属性。未设置的工具顺序执行；“可并发”由 `read_only && !exclusive` 自动推导。
如果 `config.mcp.servers` 配置了 MCP 服务器，Gateway 启动时会连接服务器、发现工具，并把 MCP 工具包装后注册到 ToolRegistry。使用 `/mcp` 查看当前连接状态和工具列表。
---
@ -224,20 +194,19 @@ Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅
## calculator — 计算器
数学表达式计算和统计函数。用 `function` 指定要执行的计算。
数学表达式计算和统计函数。
| function | 相关参数 | 说明 |
|----------|----------|------|
| `evaluate` | `expression` | 计算表达式 |
| `sum` / `count` / `range` | `values` | 求和 / 计数 / 极差 |
| `average` | `values` | 平均值 |
| `median` | `values` | 中位数 |
| `mode` | `values` | 众数 |
| `stdev` / `variance` | `values` | 标准差 / 方差 |
| `min` / `max` | `values` | 最小值 / 最大值 |
| `log` | `x`, 可选 `base` | 对数（base 默认 10） |
| `factorial` | `x` | 阶乘 |
| `round` | `x`, `decimals` | 四舍五入 |
| `percentage_change` | `a`(旧值), `b`(新值) | 变化百分比 |
| `percentile` | `values`, `p` | 百分位数（p 0–100） |
| `clamp` | `x`, `min_val`, `max_val` | 夹取到区间 |
| action | 说明 |
|--------|------|
| `evaluate` | 计算表达式 |
| `sum` | 求和 |
| `average` | 平均值 |
| `median` | 中位数 |
| `mode` | 众数 |
| `stdev` / `variance` | 标准差/方差 |
| `min` / `max` | 最小值/最大值 |
| `log` | 对数 |
| `factorial` | 阶乘 |
| `round` | 四舍五入 |
| `percentage_change` | 变化百分比 |
| `percentile` | 百分位数 |

63
resources/templates/config.example.json
View File

@ -23,8 +23,7 @@
"qwen-plus": {
"model_id": "qwen-plus",
"temperature": 0.0,
"max_tokens": 8192,
"token_limit": 128000
"max_tokens": 8192
},
"gpt-4o": {
"model_id": "gpt-4o",
@ -43,30 +42,10 @@
"default": {
"provider": "aliyun",
"model": "qwen-plus",
"max_tool_iterations": 99
"max_tool_iterations": 99,
"token_limit": 128000
}
},
"context_compaction": {
"enabled": true,
"reserve_tokens": 16384,
"keep_recent_tokens": 20000
},
"agent_orchestration": {
"definitions_dir": "agents",
"max_tree_depth": 4,
"max_runs_per_tree": 16,
"max_concurrent_runs": 6,
"max_concurrent_runs_per_session": 4,
"max_concurrent_provider_steps": 8,
"max_concurrent_provider_steps_per_session": 4,
"max_concurrent_tool_steps": 16,
"max_concurrent_tool_steps_per_session": 8,
"max_pending_inbox_events_per_session": 128,
"inbox_event_ttl_hours": 168,
"max_inbox_delivery_attempts": 8,
"max_user_turn_burst_before_inbox": 4,
"max_inbox_wait_secs": 30
},
"gateway": {
"host": "127.0.0.1",
"port": 19876,
@ -78,12 +57,6 @@
"max_files_per_message": 8,
"max_message_bytes": 67108864,
"pending_ttl_seconds": 3600
},
"scheduler": {
"enabled": true,
"poll_interval_secs": 60,
"max_concurrent": 1,
"execution_timeout_secs": 900
}
},
"client": {
@ -95,26 +68,15 @@
"app_id": "<FEISHU_APP_ID>",
"app_secret": "<FEISHU_APP_SECRET>",
"allow_from": ["*"],
"require_mention": true,
"agent": "default",
"media_dir": "~/.picobot/media/feishu",
"reaction_emoji": "Typing",
"live_updates": false,
"live_update_interval_ms": 500,
"max_image_bytes": 10485760,
"max_file_bytes": 26214400,
"media_dir_max_bytes": 536870912,
"request_timeout_secs": 30
"reaction_emoji": "Typing"
}
},
"memory": {
"consolidation_provider": null,
"consolidation_model": null,
"recall_limit": 5,
"recall_min_relevance": 0.25,
"recall_min_score": 0.25,
"recall_recency_half_life_days": 30,
"recall_timeout_ms": 1000,
"idle_consolidation_minutes": 10,
"timeline_retention_days": 90,
"max_failures_before_degrade": 3
@ -124,21 +86,10 @@
"tool_timeout_secs": 180
},
"browser": {
"enabled": true,
"command": "agent-browser",
"enabled": false,
"webdriver_url": "http://127.0.0.1:9515",
"headless": true,
"browser_executable_path": null,
"max_sessions": 4,
"idle_timeout_secs": 3600,
"command_timeout_secs": 120,
"max_output_chars": 50000,
"content_boundaries": true,
"allowed_domains": [],
"allow_private_hosts": false,
"artifact_dir": "~/.picobot/media/browser",
"persistence": {
"profile_dir": "~/.picobot/browser/profiles"
}
"chrome_path": null
},
"workspace_dir": "~/.picobot/workspace"
}

2183
src/agent/agent_loop.rs
View File

File diff suppressed because it is too large Load Diff

55
src/agent/builtin.rs
View File

@ -1,55 +0,0 @@
//! Built-in Agent definitions, released to the user config directory on
//! first run just like built-in skills. A released definition is a regular
//! user-editable file afterwards; the installer never overwrites it.
use std::path::Path;
use crate::config::LLMProviderConfig;
mod embedded {
include!(concat!(env!("OUT_DIR"), "/embedded_agents.rs"));
}
/// Install built-in Agent definitions into `<config_dir>/agents/`. Files
/// that already exist (user-modified or user-created) are left untouched.
pub fn install_builtin_agents(
config_dir: &Path,
profiles: &std::collections::HashMap<String, LLMProviderConfig>,
) {
let agents_dir = config_dir.join("agents");
if let Err(error) = std::fs::create_dir_all(&agents_dir) {
tracing::warn!(dir = %agents_dir.display(), error = %error, "Failed to create agents directory");
return;
}
for agent in embedded::EMBEDDED_AGENTS {
let path = agents_dir.join(format!("{}.md", agent.name));
if path.exists() {
continue;
}
if let Err(error) = std::fs::write(&path, agent.content) {
tracing::warn!(name = agent.name, error = %error, "Failed to install built-in Agent definition");
continue;
}
let profile = match profiles.get("default").cloned() {
Some(profile) => profile,
None => {
tracing::warn!(
name = agent.name,
"Skipping built-in Agent validation: no 'default' provider profile configured"
);
continue;
}
};
// Validate the released file immediately so a future catalog load
// cannot fail on a broken built-in. Validation failure keeps the
// file (the user can edit it) but logs loudly.
match super::definition::parse_definition(&path, std::sync::Arc::new(profile)) {
Ok(_) => {
tracing::info!(name = agent.name, dir = %path.display(), "Installed built-in Agent definition");
}
Err(error) => {
tracing::warn!(name = agent.name, error = %error, "Installed built-in Agent definition failed validation");
}
}
}
}

1097
src/agent/catalog.rs
View File

File diff suppressed because it is too large Load Diff

1490
src/agent/context_compaction.rs
View File

File diff suppressed because it is too large Load Diff

1039
src/agent/context_compressor.rs Normal file
View File

File diff suppressed because it is too large Load Diff

2108
src/agent/coordinator.rs
View File

File diff suppressed because it is too large Load Diff

671
src/agent/definition.rs
View File

@ -1,671 +0,0 @@
use std::path::{Path, PathBuf};
use std::sync::Arc;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use crate::config::LLMProviderConfig;
pub const MAX_DEFINITION_FILE_BYTES: u64 = 256 * 1024;
const MAX_DESCRIPTION_CHARS: usize = 4_096;
const MAX_ROLE_CHARS: usize = 65_536;
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default, deny_unknown_fields)]
pub struct AgentLimits {
pub timeout_secs: u64,
pub max_iterations: usize,
pub max_children: usize,
pub max_depth: u16,
pub max_concurrent_runs: usize,
pub max_concurrent_provider_steps: usize,
pub max_concurrent_tool_steps: usize,
pub max_result_chars: usize,
}
impl Default for AgentLimits {
fn default() -> Self {
Self {
timeout_secs: 900,
max_iterations: 24,
max_children: 4,
max_depth: 3,
max_concurrent_runs: 2,
max_concurrent_provider_steps: 1,
max_concurrent_tool_steps: 4,
max_result_chars: 16_000,
}
}
}
/// Signal delivery lane for background Agent events.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum SignalDelivery {
#[default]
Queue,
Steer,
}
impl SignalDelivery {
pub fn as_str(&self) -> &'static str {
match self {
Self::Queue => "queue",
Self::Steer => "steer",
}
}
}
/// Durable emit_signal contract. An Agent only sees the `emit_signal` tool
/// when this block is present; every limit below is enforced by the tool and
/// the Coordinator, not by the model.
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default, deny_unknown_fields)]
pub struct SignalContract {
pub delivery: SignalDelivery,
/// Total number of signals one run may emit.
pub max_total: u32,
/// Upper bound for the serialized `details` JSON payload of one signal.
pub max_details_bytes: usize,
/// Minimum wall time between two signals of the same run.
pub min_interval_ms: u64,
/// Maximum signals emitted within `burst_window_ms`.
pub max_burst: u32,
pub burst_window_ms: u64,
/// Allowlisted severities; anything else is rejected.
pub severity_allowlist: Vec<String>,
/// Dedupe key cooldown window; repeated keys inside the window collapse
/// to the same event, keys outside it emit again.
pub dedupe_cooldown_ms: u64,
/// Maximum JSON nesting depth of `details`.
pub max_payload_depth: usize,
}
impl Default for SignalContract {
fn default() -> Self {
Self {
delivery: SignalDelivery::Queue,
max_total: 64,
max_details_bytes: 8 * 1024,
min_interval_ms: 500,
max_burst: 5,
burst_window_ms: 10_000,
severity_allowlist: vec![
"info".to_string(),
"warning".to_string(),
"critical".to_string(),
],
dedupe_cooldown_ms: 60_000,
max_payload_depth: 16,
}
}
}
impl SignalContract {
fn validate(&self) -> Result<(), AgentDefinitionError> {
if self.max_total == 0 || self.max_total > 1024 {
return Err(AgentDefinitionError::Invalid(
"signal.max_total must be between 1 and 1024".to_string(),
));
}
if self.max_details_bytes == 0 || self.max_details_bytes > 64 * 1024 {
return Err(AgentDefinitionError::Invalid(
"signal.max_details_bytes must be between 1 and 65536".to_string(),
));
}
if self.max_burst == 0 || self.max_burst > self.max_total {
return Err(AgentDefinitionError::Invalid(
"signal.max_burst must be between 1 and max_total".to_string(),
));
}
if self.max_payload_depth == 0 || self.max_payload_depth > 64 {
return Err(AgentDefinitionError::Invalid(
"signal.max_payload_depth must be between 1 and 64".to_string(),
));
}
if self.severity_allowlist.is_empty() || self.severity_allowlist.len() > 16 {
return Err(AgentDefinitionError::Invalid(
"signal.severity_allowlist must contain 1..=16 severities".to_string(),
));
}
let mut seen = std::collections::HashSet::new();
if let Some(duplicate) = self
.severity_allowlist
.iter()
.find(|severity| !seen.insert(severity.as_str()))
{
return Err(AgentDefinitionError::Invalid(format!(
"duplicate signal severity '{duplicate}'"
)));
}
if self
.severity_allowlist
.iter()
.any(|severity| severity.trim().is_empty() || severity.len() > 64)
{
return Err(AgentDefinitionError::Invalid(
"signal severities must be non-empty and at most 64 characters".to_string(),
));
}
Ok(())
}
}
impl AgentLimits {
fn validate(&self) -> Result<(), AgentDefinitionError> {
if self.timeout_secs == 0 || self.timeout_secs > 86_400 {
return Err(AgentDefinitionError::Invalid(
"limits.timeout_secs must be between 1 and 86400".to_string(),
));
}
if self.max_iterations == 0 || self.max_iterations > 256 {
return Err(AgentDefinitionError::Invalid(
"limits.max_iterations must be between 1 and 256".to_string(),
));
}
if self.max_children == 0 || self.max_children > 128 {
return Err(AgentDefinitionError::Invalid(
"limits.max_children must be between 1 and 128".to_string(),
));
}
if self.max_depth == 0 || self.max_depth > 32 {
return Err(AgentDefinitionError::Invalid(
"limits.max_depth must be between 1 and 32".to_string(),
));
}
for (name, value, hard_max) in [
("max_concurrent_runs", self.max_concurrent_runs, 128),
(
"max_concurrent_provider_steps",
self.max_concurrent_provider_steps,
128,
),
(
"max_concurrent_tool_steps",
self.max_concurrent_tool_steps,
512,
),
] {
if value == 0 || value > hard_max {
return Err(AgentDefinitionError::Invalid(format!(
"limits.{name} must be between 1 and {hard_max}"
)));
}
}
if self.max_result_chars == 0 || self.max_result_chars > 1_000_000 {
return Err(AgentDefinitionError::Invalid(
"limits.max_result_chars must be between 1 and 1000000".to_string(),
));
}
Ok(())
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct AgentFrontmatter {
pub id: String,
pub description: String,
/// Either this (a key in `config.json`'s `agents` map) or the inline
/// `provider` + `model` pair must be present.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub llm_profile: Option<String>,
/// Inline provider/model selection; the preferred way to author an Agent
/// from the WebUI. Overrides `llm_profile` when both are present.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub provider: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub token_limit: Option<usize>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_tool_iterations: Option<usize>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub tools: Vec<String>,
/// Disabled definitions stay on disk but never load into the catalog.
#[serde(default = "default_true")]
pub enabled: bool,
/// Which Agents this one may further delegate to. `None` (field absent)
/// defaults to the built-in `general-purpose`; `Some([])` forbids further
/// delegation; a list containing `*` allows any other Agent; an explicit
/// list allows exactly those targets.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub delegates: Option<Vec<String>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub skills: Vec<String>,
#[serde(default)]
pub limits: AgentLimits,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub signal: Option<SignalContract>,
}
pub fn default_true() -> bool {
true
}
#[derive(Debug, Clone)]
pub struct AgentDefinition {
pub id: String,
pub description: String,
pub llm_profile: Option<String>,
pub provider: Option<String>,
pub model: Option<String>,
pub provider_config: Arc<LLMProviderConfig>,
pub enabled: bool,
pub tools: Vec<String>,
pub delegates: Option<Vec<String>>,
pub skills: Vec<String>,
pub limits: AgentLimits,
pub signal_contract: Option<SignalContract>,
pub role_prompt: String,
pub definition_hash: String,
pub source_path: PathBuf,
}
#[derive(Debug, thiserror::Error)]
pub enum AgentDefinitionError {
#[error("failed to read Agent definition {path}: {source}")]
Io {
path: String,
#[source]
source: std::io::Error,
},
#[error("invalid Agent definition: {0}")]
Invalid(String),
#[error("invalid YAML frontmatter: {0}")]
Yaml(#[from] serde_yaml::Error),
}
pub(crate) fn parse_definition(
path: &Path,
provider_config: Arc<LLMProviderConfig>,
) -> Result<AgentDefinition, AgentDefinitionError> {
let (frontmatter, role_prompt) = read_frontmatter(path)?;
let canonical_frontmatter = serde_json::to_vec(&frontmatter)
.map_err(|error| AgentDefinitionError::Invalid(error.to_string()))?;
let mut hasher = Sha256::new();
hasher.update(canonical_frontmatter);
hasher.update(b"\n---\n");
hasher.update(role_prompt.as_bytes());
let definition_hash = hasher
.finalize()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect();
Ok(AgentDefinition {
id: frontmatter.id,
description: frontmatter.description.trim().to_string(),
llm_profile: frontmatter.llm_profile,
provider: frontmatter.provider,
model: frontmatter.model,
provider_config,
enabled: frontmatter.enabled,
tools: frontmatter.tools,
delegates: frontmatter.delegates,
skills: frontmatter.skills,
limits: frontmatter.limits,
signal_contract: frontmatter.signal,
role_prompt,
definition_hash,
source_path: path.to_path_buf(),
})
}
/// Provider-agnostic view of a definition file, used by the management UI.
/// It carries the frontmatter plus the role body but no resolved
/// `provider_config`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AgentDefinitionInfo {
#[serde(flatten)]
pub frontmatter: AgentFrontmatter,
pub role_prompt: String,
}
/// Read and validate a definition file without resolving its provider
/// config. Used by the management API to list/validate definitions; the
/// catalog performs the full provider/tool/delegate resolution on load.
pub fn parse_definition_info(path: &Path) -> Result<AgentDefinitionInfo, AgentDefinitionError> {
let (frontmatter, role_prompt) = read_frontmatter(path)?;
Ok(AgentDefinitionInfo {
frontmatter,
role_prompt,
})
}
/// Best-effort view of a definition file for the management UI. A fully
/// valid file returns `(Some(info), None)`; a broken file returns a partial
/// `info` (id from the file stem, the role body, and whichever frontmatter
/// fields still parse) together with the parse error, so broken definitions
/// remain listed and editable instead of being silently hidden.
pub fn parse_definition_lenient(path: &Path) -> (Option<AgentDefinitionInfo>, Option<String>) {
match parse_definition_info(path) {
Ok(info) => (Some(info), None),
Err(error) => {
let stem = path
.file_stem()
.and_then(|value| value.to_str())
.unwrap_or("agent")
.to_string();
let mut role_prompt = String::new();
let mut frontmatter = AgentFrontmatter {
id: stem.clone(),
description: String::new(),
llm_profile: None,
provider: None,
model: None,
token_limit: None,
max_tool_iterations: None,
tools: Vec::new(),
enabled: false,
delegates: None,
skills: Vec::new(),
limits: AgentLimits::default(),
signal: None,
};
if let Ok(content) = std::fs::read_to_string(path) {
let normalized = content.replace("\r\n", "\n");
if let Some(rest) = normalized.strip_prefix("---\n")
&& let Some((yaml, body)) = rest.split_once("\n---\n")
{
role_prompt = body.trim().to_string();
#[derive(serde::Deserialize)]
struct Lenient {
id: Option<String>,
description: Option<String>,
llm_profile: Option<String>,
provider: Option<String>,
model: Option<String>,
token_limit: Option<usize>,
max_tool_iterations: Option<usize>,
tools: Option<Vec<String>>,
delegates: Option<Vec<String>>,
skills: Option<Vec<String>>,
}
if let Ok(parsed) = serde_yaml::from_str::<Lenient>(yaml) {
frontmatter.id = parsed.id.unwrap_or(stem);
frontmatter.description = parsed.description.unwrap_or_default();
frontmatter.llm_profile = parsed.llm_profile;
frontmatter.provider = parsed.provider;
frontmatter.model = parsed.model;
frontmatter.token_limit = parsed.token_limit;
frontmatter.max_tool_iterations = parsed.max_tool_iterations;
frontmatter.tools = parsed.tools.unwrap_or_default();
frontmatter.delegates = parsed.delegates;
frontmatter.skills = parsed.skills.unwrap_or_default();
}
}
}
(
Some(AgentDefinitionInfo {
frontmatter,
role_prompt,
}),
Some(error.to_string()),
)
}
}
}
/// Serialize a definition back to the Markdown file format.
pub fn serialize_definition(info: &AgentDefinitionInfo) -> String {
let yaml = serde_yaml::to_string(&info.frontmatter).unwrap_or_default();
format!("---\n{yaml}---\n{}\n", info.role_prompt.trim_end())
}
/// Read + validate the frontmatter and role body of a definition file,
/// shared by `parse_definition` and `parse_definition_info`.
fn read_frontmatter(path: &Path) -> Result<(AgentFrontmatter, String), AgentDefinitionError> {
let metadata = std::fs::symlink_metadata(path).map_err(|source| AgentDefinitionError::Io {
path: path.display().to_string(),
source,
})?;
if !metadata.file_type().is_file() || metadata.file_type().is_symlink() {
return Err(AgentDefinitionError::Invalid(format!(
"{} must be a regular non-symlink file",
path.display()
)));
}
if metadata.len() > MAX_DEFINITION_FILE_BYTES {
return Err(AgentDefinitionError::Invalid(format!(
"{} exceeds the {} byte limit",
path.display(),
MAX_DEFINITION_FILE_BYTES
)));
}
let content = std::fs::read_to_string(path).map_err(|source| AgentDefinitionError::Io {
path: path.display().to_string(),
source,
})?;
let normalized = content.replace("\r\n", "\n");
let mut lines = normalized.lines();
if lines.next() != Some("---") {
return Err(AgentDefinitionError::Invalid(format!(
"{} must start with a standalone --- line",
path.display()
)));
}
let mut yaml_lines = Vec::new();
let mut found_end = false;
for line in &mut lines {
if line == "---" {
found_end = true;
break;
}
yaml_lines.push(line);
}
if !found_end {
return Err(AgentDefinitionError::Invalid(format!(
"{} has no closing frontmatter delimiter",
path.display()
)));
}
let role_prompt = lines.collect::<Vec<_>>().join("\n").trim().to_string();
if role_prompt.is_empty() {
return Err(AgentDefinitionError::Invalid(format!(
"{} has an empty role body",
path.display()
)));
}
if role_prompt.chars().count() > MAX_ROLE_CHARS {
return Err(AgentDefinitionError::Invalid(format!(
"{} role body exceeds {MAX_ROLE_CHARS} characters",
path.display()
)));
}
let frontmatter: AgentFrontmatter = serde_yaml::from_str(&yaml_lines.join("\n"))?;
validate_agent_id(&frontmatter.id)?;
if frontmatter.description.trim().is_empty()
|| frontmatter.description.chars().count() > MAX_DESCRIPTION_CHARS
{
return Err(AgentDefinitionError::Invalid(format!(
"Agent '{}' description must contain 1..={MAX_DESCRIPTION_CHARS} characters",
frontmatter.id
)));
}
let has_profile = frontmatter
.llm_profile
.as_deref()
.is_some_and(|p| !p.trim().is_empty());
let has_inline = frontmatter.provider.is_some() || frontmatter.model.is_some();
if !has_profile && !has_inline {
return Err(AgentDefinitionError::Invalid(format!(
"Agent '{}' must declare either llm_profile or provider+model",
frontmatter.id
)));
}
if frontmatter.provider.is_some() != frontmatter.model.is_some() {
return Err(AgentDefinitionError::Invalid(format!(
"Agent '{}' must declare provider and model together",
frontmatter.id
)));
}
frontmatter.limits.validate()?;
if let Some(signal) = frontmatter.signal.as_ref() {
signal.validate()?;
}
reject_duplicates("tools", &frontmatter.tools)?;
if let Some(delegates) = frontmatter.delegates.as_deref() {
reject_duplicates("delegates", delegates)?;
}
reject_duplicates("skills", &frontmatter.skills)?;
let file_stem = path.file_stem().and_then(|value| value.to_str());
if file_stem != Some(frontmatter.id.as_str()) {
return Err(AgentDefinitionError::Invalid(format!(
"Agent id '{}' must match file name {}",
frontmatter.id,
path.display()
)));
}
Ok((frontmatter, role_prompt))
}
pub fn validate_agent_id(id: &str) -> Result<(), AgentDefinitionError> {
let valid = !id.is_empty()
&& id.len() <= 64
&& id
.bytes()
.next()
.is_some_and(|value| value.is_ascii_lowercase())
&& id.bytes().all(|value| {
value.is_ascii_lowercase() || value.is_ascii_digit() || value == b'_' || value == b'-'
});
if !valid || matches!(id, "root" | "main" | "default" | "general") {
return Err(AgentDefinitionError::Invalid(format!(
"invalid or reserved Agent id '{id}'"
)));
}
Ok(())
}
fn reject_duplicates(field: &str, values: &[String]) -> Result<(), AgentDefinitionError> {
let mut seen = std::collections::HashSet::new();
if let Some(value) = values.iter().find(|value| !seen.insert(value.as_str())) {
return Err(AgentDefinitionError::Invalid(format!(
"duplicate {field} entry '{value}'"
)));
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
fn provider() -> Arc<LLMProviderConfig> {
Arc::new(LLMProviderConfig {
provider_type: "openai".to_string(),
name: "test".to_string(),
base_url: "https://example.invalid/v1".to_string(),
api_key: "test".to_string(),
extra_headers: HashMap::new(),
model_id: "test-model".to_string(),
temperature: None,
max_tokens: None,
model_extra: HashMap::new(),
max_tool_iterations: 99,
token_limit: 4096,
workspace_dir: std::env::temp_dir(),
input_types: vec!["text".to_string()],
price_input_per_million: None,
price_output_per_million: None,
})
}
#[test]
fn strict_definition_parses_and_hashes_stably() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("researcher.md");
std::fs::write(
&path,
concat!(
"---\n",
"id: researcher\n",
"description: Research primary sources\n",
"llm_profile: research\n",
"tools:\n",
" - calculator\n",
"limits:\n",
" max_iterations: 12\n",
"---\n",
"# Role\n\n",
"Return evidence and uncertainty.\n"
),
)
.unwrap();
let first = parse_definition(&path, provider()).unwrap();
let second = parse_definition(&path, provider()).unwrap();
assert_eq!(first.id, "researcher");
assert_eq!(first.limits.max_iterations, 12);
assert_eq!(first.definition_hash, second.definition_hash);
assert_eq!(first.definition_hash.len(), 64);
}
#[test]
fn unknown_frontmatter_key_is_rejected() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("researcher.md");
std::fs::write(
&path,
"---\nid: researcher\ndescription: test\nllm_profile: research\nunsafe_tools: true\n---\n# Role\n",
)
.unwrap();
assert!(parse_definition(&path, provider()).is_err());
}
#[test]
fn reserved_and_mismatched_ids_are_rejected() {
assert!(validate_agent_id("root").is_err());
assert!(validate_agent_id("Uppercase").is_err());
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("researcher.md");
std::fs::write(
&path,
"---\nid: reviewer\ndescription: test\nllm_profile: research\n---\n# Role\n",
)
.unwrap();
assert!(parse_definition(&path, provider()).is_err());
}
#[test]
fn delegates_round_trip_preserves_all_forms() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("researcher.md");
for (label, delegates) in [
("unset", None),
("empty", Some(Vec::<String>::new())),
("any", Some(vec!["*".to_string()])),
(
"list",
Some(vec!["coder".to_string(), "reviewer".to_string()]),
),
] {
let info = AgentDefinitionInfo {
frontmatter: AgentFrontmatter {
id: "researcher".to_string(),
description: "test".to_string(),
llm_profile: None,
provider: Some("openai".to_string()),
model: Some("m".to_string()),
token_limit: None,
max_tool_iterations: None,
tools: Vec::new(),
enabled: true,
delegates: delegates.clone(),
skills: Vec::new(),
limits: AgentLimits::default(),
signal: None,
},
role_prompt: "# Role\n\nwork".to_string(),
};
std::fs::write(&path, serialize_definition(&info)).unwrap();
let parsed = parse_definition_info(&path).unwrap();
assert_eq!(parsed.frontmatter.delegates, delegates, "{label}");
}
}
}

347
src/agent/gate.rs
View File

@ -1,347 +0,0 @@
use std::sync::{Arc, Weak};
use dashmap::DashMap;
use tokio::sync::{OwnedSemaphorePermit, Semaphore};
use tokio_util::sync::CancellationToken;
/// Step permits are held while a provider request or tool invocation is in
/// flight and released as soon as the step finishes. Session permits are
/// released before global permits (reverse acquisition order). The fields
/// are never read; ownership alone keeps the quotas reserved (RAII).
#[derive(Debug)]
pub struct StepPermit {
#[allow(dead_code)]
session: Option<OwnedSemaphorePermit>,
#[allow(dead_code)]
global: OwnedSemaphorePermit,
}
/// Run quota permits cover a whole Agent Run from admission until its
/// terminal commit. Foreground delegation does not take run permits; the
/// quota gates background admission so a waiting parent can never deadlock
/// nested foreground children.
#[derive(Debug)]
pub struct RunPermit {
#[allow(dead_code)]
session: Option<OwnedSemaphorePermit>,
#[allow(dead_code)]
global: OwnedSemaphorePermit,
}
#[derive(Debug, thiserror::Error)]
pub enum GateError {
#[error("execution gate wait cancelled")]
Cancelled,
}
#[derive(Default)]
struct KeyedSemaphores {
permits: usize,
map: DashMap<String, Weak<Semaphore>>,
}
impl KeyedSemaphores {
fn new(permits: usize) -> Self {
Self {
permits,
map: DashMap::new(),
}
}
fn semaphore(self: &Arc<Self>, key: &str) -> Arc<Semaphore> {
if let Some(existing) = self.map.get(key)
&& let Some(semaphore) = existing.upgrade()
{
return semaphore;
}
let semaphore = Arc::new(Semaphore::new(self.permits));
self.map.insert(key.to_string(), Arc::downgrade(&semaphore));
// Drop registry entries whose owners are gone so long-lived gateways
// do not accumulate one semaphore per historical session.
self.map.retain(|_, weak| weak.strong_count() > 0);
semaphore
}
}
/// Concurrency gates shared by one runtime generation. Run quota and
/// provider/tool step permits have independent lifecycles; acquisition order
/// is always global -> session and release order is reversed.
pub struct ExecutionGate {
max_concurrent_runs: usize,
run_global: Arc<Semaphore>,
run_session: Arc<KeyedSemaphores>,
provider_global: Arc<Semaphore>,
provider_session: Arc<KeyedSemaphores>,
tool_global: Arc<Semaphore>,
tool_session: Arc<KeyedSemaphores>,
}
impl std::fmt::Debug for ExecutionGate {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ExecutionGate")
.field("run_available", &self.run_global.available_permits())
.field(
"provider_available",
&self.provider_global.available_permits(),
)
.field("tool_available", &self.tool_global.available_permits())
.finish()
}
}
impl ExecutionGate {
pub fn new(config: &crate::config::AgentOrchestrationConfig) -> Arc<Self> {
Arc::new(Self {
max_concurrent_runs: config.max_concurrent_runs,
run_global: Arc::new(Semaphore::new(config.max_concurrent_runs)),
run_session: Arc::new(KeyedSemaphores::new(config.max_concurrent_runs_per_session)),
provider_global: Arc::new(Semaphore::new(config.max_concurrent_provider_steps)),
provider_session: Arc::new(KeyedSemaphores::new(
config.max_concurrent_provider_steps_per_session,
)),
tool_global: Arc::new(Semaphore::new(config.max_concurrent_tool_steps)),
tool_session: Arc::new(KeyedSemaphores::new(
config.max_concurrent_tool_steps_per_session,
)),
})
}
/// Unlimited gate used when orchestration is disabled; root Turns still
/// route through it so the code path stays uniform.
pub fn unbounded() -> Arc<Self> {
Arc::new(Self {
max_concurrent_runs: usize::MAX,
run_global: Arc::new(Semaphore::new(Semaphore::MAX_PERMITS)),
run_session: Arc::new(KeyedSemaphores::new(Semaphore::MAX_PERMITS)),
provider_global: Arc::new(Semaphore::new(Semaphore::MAX_PERMITS)),
provider_session: Arc::new(KeyedSemaphores::new(Semaphore::MAX_PERMITS)),
tool_global: Arc::new(Semaphore::new(Semaphore::MAX_PERMITS)),
tool_session: Arc::new(KeyedSemaphores::new(Semaphore::MAX_PERMITS)),
})
}
/// Global run quota ceiling; background batches larger than this are
/// rejected up front instead of queueing indefinitely.
pub fn max_concurrent_runs(&self) -> usize {
self.max_concurrent_runs
}
pub async fn acquire_run(
self: &Arc<Self>,
session_id: &str,
cancellation: &CancellationToken,
) -> Result<RunPermit, GateError> {
let global = acquire_owned(self.run_global.clone(), cancellation).await?;
let session = acquire_keyed(&self.run_session, session_id, cancellation).await?;
Ok(RunPermit {
session: Some(session),
global,
})
}
pub async fn acquire_provider(
self: &Arc<Self>,
session_id: Option<&str>,
cancellation: &CancellationToken,
) -> Result<StepPermit, GateError> {
let global = acquire_owned(self.provider_global.clone(), cancellation).await?;
let session = match session_id {
Some(session_id) => {
Some(acquire_keyed(&self.provider_session, session_id, cancellation).await?)
}
None => None,
};
Ok(StepPermit { session, global })
}
pub async fn acquire_tool(
self: &Arc<Self>,
session_id: Option<&str>,
cancellation: &CancellationToken,
) -> Result<StepPermit, GateError> {
let global = acquire_owned(self.tool_global.clone(), cancellation).await?;
let session = match session_id {
Some(session_id) => {
Some(acquire_keyed(&self.tool_session, session_id, cancellation).await?)
}
None => None,
};
Ok(StepPermit { session, global })
}
pub fn available_provider_permits(&self) -> usize {
self.provider_global.available_permits()
}
pub fn available_tool_permits(&self) -> usize {
self.tool_global.available_permits()
}
}
async fn acquire_owned(
semaphore: Arc<Semaphore>,
cancellation: &CancellationToken,
) -> Result<OwnedSemaphorePermit, GateError> {
tokio::select! {
biased;
_ = cancellation.cancelled() => Err(GateError::Cancelled),
result = semaphore.acquire_owned() => {
result.map_err(|_| GateError::Cancelled)
}
}
}
async fn acquire_keyed(
keyed: &Arc<KeyedSemaphores>,
key: &str,
cancellation: &CancellationToken,
) -> Result<OwnedSemaphorePermit, GateError> {
acquire_owned(keyed.semaphore(key), cancellation).await
}
/// Test helper: wait until a predicate holds or fail after the timeout.
#[cfg(test)]
async fn eventually<F: Fn() -> bool>(predicate: F) {
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);
while std::time::Instant::now() < deadline {
if predicate() {
return;
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
panic!("condition did not hold within timeout");
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::Duration;
use crate::config::AgentOrchestrationConfig;
/// Global caps stay larger than session caps so a session-level waiter
/// can never exhaust the global permits while blocked (acquisition order
/// is global -> session by design).
fn gate(provider_session: usize, tool_session: usize) -> Arc<ExecutionGate> {
let config = AgentOrchestrationConfig {
max_concurrent_runs: 8,
max_concurrent_runs_per_session: 1,
max_concurrent_provider_steps: 8,
max_concurrent_provider_steps_per_session: provider_session,
max_concurrent_tool_steps: 8,
max_concurrent_tool_steps_per_session: tool_session,
..Default::default()
};
ExecutionGate::new(&config)
}
#[tokio::test]
async fn provider_permits_are_released_after_drop() {
let gate = gate(1, 1);
let token = CancellationToken::new();
let permit = gate.acquire_provider(None, &token).await.unwrap();
assert_eq!(gate.available_provider_permits(), 7);
drop(permit);
assert_eq!(gate.available_provider_permits(), 8);
}
#[tokio::test]
async fn session_quota_blocks_second_session_step_until_release() {
let gate = gate(1, 8);
let token = CancellationToken::new();
let first = gate
.acquire_provider(Some("cli:a:d1"), &token)
.await
.unwrap();
// Global capacity remains, but the same session is capped at one.
let blocked = tokio::spawn({
let gate = gate.clone();
let token = token.clone();
async move { gate.acquire_provider(Some("cli:a:d1"), &token).await }
});
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(!blocked.is_finished());
// A different session is unaffected.
let other = gate
.acquire_provider(Some("cli:b:d2"), &token)
.await
.unwrap();
drop(other);
drop(first);
let second = blocked.await.unwrap().unwrap();
drop(second);
assert_eq!(gate.available_provider_permits(), 8);
}
#[tokio::test]
async fn cancellation_aborts_permit_wait() {
let gate = gate(8, 1);
let token = CancellationToken::new();
let held = gate.acquire_tool(Some("cli:a:d1"), &token).await.unwrap();
let waiter_token = CancellationToken::new();
let waiter = tokio::spawn({
let gate = gate.clone();
let waiter_token = waiter_token.clone();
async move { gate.acquire_tool(Some("cli:a:d1"), &waiter_token).await }
});
tokio::time::sleep(Duration::from_millis(50)).await;
waiter_token.cancel();
let error = waiter.await.unwrap().unwrap_err();
assert!(matches!(error, GateError::Cancelled));
drop(held);
assert_eq!(gate.available_tool_permits(), 8);
}
#[tokio::test]
async fn run_quota_is_scoped_per_session() {
let gate = gate(8, 8);
let token = CancellationToken::new();
let held = gate.acquire_run("cli:a:d1", &token).await.unwrap();
let blocked = tokio::spawn({
let gate = gate.clone();
let token = token.clone();
async move { gate.acquire_run("cli:a:d1", &token).await }
});
tokio::time::sleep(Duration::from_millis(50)).await;
assert!(!blocked.is_finished());
let other = gate.acquire_run("cli:b:d2", &token).await.unwrap();
drop(other);
drop(held);
blocked.await.unwrap().unwrap();
}
#[tokio::test]
async fn keyed_semaphores_are_reclaimed_when_idle() {
let keyed = Arc::new(KeyedSemaphores::new(1));
let semaphore = keyed.semaphore("cli:a:d1");
assert_eq!(keyed.map.len(), 1);
drop(semaphore);
keyed.semaphore("cli:b:d2");
assert_eq!(keyed.map.len(), 1, "idle session entry must be reclaimed");
}
#[tokio::test]
async fn waiting_parent_holds_no_step_permits() {
// With a session cap of 1, a parent that merely waits for its child
// must leave the session's single provider permit free; otherwise
// nested foreground delegation would deadlock.
let gate = gate(1, 1);
let token = CancellationToken::new();
let parent_step = gate
.acquire_provider(Some("cli:a:d1"), &token)
.await
.unwrap();
drop(parent_step);
// Parent is now in the waiting_children state: no permits held.
eventually(|| gate.available_provider_permits() == 8).await;
let child = gate
.acquire_provider(Some("cli:a:d1"), &token)
.await
.unwrap();
drop(child);
assert_eq!(gate.available_provider_permits(), 8);
}
}

52
src/agent/inbox.rs
View File

@ -1,52 +0,0 @@
use std::sync::{Arc, RwLock, Weak};
/// Wake contract implemented by the SessionManager. The notifier holds only
/// a weak reference so the coordinator can be dropped independently and no
/// release cycle is formed.
#[async_trait::async_trait]
pub trait AgentInboxWakeTarget: Send + Sync {
async fn wake_agent_inbox(&self, session_id: &str, revision: i64);
}
/// Late-bound wake bus between the AgentCoordinator and Session workers.
/// Event commits never wait for a wake; the durable inbox is the source of
/// truth and a lost wake only delays the next claim until the periodic
/// re-check.
pub struct AgentInboxNotifier {
target: RwLock<Option<Weak<dyn AgentInboxWakeTarget + Send + Sync>>>,
}
impl Default for AgentInboxNotifier {
fn default() -> Self {
Self {
target: RwLock::new(None),
}
}
}
impl AgentInboxNotifier {
pub fn new() -> Arc<Self> {
Arc::new(Self::default())
}
pub fn bind(&self, target: Weak<dyn AgentInboxWakeTarget + Send + Sync>) {
*self
.target
.write()
.unwrap_or_else(|poisoned| poisoned.into_inner()) = Some(target);
}
/// Best-effort wake. A missing or dead target is not an error; the
/// periodic scanner will claim the event later.
pub async fn notify(&self, session_id: &str, revision: i64) {
let target = self
.target
.read()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.as_ref()
.and_then(|weak| weak.upgrade());
if let Some(target) = target {
target.wake_agent_inbox(session_id, revision).await;
}
}
}

30
src/agent/mod.rs
View File

@ -1,38 +1,16 @@
pub mod agent_loop;
pub mod builtin;
pub mod catalog;
pub mod context_compaction;
pub mod coordinator;
pub mod definition;
pub mod gate;
pub mod inbox;
pub mod context_compressor;
pub mod media_handler;
pub mod projection;
pub mod run;
pub mod steering;
pub mod sub_agent;
pub mod system_prompt;
pub mod turn_event;
pub use agent_loop::{AgentError, AgentLoop, AgentProcessResult};
pub use catalog::{AgentCatalog, AgentCatalogError, CatalogEntryError};
pub use context_compaction::{
CompactionCandidate, CompactionReason, ContextBudget, ContextBudgetParts, ContextCompactor,
ContextRequestKey, ContextUsageTracker, PreviousCheckpoint, SequencedMessage,
context_request_digest, estimate_tokens,
};
pub use coordinator::{AgentCoordinator, CoordinatorError, ScheduledAgentExecution};
pub use definition::{AgentDefinition, AgentLimits};
pub use gate::ExecutionGate;
pub use inbox::{AgentInboxNotifier, AgentInboxWakeTarget};
pub use projection::AgentProjectionHub;
pub use run::{AgentBudget, AgentExecutionContext};
pub use steering::{SteeringDrain, SteeringPushError, TurnInput, TurnInputSource, TurnMailbox};
pub use context_compressor::{ContextCompressor, estimate_tokens};
pub use sub_agent::{
ExecutionMode, SubAgentConfig, SubAgentError, SubAgentManager, SubAgentResult, TaskStatus,
DelegateContext, ExecutionMode, SubAgentConfig, SubAgentError, SubAgentManager, SubAgentResult,
TaskNotification, TaskStatus,
};
pub use system_prompt::{
PromptContext, PromptSection, SystemPromptBuilder, build_sub_agent_system_prompt,
build_system_prompt,
};
pub use turn_event::{AgentTurnContext, TurnEmitError, TurnEmitter, TurnEvent};

74
src/agent/projection.rs
View File

@ -1,74 +0,0 @@
//! Broadcast projection for durable Agent run/event updates.
//!
//! The hub follows the WorkManager plan-change broadcast pattern: the
//! Coordinator publishes a bounded view after each commit; the gateway
//! relays it to WebSocket clients. A lost broadcast is never fatal — the
//! client recalibrates with `GetAgentRuns`, whose `revision` comes from
//! `agent_session_state`.
use crate::protocol::{AgentEventView, AgentRunView};
/// One projected change for a root session. Exactly one of `run`/`event` is
/// present.
#[derive(Debug, Clone)]
pub struct AgentProjection {
pub session_id: String,
pub revision: i64,
pub run: Option<AgentRunView>,
pub event: Option<AgentEventView>,
}
#[derive(Debug, Clone)]
pub struct AgentProjectionHub {
tx: tokio::sync::broadcast::Sender<AgentProjection>,
}
impl Default for AgentProjectionHub {
fn default() -> Self {
Self::new()
}
}
impl AgentProjectionHub {
pub fn new() -> Self {
let (tx, _) = tokio::sync::broadcast::channel(256);
Self { tx }
}
pub fn subscribe(&self) -> tokio::sync::broadcast::Receiver<AgentProjection> {
self.tx.subscribe()
}
/// Publish a run update. Best-effort: relays lag and drop the event,
/// clients recalibrate with a full query.
pub fn publish(&self, projection: AgentProjection) {
let _ = self.tx.send(projection);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn publish_reaches_subscribers_and_drop_is_nonfatal() {
let hub = AgentProjectionHub::new();
let mut rx = hub.subscribe();
hub.publish(AgentProjection {
session_id: "cli:test:d".to_string(),
revision: 1,
run: None,
event: None,
});
let received = rx.try_recv().unwrap();
assert_eq!(received.session_id, "cli:test:d");
// No subscribers after the first is dropped; publish must not panic.
drop(rx);
hub.publish(AgentProjection {
session_id: "cli:test:d".to_string(),
revision: 2,
run: None,
event: None,
});
}
}

149
src/agent/run.rs
View File

@ -1,149 +0,0 @@
use std::sync::Arc;
use std::sync::Mutex;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio_util::sync::CancellationToken;
#[derive(Debug, Clone)]
pub struct AgentBudget {
pub remaining_runs: usize,
pub remaining_depth: u16,
}
/// A signal this run emitted, preserved so the terminal completion payload
/// can reference the signal IDs for cross-checking.
#[derive(Debug, Clone)]
pub struct EmittedSignal {
pub signal_id: String,
}
#[derive(Debug, Clone)]
pub struct AgentExecutionContext {
pub root_session_id: String,
pub root_turn_id: Option<String>,
pub run_id: String,
/// Execution attempt identifier owning conditional state transitions in
/// Storage. Equal to `run_id` for the first attempt.
pub execution_id: String,
pub parent_run_id: Option<String>,
pub caller_agent_id: String,
pub current_agent_id: String,
/// Agent IDs already present in this execution chain, including current.
pub ancestry: Vec<String>,
pub depth: u16,
pub plan_item_id: Option<String>,
pub cancellation: CancellationToken,
pub budget: AgentBudget,
/// Shared admission counter across the whole delegation tree, including
/// the root run that owns this context. Durable run accounting replaces
/// it once the Coordinator persists runs; until then it is the only
/// tree-wide enforcement of `max_runs_per_tree`.
pub tree_runs: Arc<AtomicUsize>,
/// Durable emit_signal contract; `None` means this run has no signal
/// capability and must not see the `emit_signal` tool.
pub signal_contract: Option<Arc<crate::agent::definition::SignalContract>>,
/// Signals accepted by this run so far, in emit order.
pub emitted_signals: Arc<Mutex<Vec<EmittedSignal>>>,
}
impl AgentExecutionContext {
pub fn child(
parent: &Arc<Self>,
run_id: String,
target: String,
plan_item_id: Option<String>,
cancellation: CancellationToken,
) -> Self {
let mut ancestry = parent.ancestry.clone();
ancestry.push(target.clone());
Self {
root_session_id: parent.root_session_id.clone(),
root_turn_id: parent.root_turn_id.clone(),
run_id: run_id.clone(),
execution_id: run_id,
parent_run_id: Some(parent.run_id.clone()),
caller_agent_id: parent.current_agent_id.clone(),
current_agent_id: target,
ancestry,
depth: parent.depth.saturating_add(1),
plan_item_id,
cancellation,
budget: AgentBudget {
remaining_runs: parent.budget.remaining_runs.saturating_sub(1),
remaining_depth: parent.budget.remaining_depth.saturating_sub(1),
},
tree_runs: parent.tree_runs.clone(),
signal_contract: parent.signal_contract.clone(),
emitted_signals: Arc::new(Mutex::new(Vec::new())),
}
}
/// Reserve one tree-run slot for a child that has already passed the
/// caller's budget checks. Returns the total number of runs in the tree
/// after reservation, or `None` (with the counter restored) when the
/// reservation would exceed `max_runs_per_tree`.
pub fn reserve_tree_run(&self, max_runs_per_tree: usize) -> Option<usize> {
let previous = self.tree_runs.fetch_add(1, Ordering::SeqCst);
let total = previous.saturating_add(1);
if total > max_runs_per_tree {
self.tree_runs.fetch_sub(1, Ordering::SeqCst);
return None;
}
Some(total)
}
/// Remaining tree capacity for batch admission checks.
pub fn remaining_tree_runs(&self, max_runs_per_tree: usize) -> usize {
max_runs_per_tree.saturating_sub(self.tree_runs.load(Ordering::SeqCst))
}
}
#[cfg(test)]
mod tests {
use super::*;
fn root_context() -> AgentExecutionContext {
AgentExecutionContext {
root_session_id: "cli:test:dialog".to_string(),
root_turn_id: None,
run_id: "run-root".to_string(),
execution_id: "run-root".to_string(),
parent_run_id: None,
caller_agent_id: "ROOT".to_string(),
current_agent_id: "researcher".to_string(),
ancestry: vec!["researcher".to_string()],
depth: 1,
plan_item_id: None,
cancellation: CancellationToken::new(),
budget: AgentBudget {
remaining_runs: 15,
remaining_depth: 3,
},
tree_runs: Arc::new(AtomicUsize::new(1)),
signal_contract: None,
emitted_signals: Arc::new(Mutex::new(Vec::new())),
}
}
#[test]
fn tree_run_reservation_is_shared_and_bounded() {
let parent = Arc::new(root_context());
assert_eq!(parent.remaining_tree_runs(3), 2);
assert_eq!(parent.reserve_tree_run(3), Some(2));
assert_eq!(parent.reserve_tree_run(3), Some(3));
assert_eq!(parent.reserve_tree_run(3), None);
assert_eq!(parent.tree_runs.load(Ordering::SeqCst), 3);
assert_eq!(parent.remaining_tree_runs(3), 0);
let child = AgentExecutionContext::child(
&parent,
"run-child".to_string(),
"reviewer".to_string(),
None,
CancellationToken::new(),
);
assert!(Arc::ptr_eq(&parent.tree_runs, &child.tree_runs));
assert_eq!(child.remaining_tree_runs(3), 0);
}
}

814
src/agent/steering.rs
View File

@ -1,814 +0,0 @@
//! Bounded, session-owned mailbox for same-turn steering input.
//!
//! A mailbox is intentionally separate from the session work queue. The
//! gateway can accept a normal user message or a durable Agent steer event
//! while a turn is running and place it here;
//! [`AgentLoop`](super::AgentLoop) drains it only at safe model boundaries
//! (after a complete tool batch, or before deciding that a response is
//! final). The state transition performed by
//! [`TurnMailbox::drain_or_close`] is atomic with respect to producers,
//! which means an input is either accepted by the active turn or rejected so
//! the caller can put it on the next-turn queue -- never both and never
//! neither.
//!
//! Durable steer events use a two-phase admission: the session first
//! reserves an agent-lane entry, persists the `leased → admitted` transition
//! (with the turn id), and only then activates the reservation into the
//! drainable queue. Reservations and drained-but-uncommitted entries carry
//! their storage lease token so an abandoned turn can return every admitted
//! event to `pending` instead of losing it.
use crate::bus::{ChatMessage, MediaRef, MessageSource};
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
/// Default maximum number of user steering messages accepted by one active
/// turn.
pub const DEFAULT_MAX_USER_STEERING_MESSAGES: usize = 32;
/// Default aggregate UTF-8 byte budget for pending user steering messages.
pub const DEFAULT_MAX_USER_STEERING_BYTES: usize = 64 * 1024;
/// Default maximum number of Agent steer events accepted by one active turn.
pub const DEFAULT_MAX_AGENT_STEERING_MESSAGES: usize = 8;
/// Default aggregate UTF-8 byte budget for pending Agent steer events.
pub const DEFAULT_MAX_AGENT_STEERING_BYTES: usize = 32 * 1024;
/// Origin of one mailbox input.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum TurnInputSource {
User,
AgentSignal { run_id: String, agent_id: String },
AgentCompletion { run_id: String, agent_id: String },
}
impl TurnInputSource {
pub fn is_agent(&self) -> bool {
!matches!(self, Self::User)
}
}
/// How the input reached the mailbox. Queue inputs belong to the next Turn;
/// only Steer entries are drained by the active Turn.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum InputDelivery {
Queue,
Steer,
}
/// One steering input retained by the active Turn.
#[derive(Debug, Clone)]
pub struct TurnInput {
pub id: String,
pub sequence: u64,
pub source: TurnInputSource,
pub delivery: InputDelivery,
pub content: String,
pub media_refs: Vec<MediaRef>,
/// Durable inbox event id; `Some` only for Agent steer events.
pub durable_event_id: Option<String>,
pub received_at: i64,
/// Channel attribution for user inputs, preserved through the turn.
pub message_source: Option<MessageSource>,
/// Storage lease token of a durable Agent event; the mailbox keeps it so
/// an abandoned turn can release the event back to `pending`.
pub(crate) lease_token: Option<String>,
}
impl TurnInput {
pub fn user(
id: impl Into<String>,
content: impl Into<String>,
media_refs: Vec<MediaRef>,
message_source: Option<MessageSource>,
received_at: i64,
) -> Self {
Self {
id: id.into(),
sequence: 0,
source: TurnInputSource::User,
delivery: InputDelivery::Steer,
content: content.into(),
media_refs,
durable_event_id: None,
received_at,
message_source,
lease_token: None,
}
}
/// Project this input into a provider-compatible user message. Agent
/// inputs stay hidden from client history (the Signal UI comes from the
/// durable event projection) but keep their typed source for rendering
/// and cancellation recovery.
pub fn into_chat_message(self, turn_id: String, iteration: u32) -> ChatMessage {
let (client_visibility, source) = match &self.source {
TurnInputSource::User => {
let visibility = crate::bus::ClientVisibility::Visible;
let source = self.message_source.clone();
(visibility, source)
}
TurnInputSource::AgentSignal { run_id, agent_id } => {
let source = MessageSource {
kind: crate::bus::SourceKind::AgentSignal,
from_channel: None,
from_session: None,
from_user_id: None,
system_name: None,
task_id: self.durable_event_id.clone(),
from_run_id: Some(run_id.clone()),
from_agent_id: Some(agent_id.clone()),
};
(crate::bus::ClientVisibility::Hidden, Some(source))
}
TurnInputSource::AgentCompletion { run_id, agent_id } => {
let source = MessageSource {
kind: crate::bus::SourceKind::AgentCompletion,
from_channel: None,
from_session: None,
from_user_id: None,
system_name: None,
task_id: self.durable_event_id.clone(),
from_run_id: Some(run_id.clone()),
from_agent_id: Some(agent_id.clone()),
};
(crate::bus::ClientVisibility::Hidden, Some(source))
}
};
let mut message = ChatMessage::user(self.content);
message.id = self.id;
message.turn_id = Some(turn_id);
message.iteration = Some(iteration);
message.media_refs = self.media_refs;
message.timestamp = self.received_at;
message.client_visibility = client_visibility;
message.source = source;
message
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum MailboxPhase {
Accepting,
Closed,
}
#[derive(Debug)]
struct MailboxState {
phase: MailboxPhase,
pending: VecDeque<TurnInput>,
reserved: VecDeque<TurnInput>,
/// Drained at a safe boundary but not yet committed to durable history.
/// Keeping their count/size reserved prevents concurrent producers from
/// filling the capacity that an error retry may need to restore.
in_flight: VecDeque<TurnInput>,
}
impl MailboxState {
fn user_lane_used(&self) -> (usize, usize) {
let mut count = 0usize;
let mut bytes = 0usize;
for input in self.pending.iter().chain(self.in_flight.iter()) {
if !input.source.is_agent() {
count = count.saturating_add(1);
bytes = bytes.saturating_add(input_size_bytes(input));
}
}
(count, bytes)
}
fn agent_lane_used(&self) -> (usize, usize) {
let mut count = 0usize;
let mut bytes = 0usize;
for input in self
.pending
.iter()
.chain(self.in_flight.iter())
.chain(self.reserved.iter())
{
if input.source.is_agent() {
count = count.saturating_add(1);
bytes = bytes.saturating_add(input_size_bytes(input));
}
}
(count, bytes)
}
}
/// Error returned when the active turn cannot accept a steering input.
#[derive(Debug, Clone)]
pub enum SteeringPushError {
/// The turn has reached a terminal boundary. Route the input to the
/// session's ordinary queue (or, for durable events, keep it pending).
Closed,
/// The mailbox is accepting input, but its bounded lane capacity is
/// exhausted. Route the input to the ordinary queue.
Full,
}
/// Result of the atomic final-response boundary operation.
#[derive(Debug, Clone)]
pub enum SteeringDrain {
/// One or more inputs were accepted and removed from the mailbox. The
/// mailbox remains open for a subsequent safe boundary.
Messages(Vec<TurnInput>),
/// No pending input existed. The mailbox is now closed; later producers
/// receive [`SteeringPushError::Closed`].
Closed,
}
/// What an abandoned turn returns for requeue/release handling.
#[derive(Debug, Default)]
pub struct MailboxTake {
/// Pending non-durable user inputs.
pub user_inputs: Vec<TurnInput>,
/// Durable Agent entries activated and drained or still pending:
/// `(event_id, lease_token)` of `admitted` events.
pub admitted_leases: Vec<(String, String)>,
/// Reserved-but-not-activated Agent entries: `(event_id, lease_token)`
/// of `leased` (or admitted) events.
pub reserved_leases: Vec<(String, String)>,
}
impl MailboxTake {
pub fn is_empty(&self) -> bool {
self.user_inputs.is_empty()
&& self.admitted_leases.is_empty()
&& self.reserved_leases.is_empty()
}
pub fn len(&self) -> usize {
self.user_inputs.len() + self.admitted_leases.len() + self.reserved_leases.len()
}
}
/// Shared state for same-turn steering during one active AgentLoop
/// execution.
///
/// Cloning a mailbox is cheap and shares the same mutex-protected state. In
/// practice the session stores an `Arc<TurnMailbox>` in its active-turn
/// handle and gives another clone to [`AgentTurnContext`](super::AgentTurnContext).
#[derive(Clone)]
pub struct TurnMailbox {
state: Arc<Mutex<MailboxState>>,
max_user_messages: usize,
max_user_bytes: usize,
max_agent_messages: usize,
max_agent_bytes: usize,
}
impl std::fmt::Debug for TurnMailbox {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
formatter
.debug_struct("TurnMailbox")
.field("phase", &state.phase)
.field("pending", &state.pending.len())
.field("reserved", &state.reserved.len())
.field("in_flight", &state.in_flight.len())
.field("max_user_messages", &self.max_user_messages)
.field("max_agent_messages", &self.max_agent_messages)
.finish()
}
}
impl TurnMailbox {
/// Construct a mailbox using the product defaults (32 user/8 agent).
pub fn new() -> Self {
Self::with_limits(
DEFAULT_MAX_USER_STEERING_MESSAGES,
DEFAULT_MAX_USER_STEERING_BYTES,
DEFAULT_MAX_AGENT_STEERING_MESSAGES,
DEFAULT_MAX_AGENT_STEERING_BYTES,
)
}
/// Construct a mailbox with explicit bounded capacities. Zero limits
/// are allowed and make every push return [`SteeringPushError::Full`].
pub fn with_limits(
max_user_messages: usize,
max_user_bytes: usize,
max_agent_messages: usize,
max_agent_bytes: usize,
) -> Self {
Self {
state: Arc::new(Mutex::new(MailboxState {
phase: MailboxPhase::Accepting,
pending: VecDeque::new(),
reserved: VecDeque::new(),
in_flight: VecDeque::new(),
})),
max_user_messages,
max_user_bytes,
max_agent_messages,
max_agent_bytes,
}
}
/// Return an `Arc` suitable for storing in Session and AgentTurnContext.
pub fn new_shared() -> Arc<Self> {
Arc::new(Self::new())
}
/// Try to accept one real user steering input.
///
/// This operation and the final close operation use the same mutex. A
/// producer racing with `drain_or_close` therefore receives a
/// deterministic result and can route a rejected input to the ordinary
/// queue.
pub fn try_push_user(&self, input: TurnInput) -> Result<(), SteeringPushError> {
debug_assert!(!input.source.is_agent());
let input_bytes = input_size_bytes(&input);
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
if state.phase == MailboxPhase::Closed {
return Err(SteeringPushError::Closed);
}
let (count, bytes) = state.user_lane_used();
if count >= self.max_user_messages
|| bytes.saturating_add(input_bytes) > self.max_user_bytes
{
return Err(SteeringPushError::Full);
}
state.pending.push_back(input);
Ok(())
}
/// Reserve an Agent steer event for the active Turn (two-phase admission,
/// step 2). The entry is not drainable until
/// [`activate_reserved`](Self::activate_reserved) succeeds for the same
/// Turn. `lease_token` is the storage token of the `leased` event.
pub fn try_reserve_steer(
&self,
input: TurnInput,
lease_token: String,
) -> Result<(), SteeringPushError> {
debug_assert!(input.source.is_agent());
let input_bytes = input_size_bytes(&input);
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
if state.phase == MailboxPhase::Closed {
return Err(SteeringPushError::Closed);
}
let (count, bytes) = state.agent_lane_used();
if count >= self.max_agent_messages
|| bytes.saturating_add(input_bytes) > self.max_agent_bytes
{
return Err(SteeringPushError::Full);
}
let mut input = input;
input.lease_token = Some(lease_token);
state.reserved.push_back(input);
Ok(())
}
/// Activate all reservations into the drainable queue. Returns the
/// number activated. Callers invoke this only after the durable
/// `leased → admitted` transition succeeded for the same Turn.
pub fn activate_reserved(&self) -> usize {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
let count = state.reserved.len();
let reserved: Vec<_> = state.reserved.drain(..).collect();
state.pending.extend(reserved);
count
}
/// Remove all reservations without activating them. Returns
/// `(event_id, lease_token)` pairs so the caller can release the
/// still-leased events back to `pending`.
pub fn cancel_reserved(&self) -> Vec<(String, String)> {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state
.reserved
.drain(..)
.filter_map(|input| {
let token = input.lease_token.clone()?;
Some((input.durable_event_id.unwrap_or_default(), token))
})
.collect()
}
/// Drain currently pending inputs while leaving the mailbox open.
///
/// This is used after a complete tool-call batch. It intentionally does
/// not close the mailbox: another input may steer a later iteration.
pub fn drain(&self) -> Vec<TurnInput> {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
drain_pending_locked(&mut state)
}
/// Atomically drain pending inputs, or close the mailbox if it is empty.
pub fn drain_or_close(&self) -> SteeringDrain {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
if state.pending.is_empty() {
state.phase = MailboxPhase::Closed;
SteeringDrain::Closed
} else {
SteeringDrain::Messages(drain_pending_locked(&mut state))
}
}
/// Close acceptance without dropping pending messages. Session uses
/// [`take_pending`](Self::take_pending) after AgentLoop returns to move
/// those messages to the ordinary next-turn queue (for example when the
/// maximum iteration budget is exhausted).
pub fn close(&self) {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state.phase = MailboxPhase::Closed;
}
/// Close acceptance and collect everything for requeue/release handling.
/// Any in-flight batch is returned through `admitted_leases`; `/stop`
/// uses this method to preserve its queue-clearing semantics for user
/// input while still releasing durable events back to `pending`.
pub fn close_and_take_pending(&self) -> MailboxTake {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state.phase = MailboxPhase::Closed;
take_all_locked(&mut state)
}
/// Take pending inputs without changing whether producers may still push.
///
/// Normally used after `close()`; keeping this method explicit makes it
/// possible for Session to transfer accepted-but-unprocessed input to its
/// FIFO queue without opening a race with a new turn.
pub fn take_pending(&self) -> MailboxTake {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
take_all_locked(&mut state)
}
/// Restore all inputs drained by AgentLoop since the last commit. This
/// is useful when the AgentLoop completed but Session's durable write
/// then failed: the next retry/queue operation can replay the exact
/// accepted inputs instead of silently losing them.
pub fn restore_drained(&self) {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
restore_in_flight_locked(&mut state);
}
/// Restore inputs drained by AgentLoop when a provider/tool error makes
/// the current invocation retry from persisted history. The inputs are
/// prepended in their original order and their reserved capacity is
/// released. `drain()`/`drain_or_close()` reserve capacity while a batch
/// is in-flight, so this operation cannot overflow a bounded mailbox due
/// to a racing producer.
pub fn restore_front(&self, inputs: Vec<TurnInput>) {
if inputs.is_empty() {
return;
}
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
for _ in 0..inputs.len() {
state.in_flight.pop_back();
}
for input in inputs.into_iter().rev() {
state.pending.push_front(input);
}
}
/// Mark all previously drained inputs as durably committed. Session
/// calls this only after the complete Turn persistence transaction
/// succeeds. It is a no-op when no steering batch was consumed.
pub fn commit_drained(&self) {
let mut state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state.in_flight.clear();
}
/// Durable event ids currently drained into this Turn but not yet
/// committed. Session consumes them atomically with the Turn commit.
pub fn durable_in_flight_ids(&self) -> Vec<String> {
let state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state
.in_flight
.iter()
.filter_map(|input| input.durable_event_id.clone())
.collect()
}
pub fn is_closed(&self) -> bool {
let state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state.phase == MailboxPhase::Closed
}
pub fn len(&self) -> usize {
let state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state.pending.len()
}
pub fn is_empty(&self) -> bool {
self.len() == 0
}
pub fn max_user_messages(&self) -> usize {
self.max_user_messages
}
pub fn max_agent_messages(&self) -> usize {
self.max_agent_messages
}
/// Currently pending user steer entries (wake-state hint).
pub fn user_pending_count(&self) -> usize {
let state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state
.pending
.iter()
.filter(|input| !input.source.is_agent())
.count()
}
/// Currently pending agent steer entries, including reservations
/// (wake-state hint).
pub fn agent_pending_count(&self) -> usize {
let state = self
.state
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
state
.pending
.iter()
.chain(state.reserved.iter())
.filter(|input| input.source.is_agent())
.count()
}
}
impl Default for TurnMailbox {
fn default() -> Self {
Self::new()
}
}
/// Approximate the bounded payload size without serializing the complete
/// input. Content, media paths/types and source fields are all untrusted
/// input; counting their UTF-8 bytes gives a conservative enough guard while
/// retaining the original input losslessly.
fn input_size_bytes(input: &TurnInput) -> usize {
let mut bytes = input.id.len()
+ input.content.len()
+ input.durable_event_id.as_deref().map_or(0, str::len)
+ input.lease_token.as_deref().map_or(0, str::len);
for media in &input.media_refs {
bytes = bytes.saturating_add(media.path.len() + media.media_type.len());
}
if let Some(source) = input.message_source.as_ref() {
bytes = bytes
.saturating_add(source.from_channel.as_deref().map_or(0, str::len))
.saturating_add(source.from_user_id.as_deref().map_or(0, str::len));
}
bytes
}
fn drain_pending_locked(state: &mut MailboxState) -> Vec<TurnInput> {
let inputs: Vec<_> = state.pending.drain(..).collect();
state.in_flight.extend(inputs.iter().cloned());
inputs
}
fn take_all_locked(state: &mut MailboxState) -> MailboxTake {
let mut take = MailboxTake::default();
for input in state.pending.drain(..).chain(state.in_flight.drain(..)) {
if input.source.is_agent() {
if let Some(token) = input.lease_token.clone()
&& let Some(event_id) = input.durable_event_id.clone()
{
take.admitted_leases.push((event_id, token));
}
} else {
take.user_inputs.push(input);
}
}
for input in state.reserved.drain(..) {
if let Some(token) = input.lease_token.clone()
&& let Some(event_id) = input.durable_event_id.clone()
{
take.reserved_leases.push((event_id, token));
}
}
take
}
fn restore_in_flight_locked(state: &mut MailboxState) {
if state.in_flight.is_empty() {
return;
}
let inputs: Vec<_> = state.in_flight.drain(..).collect();
for input in inputs.into_iter().rev() {
state.pending.push_front(input);
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::thread;
fn user_input(id: &str, content: &str) -> TurnInput {
TurnInput::user(id, content, Vec::new(), None, 1_000)
}
fn agent_signal_input(id: &str, event_id: &str) -> TurnInput {
TurnInput {
id: id.to_string(),
sequence: 0,
source: TurnInputSource::AgentSignal {
run_id: "run-1".to_string(),
agent_id: "researcher".to_string(),
},
delivery: InputDelivery::Steer,
content: "signal summary".to_string(),
media_refs: Vec::new(),
durable_event_id: Some(event_id.to_string()),
received_at: 1_000,
message_source: None,
lease_token: None,
}
}
#[test]
fn accepts_fifo_inputs_and_clone_shares_state() {
let mailbox = TurnMailbox::with_limits(2, 100, 8, 100);
let clone = mailbox.clone();
mailbox.try_push_user(user_input("a", "one")).unwrap();
clone.try_push_user(user_input("b", "two")).unwrap();
assert_eq!(mailbox.len(), 2);
let inputs = mailbox.drain();
assert_eq!(inputs.len(), 2);
assert!(!mailbox.is_closed());
}
#[test]
fn user_and_agent_lanes_have_independent_capacity() {
let mailbox = TurnMailbox::with_limits(1, 100_000, 1, 100_000);
mailbox.try_push_user(user_input("a", "one")).unwrap();
assert!(matches!(
mailbox.try_push_user(user_input("b", "two")),
Err(SteeringPushError::Full)
));
mailbox
.try_reserve_steer(agent_signal_input("s1", "evt-1"), "token-1".to_string())
.unwrap();
assert!(matches!(
mailbox.try_reserve_steer(agent_signal_input("s2", "evt-2"), "token-2".to_string()),
Err(SteeringPushError::Full)
));
assert_eq!(mailbox.len(), 1);
}
#[test]
fn reserved_entries_are_not_drainable_until_activation() {
let mailbox = TurnMailbox::new();
mailbox
.try_reserve_steer(agent_signal_input("s1", "evt-1"), "token-1".to_string())
.unwrap();
assert!(mailbox.drain().is_empty());
assert_eq!(mailbox.activate_reserved(), 1);
assert_eq!(mailbox.drain().len(), 1);
}
#[test]
fn cancel_reserved_returns_leases() {
let mailbox = TurnMailbox::new();
mailbox
.try_reserve_steer(agent_signal_input("s1", "evt-1"), "token-1".to_string())
.unwrap();
let leases = mailbox.cancel_reserved();
assert_eq!(leases, vec![("evt-1".to_string(), "token-1".to_string())]);
assert!(mailbox.drain().is_empty());
}
#[test]
fn close_take_releases_admitted_and_reserved_durable_events() {
let mailbox = TurnMailbox::new();
mailbox
.try_reserve_steer(agent_signal_input("s1", "evt-1"), "token-1".to_string())
.unwrap();
mailbox.activate_reserved();
mailbox
.try_reserve_steer(agent_signal_input("s2", "evt-2"), "token-2".to_string())
.unwrap();
let drained = mailbox.drain();
assert_eq!(drained.len(), 1);
mailbox.try_push_user(user_input("u", "hello")).unwrap();
let take = mailbox.close_and_take_pending();
// evt-1 was drained (admitted), evt-2 still reserved (leased), the
// user input is returned separately.
assert_eq!(take.user_inputs.len(), 1);
assert_eq!(
take.admitted_leases,
vec![("evt-1".to_string(), "token-1".to_string())]
);
assert_eq!(
take.reserved_leases,
vec![("evt-2".to_string(), "token-2".to_string())]
);
assert!(mailbox.try_push_user(user_input("late", "x")).is_err());
}
#[test]
fn drained_capacity_is_reserved_until_commit_or_restore() {
let mailbox = TurnMailbox::with_limits(1, 100_000, 8, 100_000);
mailbox.try_push_user(user_input("a", "first")).unwrap();
let drained = mailbox.drain();
assert_eq!(drained.len(), 1);
assert!(matches!(
mailbox.try_push_user(user_input("b", "second")),
Err(SteeringPushError::Full)
));
mailbox.restore_front(drained);
let take = mailbox.take_pending();
assert_eq!(take.user_inputs[0].content, "first");
mailbox.try_push_user(user_input("c", "committed")).unwrap();
let _ = mailbox.drain();
mailbox.commit_drained();
mailbox
.try_push_user(user_input("d", "after commit"))
.unwrap();
let drained = mailbox.drain();
assert_eq!(drained[0].content, "after commit");
mailbox.restore_drained();
let take = mailbox.take_pending();
assert_eq!(take.user_inputs[0].content, "after commit");
}
#[test]
fn drain_or_close_is_atomic_and_preserves_close_race_semantics() {
let mailbox = Arc::new(TurnMailbox::new());
let producer = mailbox.clone();
let close_result = thread::spawn(move || producer.drain_or_close())
.join()
.unwrap();
assert!(matches!(close_result, SteeringDrain::Closed));
assert!(matches!(
mailbox.try_push_user(user_input("late", "x")),
Err(SteeringPushError::Closed)
));
}
#[test]
fn turn_input_projects_to_hidden_agent_message() {
let input = agent_signal_input("id", "evt-1");
let message = input.into_chat_message("turn-1".to_string(), 2);
assert_eq!(message.role, "user");
assert_eq!(message.turn_id.as_deref(), Some("turn-1"));
assert_eq!(message.iteration, Some(2));
assert_eq!(
message.client_visibility,
crate::bus::ClientVisibility::Hidden
);
let source = message.source.unwrap();
assert!(matches!(source.kind, crate::bus::SourceKind::AgentSignal));
assert_eq!(source.from_run_id.as_deref(), Some("run-1"));
assert_eq!(source.task_id.as_deref(), Some("evt-1"));
}
}

1161
src/agent/sub_agent.rs
View File

File diff suppressed because it is too large Load Diff

24
src/agent/system_prompt.rs
View File

@ -57,6 +57,7 @@ impl SystemPromptBuilder {
task: &str,
timeout: &str,
skills_prompt: Option<String>,
http_get_only: bool,
) -> Self {
let mut sections: Vec<Box<dyn PromptSection>> = vec![
Box::new(SubAgentIdentitySection {
@ -65,7 +66,7 @@ impl SystemPromptBuilder {
}),
Box::new(ToolHonestySection),
Box::new(SafetySection),
Box::new(SubAgentToolsSection),
Box::new(SubAgentToolsSection { http_get_only }),
Box::new(WorkspaceSection),
];
if let Some(sp) = skills_prompt {
@ -347,10 +348,10 @@ impl PromptSection for DelegationSection {
fn build(&self, _ctx: &PromptContext<'_>) -> String {
"## 子 Agent 委托原则\n\n\
- 只有当任务可以拆成独立子任务时才委托。\n\
- 子 Agent 的工具集由其定义文件（agents/*.md 的 tools 列表）决定，不要重复说明它已有哪些工具。\n\
- 子 Agent 能否继续委托由它的 delegates 白名单决定，你不需要、也无法给它额外授权。\n\
- 子 Agent 只拿完成任务所需的最小工具集。\n\
- 永远不要把 delegate 工具再分给子 Agent。\n\
- 子任务 prompt 要直接写清目标、输出格式和限制。\n\
- 并行任务彼此不能依赖；后台等待用 background（单任务或 tasks 批量，每个 run 独立返回）。"
- 并行任务彼此不能依赖，长期任务用 background。"
.to_string()
}
}
@ -379,7 +380,7 @@ impl PromptSection for SubAgentIdentitySection {
## 规则\n\
- 只专注于这个任务，不要扩展到无关话题\n\
- 只在必要时使用工具\n\
- 只有运行时明确提供 delegate 工具时才可继续委托，并遵守已配置的目标白名单\n\
- 不要使用 delegate 工具\n\
- 无法完成时，直接说明原因\n\
- 只返回最终结果，不要描述过程\n\
- 超时：{}，接近时限时返回部分结果",
@ -389,7 +390,9 @@ impl PromptSection for SubAgentIdentitySection {
}
/// Sub-agent available tools description.
pub struct SubAgentToolsSection;
pub struct SubAgentToolsSection {
pub http_get_only: bool,
}
impl PromptSection for SubAgentToolsSection {
fn name(&self) -> &str {
@ -399,6 +402,11 @@ impl PromptSection for SubAgentToolsSection {
fn build(&self, ctx: &PromptContext<'_>) -> String {
let mut s = String::from("## 可用工具\n\n");
s.push_str(&ctx.tools.describe_for_prompt());
if self.http_get_only {
s.push_str(
"\n\n**注意**：使用 http_request 时只允许 GET 方法，禁止 POST、PUT、DELETE 等。",
);
}
s
}
}
@ -506,13 +514,15 @@ pub fn build_sub_agent_system_prompt(
workspace_dir: &Path,
model_name: &str,
skills_prompt: Option<String>,
http_get_only: bool,
) -> String {
let ctx = PromptContext {
workspace_dir,
model_name,
tools,
};
SystemPromptBuilder::with_sub_agent_defaults(task, timeout_human, skills_prompt).build(&ctx)
SystemPromptBuilder::with_sub_agent_defaults(task, timeout_human, skills_prompt, http_get_only)
.build(&ctx)
}
#[cfg(test)]

139
src/agent/turn_event.rs
View File

@ -1,139 +0,0 @@
use std::sync::{Arc, Mutex};
use thiserror::Error;
use crate::agent::steering::TurnMailbox;
use crate::providers::ToolCall;
/// Presentation facts emitted while AgentLoop processes one model turn.
///
/// Events contain no persistence or channel-delivery decisions. Session owns
/// the lifecycle around these facts and reduces them into authoritative state.
#[derive(Debug, Clone)]
pub enum TurnEvent {
ReasoningDelta {
iteration: u32,
delta: String,
},
TextDelta {
iteration: u32,
delta: String,
},
TextSegmentFinished {
iteration: u32,
},
ToolStarted {
iteration: u32,
call: ToolCall,
},
ToolFinished {
iteration: u32,
call_id: String,
success: bool,
preview: Option<String>,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum TurnEmitError {
#[error("turn is no longer active")]
Inactive,
#[error("tool call {0} already exists in this turn")]
DuplicateTool(String),
#[error("tool call {0} does not exist in this turn")]
UnknownTool(String),
}
type EmitFn = dyn Fn(TurnEvent) -> Result<(), TurnEmitError> + Send + Sync;
/// Cheap cloneable handle used by AgentLoop to report presentation facts.
#[derive(Clone)]
pub struct TurnEmitter {
emit: Arc<EmitFn>,
enabled: Arc<Mutex<bool>>,
}
/// Session-owned identity and emitter for one AgentLoop execution.
#[derive(Clone)]
pub struct AgentTurnContext {
pub turn_id: String,
pub message_id: String,
pub emitter: TurnEmitter,
/// Same-turn user input accepted while this turn is active. Session owns
/// the mailbox lifecycle; AgentLoop only drains it at safe boundaries.
pub steering: Option<Arc<TurnMailbox>>,
}
impl AgentTurnContext {
pub fn new(
turn_id: impl Into<String>,
message_id: impl Into<String>,
emitter: TurnEmitter,
) -> Self {
Self {
turn_id: turn_id.into(),
message_id: message_id.into(),
emitter,
steering: None,
}
}
/// Construct a streaming context with a shared steering mailbox.
pub fn new_with_steering(
turn_id: impl Into<String>,
message_id: impl Into<String>,
emitter: TurnEmitter,
steering: Arc<TurnMailbox>,
) -> Self {
Self {
turn_id: turn_id.into(),
message_id: message_id.into(),
emitter,
steering: Some(steering),
}
}
/// Attach a mailbox to an existing context. This builder keeps the old
/// `AgentTurnContext::new` call sites source-compatible.
pub fn with_steering(mut self, steering: Arc<TurnMailbox>) -> Self {
self.steering = Some(steering);
self
}
/// Return a clone of the shared mailbox, if steering is enabled.
pub fn steering(&self) -> Option<Arc<TurnMailbox>> {
self.steering.clone()
}
}
impl TurnEmitter {
pub(crate) fn new<F>(emit: F) -> Self
where
F: Fn(TurnEvent) -> Result<(), TurnEmitError> + Send + Sync + 'static,
{
Self {
emit: Arc::new(emit),
enabled: Arc::new(Mutex::new(true)),
}
}
pub fn emit(&self, event: TurnEvent) -> Result<(), TurnEmitError> {
let enabled = self
.enabled
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
if !*enabled {
return Ok(());
}
(self.emit)(event)
}
/// Stop forwarding new presentation facts without changing durable or
/// terminal Turn status. Session uses this before invalidating a worker.
pub fn deactivate(&self) {
*self
.enabled
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner()) = false;
}
}

238
src/bus/dispatcher.rs
View File

@ -1,14 +1,12 @@
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use tokio::sync::mpsc;
use crate::bus::{DeliveryReceipt, MessageBus, OutboundMessage};
use crate::bus::{MessageBus, OutboundMessage};
use crate::channels::ChannelManager;
use crate::channels::base::{Channel, ChannelError};
use crate::delivery::ConversationWriteLocks;
use crate::task_supervisor::TaskSupervisor;
const LANE_CAPACITY: usize = 64;
@ -22,8 +20,6 @@ pub struct OutboundDispatcher {
bus: Arc<MessageBus>,
channel_manager: ChannelManager,
task_supervisor: TaskSupervisor,
write_locks: ConversationWriteLocks,
active_lanes: Arc<AtomicUsize>,
}
impl OutboundDispatcher {
@ -31,15 +27,11 @@ impl OutboundDispatcher {
bus: Arc<MessageBus>,
channel_manager: ChannelManager,
task_supervisor: TaskSupervisor,
write_locks: ConversationWriteLocks,
active_lanes: Arc<AtomicUsize>,
) -> Self {
Self {
bus,
channel_manager,
task_supervisor,
write_locks,
active_lanes,
}
}
@ -64,14 +56,12 @@ impl OutboundDispatcher {
if sender.as_ref().is_none_or(mpsc::Sender::is_closed) {
let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else {
tracing::warn!(channel = %msg.channel, "No channel found for message");
msg.complete_delivery(DeliveryReceipt::PermanentFailure {
summary: format!("channel not found: {}", msg.channel),
});
msg.complete_delivery(Err(format!("channel not found: {}", msg.channel)));
continue;
};
let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY);
if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone()) {
msg.complete_delivery(DeliveryReceipt::DispatcherClosed);
msg.complete_delivery(Err("dispatcher is shutting down".to_string()));
continue;
}
lanes.insert(lane_key.clone(), new_sender.clone());
@ -91,9 +81,7 @@ impl OutboundDispatcher {
capacity = LANE_CAPACITY,
"Outbound lane full; rejecting message instead of blocking other destinations"
);
msg.complete_delivery(DeliveryReceipt::TransientFailure {
summary: "outbound lane is full".to_string(),
});
msg.complete_delivery(Err("outbound lane is full".to_string()));
}
Err(mpsc::error::TrySendError::Closed(msg)) => {
// The lane may have expired between the closed check and
@ -101,15 +89,13 @@ impl OutboundDispatcher {
lanes.remove(&lane_key);
let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else {
tracing::warn!(channel = %msg.channel, "No channel found for message");
msg.complete_delivery(DeliveryReceipt::PermanentFailure {
summary: format!("channel not found: {}", msg.channel),
});
msg.complete_delivery(Err(format!("channel not found: {}", msg.channel)));
continue;
};
let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY);
if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone())
{
msg.complete_delivery(DeliveryReceipt::DispatcherClosed);
msg.complete_delivery(Err("dispatcher is shutting down".to_string()));
continue;
}
match new_sender.try_send(msg) {
@ -117,9 +103,9 @@ impl OutboundDispatcher {
lanes.insert(lane_key, new_sender);
}
Err(error) => {
error
.into_inner()
.complete_delivery(DeliveryReceipt::DispatcherClosed);
error.into_inner().complete_delivery(Err(
"outbound lane could not be restarted during shutdown".to_string(),
));
}
}
}
@ -134,30 +120,24 @@ impl OutboundDispatcher {
channel_name: String,
chat_id: String,
) -> bool {
let target_lock = self.write_locks.for_target(&channel_name, &chat_id);
let active_lanes = self.active_lanes.clone();
self.task_supervisor.spawn(
format!("outbound-lane:{channel_name}:{chat_id}"),
async move {
active_lanes.fetch_add(1, Ordering::Relaxed);
let _guard = LaneGuard {
counter: active_lanes,
};
loop {
let msg = match tokio::time::timeout(LANE_IDLE_TIMEOUT, receiver.recv()).await {
Ok(Some(msg)) => msg,
Ok(None) | Err(_) => break,
};
let result = Self::send_with_retry(&*channel, &msg, &target_lock).await;
if result != DeliveryReceipt::Delivered {
let result = Self::send_with_retry(&*channel, &msg).await;
if let Err(error) = &result {
tracing::error!(
channel = %channel_name,
chat_id = %chat_id,
result = ?result,
error = %error,
"Failed to send message after retries"
);
}
msg.complete_delivery(result);
msg.complete_delivery(result.map_err(|error| error.to_string()));
}
},
)
@ -166,29 +146,25 @@ impl OutboundDispatcher {
async fn send_with_retry(
channel: &dyn Channel,
msg: &OutboundMessage,
target_lock: &tokio::sync::Mutex<()>,
) -> DeliveryReceipt {
let _guard = target_lock.lock().await;
) -> Result<(), ChannelError> {
const DELAYS: &[u64] = &[1, 2, 4];
for (attempt, &delay) in DELAYS.iter().enumerate() {
let result = tokio::time::timeout(SEND_TIMEOUT, channel.send(msg.clone())).await;
match result {
Ok(Ok(())) => return DeliveryReceipt::Delivered,
Ok(Ok(())) => return Ok(()),
Ok(Err(error)) if attempt < DELAYS.len() - 1 && error.is_transient() => {
tracing::warn!(
attempt = attempt + 1,
delay,
error_class = channel_error_class(&error),
"Send failed, retrying"
);
tracing::warn!(attempt = attempt + 1, delay, error = %error, "Send failed, retrying");
}
Ok(Err(error)) => return receipt_from_channel_error(error),
Ok(Err(error)) => return Err(error),
Err(_) if attempt < DELAYS.len() - 1 => {
tracing::warn!(attempt = attempt + 1, delay, "Send timed out, retrying");
}
Err(_) => {
return DeliveryReceipt::TimedOut;
return Err(ChannelError::Other(format!(
"send timed out after {} seconds",
SEND_TIMEOUT.as_secs()
)));
}
}
tokio::time::sleep(Duration::from_secs(delay)).await;
@ -197,48 +173,6 @@ impl OutboundDispatcher {
}
}
fn channel_error_class(error: &ChannelError) -> &'static str {
match error {
ChannelError::ConnectionError(_) => "connection",
ChannelError::SendError(_) => "send",
ChannelError::BusError(_) => "bus",
ChannelError::ConfigError(_) => "config",
ChannelError::Other(_) => "other",
}
}
fn receipt_from_channel_error(error: ChannelError) -> DeliveryReceipt {
match error {
ChannelError::ConnectionError(_) => DeliveryReceipt::TransientFailure {
summary: "channel connection failed after retries".to_string(),
},
ChannelError::SendError(_) => DeliveryReceipt::TransientFailure {
summary: "channel send failed after retries".to_string(),
},
ChannelError::BusError(_) => DeliveryReceipt::TransientFailure {
summary: "channel bus was unavailable".to_string(),
},
ChannelError::ConfigError(_) => DeliveryReceipt::PermanentFailure {
summary: "channel configuration rejected delivery".to_string(),
},
ChannelError::Other(_) => DeliveryReceipt::PermanentFailure {
summary: "channel rejected delivery".to_string(),
},
}
}
/// Decrements the active-lane counter exactly once when a lane task ends,
/// whether it exits normally, is cancelled, or is aborted.
struct LaneGuard {
counter: Arc<AtomicUsize>,
}
impl Drop for LaneGuard {
fn drop(&mut self) {
self.counter.fetch_sub(1, Ordering::Relaxed);
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -329,13 +263,7 @@ mod tests {
manager.register_channel("recording", channel.clone()).await;
let supervisor = TaskSupervisor::new();
let dispatcher = OutboundDispatcher::new(
bus.clone(),
manager,
supervisor.clone(),
ConversationWriteLocks::default(),
Arc::new(AtomicUsize::new(0)),
);
let dispatcher = OutboundDispatcher::new(bus.clone(), manager, supervisor.clone());
let task = tokio::spawn(async move { dispatcher.run().await });
bus.publish_outbound(outbound("slow", "slow-1"))
.await
@ -373,20 +301,14 @@ mod tests {
bus.clone(),
);
let supervisor = TaskSupervisor::new();
let dispatcher = OutboundDispatcher::new(
bus.clone(),
manager,
supervisor.clone(),
ConversationWriteLocks::default(),
Arc::new(AtomicUsize::new(0)),
);
let dispatcher = OutboundDispatcher::new(bus.clone(), manager, supervisor.clone());
let task = tokio::spawn(async move { dispatcher.run().await });
let mut message = outbound("missing", "not delivered");
message.channel = "missing".to_string();
let error = bus.deliver_outbound(message).await.unwrap_err();
assert!(matches!(error, crate::bus::BusError::DeliveryPermanent(_)));
assert!(matches!(error, crate::bus::BusError::DeliveryFailed(_)));
task.abort();
supervisor.shutdown(Duration::from_secs(1)).await;
}
@ -404,13 +326,7 @@ mod tests {
});
manager.register_channel("recording", channel.clone()).await;
let supervisor = TaskSupervisor::new();
let dispatcher = OutboundDispatcher::new(
bus.clone(),
manager,
supervisor.clone(),
ConversationWriteLocks::default(),
Arc::new(AtomicUsize::new(0)),
);
let dispatcher = OutboundDispatcher::new(bus.clone(), manager, supervisor.clone());
let task = tokio::spawn(async move { dispatcher.run().await });
bus.deliver_outbound(outbound("confirmed", "delivered"))
@ -422,113 +338,17 @@ mod tests {
supervisor.shutdown(Duration::from_secs(1)).await;
}
#[tokio::test]
async fn active_lane_count_tracks_task_lifetime() {
let bus = MessageBus::new(8);
let manager = ChannelManager::with_bus(
Arc::new(crate::channels::CliChatChannel::new()),
bus.clone(),
);
manager
.register_channel(
"recording",
Arc::new(RecordingChannel {
sent: Mutex::new(Vec::new()),
notify: Notify::new(),
}),
)
.await;
let supervisor = TaskSupervisor::new();
let active_lanes = Arc::new(AtomicUsize::new(0));
let dispatcher = OutboundDispatcher::new(
bus.clone(),
manager,
supervisor.clone(),
ConversationWriteLocks::default(),
active_lanes.clone(),
);
let dispatcher_task = tokio::spawn(async move { dispatcher.run().await });
bus.publish_outbound(outbound("counted", "message"))
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(1), async {
while active_lanes.load(Ordering::Relaxed) != 1 {
tokio::task::yield_now().await;
}
})
.await
.unwrap();
supervisor.shutdown(Duration::from_secs(1)).await;
assert_eq!(active_lanes.load(Ordering::Relaxed), 0);
dispatcher_task.abort();
let _ = dispatcher_task.await;
}
#[tokio::test]
async fn permanent_send_failure_is_not_retried() {
let channel = PermanentFailureChannel {
attempts: AtomicUsize::new(0),
};
let target_lock = tokio::sync::Mutex::new(());
let receipt = OutboundDispatcher::send_with_retry(
&channel,
&outbound("invalid", "message"),
&target_lock,
)
.await;
let error = OutboundDispatcher::send_with_retry(&channel, &outbound("invalid", "message"))
.await
.unwrap_err();
assert!(matches!(receipt, DeliveryReceipt::PermanentFailure { .. }));
assert!(matches!(error, ChannelError::Other(_)));
assert_eq!(channel.attempts.load(Ordering::SeqCst), 1);
}
#[test]
fn channel_errors_map_to_typed_sanitized_receipts() {
for error in [
ChannelError::ConnectionError("https://secret.example/?token=x".to_string()),
ChannelError::SendError("private response body".to_string()),
ChannelError::BusError("private queue detail".to_string()),
] {
let receipt = receipt_from_channel_error(error);
assert!(matches!(receipt, DeliveryReceipt::TransientFailure { .. }));
assert!(!format!("{receipt:?}").contains("private"));
assert!(!format!("{receipt:?}").contains("secret"));
}
for error in [
ChannelError::ConfigError("api_key=x".to_string()),
ChannelError::Other("private platform payload".to_string()),
] {
let receipt = receipt_from_channel_error(error);
assert!(matches!(receipt, DeliveryReceipt::PermanentFailure { .. }));
assert!(!format!("{receipt:?}").contains("private"));
assert!(!format!("{receipt:?}").contains("api_key"));
}
}
#[tokio::test]
async fn shared_write_lock_orders_dispatcher_with_live_turn_writes() {
let channel = RecordingChannel {
sent: Mutex::new(Vec::new()),
notify: Notify::new(),
};
let write_locks = ConversationWriteLocks::default();
let target_lock = write_locks.for_target("recording", "same-chat");
let live_write = target_lock.lock().await;
let message = outbound("same-chat", "after-live-update");
let send = OutboundDispatcher::send_with_retry(&channel, &message, &target_lock);
tokio::pin!(send);
assert!(
tokio::time::timeout(Duration::from_millis(10), &mut send)
.await
.is_err()
);
assert!(channel.sent.lock().await.is_empty());
drop(live_write);
assert_eq!(send.await, DeliveryReceipt::Delivered);
assert_eq!(channel.sent.lock().await.as_slice(), &["after-live-update"]);
}
}

247
src/bus/message.rs
View File

@ -3,52 +3,6 @@ use std::collections::HashMap;
use crate::providers::ToolCall;
/// Provider-private state required to faithfully replay an assistant message.
///
/// This is durable conversation data, but it is never presentation data. UI and
/// channel projections must not serialize it to end users.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ProviderReasoningState {
pub provider: String,
pub payload: serde_json::Value,
}
impl ProviderReasoningState {
/// Decode persisted provider state without making conversation history
/// unreadable when an old or damaged payload is encountered.
pub fn from_json_lossy(value: &str) -> Option<Self> {
serde_json::from_str(value).ok()
}
}
/// Describes whether a persisted message represents a complete model result.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum CompletionStatus {
#[default]
Completed,
Cancelled,
Interrupted,
}
impl CompletionStatus {
pub fn as_str(self) -> &'static str {
match self {
Self::Completed => "completed",
Self::Cancelled => "cancelled",
Self::Interrupted => "interrupted",
}
}
pub fn from_storage(value: &str) -> Self {
match value {
"cancelled" => Self::Cancelled,
"interrupted" => Self::Interrupted,
_ => Self::Completed,
}
}
}
// ============================================================================
// ContentBlock - Multimodal content representation (OpenAI-style)
// ============================================================================
@ -119,77 +73,18 @@ impl MediaItem {
media_type: self.media_type.clone(),
}
}
pub fn from_media_ref(media_ref: &MediaRef) -> Self {
Self::new(media_ref.path.clone(), media_ref.media_type.clone())
}
}
// ============================================================================
// ChatMessage - Used by AgentLoop for LLM conversation history
// ============================================================================
/// Whether a message may be surfaced to clients. Hidden messages exist only
/// for model replay (internal triggers) and must never appear in history,
/// projections or delivery.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum ClientVisibility {
#[default]
Visible,
Hidden,
}
impl ClientVisibility {
pub fn as_str(&self) -> &'static str {
match self {
Self::Visible => "visible",
Self::Hidden => "hidden",
}
}
}
/// Where a message Turn originated. Persisted alongside the message so
/// clients can render agent-driven continuation output without treating it as
/// a user bubble.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum TurnOrigin {
#[default]
User,
AgentContinuation,
Scheduled,
}
impl TurnOrigin {
pub fn as_str(&self) -> &'static str {
match self {
Self::User => "user",
Self::AgentContinuation => "agent_continuation",
Self::Scheduled => "scheduled",
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ChatMessage {
pub id: String,
pub role: String,
pub content: String,
pub reasoning_content: Option<String>,
/// Opaque state used only when replaying history to the same provider.
#[serde(skip_serializing_if = "Option::is_none")]
pub provider_state: Option<ProviderReasoningState>,
#[serde(skip_serializing_if = "Option::is_none")]
pub turn_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub iteration: Option<u32>,
#[serde(default)]
pub completion_status: CompletionStatus,
#[serde(default)]
pub client_visibility: ClientVisibility,
#[serde(default)]
pub turn_origin: TurnOrigin,
pub media_refs: Vec<MediaRef>,
pub timestamp: i64,
#[serde(skip_serializing_if = "Option::is_none")]
@ -204,20 +99,12 @@ pub struct ChatMessage {
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum SourceKind {
#[serde(rename = "user_input")]
UserInput,
#[serde(rename = "system_notification")]
SystemNotification,
#[serde(rename = "cross_channel")]
CrossChannel,
#[serde(rename = "external_trigger")]
ExternalTrigger,
/// A durable signal emitted by a background Agent via `emit_signal`.
#[serde(rename = "agent_signal")]
AgentSignal,
/// A durable background run completion outcome.
#[serde(rename = "agent_result")]
AgentCompletion,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@ -228,12 +115,6 @@ pub struct MessageSource {
pub from_user_id: Option<String>,
pub system_name: Option<String>,
pub task_id: Option<String>,
/// Durable Agent run identity for `agent_signal`/`agent_result` sources.
#[serde(default)]
pub from_run_id: Option<String>,
/// Agent definition id for `agent_signal`/`agent_result` sources.
#[serde(default)]
pub from_agent_id: Option<String>,
}
impl ChatMessage {
@ -243,18 +124,12 @@ impl ChatMessage {
role: "user".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs: Vec::new(),
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: None,
source: None,
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -264,18 +139,12 @@ impl ChatMessage {
role: "user".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs,
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: None,
source: None,
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -285,18 +154,12 @@ impl ChatMessage {
role: "assistant".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs: Vec::new(),
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: None,
source: None,
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -309,18 +172,12 @@ impl ChatMessage {
role: "assistant".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs: Vec::new(),
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: Some(tool_calls),
source: None,
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -330,18 +187,12 @@ impl ChatMessage {
role: "assistant".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs: Vec::new(),
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: None,
source: Some(source),
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -351,18 +202,12 @@ impl ChatMessage {
role: "system".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs: Vec::new(),
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: None,
source: None,
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -385,18 +230,12 @@ impl ChatMessage {
role: "tool".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs,
timestamp: current_timestamp(),
tool_call_id: Some(tool_call_id.into()),
tool_name: Some(tool_name.into()),
tool_calls: None,
source: None,
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
@ -406,95 +245,32 @@ impl ChatMessage {
role: "user".to_string(),
content: content.into(),
reasoning_content: None,
provider_state: None,
turn_id: None,
iteration: None,
completion_status: CompletionStatus::Completed,
media_refs: Vec::new(),
timestamp: current_timestamp(),
tool_call_id: None,
tool_name: None,
tool_calls: None,
source: Some(source),
client_visibility: ClientVisibility::Visible,
turn_origin: TurnOrigin::User,
}
}
}
#[cfg(test)]
mod conversation_message_tests {
use super::*;
#[test]
fn damaged_provider_state_is_ignored() {
assert!(ProviderReasoningState::from_json_lossy("not-json").is_none());
}
#[test]
fn unknown_completion_status_is_backward_compatible() {
assert_eq!(
CompletionStatus::from_storage("future-status"),
CompletionStatus::Completed
);
}
}
// ============================================================================
// InboundMessage - Message from Channel to Bus (user input)
// ============================================================================
/// Opaque channel-owned context that may be carried to the corresponding reply.
/// Core routing understands `reply_to`; all other platform data remains
/// private. `durable_private` holds only values the channel declares safe to
/// reuse across turns (thread/root identity); one-shot message/reaction ids
/// belong in `private`.
#[derive(Debug, Clone, Default)]
pub struct ChannelContext {
pub reply_to: Option<String>,
pub private: HashMap<String, String>,
pub durable_private: HashMap<String, String>,
}
/// Public, durable projection of a newly committed conversation message.
/// Provider replay state and source identities are deliberately excluded.
#[derive(Debug, Clone)]
pub struct CommittedMessage {
pub id: String,
pub seq: i64,
pub role: String,
pub content: String,
pub reasoning_content: Option<String>,
pub completion_status: CompletionStatus,
pub media_refs: Vec<MediaRef>,
pub created_at: i64,
pub tool_call_id: Option<String>,
pub tool_name: Option<String>,
pub tool_calls: Option<Vec<ToolCall>>,
pub turn_origin: TurnOrigin,
}
#[derive(Debug, Clone)]
pub struct CommittedTurnDelta {
pub session_id: String,
/// Highest durable message sequence included in this commit.
pub history_revision: i64,
pub messages: Vec<CommittedMessage>,
}
#[derive(Debug, Clone)]
pub struct InboundMessage {
pub channel: String,
pub sender_id: String,
pub chat_id: String,
/// Client-provided id for optimistic UI reconciliation. Channel-owned
/// inputs that do not expose a client id leave this unset; the session
/// layer may generate a durable id when it accepts the message.
pub client_message_id: Option<String>,
pub content: String,
pub received_at: i64,
pub timestamp: i64,
pub media: Vec<MediaItem>,
pub channel_context: ChannelContext,
/// Channel-specific data used internally by the channel (not forwarded).
pub metadata: HashMap<String, String>,
/// Data forwarded from inbound to outbound (copied to OutboundMessage.metadata by gateway).
pub forwarded_metadata: HashMap<String, String>,
}
// ============================================================================
@ -509,26 +285,17 @@ pub struct OutboundMessage {
pub reply_to: Option<String>,
pub media: Vec<MediaItem>,
pub metadata: HashMap<String, String>,
pub(crate) delivery: Option<tokio::sync::watch::Sender<Option<DeliveryReceipt>>>,
pub(crate) delivery: Option<tokio::sync::watch::Sender<Option<Result<(), String>>>>,
}
impl OutboundMessage {
pub(crate) fn complete_delivery(&self, result: DeliveryReceipt) {
pub(crate) fn complete_delivery(&self, result: Result<(), String>) {
if let Some(delivery) = &self.delivery {
delivery.send_replace(Some(result));
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum DeliveryReceipt {
Delivered,
TransientFailure { summary: String },
PermanentFailure { summary: String },
TimedOut,
DispatcherClosed,
}
// ============================================================================
// ControlMessage - Message for control channel (session management)
// Uses SessionCommand from session module

85
src/bus/mod.rs
View File

@ -3,9 +3,8 @@ pub mod message;
pub use dispatcher::OutboundDispatcher;
pub use message::{
ChannelContext, ChatMessage, ClientVisibility, CommittedMessage, CommittedTurnDelta,
CompletionStatus, ContentBlock, ControlMessage, DeliveryReceipt, InboundMessage, MediaItem,
MediaRef, MessageSource, OutboundMessage, ProviderReasoningState, SourceKind, TurnOrigin,
ChatMessage, ContentBlock, ControlMessage, InboundMessage, MediaItem, MediaRef, MessageSource,
OutboundMessage, SourceKind,
};
use std::sync::Arc;
@ -80,17 +79,7 @@ impl MessageBus {
loop {
delivery_rx.changed().await.map_err(|_| BusError::Closed)?;
if let Some(result) = delivery_rx.borrow().clone() {
return match result {
DeliveryReceipt::Delivered => Ok(()),
DeliveryReceipt::TransientFailure { summary } => {
Err(BusError::DeliveryTransient(summary))
}
DeliveryReceipt::PermanentFailure { summary } => {
Err(BusError::DeliveryPermanent(summary))
}
DeliveryReceipt::TimedOut => Err(BusError::DeliveryTimedOut),
DeliveryReceipt::DispatcherClosed => Err(BusError::Closed),
};
return result.map_err(BusError::DeliveryFailed);
}
}
})
@ -116,29 +105,6 @@ impl MessageBus {
pub async fn consume_control(&self) -> Option<ControlMessage> {
self.control_rx.lock().await.recv().await
}
/// Snapshot of the current depth and capacity of each bus queue.
pub fn queue_depths(&self) -> QueueDepths {
QueueDepths {
inbound_depth: (self.inbound_tx.max_capacity() - self.inbound_tx.capacity()) as u64,
inbound_cap: self.inbound_tx.max_capacity() as u64,
outbound_depth: (self.outbound_tx.max_capacity() - self.outbound_tx.capacity()) as u64,
outbound_cap: self.outbound_tx.max_capacity() as u64,
control_depth: (self.control_tx.max_capacity() - self.control_tx.capacity()) as u64,
control_cap: self.control_tx.max_capacity() as u64,
}
}
}
/// Read-only snapshot of MessageBus queue utilization.
#[derive(serde::Serialize)]
pub struct QueueDepths {
pub inbound_depth: u64,
pub inbound_cap: u64,
pub outbound_depth: u64,
pub outbound_cap: u64,
pub control_depth: u64,
pub control_cap: u64,
}
// ============================================================================
@ -148,8 +114,7 @@ pub struct QueueDepths {
#[derive(Debug)]
pub enum BusError {
Closed,
DeliveryTransient(String),
DeliveryPermanent(String),
DeliveryFailed(String),
DeliveryTimedOut,
}
@ -157,50 +122,10 @@ impl std::fmt::Display for BusError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
BusError::Closed => write!(f, "Bus channel closed"),
BusError::DeliveryTransient(error) => {
write!(f, "Transient outbound delivery failure: {error}")
}
BusError::DeliveryPermanent(error) => {
write!(f, "Permanent outbound delivery failure: {error}")
}
BusError::DeliveryFailed(error) => write!(f, "Outbound delivery failed: {error}"),
BusError::DeliveryTimedOut => write!(f, "Outbound delivery confirmation timed out"),
}
}
}
impl std::error::Error for BusError {}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
#[tokio::test]
async fn queue_depths_report_retained_sender_usage() {
let bus = MessageBus::new(3);
let empty = bus.queue_depths();
assert_eq!(empty.inbound_depth, 0);
assert_eq!(empty.inbound_cap, 3);
assert_eq!(empty.outbound_depth, 0);
assert_eq!(empty.outbound_cap, 3);
assert_eq!(empty.control_depth, 0);
assert_eq!(empty.control_cap, 3);
bus.publish_outbound(OutboundMessage {
channel: "test".to_string(),
chat_id: "chat".to_string(),
content: "queued".to_string(),
reply_to: None,
media: vec![],
metadata: HashMap::new(),
delivery: None,
})
.await
.unwrap();
assert_eq!(bus.queue_depths().outbound_depth, 1);
bus.consume_outbound().await.unwrap();
assert_eq!(bus.queue_depths().outbound_depth, 0);
}
}

67
src/channels/base.rs
View File

@ -1,33 +1,7 @@
use async_trait::async_trait;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use crate::bus::{BusError, CommittedTurnDelta, InboundMessage, MessageBus, OutboundMessage};
use crate::delivery::PresentationPolicy;
use crate::session::TurnSnapshot;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum LivePolicy {
FinalOnly,
Snapshot { min_interval: Duration },
}
#[derive(Debug, Clone)]
pub struct TurnTarget {
pub channel: String,
pub chat_id: String,
pub session_id: String,
pub reply_to: Option<String>,
pub metadata: HashMap<String, String>,
}
#[async_trait]
pub trait TurnSink: Send {
async fn update(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError>;
async fn finish(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError>;
async fn abort(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError>;
}
use crate::bus::{BusError, InboundMessage, MessageBus, OutboundMessage};
#[derive(Debug)]
pub enum ChannelError {
@ -75,41 +49,16 @@ pub trait Channel: Send + Sync + 'static {
/// Stop the channel
async fn stop(&self) -> Result<(), ChannelError>;
fn live_policy(&self) -> LivePolicy {
LivePolicy::FinalOnly
}
fn presentation_policy(&self) -> PresentationPolicy {
PresentationPolicy::external(matches!(self.live_policy(), LivePolicy::Snapshot { .. }))
}
async fn open_turn(&self, _target: TurnTarget) -> Result<Box<dyn TurnSink>, ChannelError> {
Err(ChannelError::Other(format!(
"channel {} does not support turn delivery",
self.name()
)))
}
/// Deliver a durable history delta after a Turn commit. Channels without
/// local history views intentionally ignore this event.
async fn commit_turn(
&self,
_target: &TurnTarget,
_delta: CommittedTurnDelta,
) -> Result<(), ChannelError> {
Ok(())
}
/// Whether `commit_turn` presents media references from the committed
/// assistant message to the user. Channels that return false receive a
/// separate media-only outbound delivery after the durable commit.
fn commit_turn_presents_media(&self) -> bool {
false
}
/// Send a message to the channel (called by OutboundDispatcher)
async fn send(&self, msg: OutboundMessage) -> Result<(), ChannelError>;
/// Send a streaming delta (optional, for channels that support it)
async fn send_delta(&self, chat_id: &str, delta: &str) -> Result<(), ChannelError> {
let _ = chat_id;
let _ = delta;
Ok(())
}
/// Check if a sender is allowed to use this channel
fn is_allowed(&self, _sender_id: &str) -> bool {
true

425
src/channels/cli_chat.rs
View File

@ -1,18 +1,16 @@
use async_trait::async_trait;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::{Mutex, mpsc};
use crate::bus::{CommittedTurnDelta, ControlMessage, InboundMessage, MessageBus, OutboundMessage};
use crate::bus::{ControlMessage, InboundMessage, MessageBus, OutboundMessage};
use crate::gateway::uploads::UploadRegistry;
use crate::protocol::{
HistoryMessage, MessageAttachment, SlashCommandInfo, WsInbound, WsOutbound, parse_inbound,
};
use crate::session::TurnSnapshot;
use crate::session::{SessionCommand, SessionEvent, UnifiedSessionId};
use super::base::{Channel, ChannelError, LivePolicy, TurnSink, TurnTarget};
use super::base::{Channel, ChannelError};
// ============================================================================
// Client - Connected CLI client
@ -36,7 +34,7 @@ impl Client {
pub struct CliChatChannel {
bus: std::sync::Mutex<Option<Arc<MessageBus>>>,
clients: Arc<Mutex<HashMap<String, Arc<Client>>>>,
clients: Mutex<HashMap<String, Arc<Client>>>,
uploads: UploadRegistry,
}
@ -54,7 +52,7 @@ impl CliChatChannel {
pub fn with_upload_registry(uploads: UploadRegistry) -> Self {
Self {
bus: std::sync::Mutex::new(None),
clients: Arc::new(Mutex::new(HashMap::new())),
clients: Mutex::new(HashMap::new()),
uploads,
}
}
@ -132,40 +130,6 @@ impl CliChatChannel {
}
}
/// Push a durable Agent run/event projection update to the owning
/// WebSocket client. Clients recalibrate with `GetAgentRuns` when a
/// broadcast is lagged or lost.
pub async fn publish_agent_projection(
&self,
projection: crate::agent::projection::AgentProjection,
) {
let Some(session_id) = UnifiedSessionId::parse(&projection.session_id) else {
return;
};
if session_id.channel != "cli_chat" {
return;
}
let client = self.clients.lock().await.get(&session_id.chat_id).cloned();
if let Some(client) = client {
let message = if let Some(run) = projection.run {
WsOutbound::AgentRunUpdated {
session_id: projection.session_id,
revision: projection.revision,
run,
}
} else if let Some(event) = projection.event {
WsOutbound::AgentEventUpdated {
session_id: projection.session_id,
revision: projection.revision,
event,
}
} else {
return;
};
let _ = client.sender.send(message).await;
}
}
/// Handle an inbound message from a client
pub(crate) async fn handle_inbound(&self, client: Arc<Client>, raw_msg: &str) {
match parse_inbound(raw_msg) {
@ -213,7 +177,6 @@ impl CliChatChannel {
WsInbound::UserInput {
content,
upload_ids,
client_message_id,
chat_id,
..
} => {
@ -222,15 +185,8 @@ impl CliChatChannel {
if content.trim().is_empty() && upload_ids.is_empty() {
return Err(ChannelError::Other("Message is empty".to_string()));
}
// `/queue` is deliberately allowed to carry attachments: it
// is a message-routing directive whose payload remains a
// normal user input. Other slash commands still reject
// attachments because their handlers do not consume media.
let slash_allows_attachments = crate::channels::parse_slash_command(&content)
.is_some_and(|(name, _)| name.eq_ignore_ascii_case("queue"));
if !upload_ids.is_empty()
&& crate::channels::parse_slash_command(&content).is_some()
&& !slash_allows_attachments
{
return Err(ChannelError::Other(
"Attachments cannot be sent with slash commands".to_string(),
@ -242,18 +198,6 @@ impl CliChatChannel {
"Chat does not belong to this client".to_string(),
));
}
let client_message_id =
client_message_id.and_then(|raw| match uuid::Uuid::parse_str(&raw) {
Ok(id) => Some(id.to_string()),
Err(error) => {
tracing::warn!(
client_message_id = %raw,
error = %error,
"Ignoring invalid client message id"
);
None
}
});
let uploads = self
.uploads
.take_many(&client.chat_id, &upload_ids)
@ -264,11 +208,11 @@ impl CliChatChannel {
channel: self.name().to_string(),
sender_id: "cli".to_string(),
chat_id: target_chat_id,
client_message_id,
content,
received_at: crate::bus::message::current_timestamp(),
timestamp: crate::bus::message::current_timestamp(),
media,
channel_context: Default::default(),
metadata: Default::default(),
forwarded_metadata: Default::default(),
};
if let Err(error) = bus.publish_inbound(msg).await {
self.uploads.restore(uploads).await;
@ -488,7 +432,34 @@ impl CliChatChannel {
|| message.tool_calls.is_some()
|| message.role == "tool"
})
.map(HistoryMessage::from_message_meta)
.map(|message| {
let attachments = message
.media_refs
.as_deref()
.and_then(|refs| {
serde_json::from_str::<Vec<crate::bus::MediaRef>>(refs).ok()
})
.unwrap_or_default()
.iter()
.enumerate()
.map(|(index, media_ref)| {
MessageAttachment::from_media_ref(index, media_ref)
})
.collect();
HistoryMessage {
id: message.id,
seq: message.seq,
role: message.role,
content: message.content,
created_at: message.created_at,
tool_call_id: message.tool_call_id,
tool_name: message.tool_name,
tool_calls: message
.tool_calls
.and_then(|calls| serde_json::from_str(&calls).ok()),
attachments,
}
})
.collect();
let _ = client
.sender
@ -530,96 +501,6 @@ impl CliChatChannel {
None => return Err(ChannelError::Other("Control channel closed".to_string())),
}
}
WsInbound::GetSessionStats { session_id } => {
let unified_id = Self::parse_client_session(&client, &session_id)?;
let (reply_tx, mut reply_rx) = mpsc::channel(1);
bus.publish_control(ControlMessage {
op: SessionCommand::GetSessionStats {
session_id: unified_id,
},
reply_tx,
})
.await?;
match reply_rx.recv().await {
Some(Ok(SessionEvent::SessionStats { stats })) => {
let _ = client.sender.send(WsOutbound::SessionStats { stats }).await;
}
Some(Ok(_)) => {}
Some(Err(error)) => return Err(error),
None => return Err(ChannelError::Other("Control channel closed".to_string())),
}
}
WsInbound::GetAgentRuns {
session_id,
cursor,
limit,
} => {
let unified_id = Self::parse_client_session(&client, &session_id)?;
let (reply_tx, mut reply_rx) = mpsc::channel(1);
bus.publish_control(ControlMessage {
op: SessionCommand::GetAgentRuns {
session_id: unified_id,
cursor,
limit: limit.unwrap_or(100).clamp(1, 200),
},
reply_tx,
})
.await?;
match reply_rx.recv().await {
Some(Ok(SessionEvent::AgentRuns {
session_id,
revision,
runs,
next_cursor,
})) => {
let _ = client
.sender
.send(WsOutbound::SessionAgentRuns {
session_id: session_id.to_string(),
revision,
runs,
next_cursor,
})
.await;
}
Some(Ok(_)) => {}
Some(Err(error)) => return Err(error),
None => return Err(ChannelError::Other("Control channel closed".to_string())),
}
}
WsInbound::GetAgentRun { session_id, run_id } => {
let unified_id = Self::parse_client_session(&client, &session_id)?;
let (reply_tx, mut reply_rx) = mpsc::channel(1);
bus.publish_control(ControlMessage {
op: SessionCommand::GetAgentRun {
session_id: unified_id,
run_id,
},
reply_tx,
})
.await?;
match reply_rx.recv().await {
Some(Ok(SessionEvent::AgentRun {
session_id,
revision,
run,
})) => {
let _ = client
.sender
.send(WsOutbound::AgentRunUpdated {
session_id: session_id.to_string(),
revision,
run: run.ok_or_else(|| {
ChannelError::Other("run not found".to_string())
})?,
})
.await;
}
Some(Ok(_)) => {}
Some(Err(error)) => return Err(error),
None => return Err(ChannelError::Other("Control channel closed".to_string())),
}
}
WsInbound::RenameSession { session_id, title } => {
let target = session_id
.or(current_session_guard.clone())
@ -922,54 +803,6 @@ impl CliChatChannel {
}
}
struct CliChatTurnSink {
clients: Arc<Mutex<HashMap<String, Arc<Client>>>>,
chat_id: String,
}
impl CliChatTurnSink {
async fn publish(&self, snapshot: &TurnSnapshot) {
let client = self.clients.lock().await.get(&self.chat_id).cloned();
let Some(client) = client else {
return;
};
if client
.sender
.send(WsOutbound::TurnUpdated {
snapshot: snapshot.clone(),
})
.await
.is_err()
{
let mut clients = self.clients.lock().await;
if clients
.get(&self.chat_id)
.is_some_and(|registered| Arc::ptr_eq(registered, &client))
{
clients.remove(&self.chat_id);
}
}
}
}
#[async_trait]
impl TurnSink for CliChatTurnSink {
async fn update(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError> {
self.publish(snapshot).await;
Ok(())
}
async fn finish(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError> {
self.publish(snapshot).await;
Ok(())
}
async fn abort(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError> {
self.publish(snapshot).await;
Ok(())
}
}
#[async_trait]
impl Channel for CliChatChannel {
fn name(&self) -> &str {
@ -991,50 +824,6 @@ impl Channel for CliChatChannel {
Ok(())
}
fn live_policy(&self) -> LivePolicy {
LivePolicy::Snapshot {
min_interval: Duration::from_millis(33),
}
}
fn presentation_policy(&self) -> crate::delivery::PresentationPolicy {
crate::delivery::PresentationPolicy::interactive()
}
async fn open_turn(&self, target: TurnTarget) -> Result<Box<dyn TurnSink>, ChannelError> {
Ok(Box::new(CliChatTurnSink {
clients: self.clients.clone(),
chat_id: target.chat_id,
}))
}
async fn commit_turn(
&self,
target: &TurnTarget,
delta: CommittedTurnDelta,
) -> Result<(), ChannelError> {
let client = self.clients.lock().await.get(&target.chat_id).cloned();
let Some(client) = client else {
return Ok(());
};
let frame = WsOutbound::TurnCommitted {
session_id: delta.session_id,
history_revision: delta.history_revision,
messages: delta
.messages
.into_iter()
.map(HistoryMessage::from)
.collect(),
};
client.sender.send(frame).await.map_err(|_| {
ChannelError::ConnectionError("CLI client disconnected during turn commit".to_string())
})
}
fn commit_turn_presents_media(&self) -> bool {
true
}
async fn send(&self, msg: OutboundMessage) -> Result<(), ChannelError> {
let client = self.clients.lock().await.get(&msg.chat_id).cloned();
let Some(client) = client else {
@ -1185,9 +974,8 @@ mod tests {
.handle_ws_inbound(
client,
WsInbound::UserInput {
content: "/queue 处理附件".into(),
content: "处理附件".into(),
upload_ids: vec!["upload-1".into()],
client_message_id: Some("550e8400-e29b-41d4-a716-446655440000".into()),
channel: None,
chat_id: None,
sender_id: None,
@ -1200,42 +988,6 @@ mod tests {
assert_eq!(inbound.media.len(), 1);
assert_eq!(inbound.media[0].path, "/tmp/report.pdf");
assert_eq!(inbound.media[0].media_type, "file");
assert_eq!(
inbound.client_message_id.as_deref(),
Some("550e8400-e29b-41d4-a716-446655440000")
);
assert_eq!(inbound.content, "/queue 处理附件");
}
#[tokio::test]
async fn invalid_client_message_id_is_ignored_at_channel_boundary() {
let channel = CliChatChannel::new();
let bus = MessageBus::new(4);
channel.start(bus.clone()).await.unwrap();
let (sender, _receiver) = mpsc::channel(1);
let client = Arc::new(Client {
sender,
chat_id: "client".into(),
current_session_id: Mutex::new(None),
});
channel
.handle_ws_inbound(
client,
WsInbound::UserInput {
content: "hello".into(),
upload_ids: Vec::new(),
client_message_id: Some("not-a-uuid".into()),
channel: None,
chat_id: None,
sender_id: None,
},
)
.await
.unwrap();
let inbound = bus.consume_inbound().await.unwrap();
assert_eq!(inbound.client_message_id, None);
}
#[tokio::test]
@ -1274,109 +1026,4 @@ mod tests {
other => panic!("unexpected outbound: {other:?}"),
}
}
#[tokio::test]
async fn turn_sink_sends_the_same_snapshot_shape_for_running_and_terminal_states() {
let channel = CliChatChannel::new();
let (sender, mut receiver) = mpsc::channel(2);
let client = Arc::new(Client {
sender,
chat_id: "client".into(),
current_session_id: Mutex::new(None),
});
channel.clients.lock().await.insert("client".into(), client);
let mut sink = channel
.open_turn(TurnTarget {
channel: "cli_chat".into(),
chat_id: "client".into(),
session_id: "cli_chat:client:dialog".into(),
reply_to: None,
metadata: HashMap::new(),
})
.await
.unwrap();
let (controller, emitter, _) =
crate::session::TurnController::start("cli_chat:client:dialog", "message");
emitter
.emit(crate::agent::TurnEvent::TextDelta {
iteration: 0,
delta: "stream".into(),
})
.unwrap();
let running = controller.snapshot();
sink.update(&running).await.unwrap();
controller.complete(None);
let completed = controller.snapshot();
sink.finish(&completed).await.unwrap();
match receiver.recv().await.unwrap() {
WsOutbound::TurnUpdated { snapshot } => {
assert_eq!(snapshot.status, crate::session::TurnStatus::Running);
}
other => panic!("unexpected outbound: {other:?}"),
}
match receiver.recv().await.unwrap() {
WsOutbound::TurnUpdated { snapshot } => {
assert_eq!(snapshot.status, crate::session::TurnStatus::Completed);
assert!(snapshot.revision > running.revision);
}
other => panic!("unexpected outbound: {other:?}"),
}
}
#[tokio::test]
async fn committed_turn_is_projected_as_incremental_history_frame() {
let channel = CliChatChannel::new();
let (sender, mut receiver) = mpsc::channel(1);
let client = Arc::new(Client {
sender,
chat_id: "client".into(),
current_session_id: Mutex::new(None),
});
channel.clients.lock().await.insert("client".into(), client);
let target = TurnTarget {
channel: "cli_chat".into(),
chat_id: "client".into(),
session_id: "cli_chat:client:dialog".into(),
reply_to: None,
metadata: HashMap::new(),
};
channel
.commit_turn(
&target,
CommittedTurnDelta {
session_id: target.session_id.clone(),
history_revision: 4,
messages: vec![crate::bus::CommittedMessage {
id: "message".into(),
seq: 4,
role: "assistant".into(),
content: "done".into(),
reasoning_content: None,
completion_status: crate::bus::CompletionStatus::Completed,
media_refs: Vec::new(),
created_at: 1,
tool_call_id: None,
tool_name: None,
tool_calls: None,
turn_origin: crate::bus::TurnOrigin::User,
}],
},
)
.await
.unwrap();
match receiver.recv().await.unwrap() {
WsOutbound::TurnCommitted {
history_revision,
messages,
..
} => {
assert_eq!(history_revision, 4);
assert_eq!(messages[0].id, "message");
}
other => panic!("unexpected outbound: {other:?}"),
}
}
}

1912
src/channels/feishu.rs
View File

File diff suppressed because it is too large Load Diff

2
src/channels/mod.rs
View File

@ -4,7 +4,7 @@ pub mod feishu;
pub mod manager;
pub mod slash_command;
pub use base::{Channel, ChannelError, LivePolicy, TurnSink, TurnTarget};
pub use base::{Channel, ChannelError};
pub use cli_chat::CliChatChannel;
pub use feishu::FeishuChannel;
pub use manager::ChannelManager;

66
src/client/mod.rs
View File

@ -1,10 +1,7 @@
pub use crate::protocol::{WsInbound, WsOutbound, serialize_inbound, serialize_outbound};
mod oneshot;
mod tui;
pub use oneshot::{RunOptions, read_run_prompt, run_once};
use crate::client::tui::app::{App, MessageRole};
use crate::client::tui::event::{
handle_key_event, handle_paste, request_history, request_session_list, send,
@ -98,30 +95,6 @@ fn gateway_http_base_url(gateway_url: &str) -> Result<String, Box<dyn std::error
Ok(url.to_string().trim_end_matches('/').to_string())
}
pub async fn reload_gateway(gateway_url: &str) -> Result<String, Box<dyn std::error::Error>> {
let base = gateway_http_base_url(gateway_url)?;
let mut request = reqwest::Client::new().post(format!("{base}/api/config/reload"));
if let Some(token) = load_auth_token() {
request = request.bearer_auth(token);
}
let response = request.send().await?;
let status = response.status();
let body: serde_json::Value = response.json().await?;
if !status.is_success() {
return Err(body
.get("error")
.and_then(serde_json::Value::as_str)
.unwrap_or("configuration reload failed")
.to_string()
.into());
}
Ok(body
.get("message")
.and_then(serde_json::Value::as_str)
.unwrap_or("configuration reload scheduled")
.to_string())
}
async fn exchange_pairing_code(
gateway_url: &str,
code: &str,
@ -280,38 +253,6 @@ async fn run_app(
async fn handle_ws_message(app: &mut App, outbound: WsOutbound) {
match outbound {
WsOutbound::TurnUpdated { snapshot } => {
let terminal = snapshot.status != crate::session::TurnStatus::Running;
let completed = snapshot.status == crate::session::TurnStatus::Completed;
let session_id = snapshot.session_id.clone();
if terminal {
app.pending_responses = app.pending_responses.saturating_sub(1);
}
if app.apply_turn_snapshot(snapshot) {
if terminal {
app.status_message = None;
if app.current_session_id.as_deref() == Some(&session_id) {
if !completed {
request_history(app, session_id).await;
}
} else {
app.status_message = Some("另一个会话已完成响应".to_string());
}
request_session_list(app).await;
} else {
app.status_message = Some("正在生成回复…".to_string());
}
} else if terminal && app.current_session_id.as_deref() != Some(&session_id) {
app.status_message = Some("另一个会话已完成响应".to_string());
}
}
WsOutbound::TurnCommitted {
session_id,
history_revision,
messages,
} => {
app.apply_turn_commit(&session_id, history_revision, messages);
}
WsOutbound::AssistantResponse {
id,
content,
@ -380,12 +321,7 @@ async fn handle_ws_message(app: &mut App, outbound: WsOutbound) {
} => app.set_history(&session_id, messages),
// The first Todo UI is WebUI-only. CLI keeps receiving ordinary task
// notifications and may inspect plans through /todo.
WsOutbound::SessionPlan { .. }
| WsOutbound::SessionStats { .. }
| WsOutbound::PlanUpdated { .. }
| WsOutbound::SessionAgentRuns { .. }
| WsOutbound::AgentRunUpdated { .. }
| WsOutbound::AgentEventUpdated { .. } => {}
WsOutbound::SessionPlan { .. } | WsOutbound::PlanUpdated { .. } => {}
WsOutbound::SessionRenamed { session_id, title } => {
if let Some(session) = app
.sessions

431
src/client/oneshot.rs
View File

@ -1,431 +0,0 @@
use super::{WsInbound, WsOutbound, load_auth_token};
use crate::config::get_user_config_dir;
use crate::gateway::auth::ADMIN_TOKEN_HEADER;
use crate::session::{ToolStatus, TurnBlock, TurnPhase, TurnSnapshot, TurnStatus};
use futures_util::{SinkExt, StreamExt};
use serde::Serialize;
use std::collections::HashMap;
use std::io::{self, IsTerminal, Read, Write};
use std::net::IpAddr;
use std::time::Duration;
use tokio_tungstenite::connect_async;
use tokio_tungstenite::tungstenite::{
Message,
client::IntoClientRequest,
http::{HeaderValue, header},
};
const MAX_RUN_PROMPT_BYTES: usize = 1024 * 1024;
type DynError = Box<dyn std::error::Error>;
#[derive(Debug, Clone, Copy)]
pub struct RunOptions {
pub timeout: Duration,
pub json: bool,
pub verbose: bool,
}
#[derive(Debug, Serialize)]
struct RunOutput {
session_id: String,
turn_id: String,
status: TurnStatus,
content: String,
usage: Option<crate::providers::Usage>,
error: Option<String>,
}
pub fn read_run_prompt(parts: Vec<String>) -> Result<String, DynError> {
if !parts.is_empty() {
return validate_prompt(parts.join(" "));
}
if io::stdin().is_terminal() {
return Err("provide a prompt as arguments or pipe it on stdin".into());
}
let stdin = io::stdin();
let mut locked = stdin.lock();
read_prompt_from(&mut locked)
}
pub async fn run_once(
gateway_url: &str,
prompt: String,
options: RunOptions,
) -> Result<(), Box<dyn std::error::Error>> {
if options.timeout.is_zero() {
return Err("run timeout must be greater than zero".into());
}
let prompt = validate_prompt(prompt)?;
let client_id = format!("run-{}", uuid::Uuid::new_v4().simple());
let (connect_url, local_gateway) = websocket_url(gateway_url, &client_id)?;
let admin_token = local_gateway
.then(|| std::fs::read_to_string(get_user_config_dir().join("web_admin_token")).ok())
.flatten()
.map(|token| token.trim().to_string())
.filter(|token| !token.is_empty());
let bearer_token = admin_token.is_none().then(load_auth_token).flatten();
let mut request = connect_url.into_client_request()?;
if let Some(token) = &admin_token {
let mut value = HeaderValue::from_str(token)?;
value.set_sensitive(true);
request.headers_mut().insert(ADMIN_TOKEN_HEADER, value);
} else if let Some(token) = &bearer_token {
let mut value = HeaderValue::from_str(&format!("Bearer {token}"))?;
value.set_sensitive(true);
request.headers_mut().insert(header::AUTHORIZATION, value);
}
let (stream, _) = connect_async(request).await.map_err(|error| {
if local_gateway && admin_token.is_none() {
format!(
"gateway connection failed: {error}; local admin token is unavailable at {}",
get_user_config_dir().join("web_admin_token").display()
)
} else {
format!(
"gateway connection failed: {error}. Remote gateways require an existing paired CLI token"
)
}
})?;
let (mut sender, mut receiver) = stream.split();
let operation = async {
let session_id = loop {
match receiver.next().await {
Some(Ok(Message::Text(text))) => match serde_json::from_str::<WsOutbound>(&text)? {
WsOutbound::SessionEstablished { session_id, .. } => break session_id,
WsOutbound::Error { code, message } => {
return Err::<RunOutput, DynError>(
format!("gateway error {code}: {message}").into(),
);
}
_ => {}
},
Some(Ok(Message::Close(_))) | None => {
return Err("gateway closed before establishing a session".into());
}
Some(Err(error)) => return Err(error.into()),
_ => {}
}
};
let input = WsInbound::UserInput {
content: prompt,
upload_ids: Vec::new(),
client_message_id: None,
channel: None,
chat_id: None,
sender_id: None,
};
sender
.send(Message::Text(serde_json::to_string(&input)?.into()))
.await?;
let mut turn_id = None;
let mut last_phase = None;
let mut tool_states: HashMap<String, (String, ToolStatus)> = HashMap::new();
loop {
match receiver.next().await {
Some(Ok(Message::Text(text))) => match serde_json::from_str::<WsOutbound>(&text)? {
WsOutbound::TurnUpdated { snapshot }
if snapshot.session_id == session_id
&& turn_id.as_ref().is_none_or(|id| id == &snapshot.id.0) =>
{
turn_id.get_or_insert_with(|| snapshot.id.0.clone());
if options.verbose {
report_progress(&snapshot, &mut last_phase, &mut tool_states);
}
if snapshot.status != TurnStatus::Running {
break Ok(output_from_snapshot(snapshot));
}
}
WsOutbound::Error { code, message } => {
break Err(format!("gateway error {code}: {message}").into());
}
_ => {}
},
Some(Ok(Message::Close(_))) | None => {
break Err("gateway closed before the run completed".into());
}
Some(Err(error)) => break Err(error.into()),
_ => {}
}
}
};
let output = tokio::select! {
result = tokio::time::timeout(options.timeout, operation) => {
match result {
Ok(result) => result?,
Err(_) => {
send_stop(&mut sender).await;
return Err(format!("run timed out after {} seconds", options.timeout.as_secs()).into());
}
}
}
signal = tokio::signal::ctrl_c() => {
send_stop(&mut sender).await;
signal?;
return Err("run cancelled".into());
}
};
render_output(&output, options.json)?;
match output.status {
TurnStatus::Completed => Ok(()),
TurnStatus::Cancelled => Err(output
.error
.unwrap_or_else(|| "run cancelled".to_string())
.into()),
TurnStatus::Failed => Err(output
.error
.unwrap_or_else(|| "run failed".to_string())
.into()),
TurnStatus::Running => Err("gateway returned a non-terminal run result".into()),
}
}
async fn send_stop<S>(sender: &mut S)
where
S: futures_util::Sink<Message> + Unpin,
{
let stop = WsInbound::UserInput {
content: "/stop".to_string(),
upload_ids: Vec::new(),
client_message_id: None,
channel: None,
chat_id: None,
sender_id: None,
};
if let Ok(text) = serde_json::to_string(&stop) {
let _ = sender.send(Message::Text(text.into())).await;
let _ = sender.flush().await;
}
}
fn websocket_url(
gateway_url: &str,
client_id: &str,
) -> Result<(String, bool), Box<dyn std::error::Error>> {
let mut url = reqwest::Url::parse(gateway_url)?;
let scheme = match url.scheme() {
"ws" => "ws",
"wss" => "wss",
"http" => "ws",
"https" => "wss",
other => return Err(format!("unsupported gateway URL scheme: {other}").into()),
};
url.set_scheme(scheme)
.map_err(|_| "failed to set gateway URL scheme")?;
if url.path().is_empty() || url.path() == "/" {
url.set_path("/ws");
}
url.query_pairs_mut().append_pair("client_id", client_id);
let local_gateway = url.host_str().is_some_and(|host| {
let host = host
.strip_prefix('[')
.and_then(|value| value.strip_suffix(']'))
.unwrap_or(host);
host.eq_ignore_ascii_case("localhost")
|| host
.parse::<IpAddr>()
.is_ok_and(|address| address.is_loopback())
});
Ok((url.to_string(), local_gateway))
}
fn read_prompt_from(reader: &mut impl Read) -> Result<String, Box<dyn std::error::Error>> {
let mut bytes = Vec::new();
reader
.take((MAX_RUN_PROMPT_BYTES + 1) as u64)
.read_to_end(&mut bytes)?;
if bytes.len() > MAX_RUN_PROMPT_BYTES {
return Err(format!("prompt exceeds {MAX_RUN_PROMPT_BYTES} bytes").into());
}
let prompt = String::from_utf8(bytes)?;
validate_prompt(prompt.trim_end_matches(['\r', '\n']).to_string())
}
fn validate_prompt(prompt: String) -> Result<String, Box<dyn std::error::Error>> {
if prompt.len() > MAX_RUN_PROMPT_BYTES {
return Err(format!("prompt exceeds {MAX_RUN_PROMPT_BYTES} bytes").into());
}
if prompt.trim().is_empty() {
return Err("prompt is empty".into());
}
Ok(prompt)
}
fn output_from_snapshot(snapshot: TurnSnapshot) -> RunOutput {
RunOutput {
session_id: snapshot.session_id,
turn_id: snapshot.id.0,
status: snapshot.status,
content: assistant_text(&snapshot.blocks),
usage: snapshot.usage,
error: snapshot.error,
}
}
fn assistant_text(blocks: &[TurnBlock]) -> String {
blocks
.iter()
.filter_map(|block| match block {
TurnBlock::Assistant { text, .. } if !text.is_empty() => Some(text.as_str()),
_ => None,
})
.collect::<Vec<_>>()
.join("\n\n")
}
fn report_progress(
snapshot: &TurnSnapshot,
last_phase: &mut Option<TurnPhase>,
tool_states: &mut HashMap<String, (String, ToolStatus)>,
) {
if last_phase.as_ref() != Some(&snapshot.phase) {
eprintln!("[phase: {}]", phase_name(snapshot.phase));
*last_phase = Some(snapshot.phase);
}
for block in &snapshot.blocks {
let TurnBlock::Tool {
id, name, status, ..
} = block
else {
continue;
};
let current = (name.clone(), *status);
if tool_states.get(id) != Some(&current) {
eprintln!("[tool: {name}: {}]", tool_status_name(*status));
tool_states.insert(id.clone(), current);
}
}
}
fn phase_name(phase: TurnPhase) -> &'static str {
match phase {
TurnPhase::Queued => "queued",
TurnPhase::Reasoning => "reasoning",
TurnPhase::Responding => "responding",
TurnPhase::Acting => "acting",
TurnPhase::Finalizing => "finalizing",
}
}
fn tool_status_name(status: ToolStatus) -> &'static str {
match status {
ToolStatus::Running => "running",
ToolStatus::Completed => "completed",
ToolStatus::Cancelled => "cancelled",
ToolStatus::Failed => "failed",
}
}
fn render_output(output: &RunOutput, json: bool) -> Result<(), Box<dyn std::error::Error>> {
if json {
println!("{}", serde_json::to_string(output)?);
} else if output.status == TurnStatus::Completed {
print!("{}", output.content);
if !output.content.ends_with('\n') {
println!();
}
io::stdout().flush()?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::session::{BlockId, TurnId};
#[test]
fn positional_prompt_parts_are_joined() {
assert_eq!(
validate_prompt(["hello", "world"].join(" ")).unwrap(),
"hello world"
);
}
#[test]
fn stdin_prompt_preserves_lines_and_trims_terminal_newline() {
let mut input = "first\nsecond\n".as_bytes();
assert_eq!(read_prompt_from(&mut input).unwrap(), "first\nsecond");
}
#[test]
fn empty_and_oversized_prompts_are_rejected() {
assert!(validate_prompt(" \n".to_string()).is_err());
assert!(validate_prompt("x".repeat(MAX_RUN_PROMPT_BYTES + 1)).is_err());
}
#[test]
fn assistant_blocks_are_joined_without_reasoning_or_tools() {
let blocks = vec![
TurnBlock::Reasoning {
id: BlockId("reasoning".to_string()),
iteration: 0,
text: "hidden".to_string(),
},
TurnBlock::Assistant {
id: BlockId("answer-1".to_string()),
iteration: 0,
text: "hello".to_string(),
},
TurnBlock::Tool {
id: "tool".to_string(),
iteration: 0,
name: "bash".to_string(),
arguments: serde_json::json!({}),
status: ToolStatus::Completed,
preview: None,
},
TurnBlock::Assistant {
id: BlockId("answer-2".to_string()),
iteration: 1,
text: "world".to_string(),
},
];
assert_eq!(assistant_text(&blocks), "hello\n\nworld");
}
#[test]
fn loopback_gate_uses_the_url_host() {
assert!(
websocket_url("ws://127.0.0.1:19876/ws", "run-id")
.unwrap()
.1
);
assert!(websocket_url("ws://[::1]:19876/ws", "run-id").unwrap().1);
assert!(websocket_url("http://localhost:19876", "run-id").unwrap().1);
assert!(
!websocket_url("wss://gateway.example/ws", "run-id")
.unwrap()
.1
);
}
#[test]
fn terminal_snapshot_becomes_script_output() {
let output = output_from_snapshot(TurnSnapshot {
id: TurnId("turn".to_string()),
session_id: "session".to_string(),
message_id: "message".to_string(),
revision: 1,
status: TurnStatus::Completed,
phase: TurnPhase::Finalizing,
blocks: vec![TurnBlock::Assistant {
id: BlockId("answer".to_string()),
iteration: 0,
text: "done".to_string(),
}],
usage: None,
error: None,
});
assert_eq!(output.turn_id, "turn");
assert_eq!(output.content, "done");
assert_eq!(output.status, TurnStatus::Completed);
}
}

223
src/client/tui/app.rs
View File

@ -1,7 +1,6 @@
use crate::protocol::{
HistoryMessage, MessageAttachment, SessionSummary, SlashCommandInfo, UploadDescriptor,
};
use crate::session::{TurnSnapshot, TurnStatus};
use std::collections::VecDeque;
use tokio_tungstenite::tungstenite::Message;
@ -20,8 +19,6 @@ pub struct ChatMessage {
pub id: String,
pub role: MessageRole,
pub content: String,
pub reasoning_content: Option<String>,
pub completion_status: crate::bus::CompletionStatus,
pub attachments: Vec<MessageAttachment>,
}
@ -66,8 +63,6 @@ pub struct App {
pub selected_session: usize,
pub show_archived: bool,
pub messages: VecDeque<ChatMessage>,
pub history_revision: i64,
pub active_turn: Option<TurnSnapshot>,
pub input: String,
/// UTF-8 byte offset. It is always maintained at a character boundary.
pub input_cursor_pos: usize,
@ -100,8 +95,6 @@ impl App {
selected_session: 0,
show_archived: false,
messages: VecDeque::new(),
history_revision: 0,
active_turn: None,
input: String::new(),
input_cursor_pos: 0,
focus: Focus::Input,
@ -139,8 +132,6 @@ impl App {
id,
role,
content,
reasoning_content: None,
completion_status: crate::bus::CompletionStatus::Completed,
attachments,
});
while self.messages.len() > MAX_MESSAGES {
@ -153,15 +144,6 @@ impl App {
if self.current_session_id.as_deref() != Some(session_id) {
return;
}
let history_revision = messages
.iter()
.map(|message| message.seq)
.max()
.unwrap_or(0);
let calibrates_terminal = self.active_turn.as_ref().is_some_and(|turn| {
turn.status != TurnStatus::Running
&& messages.iter().any(|message| message.id == turn.message_id)
});
self.messages = messages
.into_iter()
.filter_map(|message| {
@ -175,8 +157,6 @@ impl App {
id: message.id,
role,
content: message.content,
reasoning_content: message.reasoning_content,
completion_status: message.completion_status,
attachments: message.attachments,
})
})
@ -185,63 +165,7 @@ impl App {
self.messages.pop_front();
}
self.chat_scroll_from_bottom = 0;
self.history_revision = history_revision;
self.status_message = None;
if calibrates_terminal {
self.active_turn = None;
}
}
pub fn apply_turn_commit(
&mut self,
session_id: &str,
history_revision: i64,
messages: Vec<HistoryMessage>,
) -> bool {
if self.current_session_id.as_deref() != Some(session_id)
|| history_revision <= self.history_revision
{
return false;
}
let calibrates_terminal = self.active_turn.as_ref().is_some_and(|turn| {
turn.status != TurnStatus::Running
&& messages.iter().any(|message| message.id == turn.message_id)
});
for message in messages {
let role = match message.role.as_str() {
"user" => MessageRole::User,
"assistant" => MessageRole::Assistant,
"system" | "tool" => MessageRole::System,
_ => continue,
};
let projected = ChatMessage {
id: message.id.clone(),
role,
content: message.content,
reasoning_content: message.reasoning_content,
completion_status: message.completion_status,
attachments: message.attachments,
};
if let Some(existing) = self
.messages
.iter_mut()
.find(|existing| existing.id == message.id)
{
*existing = projected;
} else {
self.messages.push_back(projected);
}
}
while self.messages.len() > MAX_MESSAGES {
self.messages.pop_front();
}
self.history_revision = history_revision;
self.chat_scroll_from_bottom = 0;
self.status_message = None;
if calibrates_terminal {
self.active_turn = None;
}
true
}
pub fn set_sessions(&mut self, sessions: Vec<SessionSummary>) {
@ -261,8 +185,6 @@ impl App {
if self.current_session_id != session_id {
self.current_session_id = session_id;
self.messages.clear();
self.active_turn = None;
self.history_revision = 0;
self.pending_uploads.clear();
self.chat_scroll_from_bottom = 0;
}
@ -276,26 +198,6 @@ impl App {
}
}
pub fn apply_turn_snapshot(&mut self, snapshot: TurnSnapshot) -> bool {
if self.current_session_id.as_deref() != Some(&snapshot.session_id) {
return false;
}
if let Some(current) = &self.active_turn
&& current.id == snapshot.id
&& current.revision >= snapshot.revision
{
return false;
}
let already_committed = snapshot.status != TurnStatus::Running
&& self
.messages
.iter()
.any(|message| message.id == snapshot.message_id);
self.active_turn = (!already_committed).then_some(snapshot);
self.chat_scroll_from_bottom = 0;
true
}
pub fn current_title(&self) -> &str {
self.current_session_id
.as_ref()
@ -481,21 +383,6 @@ fn next_boundary(value: &str, offset: usize) -> Option<usize> {
#[cfg(test)]
mod tests {
use super::*;
use crate::session::{TurnId, TurnPhase, TurnState};
fn turn(revision: u64, status: TurnStatus) -> TurnSnapshot {
TurnState {
id: TurnId("turn".into()),
session_id: "current".into(),
message_id: "message".into(),
revision,
status,
phase: TurnPhase::Responding,
blocks: Vec::new(),
usage: None,
error: None,
}
}
#[test]
fn unicode_cursor_edits_only_at_character_boundaries() {
@ -517,114 +404,4 @@ mod tests {
app.set_history("old", Vec::new());
assert_eq!(app.messages.len(), 1);
}
#[test]
fn active_turn_ignores_stale_revisions_and_other_sessions() {
let mut app = App::new();
app.set_current_session(Some("current".into()));
assert!(app.apply_turn_snapshot(turn(2, TurnStatus::Running)));
assert!(!app.apply_turn_snapshot(turn(1, TurnStatus::Running)));
let mut other = turn(3, TurnStatus::Running);
other.session_id = "other".into();
assert!(!app.apply_turn_snapshot(other));
assert_eq!(app.active_turn.as_ref().unwrap().revision, 2);
}
#[test]
fn terminal_turn_remains_visible_until_history_calibrates_it() {
let mut app = App::new();
app.set_current_session(Some("current".into()));
app.apply_turn_snapshot(turn(3, TurnStatus::Completed));
assert!(app.active_turn.is_some());
app.set_history(
"current",
vec![HistoryMessage {
id: "message".into(),
seq: 1,
role: "assistant".into(),
content: "done".into(),
reasoning_content: None,
completion_status: crate::bus::CompletionStatus::Completed,
created_at: 1,
tool_call_id: None,
tool_name: None,
tool_calls: None,
attachments: Vec::new(),
turn_origin: crate::bus::TurnOrigin::User,
}],
);
assert!(app.active_turn.is_none());
}
#[test]
fn committed_delta_calibrates_terminal_without_reloading_history() {
let mut app = App::new();
app.set_current_session(Some("current".into()));
app.apply_turn_snapshot(turn(3, TurnStatus::Completed));
assert!(app.apply_turn_commit(
"current",
2,
vec![HistoryMessage {
id: "message".into(),
seq: 2,
role: "assistant".into(),
content: "done".into(),
reasoning_content: None,
completion_status: crate::bus::CompletionStatus::Completed,
created_at: 1,
tool_call_id: None,
tool_name: None,
tool_calls: None,
attachments: Vec::new(),
turn_origin: crate::bus::TurnOrigin::User,
}],
));
assert!(app.active_turn.is_none());
assert_eq!(app.messages.back().unwrap().content, "done");
assert!(!app.apply_turn_commit("current", 2, Vec::new()));
}
#[test]
fn terminal_snapshot_calibrates_when_commit_arrived_first() {
let mut app = App::new();
app.set_current_session(Some("current".into()));
assert!(app.apply_turn_commit(
"current",
2,
vec![HistoryMessage {
id: "message".into(),
seq: 2,
role: "assistant".into(),
content: "done".into(),
reasoning_content: None,
completion_status: crate::bus::CompletionStatus::Completed,
created_at: 1,
tool_call_id: None,
tool_name: None,
tool_calls: None,
attachments: Vec::new(),
turn_origin: crate::bus::TurnOrigin::User,
}],
));
assert!(app.apply_turn_snapshot(turn(3, TurnStatus::Completed)));
assert!(app.active_turn.is_none());
}
#[test]
fn failed_turn_without_a_durable_message_remains_visible_after_history_refresh() {
let mut app = App::new();
app.set_current_session(Some("current".into()));
app.apply_turn_snapshot(turn(3, TurnStatus::Failed));
app.set_history("current", Vec::new());
assert_eq!(
app.active_turn.as_ref().map(|turn| turn.status),
Some(TurnStatus::Failed)
);
}
}

99
src/client/tui/components/chat_history.rs
View File

@ -1,5 +1,4 @@
use crate::client::tui::app::{App, MessageRole};
use crate::session::{ToolStatus, TurnBlock, TurnPhase, TurnStatus};
use ratatui::{
Frame,
layout::Rect,
@ -24,15 +23,6 @@ pub fn render(frame: &mut Frame, area: Rect, app: &App) {
label,
Style::default().fg(color).add_modifier(Modifier::BOLD),
)));
if let Some(reasoning) = &message.reasoning_content {
lines.push(Line::from(Span::styled(
"思考过程",
Style::default()
.fg(Color::DarkGray)
.add_modifier(Modifier::ITALIC),
)));
push_wrapped(&mut lines, reasoning, content_width, Color::DarkGray);
}
for source_line in message.content.lines() {
let wrapped = textwrap::wrap(source_line, content_width);
if wrapped.is_empty() {
@ -45,12 +35,6 @@ pub fn render(frame: &mut Frame, area: Rect, app: &App) {
);
}
}
if message.completion_status != crate::bus::CompletionStatus::Completed {
lines.push(Line::from(Span::styled(
format!("[{}]", message.completion_status.as_str()),
Style::default().fg(Color::Yellow),
)));
}
for attachment in &message.attachments {
lines.push(Line::from(Span::styled(
format!(" [附件 {}] {}", attachment.index + 1, attachment.name),
@ -59,75 +43,7 @@ pub fn render(frame: &mut Frame, area: Rect, app: &App) {
}
lines.push(Line::from(""));
}
if let Some(turn) = &app.active_turn {
lines.push(Line::from(Span::styled(
"PicoBot",
Style::default()
.fg(Color::Green)
.add_modifier(Modifier::BOLD),
)));
for block in &turn.blocks {
match block {
TurnBlock::Reasoning { text, .. } => {
lines.push(Line::from(Span::styled(
"思考过程",
Style::default()
.fg(Color::DarkGray)
.add_modifier(Modifier::ITALIC),
)));
push_wrapped(&mut lines, text, content_width, Color::DarkGray);
}
TurnBlock::Assistant { text, .. } => {
push_wrapped(&mut lines, text, content_width, Color::Reset);
}
TurnBlock::Tool {
name,
status,
preview,
..
} => {
let status = match status {
ToolStatus::Running => "执行中",
ToolStatus::Completed => "已完成",
ToolStatus::Cancelled => "已停止",
ToolStatus::Failed => "失败",
};
lines.push(Line::from(Span::styled(
format!("工具 · {name} · {status}"),
Style::default().fg(Color::Magenta),
)));
if let Some(preview) = preview {
push_wrapped(&mut lines, preview, content_width, Color::DarkGray);
}
}
}
}
let phase = match turn.phase {
TurnPhase::Queued => "排队中",
TurnPhase::Reasoning => "思考中",
TurnPhase::Responding => "生成中",
TurnPhase::Acting => "调用工具中",
TurnPhase::Finalizing => "收尾中",
};
let status = match turn.status {
TurnStatus::Running => phase,
TurnStatus::Completed => "已完成",
TurnStatus::Cancelled => "已停止",
TurnStatus::Failed => "失败",
};
lines.push(Line::from(Span::styled(
format!("● {status}"),
Style::default().fg(if turn.status == TurnStatus::Failed {
Color::Red
} else {
Color::Cyan
}),
)));
if let Some(error) = &turn.error {
push_wrapped(&mut lines, error, content_width, Color::Red);
}
lines.push(Line::from(""));
} else if app.pending_responses > 0 {
if app.pending_responses > 0 {
lines.push(Line::from(Span::styled(
"● 正在思考…",
Style::default().fg(Color::Cyan),
@ -150,16 +66,3 @@ pub fn render(frame: &mut Frame, area: Rect, app: &App) {
area,
);
}
fn push_wrapped(lines: &mut Vec<Line<'static>>, text: &str, width: usize, color: Color) {
for source_line in text.lines() {
let wrapped = textwrap::wrap(source_line, width);
if wrapped.is_empty() {
lines.push(Line::from(""));
} else {
lines.extend(wrapped.into_iter().map(|line| {
Line::from(Span::styled(line.into_owned(), Style::default().fg(color)))
}));
}
}
}

1
src/client/tui/event.rs
View File

@ -206,7 +206,6 @@ async fn handle_input_key(app: &mut App, key: KeyEvent) {
WsInbound::UserInput {
content: input,
upload_ids,
client_message_id: None,
channel: None,
// Session routing is owned by the server. A full session
// id is not a chat id and must never be sent here.

34
src/client/tui/ui.rs
View File

@ -189,38 +189,4 @@ mod tests {
);
terminal.draw(|frame| render_ui(frame, &app)).unwrap();
}
#[test]
fn active_reasoning_text_and_tool_snapshot_renders_without_panic() {
let backend = TestBackend::new(96, 24);
let mut terminal = Terminal::new(backend).unwrap();
let mut app = App::new();
app.set_current_session(Some("session".into()));
let (controller, emitter, _) = crate::session::TurnController::start("session", "message");
emitter
.emit(crate::agent::TurnEvent::ReasoningDelta {
iteration: 0,
delta: "先检查状态".into(),
})
.unwrap();
emitter
.emit(crate::agent::TurnEvent::TextDelta {
iteration: 0,
delta: "正在处理".into(),
})
.unwrap();
emitter
.emit(crate::agent::TurnEvent::ToolStarted {
iteration: 0,
call: crate::providers::ToolCall {
id: "call".into(),
name: "bash".into(),
arguments: serde_json::json!({"cmd": "pwd"}),
},
})
.unwrap();
app.apply_turn_snapshot((*controller.snapshot()).clone());
terminal.draw(|frame| render_ui(frame, &app)).unwrap();
}
}

1405
src/config/mod.rs
View File

File diff suppressed because it is too large Load Diff

680
src/delivery/coordinator.rs
View File

@ -1,680 +0,0 @@
use std::collections::HashMap;
use std::sync::{Arc, Mutex, Weak};
use std::time::Duration;
use tokio::sync::{Mutex as AsyncMutex, oneshot, watch};
use tokio::time::{Instant, sleep_until, timeout};
use crate::channels::{Channel, ChannelError, LivePolicy, TurnSink, TurnTarget};
use crate::delivery::{PresentationPolicy, project_snapshot};
use crate::session::{TurnSnapshot, TurnStatus};
use crate::task_supervisor::TaskSupervisor;
const SINK_CALL_TIMEOUT: Duration = Duration::from_secs(30);
const FINAL_RETRY_DELAYS: &[Duration] = &[
Duration::from_secs(1),
Duration::from_secs(2),
Duration::from_secs(4),
];
#[derive(Debug)]
pub enum DeliveryError {
ChannelNotFound(String),
OpenFailed(ChannelError),
SnapshotStreamClosed,
SupervisorStopping,
CompletionLost,
FinalTimedOut,
FinalFailed(ChannelError),
}
impl std::fmt::Display for DeliveryError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::ChannelNotFound(channel) => write!(formatter, "channel not found: {channel}"),
Self::OpenFailed(error) => write!(formatter, "failed to open turn sink: {error}"),
Self::SnapshotStreamClosed => {
formatter.write_str("turn snapshot stream closed before a terminal state")
}
Self::SupervisorStopping => {
formatter.write_str("cannot start turn delivery while Gateway is stopping")
}
Self::CompletionLost => {
formatter.write_str("turn delivery task stopped without reporting completion")
}
Self::FinalTimedOut => formatter.write_str("final turn delivery timed out"),
Self::FinalFailed(error) => write!(formatter, "final turn delivery failed: {error}"),
}
}
}
impl std::error::Error for DeliveryError {}
/// Shared ordering boundary for writes to one `(channel, chat_id)` target.
///
/// The registry stores weak references so inactive conversations disappear
/// without a cleanup task. Callers hold the returned lock only around one
/// external write, never for the lifetime of a Turn.
#[derive(Clone, Default)]
pub struct ConversationWriteLocks {
locks: Arc<Mutex<HashMap<String, Weak<AsyncMutex<()>>>>>,
}
impl ConversationWriteLocks {
pub fn for_target(&self, channel: &str, chat_id: &str) -> Arc<AsyncMutex<()>> {
let key = format!("{channel}\0{chat_id}");
let mut locks = self
.locks
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
if let Some(existing) = locks.get(&key).and_then(Weak::upgrade) {
return existing;
}
let lock = Arc::new(AsyncMutex::new(()));
locks.insert(key, Arc::downgrade(&lock));
lock
}
}
#[derive(Clone)]
pub struct DeliveryCoordinator {
write_locks: ConversationWriteLocks,
sink_call_timeout: Duration,
final_retry_delays: Arc<[Duration]>,
}
pub(crate) struct SinkRoute {
pub channel: String,
pub chat_id: String,
pub live_policy: LivePolicy,
pub presentation: PresentationPolicy,
}
impl DeliveryCoordinator {
pub fn new(write_locks: ConversationWriteLocks) -> Self {
Self {
write_locks,
sink_call_timeout: SINK_CALL_TIMEOUT,
final_retry_delays: FINAL_RETRY_DELAYS.into(),
}
}
#[cfg(test)]
fn for_test(
sink_call_timeout: Duration,
final_retry_delays: impl Into<Arc<[Duration]>>,
) -> Self {
Self {
write_locks: ConversationWriteLocks::default(),
sink_call_timeout,
final_retry_delays: final_retry_delays.into(),
}
}
pub fn write_locks(&self) -> ConversationWriteLocks {
self.write_locks.clone()
}
pub async fn open_and_deliver(
&self,
channel: Arc<dyn Channel + Send + Sync>,
target: TurnTarget,
presentation: PresentationPolicy,
snapshots: watch::Receiver<Arc<TurnSnapshot>>,
) -> Result<(), DeliveryError> {
let live_policy = channel.live_policy();
let sink = channel
.open_turn(target.clone())
.await
.map_err(DeliveryError::OpenFailed)?;
self.deliver(
&target.channel,
&target.chat_id,
live_policy,
presentation,
snapshots,
sink,
)
.await
}
/// Start one sink lifecycle under the Gateway's task owner and return a
/// bounded completion report to the caller.
pub fn spawn(
&self,
supervisor: &TaskSupervisor,
channel: Arc<dyn Channel + Send + Sync>,
target: TurnTarget,
presentation: PresentationPolicy,
snapshots: watch::Receiver<Arc<TurnSnapshot>>,
) -> Result<oneshot::Receiver<Result<(), DeliveryError>>, DeliveryError> {
let (result_tx, result_rx) = oneshot::channel();
let coordinator = self.clone();
let task_name = format!("turn-delivery:{}:{}", target.channel, target.chat_id);
let spawned = supervisor.spawn(task_name, async move {
let result = coordinator
.open_and_deliver(channel, target, presentation, snapshots)
.await;
if let Err(error) = &result {
tracing::error!(error = %error, "Turn delivery failed");
}
let _ = result_tx.send(result);
});
if !spawned {
return Err(DeliveryError::SupervisorStopping);
}
Ok(result_rx)
}
pub(crate) fn spawn_sink(
&self,
supervisor: &TaskSupervisor,
route: SinkRoute,
mut snapshots: watch::Receiver<Arc<TurnSnapshot>>,
mut sink: Box<dyn TurnSink>,
) -> Result<oneshot::Receiver<Result<(), DeliveryError>>, DeliveryError> {
let SinkRoute {
channel,
chat_id,
live_policy,
presentation,
} = route;
let (result_tx, result_rx) = oneshot::channel();
let coordinator = self.clone();
let task_name = format!("turn-delivery:{channel}:{chat_id}");
let cancellation = supervisor.cancellation_token();
let shutdown_snapshot = snapshots.clone();
let spawned = supervisor.spawn_graceful(task_name, async move {
let mut delivery = Box::pin(coordinator.deliver_sink(
&channel,
&chat_id,
live_policy,
presentation,
&mut snapshots,
&mut *sink,
));
let result = tokio::select! {
result = &mut delivery => result,
() = cancellation.cancelled() => {
drop(delivery);
let snapshot = shutdown_snapshot.borrow().clone();
let projected = project_snapshot(&snapshot, presentation);
coordinator
.abort_for_shutdown(&channel, &chat_id, &mut *sink, &projected)
.await
}
};
if let Err(error) = &result {
tracing::error!(channel, chat_id, error = %error, "Turn delivery failed");
}
let _ = result_tx.send(result);
});
if !spawned {
return Err(DeliveryError::SupervisorStopping);
}
Ok(result_rx)
}
pub async fn deliver(
&self,
channel: &str,
chat_id: &str,
live_policy: LivePolicy,
presentation: PresentationPolicy,
mut snapshots: watch::Receiver<Arc<TurnSnapshot>>,
mut sink: Box<dyn TurnSink>,
) -> Result<(), DeliveryError> {
self.deliver_sink(
channel,
chat_id,
live_policy,
presentation,
&mut snapshots,
&mut *sink,
)
.await
}
async fn deliver_sink(
&self,
channel: &str,
chat_id: &str,
live_policy: LivePolicy,
presentation: PresentationPolicy,
snapshots: &mut watch::Receiver<Arc<TurnSnapshot>>,
sink: &mut dyn TurnSink,
) -> Result<(), DeliveryError> {
let target_lock = self.write_locks.for_target(channel, chat_id);
let min_interval = match live_policy {
LivePolicy::FinalOnly => None,
LivePolicy::Snapshot { min_interval } if presentation.live => Some(min_interval),
LivePolicy::Snapshot { .. } => None,
};
let mut next_update_at = Instant::now();
loop {
let snapshot = snapshots.borrow_and_update().clone();
if snapshot.status != TurnStatus::Running {
let projected = project_snapshot(&snapshot, presentation);
return self.deliver_terminal(&target_lock, sink, &projected).await;
}
if let Some(interval) = min_interval {
while Instant::now() < next_update_at {
tokio::select! {
changed = snapshots.changed() => {
changed.map_err(|_| DeliveryError::SnapshotStreamClosed)?;
let latest = snapshots.borrow_and_update().clone();
if latest.status != TurnStatus::Running {
let projected = project_snapshot(&latest, presentation);
return self.deliver_terminal(&target_lock, sink, &projected).await;
}
}
() = sleep_until(next_update_at) => break,
}
}
let latest = snapshots.borrow_and_update().clone();
if latest.status != TurnStatus::Running {
let projected = project_snapshot(&latest, presentation);
return self.deliver_terminal(&target_lock, sink, &projected).await;
}
let projected = project_snapshot(&latest, presentation);
let _guard = target_lock.lock().await;
match timeout(self.sink_call_timeout, sink.update(&projected)).await {
Ok(Ok(())) => {}
Ok(Err(error)) => {
tracing::warn!(error = %error, revision = projected.revision, "Live turn update failed; waiting for a newer snapshot");
}
Err(_) => {
tracing::warn!(
revision = projected.revision,
"Live turn update timed out; waiting for a newer snapshot"
);
}
}
next_update_at = Instant::now() + interval;
}
snapshots
.changed()
.await
.map_err(|_| DeliveryError::SnapshotStreamClosed)?;
}
}
async fn abort_for_shutdown(
&self,
channel: &str,
chat_id: &str,
sink: &mut dyn TurnSink,
snapshot: &TurnSnapshot,
) -> Result<(), DeliveryError> {
let target_lock = self.write_locks.for_target(channel, chat_id);
let _guard = target_lock.lock().await;
match timeout(self.sink_call_timeout, sink.abort(snapshot)).await {
Ok(Ok(())) => Ok(()),
Ok(Err(error)) => Err(DeliveryError::FinalFailed(error)),
Err(_) => Err(DeliveryError::FinalTimedOut),
}
}
async fn deliver_terminal(
&self,
target_lock: &Arc<AsyncMutex<()>>,
sink: &mut dyn TurnSink,
snapshot: &TurnSnapshot,
) -> Result<(), DeliveryError> {
let attempts = self.final_retry_delays.len() + 1;
for attempt in 0..attempts {
let _guard = target_lock.lock().await;
let result = if snapshot.status == TurnStatus::Completed {
timeout(self.sink_call_timeout, sink.finish(snapshot)).await
} else {
timeout(self.sink_call_timeout, sink.abort(snapshot)).await
};
drop(_guard);
match result {
Ok(Ok(())) => return Ok(()),
Ok(Err(error))
if error.is_transient() && attempt < self.final_retry_delays.len() =>
{
sleep_until(Instant::now() + self.final_retry_delays[attempt]).await;
}
Ok(Err(error)) => return Err(DeliveryError::FinalFailed(error)),
Err(_) if attempt < self.final_retry_delays.len() => {
sleep_until(Instant::now() + self.final_retry_delays[attempt]).await;
}
Err(_) => return Err(DeliveryError::FinalTimedOut),
}
}
unreachable!()
}
}
#[cfg(test)]
mod tests {
use super::*;
use async_trait::async_trait;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::sync::{Mutex as TokioMutex, Notify};
use crate::agent::TurnEvent;
use crate::bus::{MessageBus, OutboundMessage};
use crate::channels::{Channel, TurnSink};
use crate::session::{TurnBlock, TurnController};
#[derive(Default)]
struct SinkState {
updates: TokioMutex<Vec<TurnSnapshot>>,
terminal: TokioMutex<Vec<TurnSnapshot>>,
update_started: Notify,
release_update: Notify,
block_first_update: bool,
fail_updates: AtomicUsize,
fail_finish: AtomicUsize,
}
struct RecordingSink(Arc<SinkState>);
struct SinkChannel {
state: Arc<SinkState>,
opened: AtomicUsize,
}
#[async_trait]
impl Channel for SinkChannel {
fn name(&self) -> &str {
"sink-channel"
}
fn is_running(&self) -> bool {
true
}
async fn start(&self, _bus: Arc<MessageBus>) -> Result<(), ChannelError> {
Ok(())
}
async fn stop(&self) -> Result<(), ChannelError> {
Ok(())
}
fn live_policy(&self) -> LivePolicy {
LivePolicy::FinalOnly
}
async fn open_turn(&self, _target: TurnTarget) -> Result<Box<dyn TurnSink>, ChannelError> {
self.opened.fetch_add(1, Ordering::SeqCst);
Ok(sink(self.state.clone()))
}
async fn send(&self, _msg: OutboundMessage) -> Result<(), ChannelError> {
Ok(())
}
}
#[async_trait]
impl TurnSink for RecordingSink {
async fn update(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError> {
self.0.update_started.notify_waiters();
if self.0.block_first_update && self.0.updates.lock().await.is_empty() {
self.0.release_update.notified().await;
}
if self
.0
.fail_updates
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |remaining| {
remaining.checked_sub(1)
})
.is_ok()
{
return Err(ChannelError::SendError("update".into()));
}
self.0.updates.lock().await.push(snapshot.clone());
Ok(())
}
async fn finish(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError> {
if self
.0
.fail_finish
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |remaining| {
remaining.checked_sub(1)
})
.is_ok()
{
return Err(ChannelError::SendError("finish".into()));
}
self.0.terminal.lock().await.push(snapshot.clone());
Ok(())
}
async fn abort(&mut self, snapshot: &TurnSnapshot) -> Result<(), ChannelError> {
self.0.terminal.lock().await.push(snapshot.clone());
Ok(())
}
}
fn sink(state: Arc<SinkState>) -> Box<dyn TurnSink> {
Box::new(RecordingSink(state))
}
#[tokio::test]
async fn slow_sink_observes_latest_snapshot_and_terminal_bypasses_throttle() {
let state = Arc::new(SinkState {
block_first_update: true,
..SinkState::default()
});
let (controller, emitter, receiver) = TurnController::start("session", "message");
let coordinator = DeliveryCoordinator::for_test(Duration::from_secs(30), []);
let task = tokio::spawn({
let state = state.clone();
async move {
coordinator
.deliver(
"cli_chat",
"chat",
LivePolicy::Snapshot {
min_interval: Duration::from_secs(10),
},
PresentationPolicy::interactive(),
receiver,
sink(state),
)
.await
}
});
state.update_started.notified().await;
emitter
.emit(TurnEvent::TextDelta {
iteration: 0,
delta: "a".into(),
})
.unwrap();
emitter
.emit(TurnEvent::TextDelta {
iteration: 0,
delta: "b".into(),
})
.unwrap();
state.release_update.notify_waiters();
tokio::task::yield_now().await;
controller.complete(None);
assert!(task.await.unwrap().is_ok());
let terminal = state.terminal.lock().await;
assert_eq!(terminal.len(), 1);
assert_eq!(terminal[0].status, TurnStatus::Completed);
assert!(
matches!(&terminal[0].blocks[0], TurnBlock::Assistant { text, .. } if text == "ab")
);
}
#[tokio::test]
async fn hidden_reasoning_is_removed_before_sink_and_failed_update_recovers() {
let state = Arc::new(SinkState {
fail_updates: AtomicUsize::new(1),
..SinkState::default()
});
let (controller, emitter, receiver) = TurnController::start("session", "message");
let coordinator = DeliveryCoordinator::for_test(Duration::from_secs(30), []);
let task = tokio::spawn({
let state = state.clone();
async move {
coordinator
.deliver(
"feishu",
"chat",
LivePolicy::Snapshot {
min_interval: Duration::ZERO,
},
PresentationPolicy::external(true),
receiver,
sink(state),
)
.await
}
});
emitter
.emit(TurnEvent::ReasoningDelta {
iteration: 0,
delta: "secret".into(),
})
.unwrap();
tokio::task::yield_now().await;
emitter
.emit(TurnEvent::TextDelta {
iteration: 0,
delta: "public".into(),
})
.unwrap();
tokio::task::yield_now().await;
controller.complete(None);
assert!(task.await.unwrap().is_ok());
let terminal = state.terminal.lock().await;
assert!(
terminal[0]
.blocks
.iter()
.all(|block| !matches!(block, TurnBlock::Reasoning { .. }))
);
}
#[tokio::test]
async fn final_only_skips_updates_and_retries_transient_finish() {
let state = Arc::new(SinkState {
fail_finish: AtomicUsize::new(2),
..SinkState::default()
});
let (controller, emitter, receiver) = TurnController::start("session", "message");
let coordinator = DeliveryCoordinator::for_test(
Duration::from_secs(30),
[Duration::from_millis(1), Duration::from_millis(2)],
);
let task = tokio::spawn({
let state = state.clone();
async move {
coordinator
.deliver(
"channel",
"chat",
LivePolicy::FinalOnly,
PresentationPolicy::unattended(),
receiver,
sink(state),
)
.await
}
});
emitter
.emit(TurnEvent::TextDelta {
iteration: 0,
delta: "done".into(),
})
.unwrap();
controller.complete(None);
assert!(task.await.unwrap().is_ok());
assert!(state.updates.lock().await.is_empty());
assert_eq!(state.terminal.lock().await.len(), 1);
}
#[tokio::test]
async fn open_and_deliver_owns_sink_creation_and_terminal_lifecycle() {
let state = Arc::new(SinkState::default());
let channel = Arc::new(SinkChannel {
state: state.clone(),
opened: AtomicUsize::new(0),
});
let (controller, emitter, receiver) = TurnController::start("session", "message");
let coordinator = DeliveryCoordinator::for_test(Duration::from_secs(30), []);
let target = TurnTarget {
channel: "sink-channel".into(),
chat_id: "chat".into(),
session_id: "session".into(),
reply_to: None,
metadata: HashMap::new(),
};
let task = tokio::spawn({
let channel = channel.clone();
async move {
coordinator
.open_and_deliver(channel, target, PresentationPolicy::unattended(), receiver)
.await
}
});
emitter
.emit(TurnEvent::TextDelta {
iteration: 0,
delta: "done".into(),
})
.unwrap();
controller.complete(None);
assert!(task.await.unwrap().is_ok());
assert_eq!(channel.opened.load(Ordering::SeqCst), 1);
assert_eq!(state.terminal.lock().await.len(), 1);
}
#[tokio::test]
async fn supervisor_shutdown_aborts_sink_and_waits_for_cleanup() {
let state = Arc::new(SinkState::default());
let (_controller, emitter, receiver) = TurnController::start("session", "message");
emitter
.emit(TurnEvent::TextDelta {
iteration: 0,
delta: "partial".into(),
})
.unwrap();
let supervisor = TaskSupervisor::new();
let coordinator = DeliveryCoordinator::for_test(Duration::from_secs(1), []);
let result = coordinator
.spawn_sink(
&supervisor,
SinkRoute {
channel: "channel".into(),
chat_id: "chat".into(),
live_policy: LivePolicy::FinalOnly,
presentation: PresentationPolicy::unattended(),
},
receiver,
sink(state.clone()),
)
.unwrap();
tokio::task::yield_now().await;
supervisor.shutdown(Duration::from_secs(1)).await;
assert!(result.await.unwrap().is_ok());
let terminal = state.terminal.lock().await;
assert_eq!(terminal.len(), 1);
assert_eq!(terminal[0].status, TurnStatus::Running);
}
}

7
src/delivery/mod.rs
View File

@ -1,7 +0,0 @@
mod coordinator;
mod policy;
mod service;
pub use coordinator::{ConversationWriteLocks, DeliveryCoordinator, DeliveryError};
pub use policy::{PresentationPolicy, ReasoningVisibility, ToolVisibility, project_snapshot};
pub use service::{TurnDeliveryHandle, TurnDeliveryService};

138
src/delivery/policy.rs
View File

@ -1,138 +0,0 @@
use crate::session::{TurnBlock, TurnSnapshot};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ReasoningVisibility {
Hidden,
Collapsed,
Expanded,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ToolVisibility {
Hidden,
Compact,
Detailed,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct PresentationPolicy {
pub live: bool,
pub reasoning: ReasoningVisibility,
pub tools: ToolVisibility,
}
impl PresentationPolicy {
pub const fn interactive() -> Self {
Self {
live: true,
reasoning: ReasoningVisibility::Collapsed,
tools: ToolVisibility::Detailed,
}
}
pub const fn external(live: bool) -> Self {
Self {
live,
reasoning: ReasoningVisibility::Hidden,
tools: ToolVisibility::Compact,
}
}
pub const fn unattended() -> Self {
Self {
live: false,
reasoning: ReasoningVisibility::Hidden,
tools: ToolVisibility::Hidden,
}
}
}
/// Produce the immutable view that is allowed to leave the Gateway core.
///
/// Presentation filtering deliberately clones the snapshot. Conversation
/// history and the authoritative TurnController state remain untouched.
pub fn project_snapshot(snapshot: &TurnSnapshot, policy: PresentationPolicy) -> TurnSnapshot {
let mut projected = snapshot.clone();
projected.blocks.retain_mut(|block| match block {
TurnBlock::Reasoning { .. } => policy.reasoning != ReasoningVisibility::Hidden,
TurnBlock::Assistant { .. } => true,
TurnBlock::Tool {
arguments, preview, ..
} => match policy.tools {
ToolVisibility::Hidden => false,
ToolVisibility::Compact => {
*arguments = serde_json::Value::Null;
*preview = None;
true
}
ToolVisibility::Detailed => true,
},
});
projected
}
#[cfg(test)]
mod tests {
use super::*;
use crate::session::{BlockId, ToolStatus, TurnId, TurnPhase, TurnState, TurnStatus};
fn snapshot() -> TurnSnapshot {
TurnState {
id: TurnId("turn".into()),
session_id: "session".into(),
message_id: "message".into(),
revision: 3,
status: TurnStatus::Running,
phase: TurnPhase::Acting,
blocks: vec![
TurnBlock::Reasoning {
id: BlockId("reasoning".into()),
iteration: 0,
text: "private chain".into(),
},
TurnBlock::Assistant {
id: BlockId("text".into()),
iteration: 0,
text: "visible".into(),
},
TurnBlock::Tool {
id: "tool".into(),
iteration: 0,
name: "bash".into(),
arguments: serde_json::json!({"token": "secret"}),
status: ToolStatus::Completed,
preview: Some("sensitive output".into()),
},
],
usage: None,
error: None,
}
}
#[test]
fn external_projection_removes_reasoning_and_tool_details_without_mutating_source() {
let source = snapshot();
let projected = project_snapshot(&source, PresentationPolicy::external(true));
assert_eq!(projected.blocks.len(), 2);
assert!(matches!(projected.blocks[0], TurnBlock::Assistant { .. }));
assert!(matches!(
&projected.blocks[1],
TurnBlock::Tool {
arguments: serde_json::Value::Null,
preview: None,
..
}
));
assert_eq!(source.blocks.len(), 3);
assert!(matches!(source.blocks[0], TurnBlock::Reasoning { .. }));
}
#[test]
fn unattended_projection_keeps_only_assistant_blocks() {
let projected = project_snapshot(&snapshot(), PresentationPolicy::unattended());
assert_eq!(projected.blocks.len(), 1);
assert!(matches!(projected.blocks[0], TurnBlock::Assistant { .. }));
}
}

123
src/delivery/service.rs
View File

@ -1,123 +0,0 @@
use std::sync::Arc;
use tokio::sync::{oneshot, watch};
use crate::bus::CommittedTurnDelta;
use crate::channels::{ChannelManager, TurnTarget};
use crate::delivery::{DeliveryCoordinator, DeliveryError};
use crate::session::TurnSnapshot;
use crate::task_supervisor::TaskSupervisor;
use super::coordinator::SinkRoute;
/// Gateway-owned facade that resolves a target Channel and starts exactly one
/// TurnSink lifecycle. Session workers depend on this abstraction rather than
/// on Channel implementations or WebSocket/Feishu protocols.
#[derive(Clone)]
pub struct TurnDeliveryService {
coordinator: DeliveryCoordinator,
channels: ChannelManager,
supervisor: TaskSupervisor,
}
/// Completion handle for one TurnSink lifecycle.
///
/// Creating a sink only proves that delivery started. Callers consume this
/// handle after publishing a terminal snapshot to learn whether the terminal
/// write actually reached the Channel.
pub struct TurnDeliveryHandle {
pub(crate) completion: oneshot::Receiver<Result<(), DeliveryError>>,
}
impl TurnDeliveryHandle {
pub async fn wait(self) -> Result<(), DeliveryError> {
self.completion
.await
.unwrap_or(Err(DeliveryError::CompletionLost))
}
}
impl TurnDeliveryService {
pub fn new(
coordinator: DeliveryCoordinator,
channels: ChannelManager,
supervisor: TaskSupervisor,
) -> Self {
Self {
coordinator,
channels,
supervisor,
}
}
pub async fn start(
&self,
target: TurnTarget,
snapshots: watch::Receiver<Arc<TurnSnapshot>>,
) -> Result<TurnDeliveryHandle, DeliveryError> {
let channel = self
.channels
.get_channel(&target.channel)
.await
.ok_or_else(|| DeliveryError::ChannelNotFound(target.channel.clone()))?;
let live_policy = channel.live_policy();
let presentation = channel.presentation_policy();
let sink = channel
.open_turn(target.clone())
.await
.map_err(DeliveryError::OpenFailed)?;
let completion = self.coordinator.spawn_sink(
&self.supervisor,
SinkRoute {
channel: target.channel,
chat_id: target.chat_id,
live_policy,
presentation,
},
snapshots,
sink,
)?;
Ok(TurnDeliveryHandle { completion })
}
pub async fn commit(
&self,
target: &TurnTarget,
delta: CommittedTurnDelta,
) -> Result<bool, DeliveryError> {
let channel = self
.channels
.get_channel(&target.channel)
.await
.ok_or_else(|| DeliveryError::ChannelNotFound(target.channel.clone()))?;
let presents_media = channel.commit_turn_presents_media();
channel
.commit_turn(target, delta)
.await
.map_err(DeliveryError::FinalFailed)?;
Ok(presents_media)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn handle_reports_terminal_delivery_failure() {
let (sender, completion) = oneshot::channel();
sender.send(Err(DeliveryError::FinalTimedOut)).unwrap();
let result = TurnDeliveryHandle { completion }.wait().await;
assert!(matches!(result, Err(DeliveryError::FinalTimedOut)));
}
#[tokio::test]
async fn handle_reports_lost_delivery_task() {
let (sender, completion) = oneshot::channel();
drop(sender);
let result = TurnDeliveryHandle { completion }.wait().await;
assert!(matches!(result, Err(DeliveryError::CompletionLost)));
}
}

103
src/gateway/auth.rs
View File

@ -21,7 +21,6 @@ const MAX_FAILED_ATTEMPTS: u32 = 5;
const MAX_TRACKED_CLIENTS: usize = 4096;
const MAX_PAIRED_TOKENS: usize = 128;
const AUTH_COOKIE: &str = "picobot_auth";
pub const ADMIN_TOKEN_HEADER: &str = "X-Picobot-Admin-Token";
#[derive(Debug, Clone, Serialize, Deserialize)]
struct AuthStore {
@ -66,12 +65,8 @@ pub struct AuthManager {
state: Arc<Mutex<AuthState>>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AuthIdentity {
PairingDisabled,
Paired { token_hash: String },
LocalAdmin,
}
#[derive(Debug, Clone)]
pub struct AuthIdentity(pub Option<String>);
#[derive(Debug)]
pub enum PairError {
@ -115,7 +110,7 @@ impl AuthManager {
pub async fn authenticate(&self, token: Option<&str>) -> Option<AuthIdentity> {
if !self.required {
return Some(AuthIdentity::PairingDisabled);
return Some(AuthIdentity(None));
}
let hash = hash_token(token?);
self.state
@ -123,17 +118,17 @@ impl AuthManager {
.await
.token_hashes
.contains(&hash)
.then_some(AuthIdentity::Paired { token_hash: hash })
.then_some(AuthIdentity(Some(hash)))
}
pub async fn identity_is_active(&self, identity: &AuthIdentity) -> bool {
match identity {
AuthIdentity::PairingDisabled => !self.required,
AuthIdentity::LocalAdmin => true,
AuthIdentity::Paired { token_hash } => {
self.required && self.state.lock().await.token_hashes.contains(token_hash)
}
if !self.required {
return true;
}
let Some(hash) = identity.0.as_ref() else {
return false;
};
self.state.lock().await.token_hashes.contains(hash)
}
pub fn authenticate_admin(&self, token: Option<&str>) -> bool {
@ -215,24 +210,9 @@ pub async fn require_auth(
mut request: Request<Body>,
next: Next,
) -> Response {
let mut identity = auth
let identity = auth
.authenticate(token_from_headers(request.headers()))
.await;
if identity.is_none()
&& request.uri().path() == "/ws"
&& request
.extensions()
.get::<ConnectInfo<SocketAddr>>()
.is_some_and(|ConnectInfo(peer)| peer.ip().is_loopback())
&& auth.authenticate_admin(
request
.headers()
.get(ADMIN_TOKEN_HEADER)
.and_then(|value| value.to_str().ok()),
)
{
identity = Some(AuthIdentity::LocalAdmin);
}
let Some(identity) = identity else {
return (
StatusCode::UNAUTHORIZED,
@ -334,7 +314,7 @@ pub async fn issue_code(
headers: HeaderMap,
) -> Response {
let admin_token = headers
.get(ADMIN_TOKEN_HEADER)
.get("X-Picobot-Admin-Token")
.and_then(|value| value.to_str().ok());
if !peer.ip().is_loopback() || !state.auth.authenticate_admin(admin_token) {
return (
@ -508,7 +488,7 @@ async fn load_or_create_admin_token(path: &Path) -> Result<String, std::io::Erro
#[cfg(test)]
mod tests {
use super::*;
use axum::{Extension, Router, middleware, routing};
use axum::{Router, middleware, routing};
use tower::ServiceExt;
#[tokio::test]
@ -613,61 +593,4 @@ mod tests {
.unwrap();
assert_eq!(authorized.status(), StatusCode::OK);
}
#[tokio::test]
async fn local_admin_auth_is_limited_to_loopback_websockets() {
let dir = tempfile::tempdir().unwrap();
let manager = AuthManager::load(true, dir.path().join("auth.json"))
.await
.unwrap();
let admin_token = tokio::fs::read_to_string(dir.path().join("web_admin_token"))
.await
.unwrap();
let app = Router::new()
.route(
"/ws",
routing::get(|Extension(identity): Extension<AuthIdentity>| async move {
assert_eq!(identity, AuthIdentity::LocalAdmin);
StatusCode::OK
}),
)
.route("/protected", routing::get(|| async { StatusCode::OK }))
.route_layer(middleware::from_fn_with_state(manager, require_auth));
let mut local_ws = Request::get("/ws")
.header(ADMIN_TOKEN_HEADER, admin_token.trim())
.body(Body::empty())
.unwrap();
local_ws
.extensions_mut()
.insert(ConnectInfo(SocketAddr::from(([127, 0, 0, 1], 42000))));
assert_eq!(
app.clone().oneshot(local_ws).await.unwrap().status(),
StatusCode::OK
);
let mut remote_ws = Request::get("/ws")
.header(ADMIN_TOKEN_HEADER, admin_token.trim())
.body(Body::empty())
.unwrap();
remote_ws
.extensions_mut()
.insert(ConnectInfo(SocketAddr::from(([192, 0, 2, 10], 42000))));
assert_eq!(
app.clone().oneshot(remote_ws).await.unwrap().status(),
StatusCode::UNAUTHORIZED
);
let mut local_api = Request::get("/protected")
.header(ADMIN_TOKEN_HEADER, admin_token.trim())
.body(Body::empty())
.unwrap();
local_api
.extensions_mut()
.insert(ConnectInfo(SocketAddr::from(([127, 0, 0, 1], 42000))));
assert_eq!(
app.oneshot(local_api).await.unwrap().status(),
StatusCode::UNAUTHORIZED
);
}
}

1036
src/gateway/http.rs
View File

File diff suppressed because it is too large Load Diff

775
src/gateway/mod.rs
View File

@ -1,96 +1,42 @@
pub mod auth;
pub mod http;
pub(crate) mod reload;
mod router;
pub mod uploads;
pub mod ws;
use axum::{Router, middleware, routing};
use std::net::SocketAddr;
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use tokio::net::TcpListener;
use crate::bus::{MessageBus, OutboundDispatcher};
use crate::bus::{ControlMessage, MessageBus, OutboundDispatcher};
use crate::channels::base::ChannelError;
use crate::channels::{ChannelManager, CliChatChannel};
use crate::config::{Config, ConfigLoadContext, ensure_workspace_dir, expand_path};
use crate::delivery::{ConversationWriteLocks, DeliveryCoordinator, TurnDeliveryService};
use crate::config::{Config, ensure_workspace_dir, expand_path};
use crate::logging;
use crate::mcp;
use crate::memory::MemoryManager;
use crate::scheduler::Scheduler;
use crate::session::{AgentCatalogPreparation, SessionManager, SessionManagerServices};
use crate::session::SessionManager;
use crate::task_supervisor::TaskSupervisor;
/// Process boot clock. A process-level static so uptime survives config reload,
/// which swaps GatewayState generations without restarting the process.
static STARTED: std::sync::OnceLock<std::time::Instant> = std::sync::OnceLock::new();
/// Seconds elapsed since the gateway process started.
pub fn process_uptime_secs() -> u64 {
STARTED
.get_or_init(std::time::Instant::now)
.elapsed()
.as_secs()
}
pub struct GatewayState {
pub config: Config,
pub config_path: std::path::PathBuf,
pub(crate) config_load_context: Arc<ConfigLoadContext>,
pub(crate) config_write_lock: Arc<tokio::sync::Mutex<()>>,
pub workspace_dir: std::path::PathBuf,
pub(crate) health: Arc<crate::health::HealthService>,
pub session_manager: Arc<SessionManager>,
pub channel_manager: ChannelManager,
pub storage: Arc<crate::storage::Storage>,
pub task_supervisor: TaskSupervisor,
pub delivery_coordinator: DeliveryCoordinator,
pub connection_shutdown: tokio_util::sync::CancellationToken,
pub auth: auth::AuthManager,
pub uploads: uploads::UploadRegistry,
/// Live WebSocket connection count.
pub ws_connections: Arc<AtomicUsize>,
/// Active outbound dispatcher lane count (shared with the dispatcher).
pub outbound_lanes: Arc<AtomicUsize>,
pub(crate) reload: reload::ReloadHandle,
pub(crate) admission: reload::RuntimeAdmission,
pub agent_catalog: Arc<crate::agent::AgentCatalog>,
/// Directory holding Agent definition files (resolved definitions_dir).
pub agents_dir: std::path::PathBuf,
}
impl GatewayState {
/// Construct a standalone state. Configuration reload is available only
/// when the state is owned by [`run`], which owns the generation loop.
pub async fn new() -> Result<Self, Box<dyn std::error::Error>> {
let config_path = crate::config::resolve_default_config_path();
let config_load_context = Arc::new(Config::load_context());
let config = Config::load_for_startup(&config_path, &config_load_context)?;
Self::from_config(
config,
config_path,
config_load_context,
Arc::new(tokio::sync::Mutex::new(())),
reload::ReloadHandle::unavailable(),
true,
1,
)
.await
}
async fn from_config(
config: Config,
config_path: std::path::PathBuf,
config_load_context: Arc<ConfigLoadContext>,
config_write_lock: Arc<tokio::sync::Mutex<()>>,
reload: reload::ReloadHandle,
initialize_process: bool,
runtime_generation: u64,
) -> Result<Self, Box<dyn std::error::Error>> {
let config = Config::load_default()?;
let task_supervisor = TaskSupervisor::new();
let admission = reload::RuntimeAdmission::open();
let delivery_coordinator = DeliveryCoordinator::new(ConversationWriteLocks::default());
let connection_shutdown = tokio_util::sync::CancellationToken::new();
let auth = auth::AuthManager::load(
config.gateway.require_pairing,
@ -103,9 +49,7 @@ impl GatewayState {
let workspace_path = expand_path(&config.workspace_dir);
let workspace_path = ensure_workspace_dir(&workspace_path)?;
if initialize_process {
// Startup is single-threaded. Reload candidates reuse the already
// selected workspace and must not mutate process-global cwd.
// Switch current working directory to workspace
std::env::set_current_dir(&workspace_path).map_err(|e| {
format!(
"Failed to switch to workspace directory {}: {}",
@ -113,14 +57,11 @@ impl GatewayState {
e
)
})?;
}
tracing::info!("Using workspace directory: {}", workspace_path.display());
// Release default AGENTS.md and USER.md to ~/.picobot/ if not exist
if initialize_process {
ensure_default_config_files();
}
// Get provider config for SessionManager
let mut provider_config = config.get_provider_config("default")?;
@ -131,17 +72,8 @@ impl GatewayState {
let db_path = if let Some(ref path) = config.gateway.session_db_path {
std::path::PathBuf::from(path)
} else {
crate::config::get_default_db_path()
workspace_path.join("picobot.db")
};
if let Some(parent) = db_path.parent() {
std::fs::create_dir_all(parent).map_err(|e| {
format!(
"Failed to create database directory {}: {}",
parent.display(),
e
)
})?;
}
let storage = Arc::new(
crate::storage::Storage::new(&db_path)
.await
@ -156,16 +88,11 @@ impl GatewayState {
let consolidation_model = config
.memory
.resolve_consolidation_model(&provider_config.model_id);
let memory_manager = Arc::new(
MemoryManager::new(
let memory_manager = Arc::new(MemoryManager::new(
storage.clone(),
consolidation_provider,
consolidation_model,
)
.with_recall(crate::memory::recall::RecallConfig::from_memory_config(
&config.memory,
)),
);
));
tracing::info!(
consolidation_provider = %memory_manager.consolidation_provider,
consolidation_model = %memory_manager.consolidation_model,
@ -175,88 +102,34 @@ impl GatewayState {
// Create MessageBus first (shared by SessionManager and ChannelManager)
let bus = MessageBus::new(100);
// Channels are resolved by TurnDeliveryService, while Session workers
// depend only on that protocol-neutral delivery facade.
let cli_chat_channel = Arc::new(CliChatChannel::with_upload_registry(uploads.clone()));
let channel_manager = ChannelManager::with_bus(cli_chat_channel, bus.clone());
channel_manager
.init(&config, workspace_path.clone())
.await
.map_err(|e| format!("Failed to init channels: {}", e))?;
let available_channels = channel_manager.list_channel_names().await;
let turn_delivery = TurnDeliveryService::new(
delivery_coordinator.clone(),
channel_manager.clone(),
task_supervisor.clone(),
);
let browser_config = if config.browser.enabled {
Some(config.browser.clone())
} else {
None
};
let health = Arc::new(
crate::health::HealthService::new(config.clone())
.with_scheduler_runtime(storage.clone(), available_channels.clone()),
);
let provider_profiles: std::collections::HashMap<String, _> = config
.agents
.keys()
.filter_map(|name| {
config
.get_provider_config(name)
.ok()
.map(|profile| (name.clone(), profile))
})
.collect();
let config_dir = config_path
.parent()
.unwrap_or_else(|| std::path::Path::new("."))
.to_path_buf();
// Resolve the Agent definitions directory exactly like the catalog
// does (relative paths stay inside the trusted config dir).
let agents_dir = {
let configured =
crate::config::expand_path(&config.agent_orchestration.definitions_dir);
if configured.is_absolute() {
configured
} else {
config_dir.join(configured)
}
};
// Create SessionManager with bus injection
let session_manager = SessionManager::new(
provider_config.clone(),
config.context_compaction.clone(),
AgentCatalogPreparation {
provider_profiles,
providers: config.providers.clone(),
models: config.models.clone(),
workspace_dir: crate::config::expand_path(&config.workspace_dir),
config: config.agent_orchestration.clone(),
config_dir,
runtime_generation,
},
storage.clone(),
SessionManagerServices::new(
bus.clone(),
memory_manager,
task_supervisor.clone(),
turn_delivery,
reload.clone(),
)
.with_admission(admission.clone()),
browser_config,
health.clone(),
config.gateway.max_concurrent_background_tasks,
task_supervisor.clone(),
)?;
let session_manager = Arc::new(session_manager);
session_manager.bind_inbox_wake();
let agent_catalog = session_manager.agent_catalog();
health.bind_agent_catalog(agent_catalog.clone());
// Create ChannelManager and init channels
let cli_chat_channel = Arc::new(CliChatChannel::with_upload_registry(uploads.clone()));
let channel_manager = ChannelManager::with_bus(cli_chat_channel, bus);
channel_manager
.init(&config, workspace_path.clone())
.await
.map_err(|e| format!("Failed to init channels: {}", e))?;
// Register send_message tool with available channel names
let available_channels = channel_manager.list_channel_names().await;
let valid_channels = available_channels.clone();
session_manager.register_outbound_tool(available_channels);
@ -268,6 +141,21 @@ impl GatewayState {
valid_channels.clone(),
));
// Initialize MCP servers — connect and register discovered tools
if !config.mcp.servers.is_empty() {
let mcp_tools = mcp::connect_all(&config.mcp).await;
for tool_info in mcp_tools {
let wrapper = mcp::McpToolWrapper::new(
&tool_info.server_name,
tool_info.tool_name,
tool_info.description,
tool_info.schema,
tool_info.connection,
);
session_manager.tools().register(wrapper);
}
}
// Initialize scheduler if enabled in config
let scheduler_config = config.gateway.scheduler.clone().unwrap_or_default();
if scheduler_config.enabled {
@ -286,15 +174,11 @@ impl GatewayState {
.tools()
.register(crate::tools::cron::CronAddTool::new(
storage.clone(),
valid_channels.clone(),
agent_catalog.clone(),
valid_channels,
));
session_manager
.tools()
.register(crate::tools::cron::CronListTool::new(storage.clone()));
session_manager
.tools()
.register(crate::tools::cron::CronRunsTool::new(storage.clone()));
session_manager
.tools()
.register(crate::tools::cron::CronRemoveTool::new(storage.clone()));
@ -306,35 +190,21 @@ impl GatewayState {
.register(crate::tools::cron::CronDisableTool::new(storage.clone()));
session_manager
.tools()
.register(crate::tools::cron::CronUpdateTool::new(
storage.clone(),
valid_channels,
agent_catalog.clone(),
));
.register(crate::tools::cron::CronUpdateTool::new(storage.clone()));
tracing::info!("Cron tools registered");
}
Ok(Self {
config,
config_path,
config_load_context,
config_write_lock,
workspace_dir: workspace_path,
health,
session_manager: session_manager.clone(),
channel_manager,
storage,
task_supervisor,
delivery_coordinator,
connection_shutdown,
auth,
uploads,
ws_connections: Arc::new(AtomicUsize::new(0)),
outbound_lanes: Arc::new(AtomicUsize::new(0)),
reload,
admission,
agent_catalog,
agents_dir,
})
}
@ -349,66 +219,7 @@ impl GatewayState {
}
/// Start the message processing loops
pub async fn start_message_processing(&self) -> Result<(), String> {
match self
.storage
.recover_scheduled_runs(chrono::Utc::now().timestamp_millis())
.await
{
Ok(recovered) if recovered > 0 => {
tracing::warn!(
recovered,
"Scheduled runs recovered as unknown on activation"
);
}
Ok(_) => {}
Err(error) => {
return Err(format!(
"Scheduled run recovery failed on activation: {error}"
));
}
}
// Recover durable Agent state for this runtime generation: interrupt
// runs of older generations, expire stale inbox leases and reconcile
// capacity rows. Runs never recover while the generation is still a candidate.
if let Some(coordinator) = self.session_manager.agent_coordinator() {
match coordinator.recover_on_activation().await {
Ok(report) => {
if report.interrupted_runs > 0 || report.leases_expired > 0 {
tracing::warn!(
interrupted = report.interrupted_runs,
completion_events = report.completion_events_generated,
leases_expired = report.leases_expired,
dead_lettered = report.dead_lettered,
sessions_reconciled = report.sessions_reconciled,
"Agent state recovered on activation"
);
}
}
Err(error) => {
return Err(format!(
"Agent state recovery failed on activation: {error}"
));
}
}
}
// MCP connections have external/process-wide side effects. Activate
// them only after this generation becomes current, never while it is
// merely a reload candidate.
let mcp_tools = mcp::connect_all(&self.config.mcp).await;
for tool_info in mcp_tools {
let wrapper = mcp::McpToolWrapper::new(
&tool_info.server_name,
tool_info.tool_name,
tool_info.description,
tool_info.schema,
tool_info.connection,
tool_info.settings,
);
self.session_manager.tools().register(wrapper);
}
pub async fn start_message_processing(&self) {
let bus = self.bus();
let bus_for_outbound = bus.clone();
let session_manager = self.session_manager.clone();
@ -445,37 +256,85 @@ impl GatewayState {
}
});
// Relay durable Agent run/event projections. A lagged or lost
// broadcast is non-fatal: the client recalibrates with GetAgentRuns.
let mut agent_projection_events = self.session_manager.projection_hub().subscribe();
let cli_chat = self.cli_chat_channel();
self.task_supervisor
.spawn("agent-projection-events", async move {
// Spawn unified message processor
// This handles both inbound AI messages and control messages in one loop
self.task_supervisor.spawn("message-processor", async move {
tracing::info!("Message processor started");
loop {
match agent_projection_events.recv().await {
Ok(event) => cli_chat.publish_agent_projection(event).await,
Err(tokio::sync::broadcast::error::RecvError::Lagged(skipped)) => {
tracing::warn!(skipped, "Agent projection relay lagged");
tokio::select! {
// Inbound: AI message flow
inbound = bus.consume_inbound() => {
let Some(inbound) = inbound else {
tracing::warn!("Message processor stopping because inbound bus closed");
break;
};
match session_manager.handle_message(
&inbound.channel,
&inbound.sender_id,
&inbound.chat_id,
&inbound.content,
inbound.media,
).await {
Ok(crate::session::session::HandleResult::AgentResponse(content)) => {
let outbound = crate::bus::OutboundMessage {
channel: inbound.channel.clone(),
chat_id: inbound.chat_id.clone(),
content,
reply_to: None,
media: vec![],
metadata: inbound.forwarded_metadata,
delivery: None,
};
if let Err(e) = bus.publish_outbound(outbound).await {
tracing::error!(error = %e, "Failed to publish outbound");
}
}
Ok(crate::session::session::HandleResult::CommandOutput(content)) => {
let mut metadata = inbound.forwarded_metadata;
metadata.insert("_type".to_string(), "command".to_string());
let outbound = crate::bus::OutboundMessage {
channel: inbound.channel.clone(),
chat_id: inbound.chat_id.clone(),
content,
reply_to: None,
media: vec![],
metadata,
delivery: None,
};
if let Err(e) = bus.publish_outbound(outbound).await {
tracing::error!(error = %e, "Failed to publish outbound");
}
}
Ok(crate::session::session::HandleResult::AgentProcessing) => {
// Agent is processing in background; response will be
// sent via bus directly from the spawned task.
// The select loop remains free to handle subsequent
// messages (including slash commands).
}
Err(e) => {
tracing::error!(error = %e, "Failed to handle message");
}
}
}
// Control: session management operations
msg = bus.consume_control() => {
let Some(msg) = msg else {
tracing::warn!("Message processor stopping because control bus closed");
break;
};
Self::handle_control_message(&session_manager, msg).await;
}
Err(tokio::sync::broadcast::error::RecvError::Closed) => break,
}
}
});
router::spawn_message_routers(
bus.clone(),
session_manager,
self.task_supervisor.clone(),
self.admission.clone(),
);
// Spawn outbound dispatcher
let dispatcher = OutboundDispatcher::new(
bus_for_outbound,
self.channel_manager.clone(),
self.task_supervisor.clone(),
self.delivery_coordinator.write_locks(),
self.outbound_lanes.clone(),
);
self.task_supervisor
@ -487,18 +346,122 @@ impl GatewayState {
// Spawn scheduler background task if enabled
let scheduler_config = self.config.gateway.scheduler.clone().unwrap_or_default();
if scheduler_config.enabled {
let sched = Arc::new(Scheduler::with_admission(
let sched = Arc::new(Scheduler::new(
self.storage.clone(),
self.session_manager.clone(),
scheduler_config,
self.admission.clone(),
));
self.task_supervisor.spawn("scheduler", async move {
sched.run().await;
});
tracing::info!("Scheduler background task spawned");
}
Ok(())
}
/// Handle control messages (session management operations)
async fn handle_control_message(session_manager: &SessionManager, msg: ControlMessage) {
use crate::session::{SessionCommand::*, SessionEvent};
let reply_tx = msg.reply_tx;
let result: Result<SessionEvent, ChannelError> = match msg.op {
CreateDialog {
channel,
chat_id,
title,
} => session_manager
.create_dialog(&channel, &chat_id, title.as_deref())
.await
.map(|(session_id, title)| SessionEvent::DialogCreated { session_id, title })
.map_err(|e| ChannelError::Other(e.to_string())),
ListDialogs {
channel,
chat_id,
include_archived,
} => session_manager
.list_dialogs(&channel, &chat_id, include_archived)
.await
.map(|(dialogs, current_dialog_id)| SessionEvent::DialogList {
dialogs,
current_dialog_id,
})
.map_err(|e| ChannelError::Other(e.to_string())),
GetCurrentDialog { channel, chat_id } => session_manager
.get_current_dialog(&channel, &chat_id)
.await
.map(|session_id| SessionEvent::CurrentDialog { session_id })
.map_err(|e| ChannelError::Other(e.to_string())),
SwitchDialog {
channel,
chat_id,
dialog_id,
} => session_manager
.switch_dialog(&channel, &chat_id, &dialog_id)
.await
.map(|session_id| SessionEvent::DialogSwitched { session_id })
.map_err(|e| ChannelError::Other(e.to_string())),
GetDialogHistory { session_id, limit } => session_manager
.get_dialog_history(&session_id, limit)
.await
.map(|messages| SessionEvent::DialogHistory {
session_id,
messages,
})
.map_err(|e| ChannelError::Other(e.to_string())),
GetTaskPlan { session_id } => session_manager
.get_task_plan(&session_id)
.await
.map(|plan| SessionEvent::TaskPlan { session_id, plan })
.map_err(|e| ChannelError::Other(e.to_string())),
RenameDialog { session_id, title } => session_manager
.rename_dialog(&session_id, &title)
.await
.map(|()| SessionEvent::DialogRenamed { session_id, title })
.map_err(|e| ChannelError::Other(e.to_string())),
ArchiveDialog { session_id } => session_manager
.archive_dialog(&session_id)
.await
.map(|()| SessionEvent::DialogArchived { session_id })
.map_err(|e| ChannelError::Other(e.to_string())),
DeleteDialog { session_id } => session_manager
.delete_dialog(&session_id)
.await
.map(|()| SessionEvent::DialogDeleted { session_id })
.map_err(|e| ChannelError::Other(e.to_string())),
ClearHistory { session_id } => session_manager
.clear_dialog_history(&session_id)
.await
.map(|()| SessionEvent::HistoryCleared { session_id })
.map_err(|e| ChannelError::Other(e.to_string())),
GetSlashCommands {
channel: _,
chat_id: _,
} => {
let commands = session_manager.get_slash_commands().to_vec();
Ok(SessionEvent::SlashCommandsList { commands })
}
ExecuteSlashCommand {
command,
args,
channel,
chat_id,
current_session_id,
} => session_manager
.execute_slash_command(
&command,
args.as_deref(),
&channel,
&chat_id,
current_session_id.as_ref(),
)
.await
.map(|(new_id, msg)| SessionEvent::SlashCommandExecuted {
new_session_id: new_id,
message: msg,
})
.map_err(|e| ChannelError::Other(e.to_string())),
};
let _ = reply_tx.send(result).await;
}
}
@ -506,295 +469,37 @@ pub async fn run(
host: Option<String>,
port: Option<u16>,
) -> Result<(), Box<dyn std::error::Error>> {
STARTED.get_or_init(std::time::Instant::now);
let config_path = crate::config::resolve_default_config_path();
let config_load_context = Arc::new(Config::load_context());
let config = Config::load_for_startup(&config_path, &config_load_context)?;
// Initialize logging
logging::init_logging();
tracing::info!(config_path = %config_path.display(), "Starting PicoBot Gateway");
if !config.diagnostics.is_empty() {
let paths = config
.diagnostics
.iter()
.take(8)
.map(|diagnostic| diagnostic.path.as_str())
.collect::<Vec<_>>()
.join(", ");
tracing::warn!(
count = config.diagnostics.len(),
paths,
"Gateway started with recoverable configuration entries ignored"
);
}
tracing::info!("Starting PicoBot Gateway");
let mut reload_controller = reload::ReloadController::new(
config_load_context.process_env.clone(),
config_load_context.startup_cwd.clone(),
);
let config_write_lock = Arc::new(tokio::sync::Mutex::new(()));
let mut state = Arc::new(
GatewayState::from_config(
config,
config_path.clone(),
config_load_context.clone(),
config_write_lock.clone(),
reload_controller.handle.clone(),
true,
1,
)
.await?,
);
let state = Arc::new(GatewayState::new().await?);
// Start all channels (init already done in GatewayState::new)
state.channel_manager.start_all().await?;
// Start message processing (inbound processor + control processor + outbound dispatcher)
state.start_message_processing().await;
// CLI args override config file values
let bind_host = host.unwrap_or_else(|| state.config.gateway.host.clone());
let bind_port = port.unwrap_or(state.config.gateway.port);
let addr = format!("{}:{}", bind_host, bind_port);
let listener = std::net::TcpListener::bind(&addr)?;
listener.set_nonblocking(true)?;
tracing::info!(address = %addr, "Gateway listening");
let process_signal = wait_for_shutdown_signal();
tokio::pin!(process_signal);
let mut current_generation = 1_u64;
loop {
if let Err(error) = state.channel_manager.start_all().await {
reload_controller.set_failed(current_generation, error.to_string());
return Err(error.into());
}
state.start_message_processing().await?;
reload_controller.set_phase(current_generation, reload::ReloadPhase::Active);
let app = build_router(state.clone());
let generation_listener = TcpListener::from_std(listener.try_clone()?)?;
let generation_shutdown = tokio_util::sync::CancellationToken::new();
let shutdown_wait = generation_shutdown.clone();
let mut serve_task = tokio::spawn(async move {
axum::serve(
generation_listener,
app.into_make_service_with_connect_info::<SocketAddr>(),
)
.with_graceful_shutdown(async move { shutdown_wait.cancelled().await })
.await
});
let mut next_state = None;
let mut serve_result = None;
'generation: loop {
tokio::select! {
result = &mut serve_task => {
serve_result = Some(result);
state.connection_shutdown.cancel();
generation_shutdown.cancel();
break 'generation;
}
_ = &mut process_signal => {
tracing::info!("Shutdown signal received");
state.admission.close();
state.connection_shutdown.cancel();
generation_shutdown.cancel();
break 'generation;
}
request = reload_controller.receiver.recv() => {
let Some(request) = request else {
state.admission.close();
state.connection_shutdown.cancel();
generation_shutdown.cancel();
break 'generation;
};
let requested_generation = request.generation;
reload_controller.set_phase(requested_generation, reload::ReloadPhase::Preparing);
let candidate_result = {
let _write_guard = state.config_write_lock.lock().await;
reload::load_candidate(
&config_path,
&reload_controller.startup_process_env,
&reload_controller.startup_cwd,
&state.config,
&state.workspace_dir,
)
};
let candidate = match candidate_result {
Ok(candidate) => candidate,
Err(error) => {
reload_controller.set_failed(requested_generation, error.to_string());
let _ = request.response.send(Err(error));
continue;
}
};
let preparation = GatewayState::from_config(
candidate,
config_path.clone(),
config_load_context.clone(),
config_write_lock.clone(),
reload_controller.handle.clone(),
false,
requested_generation,
);
tokio::pin!(preparation);
let prepared = match tokio::select! {
result = &mut preparation => Some(result),
result = &mut serve_task => {
serve_result = Some(result);
None
}
_ = &mut process_signal => None,
} {
None => {
let error = reload::ReloadError::ShuttingDown;
reload_controller.set_failed(requested_generation, error.to_string());
let _ = request.response.send(Err(error));
state.admission.close();
state.connection_shutdown.cancel();
generation_shutdown.cancel();
break 'generation;
}
Some(result) => match result {
Ok(prepared) => Arc::new(prepared),
Err(error) => {
let error = reload::ReloadError::PreparationFailed(format!(
"configuration reload failed: {error}"
));
reload_controller.set_failed(requested_generation, error.to_string());
let _ = request.response.send(Err(error));
continue;
}
}};
state.admission.close();
reload_controller.set_phase(requested_generation, reload::ReloadPhase::Draining);
let message = "配置校验通过；Gateway 将在当前任务结束后切换到新配置。".to_string();
let _ = request.response.send(Ok(reload::ReloadAccepted {
generation: requested_generation,
message,
}));
let drain = async {
tokio::join!(
state.admission.wait_for_idle(),
state.session_manager.wait_until_idle(std::time::Duration::from_secs(60)),
)
};
let drain_result = tokio::select! {
result = tokio::time::timeout(std::time::Duration::from_secs(60), drain) => {
Some(matches!(result, Ok(((), true))))
}
result = &mut serve_task => {
serve_result = Some(result);
None
}
_ = &mut process_signal => None,
};
let Some(drained) = drain_result else {
reload_controller.set_failed(
requested_generation,
"gateway stopped while draining configuration reload",
);
state.connection_shutdown.cancel();
generation_shutdown.cancel();
break 'generation;
};
if !drained {
tracing::warn!("Reload drain period ended before all work became idle");
}
tracing::info!(path = %config_path.display(), "Switching to reloaded configuration");
reload_controller.set_phase(requested_generation, reload::ReloadPhase::Activating);
state.connection_shutdown.cancel();
generation_shutdown.cancel();
next_state = Some((prepared, requested_generation));
break 'generation;
}
}
}
if serve_result.is_none() {
serve_result =
match tokio::time::timeout(std::time::Duration::from_secs(10), &mut serve_task)
.await
{
Ok(result) => Some(result),
Err(_) => {
tracing::warn!("Aborting Axum generation after shutdown timeout");
serve_task.abort();
Some(serve_task.await)
}
};
}
if let Err(error) = state.channel_manager.stop_all().await {
tracing::error!(error = %error, "Failed to stop channels cleanly");
}
state.task_supervisor.cancel();
state
.task_supervisor
.shutdown(std::time::Duration::from_secs(10))
.await;
if let Some(result) = serve_result {
match result {
Ok(Ok(())) => {}
Ok(Err(error)) => return Err(error.into()),
Err(error) if error.is_cancelled() => {}
Err(error) => return Err(format!("Gateway server task failed: {error}").into()),
}
}
match next_state.take() {
Some((prepared, generation)) => {
state = prepared;
current_generation = generation;
}
None => break,
}
}
Ok(())
}
fn build_router(state: Arc<GatewayState>) -> Router {
let protected = Router::new()
.route("/api/health", routing::get(http::health_report))
.route("/api/health", routing::get(http::health))
.route(
"/api/config",
routing::get(http::get_config).put(http::put_config),
)
.route(
"/api/config/cleanup-invalid",
routing::post(http::cleanup_invalid_config_entries),
)
.route("/api/config/reload", routing::post(http::reload_config))
.route(
"/api/config/reload/status",
routing::get(http::reload_status),
)
.route(
"/api/profiles/{name}",
routing::get(http::get_profile).put(http::put_profile),
)
.route("/api/logs", routing::get(http::get_logs))
.route("/api/tasks", routing::get(http::get_tasks))
.route("/api/status", routing::get(http::get_status))
.route("/api/tools", routing::get(http::get_tools))
.route("/api/skills", routing::get(http::get_skills))
.route("/api/skills/{name}", routing::put(http::put_skill_enabled))
.route("/api/jobs", routing::get(http::get_jobs))
.route("/api/jobs/{id}/runs", routing::get(http::get_job_runs))
.route(
"/api/agents",
routing::get(http::list_agents).post(http::put_agent),
)
.route("/api/agents/options", routing::get(http::get_agent_options))
.route("/api/agents/{id}", routing::delete(http::delete_agent))
.route("/api/agent-runs", routing::get(http::get_agent_runs))
.route(
"/api/agent-runs/{id}",
routing::get(http::get_agent_run).post(http::cancel_agent_run),
)
.route(
"/api/agent-runs/{id}/events",
routing::get(http::get_agent_run_events),
)
.route("/api/memories", routing::get(http::get_memories))
.route(
"/api/memories/{key}",
routing::put(http::put_memory).delete(http::delete_memory),
)
.route(
"/api/chat/{client_id}/uploads",
routing::post(http::upload_file).layer(axum::extract::DefaultBodyLimit::disable()),
@ -804,24 +509,50 @@ fn build_router(state: Arc<GatewayState>) -> Router {
routing::get(http::download_attachment),
)
.route("/ws", routing::get(ws::ws_handler))
.route("/ws/logs", routing::get(ws::ws_logs_handler))
.route_layer(middleware::from_fn_with_state(
state.auth.clone(),
auth::require_auth,
));
Router::new()
let app = Router::new()
.route("/", routing::get(http::webui_index))
.route("/app.js", routing::get(http::webui_script))
.route("/styles.css", routing::get(http::webui_styles))
.route("/theme-init.js", routing::get(http::webui_theme_init))
.route("/fonts/{name}", routing::get(http::webui_font))
.route("/health", routing::get(http::health))
.route("/api/auth/status", routing::get(auth::status))
.route("/api/auth/pair", routing::post(auth::pair))
.route("/api/auth/code", routing::post(auth::issue_code))
.merge(protected)
.with_state(state)
.with_state(state.clone());
let addr = format!("{}:{}", bind_host, bind_port);
let listener = TcpListener::bind(&addr).await?;
tracing::info!(address = %addr, "Gateway listening");
let connection_shutdown = state.connection_shutdown.clone();
let serve_result = axum::serve(
listener,
app.into_make_service_with_connect_info::<SocketAddr>(),
)
.with_graceful_shutdown(async move {
wait_for_shutdown_signal().await;
tracing::info!("Shutdown signal received");
connection_shutdown.cancel();
})
.await;
// Stop external intake before waiting for internal work to finish.
if let Err(error) = state.channel_manager.stop_all().await {
tracing::error!(error = %error, "Failed to stop channels cleanly");
}
state.task_supervisor.cancel();
state
.task_supervisor
.shutdown(std::time::Duration::from_secs(10))
.await;
serve_result?;
Ok(())
}
async fn wait_for_shutdown_signal() {

512
src/gateway/reload.rs
View File

@ -1,512 +0,0 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::{Arc, RwLock};
use serde::Serialize;
use tokio::sync::{Notify, mpsc, oneshot};
use crate::config::Config;
const RELOAD_QUEUE_CAPACITY: usize = 8;
#[derive(Clone)]
pub(crate) struct RuntimeAdmission {
inner: Arc<AdmissionInner>,
}
struct AdmissionInner {
accepting: AtomicBool,
active: AtomicUsize,
idle: Notify,
}
pub(crate) struct ActivityGuard {
admission: RuntimeAdmission,
}
impl RuntimeAdmission {
pub fn open() -> Self {
Self {
inner: Arc::new(AdmissionInner {
accepting: AtomicBool::new(true),
active: AtomicUsize::new(0),
idle: Notify::new(),
}),
}
}
pub fn try_enter(&self) -> Option<ActivityGuard> {
if !self.inner.accepting.load(Ordering::Acquire) {
return None;
}
self.inner.active.fetch_add(1, Ordering::AcqRel);
if !self.inner.accepting.load(Ordering::Acquire) {
self.leave();
return None;
}
Some(ActivityGuard {
admission: self.clone(),
})
}
pub fn close(&self) {
self.inner.accepting.store(false, Ordering::Release);
if self.inner.active.load(Ordering::Acquire) == 0 {
self.inner.idle.notify_waiters();
}
}
pub fn is_accepting(&self) -> bool {
self.inner.accepting.load(Ordering::Acquire)
}
pub async fn wait_for_idle(&self) {
loop {
let notified = self.inner.idle.notified();
if self.inner.active.load(Ordering::Acquire) == 0 {
return;
}
notified.await;
}
}
fn leave(&self) {
if self.inner.active.fetch_sub(1, Ordering::AcqRel) == 1 {
self.inner.idle.notify_waiters();
}
}
}
impl Drop for ActivityGuard {
fn drop(&mut self) {
self.admission.leave();
}
}
pub(crate) struct ReloadRequest {
pub generation: u64,
pub response: oneshot::Sender<Result<ReloadAccepted, ReloadError>>,
}
#[derive(Clone)]
pub struct ReloadHandle {
sender: mpsc::Sender<ReloadRequest>,
next_generation: Arc<AtomicU64>,
pending: Arc<AtomicBool>,
status: Arc<RwLock<ReloadStatus>>,
}
pub(crate) struct ReloadController {
pub handle: ReloadHandle,
pub receiver: mpsc::Receiver<ReloadRequest>,
pub startup_process_env: HashMap<String, String>,
pub startup_cwd: PathBuf,
status: Arc<RwLock<ReloadStatus>>,
}
#[derive(Debug, Clone, Serialize)]
pub struct ReloadAccepted {
pub generation: u64,
pub message: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ReloadPhase {
Active,
Preparing,
Draining,
Activating,
Failed,
}
#[derive(Debug, Clone, Serialize)]
pub struct ReloadStatus {
pub generation: u64,
pub phase: ReloadPhase,
pub requested_at: Option<i64>,
pub activated_at: Option<i64>,
pub last_error: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ReloadError {
AlreadyPending,
ShuttingDown,
InvalidConfig(String),
ImmutableField(String),
PreparationFailed(String),
}
impl std::fmt::Display for ReloadError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::AlreadyPending => {
write!(formatter, "another configuration reload is already pending")
}
Self::ShuttingDown => write!(formatter, "gateway is shutting down"),
Self::InvalidConfig(error)
| Self::ImmutableField(error)
| Self::PreparationFailed(error) => formatter.write_str(error),
}
}
}
impl std::error::Error for ReloadError {}
impl ReloadController {
pub fn new(startup_process_env: HashMap<String, String>, startup_cwd: PathBuf) -> Self {
let (sender, receiver) = mpsc::channel(RELOAD_QUEUE_CAPACITY);
let status = Arc::new(RwLock::new(ReloadStatus {
generation: 1,
phase: ReloadPhase::Active,
requested_at: None,
activated_at: Some(chrono::Utc::now().timestamp_millis()),
last_error: None,
}));
Self {
handle: ReloadHandle {
sender,
next_generation: Arc::new(AtomicU64::new(2)),
pending: Arc::new(AtomicBool::new(false)),
status: status.clone(),
},
receiver,
startup_process_env,
startup_cwd,
status,
}
}
pub fn set_phase(&self, generation: u64, phase: ReloadPhase) {
let mut status = self
.status
.write()
.unwrap_or_else(|error| error.into_inner());
status.generation = generation;
status.phase = phase;
if phase == ReloadPhase::Preparing {
status.requested_at = Some(chrono::Utc::now().timestamp_millis());
status.activated_at = None;
status.last_error = None;
}
if phase == ReloadPhase::Active {
status.activated_at = Some(chrono::Utc::now().timestamp_millis());
self.handle.pending.store(false, Ordering::Release);
}
}
pub fn set_failed(&self, generation: u64, error: impl Into<String>) {
let mut status = self
.status
.write()
.unwrap_or_else(|error| error.into_inner());
status.generation = generation;
status.phase = ReloadPhase::Failed;
status.last_error = Some(error.into());
self.handle.pending.store(false, Ordering::Release);
}
}
impl ReloadHandle {
pub(crate) fn unavailable() -> Self {
let (sender, receiver) = mpsc::channel(1);
drop(receiver);
Self {
sender,
next_generation: Arc::new(AtomicU64::new(1)),
pending: Arc::new(AtomicBool::new(false)),
status: Arc::new(RwLock::new(ReloadStatus {
generation: 0,
phase: ReloadPhase::Failed,
requested_at: None,
activated_at: None,
last_error: Some("reload controller is unavailable".to_string()),
})),
}
}
pub async fn request(&self) -> Result<ReloadAccepted, ReloadError> {
self.pending
.compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
.map_err(|_| ReloadError::AlreadyPending)?;
let generation = self.next_generation.fetch_add(1, Ordering::Relaxed);
let (response, receiver) = oneshot::channel();
if let Err(error) = self.sender.try_send(ReloadRequest {
generation,
response,
}) {
self.pending.store(false, Ordering::Release);
return Err(match error {
mpsc::error::TrySendError::Full(_) => ReloadError::AlreadyPending,
mpsc::error::TrySendError::Closed(_) => ReloadError::ShuttingDown,
});
}
match receiver.await {
Ok(result) => result,
Err(_) => {
self.pending.store(false, Ordering::Release);
Err(ReloadError::ShuttingDown)
}
}
}
pub fn status(&self) -> ReloadStatus {
self.status
.read()
.unwrap_or_else(|error| error.into_inner())
.clone()
}
}
pub(crate) fn load_candidate(
config_path: &Path,
startup_process_env: &HashMap<String, String>,
startup_cwd: &std::path::Path,
current: &Config,
current_workspace: &std::path::Path,
) -> Result<Config, ReloadError> {
let mut candidate = Config::load_for_reload(config_path, startup_process_env, startup_cwd)
.map_err(|error| {
ReloadError::InvalidConfig(format!("configuration reload failed: {error}"))
})?;
candidate
.get_provider_config("default")
.map_err(|error| ReloadError::InvalidConfig(format!("invalid default agent: {error}")))?;
if let Some(feishu) = candidate.channels.get("feishu")
&& feishu.enabled
&& (feishu.app_id.trim().is_empty() || feishu.app_secret.trim().is_empty())
{
return Err(ReloadError::InvalidConfig(
"enabled channels.feishu requires non-empty app_id and app_secret".to_string(),
));
}
let mut candidate_workspace = crate::config::expand_path(&candidate.workspace_dir);
if candidate_workspace.is_relative() {
candidate_workspace = startup_cwd.join(candidate_workspace);
}
let candidate_workspace = candidate_workspace
.canonicalize()
.unwrap_or(candidate_workspace);
if current_workspace != candidate_workspace {
return Err(ReloadError::ImmutableField(
"workspace_dir cannot be reloaded; restart the gateway".to_string(),
));
}
candidate.workspace_dir = current_workspace.to_string_lossy().to_string();
if effective_db_path(current, current_workspace)
!= effective_db_path(&candidate, current_workspace)
{
return Err(ReloadError::ImmutableField(
"gateway.session_db_path cannot be reloaded; restart the gateway".to_string(),
));
}
if current.gateway.host != candidate.gateway.host
|| current.gateway.port != candidate.gateway.port
{
return Err(ReloadError::ImmutableField(
"gateway.host and gateway.port cannot be reloaded; restart the gateway".to_string(),
));
}
Ok(candidate)
}
fn effective_db_path(config: &Config, workspace: &Path) -> PathBuf {
let path = config
.gateway
.session_db_path
.as_deref()
.map(crate::config::expand_path)
.unwrap_or_else(crate::config::get_default_db_path);
let path = if path.is_relative() {
workspace.join(path)
} else {
path
};
path.canonicalize().unwrap_or(path)
}
#[cfg(test)]
mod tests {
use super::*;
fn config_json(workspace: &std::path::Path, model_id: &str) -> String {
serde_json::json!({
"providers": {
"provider": {
"type": "openai",
"base_url": "https://example.invalid/v1",
"api_key": "test"
}
},
"models": { "model": { "model_id": model_id } },
"agents": {
"default": { "provider": "provider", "model": "model" }
},
"workspace_dir": workspace
})
.to_string()
}
#[test]
fn candidate_accepts_runtime_changes_and_rejects_workspace_changes() {
let temp = tempfile::tempdir().unwrap();
let workspace = temp.path().join("workspace");
std::fs::create_dir_all(&workspace).unwrap();
let config_path = temp.path().join("config.json");
let current: Config = serde_json::from_str(&config_json(&workspace, "old-model")).unwrap();
std::fs::write(
&config_path,
config_json(std::path::Path::new("workspace"), "new-model"),
)
.unwrap();
let candidate = load_candidate(
&config_path,
&HashMap::new(),
temp.path(),
&current,
&workspace,
)
.unwrap();
assert_eq!(
candidate.get_provider_config("default").unwrap().model_id,
"new-model"
);
let other_workspace = temp.path().join("other");
std::fs::create_dir_all(&other_workspace).unwrap();
std::fs::write(&config_path, config_json(&other_workspace, "new-model")).unwrap();
let error = load_candidate(
&config_path,
&HashMap::new(),
temp.path(),
&current,
&workspace,
)
.unwrap_err();
assert!(
error
.to_string()
.contains("workspace_dir cannot be reloaded")
);
}
#[test]
fn default_database_path_equivalence_is_reloadable() {
let temp = tempfile::tempdir().unwrap();
let workspace = temp.path().join("workspace");
std::fs::create_dir_all(&workspace).unwrap();
let config_path = temp.path().join("config.json");
let current: Config = serde_json::from_str(&config_json(&workspace, "old-model")).unwrap();
// The implicit default (no session_db_path) is the config-dir data
// path; an explicit path resolving to the same file stays reloadable.
let default_db = crate::config::get_default_db_path();
let mut candidate: serde_json::Value =
serde_json::from_str(&config_json(&workspace, "new-model")).unwrap();
candidate["gateway"] = serde_json::json!({
"session_db_path": default_db.to_string_lossy()
});
std::fs::write(&config_path, serde_json::to_vec(&candidate).unwrap()).unwrap();
load_candidate(
&config_path,
&HashMap::new(),
temp.path(),
&current,
&workspace,
)
.unwrap();
// A different explicit path (e.g. the old workspace location) must be
// rejected as restart-only to protect the shared database identity.
let mut candidate: serde_json::Value =
serde_json::from_str(&config_json(&workspace, "new-model")).unwrap();
candidate["gateway"] = serde_json::json!({
"session_db_path": workspace.join("picobot.db").to_string_lossy()
});
std::fs::write(&config_path, serde_json::to_vec(&candidate).unwrap()).unwrap();
let error = load_candidate(
&config_path,
&HashMap::new(),
temp.path(),
&current,
&workspace,
)
.unwrap_err();
assert!(
error
.to_string()
.contains("session_db_path cannot be reloaded")
);
}
#[tokio::test]
async fn admission_closes_and_waits_for_existing_activity() {
let admission = RuntimeAdmission::open();
let activity = admission.try_enter().unwrap();
admission.close();
assert!(admission.try_enter().is_none());
let waiting = admission.wait_for_idle();
tokio::pin!(waiting);
assert!(
tokio::time::timeout(std::time::Duration::from_millis(10), &mut waiting)
.await
.is_err()
);
drop(activity);
tokio::time::timeout(std::time::Duration::from_secs(1), waiting)
.await
.unwrap();
}
#[tokio::test]
async fn one_reload_remains_pending_until_its_generation_is_terminal() {
let mut controller = ReloadController::new(HashMap::new(), PathBuf::from("."));
let handle = controller.handle.clone();
let first = tokio::spawn({
let handle = handle.clone();
async move { handle.request().await }
});
let request = controller.receiver.recv().await.unwrap();
assert_eq!(request.generation, 2);
request
.response
.send(Ok(ReloadAccepted {
generation: 2,
message: "accepted".to_string(),
}))
.unwrap();
first.await.unwrap().unwrap();
assert_eq!(
handle.request().await.unwrap_err(),
ReloadError::AlreadyPending
);
controller.set_phase(2, ReloadPhase::Active);
let next = tokio::spawn({
let handle = handle.clone();
async move { handle.request().await }
});
let request = controller.receiver.recv().await.unwrap();
assert_eq!(request.generation, 3);
controller.set_failed(3, "invalid candidate");
request
.response
.send(Err(ReloadError::InvalidConfig(
"invalid candidate".to_string(),
)))
.unwrap();
assert!(matches!(
next.await.unwrap(),
Err(ReloadError::InvalidConfig(_))
));
let status = handle.status();
assert_eq!(status.generation, 3);
assert_eq!(status.phase, ReloadPhase::Failed);
}
}

513
src/gateway/router.rs
View File

@ -1,513 +0,0 @@
use std::collections::HashMap;
use std::future::Future;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::{Semaphore, mpsc};
use crate::bus::{ControlMessage, InboundMessage, MessageBus, OutboundMessage};
use crate::channels::ChannelError;
use crate::channels::parse_slash_command;
use crate::gateway::reload::{ActivityGuard, RuntimeAdmission};
use crate::session::{SessionCommand, SessionEvent, SessionManager};
use crate::task_supervisor::TaskSupervisor;
const INBOUND_LANE_CAPACITY: usize = 32;
const INBOUND_LANE_IDLE_TIMEOUT: Duration = Duration::from_secs(300);
const CONTROL_MAX_IN_FLIGHT: usize = 64;
pub(super) fn spawn_message_routers(
bus: Arc<MessageBus>,
session_manager: Arc<SessionManager>,
supervisor: TaskSupervisor,
admission: RuntimeAdmission,
) {
spawn_inbound_router(
bus.clone(),
session_manager.clone(),
supervisor.clone(),
admission,
);
spawn_control_router(bus, session_manager, supervisor);
}
fn spawn_inbound_router(
bus: Arc<MessageBus>,
session_manager: Arc<SessionManager>,
supervisor: TaskSupervisor,
admission: RuntimeAdmission,
) {
let lane_supervisor = supervisor.clone();
supervisor.spawn("inbound-router", async move {
tracing::info!(lane_capacity = INBOUND_LANE_CAPACITY, "Inbound router started");
let mut lanes: HashMap<String, mpsc::Sender<AdmittedInbound>> = HashMap::new();
let mut messages_seen = 0_u64;
while let Some(inbound) = bus.consume_inbound().await {
messages_seen = messages_seen.wrapping_add(1);
if messages_seen.is_multiple_of(128) {
lanes.retain(|_, sender| !sender.is_closed());
}
let Some(activity) = admission.try_enter() else {
publish_command_output(
&bus,
inbound,
"Gateway 正在重新加载配置，请稍后重试。".to_string(),
)
.await;
continue;
};
let inbound = AdmittedInbound { inbound, activity };
// Stop must be able to invalidate a running worker even when an
// earlier slow slash command occupies this conversation's lane.
if is_priority_stop(&inbound.inbound.content) {
let request_bus = bus.clone();
let request_manager = session_manager.clone();
let task_name = format!(
"inbound-stop:{}:{}",
inbound.inbound.channel, inbound.inbound.chat_id
);
if !lane_supervisor.spawn(task_name, async move {
process_inbound(request_bus, request_manager, inbound).await;
}) {
break;
}
continue;
}
let key = conversation_key(&inbound.inbound.channel, &inbound.inbound.chat_id);
let mut sender = lanes.get(&key).cloned();
if sender.as_ref().is_none_or(mpsc::Sender::is_closed) {
let (new_sender, receiver) = mpsc::channel(INBOUND_LANE_CAPACITY);
if !spawn_inbound_lane(
&lane_supervisor,
bus.clone(),
session_manager.clone(),
inbound.inbound.channel.clone(),
inbound.inbound.chat_id.clone(),
receiver,
) {
tracing::warn!("Inbound router is stopping");
break;
}
lanes.insert(key.clone(), new_sender.clone());
sender = Some(new_sender);
}
let Some(sender) = sender else {
tracing::error!("Inbound lane creation did not produce a sender");
continue;
};
match sender.try_send(inbound) {
Ok(()) => {}
Err(mpsc::error::TrySendError::Full(inbound)) => {
tracing::warn!(channel = %inbound.inbound.channel, chat_id = %inbound.inbound.chat_id, "Inbound conversation lane is full");
publish_command_output(
&bus,
inbound.inbound,
"当前对话入口队列已满，请稍后重试。".to_string(),
)
.await;
}
Err(mpsc::error::TrySendError::Closed(inbound)) => {
// The lane may have exited on its idle boundary between the
// closed check and try_send. Recreate it once without
// dropping this input.
let (new_sender, receiver) = mpsc::channel(INBOUND_LANE_CAPACITY);
if !spawn_inbound_lane(
&lane_supervisor,
bus.clone(),
session_manager.clone(),
inbound.inbound.channel.clone(),
inbound.inbound.chat_id.clone(),
receiver,
) {
break;
}
lanes.insert(key, new_sender.clone());
if new_sender.try_send(inbound).is_err() {
tracing::error!("Failed to enqueue input into replacement lane");
}
}
}
}
tracing::warn!("Inbound router stopped because inbound bus closed");
});
}
fn spawn_inbound_lane(
supervisor: &TaskSupervisor,
bus: Arc<MessageBus>,
session_manager: Arc<SessionManager>,
channel: String,
chat_id: String,
receiver: mpsc::Receiver<AdmittedInbound>,
) -> bool {
supervisor.spawn(format!("inbound-lane:{channel}:{chat_id}"), async move {
run_ordered_lane(receiver, INBOUND_LANE_IDLE_TIMEOUT, move |inbound| {
process_inbound(bus.clone(), session_manager.clone(), inbound)
})
.await;
})
}
async fn run_ordered_lane<T, F, Fut>(
mut receiver: mpsc::Receiver<T>,
idle_timeout: Duration,
mut handler: F,
) where
T: Send + 'static,
F: FnMut(T) -> Fut,
Fut: Future<Output = ()>,
{
loop {
let item = match tokio::time::timeout(idle_timeout, receiver.recv()).await {
Ok(Some(item)) => item,
Ok(None) | Err(_) => break,
};
handler(item).await;
}
}
async fn process_inbound(
bus: Arc<MessageBus>,
session_manager: Arc<SessionManager>,
admitted: AdmittedInbound,
) {
let AdmittedInbound {
inbound,
activity: _activity,
} = admitted;
let result = session_manager.handle_message(&inbound).await;
match result {
Ok(crate::session::session::HandleResult::AgentResponse(content)) => {
publish_assistant_output(&bus, inbound, content).await;
}
Ok(crate::session::session::HandleResult::CommandOutput(content)) => {
publish_command_output(&bus, inbound, content).await;
}
Ok(crate::session::session::HandleResult::AgentProcessing) => {}
Err(error) => {
tracing::error!(channel = %inbound.channel, chat_id = %inbound.chat_id, error = %error, "Failed to handle inbound message");
publish_command_output(&bus, inbound, "消息处理失败，请稍后重试。".to_string()).await;
}
}
}
async fn publish_assistant_output(bus: &MessageBus, inbound: InboundMessage, content: String) {
publish_output(bus, inbound, content, false, false).await;
}
async fn publish_command_output(bus: &MessageBus, inbound: InboundMessage, content: String) {
publish_output(bus, inbound, content, true, true).await;
}
async fn publish_output(
bus: &MessageBus,
inbound: InboundMessage,
content: String,
command: bool,
confirmed: bool,
) {
let mut metadata = inbound.channel_context.private;
if command {
metadata.insert("_type".to_string(), "command".to_string());
}
let outbound = OutboundMessage {
channel: inbound.channel,
chat_id: inbound.chat_id,
content,
reply_to: inbound.channel_context.reply_to,
media: vec![],
metadata,
delivery: None,
};
let result = if confirmed {
bus.deliver_outbound(outbound).await
} else {
bus.publish_outbound(outbound).await
};
if let Err(error) = result {
tracing::error!(error = %error, "Failed to publish routed outbound message");
}
}
struct AdmittedInbound {
inbound: InboundMessage,
activity: ActivityGuard,
}
fn spawn_control_router(
bus: Arc<MessageBus>,
session_manager: Arc<SessionManager>,
supervisor: TaskSupervisor,
) {
let request_supervisor = supervisor.clone();
supervisor.spawn("control-router", async move {
tracing::info!(
max_in_flight = CONTROL_MAX_IN_FLIGHT,
"Control router started"
);
let permits = Arc::new(Semaphore::new(CONTROL_MAX_IN_FLIGHT));
loop {
let permit = match permits.clone().acquire_owned().await {
Ok(permit) => permit,
Err(_) => break,
};
let Some(message) = bus.consume_control().await else {
break;
};
let manager = session_manager.clone();
if !request_supervisor.spawn("control-request", async move {
let _permit = permit;
handle_control_message(&manager, message).await;
}) {
break;
}
}
tracing::warn!("Control router stopped because control bus closed");
});
}
async fn handle_control_message(session_manager: &SessionManager, message: ControlMessage) {
use SessionCommand::*;
let reply_tx = message.reply_tx;
let result: Result<SessionEvent, ChannelError> = match message.op {
CreateDialog {
channel,
chat_id,
title,
} => session_manager
.create_dialog(&channel, &chat_id, title.as_deref())
.await
.map(|(session_id, title)| SessionEvent::DialogCreated { session_id, title })
.map_err(|error| ChannelError::Other(error.to_string())),
ListDialogs {
channel,
chat_id,
include_archived,
} => session_manager
.list_dialogs(&channel, &chat_id, include_archived)
.await
.map(|(dialogs, current_dialog_id)| SessionEvent::DialogList {
dialogs,
current_dialog_id,
})
.map_err(|error| ChannelError::Other(error.to_string())),
GetCurrentDialog { channel, chat_id } => session_manager
.get_current_dialog(&channel, &chat_id)
.await
.map(|session_id| SessionEvent::CurrentDialog { session_id })
.map_err(|error| ChannelError::Other(error.to_string())),
SwitchDialog {
channel,
chat_id,
dialog_id,
} => session_manager
.switch_dialog(&channel, &chat_id, &dialog_id)
.await
.map(|session_id| SessionEvent::DialogSwitched { session_id })
.map_err(|error| ChannelError::Other(error.to_string())),
GetDialogHistory { session_id, limit } => session_manager
.get_dialog_history(&session_id, limit)
.await
.map(|messages| SessionEvent::DialogHistory {
session_id,
messages,
})
.map_err(|error| ChannelError::Other(error.to_string())),
GetTaskPlan { session_id } => session_manager
.get_task_plan(&session_id)
.await
.map(|plan| SessionEvent::TaskPlan { session_id, plan })
.map_err(|error| ChannelError::Other(error.to_string())),
GetSessionStats { session_id } => session_manager
.get_session_stats(&session_id)
.await
.map(|stats| SessionEvent::SessionStats { stats })
.map_err(|error| ChannelError::Other(error.to_string())),
GetAgentRuns {
session_id,
cursor,
limit,
} => session_manager
.get_agent_runs(&session_id, cursor, limit)
.await
.map_err(|error| ChannelError::Other(error.to_string())),
GetAgentRun { session_id, run_id } => session_manager
.get_agent_run(&session_id, &run_id)
.await
.map_err(|error| ChannelError::Other(error.to_string())),
RenameDialog { session_id, title } => session_manager
.rename_dialog(&session_id, &title)
.await
.map(|()| SessionEvent::DialogRenamed { session_id, title })
.map_err(|error| ChannelError::Other(error.to_string())),
ArchiveDialog { session_id } => session_manager
.archive_dialog(&session_id)
.await
.map(|()| SessionEvent::DialogArchived { session_id })
.map_err(|error| ChannelError::Other(error.to_string())),
DeleteDialog { session_id } => session_manager
.delete_dialog(&session_id)
.await
.map(|()| SessionEvent::DialogDeleted { session_id })
.map_err(|error| ChannelError::Other(error.to_string())),
ClearHistory { session_id } => session_manager
.clear_dialog_history(&session_id)
.await
.map(|()| SessionEvent::HistoryCleared { session_id })
.map_err(|error| ChannelError::Other(error.to_string())),
GetSlashCommands { .. } => Ok(SessionEvent::SlashCommandsList {
commands: session_manager.get_slash_commands().to_vec(),
}),
ExecuteSlashCommand {
command,
args,
channel,
chat_id,
current_session_id,
} => session_manager
.execute_slash_command(
&command,
args.as_deref(),
&channel,
&chat_id,
current_session_id.as_ref(),
)
.await
.map(
|(new_session_id, message)| SessionEvent::SlashCommandExecuted {
new_session_id,
message,
},
)
.map_err(|error| ChannelError::Other(error.to_string())),
};
let _ = reply_tx.send(result).await;
}
fn conversation_key(channel: &str, chat_id: &str) -> String {
format!("{channel}\0{chat_id}")
}
fn is_priority_stop(content: &str) -> bool {
parse_slash_command(content).is_some_and(|(command, _)| command == "stop")
}
#[cfg(test)]
mod tests {
use super::*;
use crate::bus::ChannelContext;
use std::collections::HashSet;
use tokio::sync::Notify;
#[test]
fn only_stop_bypasses_a_conversation_lane() {
assert!(is_priority_stop("/stop"));
assert!(is_priority_stop(" /stop "));
assert!(!is_priority_stop("/compact"));
assert!(!is_priority_stop("normal message"));
}
#[test]
fn conversation_keys_do_not_alias() {
let keys = HashSet::from([
conversation_key("a", "bc"),
conversation_key("ab", "c"),
conversation_key("a", "bd"),
]);
assert_eq!(keys.len(), 3);
}
#[tokio::test]
async fn routed_output_preserves_reply_target_and_private_context() {
let bus = MessageBus::new(2);
let inbound = InboundMessage {
channel: "test".to_string(),
sender_id: "user".to_string(),
chat_id: "chat".to_string(),
client_message_id: None,
content: "hello".to_string(),
received_at: 123,
media: vec![],
channel_context: ChannelContext {
reply_to: Some("parent".to_string()),
private: HashMap::from([("opaque".to_string(), "value".to_string())]),
durable_private: HashMap::new(),
},
};
let publish_task = tokio::spawn({
let bus = bus.clone();
async move { publish_command_output(&bus, inbound, "done".to_string()).await }
});
let output = bus.consume_outbound().await.unwrap();
assert_eq!(output.reply_to.as_deref(), Some("parent"));
assert_eq!(
output.metadata.get("opaque").map(String::as_str),
Some("value")
);
assert_eq!(
output.metadata.get("_type").map(String::as_str),
Some("command")
);
assert!(!publish_task.is_finished());
output.complete_delivery(crate::bus::DeliveryReceipt::Delivered);
publish_task.await.unwrap();
}
#[tokio::test]
async fn slow_conversation_lane_does_not_block_another_lane() {
let (slow_tx, slow_rx) = mpsc::channel(2);
let (fast_tx, fast_rx) = mpsc::channel(2);
let slow_started = Arc::new(Notify::new());
let release_slow = Arc::new(Notify::new());
let fast_finished = Arc::new(Notify::new());
let slow_task = tokio::spawn({
let slow_started = slow_started.clone();
let release_slow = release_slow.clone();
async move {
run_ordered_lane(slow_rx, Duration::from_secs(1), move |_| {
let slow_started = slow_started.clone();
let release_slow = release_slow.clone();
async move {
slow_started.notify_one();
release_slow.notified().await;
}
})
.await;
}
});
let fast_task = tokio::spawn({
let fast_finished = fast_finished.clone();
async move {
run_ordered_lane(fast_rx, Duration::from_secs(1), move |_| {
let fast_finished = fast_finished.clone();
async move { fast_finished.notify_one() }
})
.await;
}
});
slow_tx.send("slow").await.unwrap();
slow_started.notified().await;
fast_tx.send("fast").await.unwrap();
tokio::time::timeout(Duration::from_millis(100), fast_finished.notified())
.await
.expect("fast lane was blocked by unrelated slow lane");
release_slow.notify_one();
drop(slow_tx);
drop(fast_tx);
slow_task.await.unwrap();
fast_task.await.unwrap();
}
}

132
src/gateway/ws.rs
View File

@ -7,7 +7,6 @@ use axum::response::Response;
use futures_util::{SinkExt, StreamExt};
use serde::Deserialize;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::sync::mpsc;
use tokio::time::{Duration, timeout};
@ -43,8 +42,6 @@ async fn handle_socket(
client_id: Option<String>,
identity: super::auth::AuthIdentity,
) {
let _connection_guard = ConnectionGuard::new(state.ws_connections.clone());
// Create channel for sending outbound messages to this client
let (sender, mut receiver) = mpsc::channel::<WsOutbound>(100);
@ -59,12 +56,10 @@ async fn handle_socket(
let _ = sender
.send(WsOutbound::SessionEstablished {
session_id: session_id.clone(),
capabilities: {
let mut capabilities = vec!["turn_snapshots_v1".to_string()];
if state.uploads.enabled() {
capabilities.push("file_transfer_v1".to_string());
}
capabilities
capabilities: if state.uploads.enabled() {
vec!["file_transfer_v1".to_string()]
} else {
Vec::new()
},
})
.await;
@ -134,115 +129,6 @@ async fn handle_socket(
tracing::info!(session_id = %session_id, "CLI session ended");
}
struct ConnectionGuard {
counter: Arc<AtomicUsize>,
}
impl ConnectionGuard {
fn new(counter: Arc<AtomicUsize>) -> Self {
counter.fetch_add(1, Ordering::Relaxed);
Self { counter }
}
}
impl Drop for ConnectionGuard {
fn drop(&mut self) {
self.counter.fetch_sub(1, Ordering::Relaxed);
}
}
#[derive(Debug, Default, Deserialize)]
pub struct WsLogsQuery {
level: Option<String>,
search: Option<String>,
}
pub async fn ws_logs_handler(
ws: WebSocketUpgrade,
Query(query): Query<WsLogsQuery>,
Extension(_identity): Extension<super::auth::AuthIdentity>,
) -> Response {
ws.on_upgrade(|socket| async move {
handle_logs_socket(socket, query).await;
})
}
async fn handle_logs_socket(ws: WebSocket, query: WsLogsQuery) {
let Some(tx) = crate::logging::log_sender() else {
return;
};
let mut rx = tx.subscribe();
let (mut ws_sender, mut ws_receiver) = ws.split();
let min_level = query.level.as_deref().map(parse_min_level).unwrap_or(0);
let search = query
.search
.filter(|s| !s.is_empty())
.map(|s| s.to_ascii_lowercase());
loop {
tokio::select! {
result = rx.recv() => {
match result {
Ok(event) => {
if level_rank(&event.level) < min_level {
continue;
}
if let Some(needle) = &search
&& !event.message.to_ascii_lowercase().contains(needle)
&& !event.target.to_ascii_lowercase().contains(needle)
{
continue;
}
let json = serde_json::json!({
"ts": event.ts,
"level": event.level,
"target": event.target,
"message": event.message,
});
if ws_sender
.send(WsMessage::Text(json.to_string().into()))
.await
.is_err()
{
break;
}
}
Err(tokio::sync::broadcast::error::RecvError::Lagged(_)) => continue,
Err(tokio::sync::broadcast::error::RecvError::Closed) => break,
}
}
msg = ws_receiver.next() => {
match msg {
Some(Ok(WsMessage::Close(_))) | None => break,
_ => {}
}
}
}
}
}
fn parse_min_level(level: &str) -> u8 {
match level.to_ascii_uppercase().as_str() {
"DEBUG" => 1,
"INFO" => 2,
"WARN" => 3,
"ERROR" => 4,
_ => 0,
}
}
fn level_rank(level: &str) -> u8 {
match level {
"TRACE" => 0,
"DEBUG" => 1,
"INFO" => 2,
"WARN" => 3,
"ERROR" => 4,
_ => 2,
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -257,14 +143,4 @@ mod tests {
assert!(valid_client_id(Some("x".repeat(65))).is_none());
assert!(valid_client_id(Some(String::new())).is_none());
}
#[test]
fn connection_guard_tracks_its_scope() {
let connections = Arc::new(AtomicUsize::new(0));
{
let _guard = ConnectionGuard::new(connections.clone());
assert_eq!(connections.load(Ordering::Relaxed), 1);
}
assert_eq!(connections.load(Ordering::Relaxed), 0);
}
}

1103
src/health.rs
View File

File diff suppressed because it is too large Load Diff

2
src/lib.rs
View File

@ -3,9 +3,7 @@ pub mod bus;
pub mod channels;
pub mod client;
pub mod config;
pub mod delivery;
pub mod gateway;
pub mod health;
pub mod logging;
pub mod mcp;
pub mod memory;

79
src/logging.rs
View File

@ -1,93 +1,27 @@
use std::path::PathBuf;
use std::sync::OnceLock;
use tokio::sync::broadcast;
use tracing::field::{Field, Visit};
use tracing::{Event, Subscriber};
use tracing_appender::rolling::{RollingFileAppender, Rotation};
use tracing_subscriber::Layer;
use tracing_subscriber::layer::Context;
use tracing_subscriber::registry::LookupSpan;
use tracing_subscriber::{
EnvFilter, fmt, fmt::time::LocalTime, layer::SubscriberExt, util::SubscriberInitExt,
};
#[derive(Clone, Debug)]
pub struct LogEvent {
pub ts: String,
pub level: String,
pub target: String,
pub message: String,
}
static LOG_TX: OnceLock<broadcast::Sender<LogEvent>> = OnceLock::new();
pub fn log_sender() -> Option<broadcast::Sender<LogEvent>> {
LOG_TX.get().cloned()
}
struct MessageVisitor(String);
impl Visit for MessageVisitor {
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
if field.name() == "message" {
self.0 = format!("{value:?}");
}
}
fn record_str(&mut self, field: &Field, value: &str) {
if field.name() == "message" {
self.0 = value.to_string();
}
}
}
struct BroadcastLayer;
impl<S> Layer<S> for BroadcastLayer
where
S: Subscriber + for<'a> LookupSpan<'a>,
{
fn on_event(&self, event: &Event<'_>, _ctx: Context<'_, S>) {
let Some(tx) = LOG_TX.get() else {
return;
};
if tx.receiver_count() == 0 {
return;
}
let mut visitor = MessageVisitor(String::new());
event.record(&mut visitor);
let ts = time::OffsetDateTime::now_local()
.unwrap_or_else(|_| time::OffsetDateTime::now_utc())
.format(&time::format_description::well_known::Rfc3339)
.unwrap_or_default();
let _ = tx.send(LogEvent {
ts,
level: event.metadata().level().to_string(),
target: event.metadata().target().to_string(),
message: visitor.0,
});
}
}
/// Get the default log directory path: ~/.picobot/logs
pub fn get_default_log_dir() -> PathBuf {
let home = dirs::home_dir().unwrap_or_else(|| PathBuf::from("."));
home.join(".picobot").join("logs")
}
/// Get the default config file path: ~/.picobot/config.json
pub fn get_default_config_path() -> PathBuf {
let home = dirs::home_dir().unwrap_or_else(|| PathBuf::from("."));
home.join(".picobot").join("config.json")
}
/// Initialize logging with file appender
/// Logs are written to ~/.picobot/logs/ with daily rotation
pub fn init_logging() {
let (tx, _) = broadcast::channel::<LogEvent>(1024);
let _ = LOG_TX.set(tx);
let log_dir = get_default_log_dir();
// Create log directory if it doesn't exist
if !log_dir.exists()
&& let Err(e) = std::fs::create_dir_all(&log_dir)
{
@ -98,8 +32,10 @@ pub fn init_logging() {
);
}
// Create file appender with daily rotation
let file_appender = RollingFileAppender::new(Rotation::DAILY, &log_dir, "picobot.log");
// Build subscriber with both console and file output
let env_filter = EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info"));
let file_layer = fmt::layer()
@ -119,7 +55,6 @@ pub fn init_logging() {
.with(env_filter)
.with(console_layer)
.with(file_layer)
.with(BroadcastLayer)
.init();
tracing::info!("Logging initialized. Log directory: {}", log_dir.display());

83
src/main.rs
View File

@ -19,7 +19,7 @@ enum ServiceCommand {
#[derive(Parser)]
#[command(name = "picobot")]
#[command(about = "A CLI chatbot", long_about = None)]
#[command(version)]
#[command(version = "1.1.1")]
enum Command {
/// Connect to gateway
Chat {
@ -30,23 +30,6 @@ enum Command {
#[arg(long)]
pair_code: Option<String>,
},
/// Send one prompt through the gateway, print the final response, and exit
Run {
/// Prompt text; when omitted, read it from stdin
prompt: Vec<String>,
/// Gateway WebSocket or HTTP URL
#[arg(long)]
gateway_url: Option<String>,
/// Maximum time to wait for the turn, in seconds
#[arg(long, default_value_t = 300)]
timeout: u64,
/// Print the terminal turn as one JSON object
#[arg(long)]
json: bool,
/// Print phase and tool progress to stderr
#[arg(long)]
verbose: bool,
},
/// Start gateway server
Gateway {
/// Host to bind to
@ -56,18 +39,6 @@ enum Command {
#[arg(long)]
port: Option<u16>,
},
/// Reload a running gateway's configuration
Reload {
/// Gateway WebSocket or HTTP URL
#[arg(long)]
gateway_url: Option<String>,
},
/// Check PicoBot runtime and configured external dependencies
Health {
/// Print the structured report as JSON
#[arg(long)]
json: bool,
},
/// Generate a one-time browser pairing code from the local gateway
Pair {
/// Gateway WebSocket or HTTP URL
@ -106,56 +77,9 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
.unwrap_or_else(|| "ws://127.0.0.1:19876/ws".to_string());
picobot::client::run(&url, pair_code.as_deref()).await?;
}
Command::Run {
prompt,
gateway_url,
timeout,
json,
verbose,
} => {
if timeout == 0 {
return Err("--timeout must be greater than zero".into());
}
let config = picobot::config::Config::load_default().ok();
let url = gateway_url
.or_else(|| config.as_ref().map(|c| c.client.gateway_url.clone()))
.unwrap_or_else(|| "ws://127.0.0.1:19876/ws".to_string());
let prompt = picobot::client::read_run_prompt(prompt)?;
picobot::client::run_once(
&url,
prompt,
picobot::client::RunOptions {
timeout: std::time::Duration::from_secs(timeout),
json,
verbose,
},
)
.await?;
}
Command::Gateway { host, port } => {
picobot::gateway::run(host, port).await?;
}
Command::Reload { gateway_url } => {
let config = picobot::config::Config::load_default().ok();
let url = gateway_url
.or_else(|| config.as_ref().map(|c| c.client.gateway_url.clone()))
.unwrap_or_else(|| "ws://127.0.0.1:19876/ws".to_string());
println!("{}", picobot::client::reload_gateway(&url).await?);
}
Command::Health { json } => {
let report = match picobot::config::Config::load_default() {
Ok(config) => picobot::health::HealthService::new(config).check().await,
Err(error) => picobot::health::HealthReport::configuration_error(error.to_string()),
};
if json {
println!("{}", serde_json::to_string_pretty(&report)?);
} else {
println!("{}", report.render_text());
}
if !report.is_usable() {
std::process::exit(1);
}
}
Command::Pair {
gateway_url,
revoke_all,
@ -177,10 +101,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
})?;
let response = reqwest::Client::new()
.post(endpoint)
.header(
picobot::gateway::auth::ADMIN_TOKEN_HEADER,
admin_token.trim(),
)
.header("X-Picobot-Admin-Token", admin_token.trim())
.send()
.await?;
let status = response.status();

63
src/mcp/mod.rs
View File

@ -5,38 +5,22 @@ use std::sync::{Arc, Mutex};
use anyhow::Context;
use http::{HeaderName, HeaderValue};
use rmcp::model::{CallToolRequestParams, ContentBlock};
use rmcp::model::{CallToolRequestParams, RawContent};
use rmcp::transport::streamable_http_client::StreamableHttpClientTransportConfig;
use rmcp::transport::{StreamableHttpClientTransport, TokioChildProcess};
use rmcp::{Peer, RoleClient, ServiceExt};
use tokio::process::Command;
use crate::config::{McpConfig, McpServerConfig, McpToolSettings, McpTransport};
use crate::config::{McpConfig, McpServerConfig, McpTransport};
use crate::tools::ToolResult;
pub use tool_wrapper::McpToolWrapper;
/// Reserved prefix for tool names registered from MCP servers.
pub const MCP_TOOL_PREFIX: &str = "mcp_";
/// Build the public ToolRegistry name for an MCP-discovered tool.
pub fn qualified_tool_name(server_name: &str, tool_name: &str) -> String {
format!("{MCP_TOOL_PREFIX}{server_name}_{tool_name}")
}
/// Whether a ToolRegistry name belongs to the MCP namespace.
pub fn is_mcp_tool_name(name: &str) -> bool {
name.starts_with(MCP_TOOL_PREFIX)
}
/// Status of a single MCP tool.
#[derive(Debug, Clone)]
pub struct McpToolStatus {
pub name: String,
pub description: String,
pub read_only: bool,
pub exclusive: bool,
pub concurrency_safe: bool,
}
/// Status of a single MCP server.
@ -101,14 +85,14 @@ impl McpConnection {
fn extract_text(result: &rmcp::model::CallToolResult) -> String {
let mut parts = Vec::new();
for content in &result.content {
match content {
ContentBlock::Text(text) => {
match &**content {
RawContent::Text(text) => {
parts.push(text.text.clone());
}
ContentBlock::Image(image) => {
RawContent::Image(image) => {
parts.push(format!("[image: {}]", image.mime_type,));
}
ContentBlock::Resource(resource) => match &resource.resource {
RawContent::Resource(resource) => match &resource.resource {
rmcp::model::ResourceContents::TextResourceContents { text, .. } => {
parts.push(format!(
"[resource text: {}]",
@ -118,7 +102,6 @@ fn extract_text(result: &rmcp::model::CallToolResult) -> String {
rmcp::model::ResourceContents::BlobResourceContents { uri, .. } => {
parts.push(format!("[resource blob: {}]", uri));
}
_ => parts.push("[unsupported resource]".to_string()),
},
_ => {
parts.push("[unsupported content]".to_string());
@ -138,7 +121,6 @@ pub struct ToolInfo {
pub description: String,
pub schema: serde_json::Value,
pub connection: Arc<McpConnection>,
pub settings: McpToolSettings,
}
pub async fn connect_all(config: &McpConfig) -> Vec<ToolInfo> {
@ -146,14 +128,6 @@ pub async fn connect_all(config: &McpConfig) -> Vec<ToolInfo> {
let mut server_statuses = Vec::new();
for server_config in &config.servers {
if !server_config.enabled {
tracing::info!(
server = %server_config.name,
"MCP server disabled by config, skipping"
);
continue;
}
let transport_str = match server_config.transport {
McpTransport::Stdio => "stdio",
McpTransport::Sse => "sse",
@ -172,15 +146,9 @@ pub async fn connect_all(config: &McpConfig) -> Vec<ToolInfo> {
);
let tool_statuses: Vec<McpToolStatus> = server_tools
.iter()
.map(|(name, desc, _)| {
let settings = server_config.tool_settings_for(name);
McpToolStatus {
.map(|(name, desc, _)| McpToolStatus {
name: name.clone(),
description: desc.clone(),
read_only: settings.read_only,
exclusive: settings.exclusive,
concurrency_safe: settings.concurrency_safe(),
}
})
.collect();
server_statuses.push(McpServerStatus {
@ -191,14 +159,12 @@ pub async fn connect_all(config: &McpConfig) -> Vec<ToolInfo> {
tools: tool_statuses,
});
for (orig_name, desc, schema) in server_tools {
let settings = server_config.tool_settings_for(&orig_name);
tools.push(ToolInfo {
server_name: server_config.name.clone(),
tool_name: orig_name,
description: desc,
schema,
connection: connection.clone(),
settings,
});
}
}
@ -326,18 +292,3 @@ async fn list_tools(
})
.collect())
}
#[cfg(test)]
mod tests {
use super::{is_mcp_tool_name, qualified_tool_name};
#[test]
fn qualified_tool_names_use_the_mcp_namespace() {
assert_eq!(
qualified_tool_name("filesystem", "read_file"),
"mcp_filesystem_read_file"
);
assert!(is_mcp_tool_name("mcp_filesystem_read_file"));
assert!(!is_mcp_tool_name("file_read"));
}
}

16
src/mcp/tool_wrapper.rs
View File

@ -2,10 +2,9 @@ use std::sync::Arc;
use async_trait::async_trait;
use crate::config::McpToolSettings;
use crate::tools::{Tool, ToolResult};
use super::{McpConnection, qualified_tool_name};
use super::McpConnection;
pub struct McpToolWrapper {
full_name: String,
@ -13,7 +12,6 @@ pub struct McpToolWrapper {
parameters_schema: serde_json::Value,
original_tool_name: String,
connection: Arc<McpConnection>,
settings: McpToolSettings,
}
impl McpToolWrapper {
@ -23,15 +21,13 @@ impl McpToolWrapper {
description: String,
parameters_schema: serde_json::Value,
connection: Arc<McpConnection>,
settings: McpToolSettings,
) -> Self {
Self {
full_name: qualified_tool_name(server_name, &original_tool_name),
full_name: format!("{}__{}", server_name, original_tool_name),
description,
parameters_schema,
original_tool_name,
connection,
settings,
}
}
}
@ -50,14 +46,6 @@ impl Tool for McpToolWrapper {
self.parameters_schema.clone()
}
fn read_only(&self) -> bool {
self.settings.read_only
}
fn exclusive(&self) -> bool {
self.settings.exclusive
}
async fn execute(&self, args: serde_json::Value) -> anyhow::Result<ToolResult> {
self.connection
.call_tool(&self.original_tool_name, args)

118
src/memory/mod.rs
View File

@ -1,4 +1,3 @@
pub mod recall;
pub mod types;
use std::sync::Arc;
@ -7,8 +6,6 @@ use uuid::Uuid;
use crate::storage::Storage;
pub use types::{ConsolidationFact, ConsolidationResult, MemoryCategory, MemoryEntry};
use recall::RecallConfig;
/// MemoryManager provides high-level memory operations.
/// Wraps the Storage SQLite layer with semantic methods.
#[derive(Clone)]
@ -16,7 +13,6 @@ pub struct MemoryManager {
storage: Arc<Storage>,
pub consolidation_provider: String,
pub consolidation_model: String,
recall: RecallConfig,
}
impl MemoryManager {
@ -29,57 +25,9 @@ impl MemoryManager {
storage,
consolidation_provider,
consolidation_model,
recall: RecallConfig::default(),
}
}
/// Override the automatic-recall knobs (derived from `config.memory`).
pub fn with_recall(mut self, config: RecallConfig) -> Self {
self.recall = config;
self
}
/// Automatic per-turn recall for context injection.
///
/// Returns up to `recall.limit` Knowledge entries that clear both the
/// relevance and combined-score gates. The search is bounded by a hard
/// timeout: on expiry, error, or an empty normalized query it returns an
/// empty vector so the turn is never delayed by memory lookup.
pub async fn recall_for_context(&self, query: &str) -> Vec<MemoryEntry> {
let terms = recall::normalize_query(query);
if terms.is_empty() {
return Vec::new();
}
let candidate_limit = (self.recall.limit * recall::CANDIDATE_FACTOR)
.min(recall::MAX_CANDIDATES)
.max(self.recall.limit);
let candidates = tokio::time::timeout(
self.recall.timeout,
self.storage.search_memories_by_terms(
&terms,
Some(&MemoryCategory::Knowledge),
None,
candidate_limit,
),
)
.await;
let candidates = match candidates {
Ok(Ok(entries)) => entries,
Ok(Err(error)) => {
tracing::debug!(error = %error, "memory recall failed; skipping injection");
return Vec::new();
}
Err(_) => {
tracing::debug!("memory recall timed out; skipping injection");
return Vec::new();
}
};
let now_ms = chrono::Utc::now().timestamp_millis();
let mut ranked = recall::rank_and_gate(candidates, &terms, &self.recall, now_ms);
ranked.truncate(self.recall.limit);
ranked
}
/// Store or update a memory entry. Generates timestamp and UUID.
pub async fn store(
&self,
@ -140,10 +88,7 @@ impl MemoryManager {
/// Check if the memory system has any entries (for testing/health check).
pub async fn is_empty(&self) -> Result<bool, crate::storage::StorageError> {
self.storage
.list_memories(None, None, 1)
.await
.map(|entries| entries.is_empty())
self.recall("*", 1, None, None).await.map(|r| r.is_empty())
}
}
@ -312,65 +257,4 @@ mod tests {
assert_eq!(scoped[0].key, "tl_a");
assert_eq!(scoped[0].session_id.as_deref(), Some("chan:chat:dialog_a"));
}
#[tokio::test]
async fn recall_for_context_returns_empty_for_stopword_only_query() {
let (mm, _dir) = setup_memory_manager().await;
mm.store(
"user_pref",
"user prefers python",
MemoryCategory::Knowledge,
None,
Some(0.9),
)
.await
.unwrap();
// A query made entirely of stopwords must not surface unrelated memories.
assert!(mm.recall_for_context("你好吗请吧").await.is_empty());
}
#[tokio::test]
async fn recall_for_context_recalls_relevant_and_gates_irrelevant() {
let (mm, _dir) = setup_memory_manager().await;
mm.store(
"user_pref",
"user prefers python for scripting",
MemoryCategory::Knowledge,
None,
Some(0.9),
)
.await
.unwrap();
mm.store(
"favorite_color",
"user favorite color is blue",
MemoryCategory::Knowledge,
None,
Some(0.9),
)
.await
.unwrap();
let hits = mm.recall_for_context("帮我写一个 python 脚本").await;
assert_eq!(hits.len(), 1);
assert_eq!(hits[0].key, "user_pref");
}
#[tokio::test]
async fn recall_for_context_respects_limit() {
let (mm, _dir) = setup_memory_manager().await;
for i in 0..8 {
mm.store(
&format!("pref_{i}"),
format!("user preference about python number {i}").as_str(),
MemoryCategory::Knowledge,
None,
Some(0.9),
)
.await
.unwrap();
}
let hits = mm.recall_for_context("python preference").await;
assert!(hits.len() <= 5);
}
}

242
src/memory/recall.rs
View File

@ -1,242 +0,0 @@
//! Deterministic, relevance-gated memory recall for per-turn context injection.
//!
//! This is the *automatic* recall path (as opposed to the user-facing
//! `memory_recall` tool, which performs an unfiltered search). It tokenizes
//! and normalizes the query, retrieves a small candidate set through FTS5,
//! then ranks and gates candidates by lexical relevance, importance, and
//! recency. Everything here is bounded and synchronous so a turn is never
//! delayed by a slow search.
use std::collections::HashSet;
use std::sync::OnceLock;
use std::time::Duration;
use jieba_rs::Jieba;
use super::MemoryEntry;
/// Weights for the combined score (sum to 1.0 for a clean 0..1 range).
const WEIGHT_RELEVANCE: f64 = 0.5;
const WEIGHT_IMPORTANCE: f64 = 0.3;
const WEIGHT_RECENCY: f64 = 0.2;
/// Candidate set is `limit * CANDIDATE_FACTOR`, capped by `MAX_CANDIDATES`.
pub(crate) const CANDIDATE_FACTOR: usize = 4;
pub(crate) const MAX_CANDIDATES: usize = 50;
/// Hard bounds on the query to keep tokenization and the FTS5 MATCH cheap.
const MAX_QUERY_CHARS: usize = 512;
const MAX_TERMS: usize = 12;
const STOPWORDS: &[&str] = &[
"的", "了", "是", "在", "我", "你", "他", "她", "它", "们", "吗", "呢", "啊", "吧", "这", "那",
"什么", "怎么", "为什么", "帮我", "一下", "请", "要", "想", "能", "可以", "这个", "那个", "今天",
"现在", "以及", "还是", "因为", "所以", "如果", "但是", "就", "都", "也", "还", "很", "被", "把",
"和", "与", "或", "有", "没有", "一个", "一种", "一些", "哪些", "如何",
"a", "an", "the", "to", "of", "for", "and", "or", "what", "how", "please", "can", "do", "is",
"are", "my", "you", "i", "we", "this", "that", "me", "it", "on", "in", "at", "with", "about",
];
/// Runtime knobs for automatic recall, derived from `config.memory`.
#[derive(Debug, Clone)]
pub struct RecallConfig {
pub limit: usize,
pub min_relevance: f64,
pub min_score: f64,
pub recency_half_life_days: f64,
pub timeout: Duration,
}
impl Default for RecallConfig {
fn default() -> Self {
Self {
limit: 5,
min_relevance: 0.25,
min_score: 0.25,
recency_half_life_days: 30.0,
timeout: Duration::from_millis(1000),
}
}
}
impl RecallConfig {
pub fn from_memory_config(memory: &crate::config::MemoryConfig) -> Self {
Self {
limit: memory.recall_limit,
min_relevance: memory.recall_min_relevance,
min_score: memory.recall_min_score,
recency_half_life_days: memory.recall_recency_half_life_days as f64,
timeout: Duration::from_millis(memory.recall_timeout_ms),
}
}
}
fn jieba() -> &'static Jieba {
static INSTANCE: OnceLock<Jieba> = OnceLock::new();
INSTANCE.get_or_init(Jieba::new)
}
/// Tokenize with jieba and drop tokens that are too short to be meaningful.
pub(crate) fn tokenize(query: &str) -> Vec<String> {
jieba()
.cut(query, true)
.into_iter()
.map(|token| token.word)
.filter(|word| word.len() > 1 || word.bytes().any(|b| b > 127))
.map(str::to_string)
.collect()
}
/// Normalize a user query into deduplicated, stopword-free search terms.
pub(crate) fn normalize_query(query: &str) -> Vec<String> {
let bounded: String = if query.chars().count() > MAX_QUERY_CHARS {
query.chars().take(MAX_QUERY_CHARS).collect()
} else {
query.to_string()
};
let mut seen = HashSet::new();
let mut terms: Vec<String> = Vec::new();
for term in tokenize(&bounded) {
let lower = term.to_lowercase();
if STOPWORDS.contains(&lower.as_str()) {
continue;
}
if seen.insert(lower) {
terms.push(term);
}
}
terms.truncate(MAX_TERMS);
terms
}
/// Fraction of distinct query terms present in the entry's key or content.
fn relevance(entry: &MemoryEntry, terms: &[String]) -> f64 {
let key = entry.key.to_lowercase();
let content = entry.content.to_lowercase();
let matched = terms
.iter()
.filter(|term| {
let term = term.to_lowercase();
key.contains(&term) || content.contains(&term)
})
.count();
matched as f64 / terms.len() as f64
}
/// Exponential recency decay in [0,1]; 1.0 for just-updated, 0.0 for very old.
fn recency(updated_at: &str, half_life_days: f64, now_ms: i64) -> f64 {
let Ok(dt) = chrono::DateTime::parse_from_rfc3339(updated_at) else {
return 0.0;
};
let half_life = half_life_days.max(1.0);
let age_ms = (now_ms - dt.timestamp_millis()).max(0);
let age_days = age_ms as f64 / 86_400_000.0;
(-age_days / half_life).exp()
}
/// Rank candidates by combined score and drop those below the gates.
pub(crate) fn rank_and_gate(
candidates: Vec<MemoryEntry>,
terms: &[String],
config: &RecallConfig,
now_ms: i64,
) -> Vec<MemoryEntry> {
let mut scored: Vec<(f64, f64, MemoryEntry)> = Vec::with_capacity(candidates.len());
for entry in candidates {
let rel = relevance(&entry, terms);
if rel < config.min_relevance {
continue;
}
let importance = entry.importance.clamp(0.0, 1.0);
let rec = recency(&entry.updated_at, config.recency_half_life_days, now_ms);
let score = WEIGHT_RELEVANCE * rel + WEIGHT_IMPORTANCE * importance + WEIGHT_RECENCY * rec;
if score < config.min_score {
continue;
}
scored.push((score, importance, entry));
}
scored.sort_by(|a, b| {
b.0.partial_cmp(&a.0)
.unwrap_or(std::cmp::Ordering::Equal)
.then_with(|| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal))
});
scored.into_iter().map(|(_, _, entry)| entry).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::memory::MemoryCategory;
fn entry(key: &str, content: &str, importance: f64, updated_at: &str) -> MemoryEntry {
MemoryEntry {
id: format!("id-{key}"),
key: key.to_string(),
content: content.to_string(),
category: MemoryCategory::Knowledge,
importance,
session_id: None,
created_at: updated_at.to_string(),
updated_at: updated_at.to_string(),
}
}
#[test]
fn normalize_query_removes_stopwords_and_deduplicates() {
let terms = normalize_query("请帮我写一个 Python 脚本，用 python 处理");
assert!(!terms.contains(&"请".to_string()));
assert!(!terms.contains(&"一个".to_string()));
// "python" appears twice but is deduplicated (case-insensitive).
assert_eq!(terms.iter().filter(|t| t.to_lowercase() == "python").count(), 1);
}
#[test]
fn normalize_query_returns_empty_for_all_stopwords() {
assert!(normalize_query("呢吗啊吧").is_empty());
}
#[test]
fn relevance_counts_distinct_term_matches() {
let e = entry("py", "the user prefers python", 0.5, "2024-01-01T00:00:00Z");
let terms = vec!["python".to_string(), "rust".to_string()];
assert!((relevance(&e, &terms) - 0.5).abs() < 1e-9);
}
#[test]
fn gate_drops_low_relevance() {
let e = entry("unrelated", "something else entirely", 1.0, "2024-01-01T00:00:00Z");
let terms = vec!["python".to_string(), "script".to_string()];
let cfg = RecallConfig::default();
let out = rank_and_gate(vec![e], &terms, &cfg, 1_704_067_200_000);
assert!(out.is_empty());
}
#[test]
fn gate_keeps_high_relevance_and_sorts_by_score() {
let strong = entry("strong", "python script", 0.5, "2024-01-01T00:00:00Z");
let weak = entry("weak", "a python note", 0.9, "2024-01-01T00:00:00Z");
let terms = vec!["python".to_string(), "script".to_string()];
let cfg = RecallConfig::default();
let out = rank_and_gate(vec![weak, strong], &terms, &cfg, 1_704_067_200_000);
assert_eq!(out.len(), 2);
assert_eq!(out[0].key, "strong");
}
#[test]
fn importance_breaks_relevance_ties() {
let a = entry("a", "python script", 0.2, "2024-01-01T00:00:00Z");
let b = entry("b", "python script", 0.9, "2024-01-01T00:00:00Z");
let terms = vec!["python".to_string(), "script".to_string()];
let cfg = RecallConfig::default();
let out = rank_and_gate(vec![a, b], &terms, &cfg, 1_704_067_200_000);
assert_eq!(out[0].key, "b");
}
#[test]
fn recency_decays_with_age() {
let recent = recency("2024-01-10T00:00:00Z", 30.0, 1_704_067_200_000);
let old = recency("2023-01-10T00:00:00Z", 30.0, 1_704_067_200_000);
assert!(recent > old);
assert!((0.0..=1.0).contains(&recent));
}
}

302
src/observability/metrics.rs
View File

@ -1,302 +0,0 @@
use std::collections::{HashMap, VecDeque};
use std::sync::atomic::{AtomicU64, Ordering::Relaxed};
use std::sync::{Arc, Mutex, OnceLock};
use crate::providers::Usage;
const WINDOW: usize = 100;
const DEGRADE_THRESHOLD: usize = 3;
const DEGRADE_WINDOW: usize = 10;
#[derive(Default)]
struct ProviderStat {
model: String,
tokens_in: u64,
tokens_out: u64,
cost: f64,
calls: u64,
last_latency_ms: u64,
latencies: VecDeque<u64>,
recent_results: VecDeque<bool>,
}
pub struct Metrics {
tokens_in: AtomicU64,
tokens_out: AtomicU64,
turns: AtomicU64,
tool_calls: AtomicU64,
per_tool: Mutex<HashMap<String, u64>>,
turn_latencies: Mutex<VecDeque<u64>>,
providers: Mutex<HashMap<String, ProviderStat>>,
}
#[derive(serde::Serialize)]
pub struct MetricsSnapshot {
pub tokens_in: u64,
pub tokens_out: u64,
pub cost: f64,
pub turns: u64,
pub tool_calls: u64,
pub turn_latency_p95_ms: u64,
pub per_tool: HashMap<String, u64>,
pub providers: Vec<ProviderSnapshot>,
}
#[derive(serde::Serialize)]
pub struct ProviderSnapshot {
pub name: String,
pub model: String,
pub status: String,
pub latency_ms: u64,
pub latencies: Vec<u64>,
pub tokens_in: u64,
pub tokens_out: u64,
pub cost: f64,
}
impl Default for Metrics {
fn default() -> Self {
Self::new()
}
}
impl Metrics {
pub fn new() -> Self {
Self {
tokens_in: AtomicU64::new(0),
tokens_out: AtomicU64::new(0),
turns: AtomicU64::new(0),
tool_calls: AtomicU64::new(0),
per_tool: Mutex::new(HashMap::new()),
turn_latencies: Mutex::new(VecDeque::new()),
providers: Mutex::new(HashMap::new()),
}
}
pub fn record_turn(&self, usage: Option<&Usage>, latency_ms: u64) {
self.turns.fetch_add(1, Relaxed);
if let Some(u) = usage {
self.tokens_in
.fetch_add(u64::from(u.prompt_tokens), Relaxed);
self.tokens_out
.fetch_add(u64::from(u.completion_tokens), Relaxed);
}
let mut q = self
.turn_latencies
.lock()
.unwrap_or_else(|e| e.into_inner());
q.push_back(latency_ms);
while q.len() > WINDOW {
q.pop_front();
}
}
pub fn record_tool_call(&self, name: &str, _success: bool) {
self.tool_calls.fetch_add(1, Relaxed);
*self
.per_tool
.lock()
.unwrap_or_else(|e| e.into_inner())
.entry(name.to_string())
.or_insert(0) += 1;
}
pub fn record_provider(
&self,
name: &str,
model: &str,
cost: Option<f64>,
latency_ms: u64,
is_error: bool,
) {
let mut map = self.providers.lock().unwrap_or_else(|e| e.into_inner());
let stat = map.entry(name.to_string()).or_insert_with(|| ProviderStat {
model: model.to_string(),
..Default::default()
});
stat.model = model.to_string();
stat.calls += 1;
stat.last_latency_ms = latency_ms;
stat.latencies.push_back(latency_ms);
while stat.latencies.len() > WINDOW {
stat.latencies.pop_front();
}
stat.recent_results.push_back(is_error);
while stat.recent_results.len() > DEGRADE_WINDOW {
stat.recent_results.pop_front();
}
if let Some(c) = cost {
stat.cost += c;
}
}
pub fn record_provider_tokens(&self, name: &str, usage: &Usage) {
let mut map = self.providers.lock().unwrap_or_else(|e| e.into_inner());
let stat = map.entry(name.to_string()).or_default();
stat.tokens_in += u64::from(usage.prompt_tokens);
stat.tokens_out += u64::from(usage.completion_tokens);
}
pub fn tool_call_count(&self, name: &str) -> u64 {
*self
.per_tool
.lock()
.unwrap_or_else(|e| e.into_inner())
.get(name)
.unwrap_or(&0)
}
pub fn snapshot(&self) -> MetricsSnapshot {
let latencies = self
.turn_latencies
.lock()
.unwrap_or_else(|e| e.into_inner());
let p95 = percentile_95(&latencies);
let providers = self.providers.lock().unwrap_or_else(|e| e.into_inner());
let mut cost = 0.0;
let provider_snaps = providers
.iter()
.map(|(name, s)| {
cost += s.cost;
let errors = s.recent_results.iter().filter(|e| **e).count();
ProviderSnapshot {
name: name.clone(),
model: s.model.clone(),
status: if s.recent_results.len() >= DEGRADE_WINDOW
&& errors >= DEGRADE_THRESHOLD
{
"degraded".to_string()
} else {
"ok".to_string()
},
latency_ms: s.last_latency_ms,
latencies: s.latencies.iter().copied().collect(),
tokens_in: s.tokens_in,
tokens_out: s.tokens_out,
cost: s.cost,
}
})
.collect();
MetricsSnapshot {
tokens_in: self.tokens_in.load(Relaxed),
tokens_out: self.tokens_out.load(Relaxed),
cost,
turns: self.turns.load(Relaxed),
tool_calls: self.tool_calls.load(Relaxed),
turn_latency_p95_ms: p95,
per_tool: self
.per_tool
.lock()
.unwrap_or_else(|e| e.into_inner())
.clone(),
providers: provider_snaps,
}
}
}
fn percentile_95(values: &VecDeque<u64>) -> u64 {
if values.is_empty() {
return 0;
}
let mut sorted: Vec<u64> = values.iter().copied().collect();
sorted.sort_unstable();
let idx = ((sorted.len() as f64 * 0.95).ceil() as usize)
.saturating_sub(1)
.min(sorted.len() - 1);
sorted[idx]
}
static GLOBAL: OnceLock<Arc<Metrics>> = OnceLock::new();
pub fn global_metrics() -> Arc<Metrics> {
GLOBAL.get_or_init(|| Arc::new(Metrics::new())).clone()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn records_turns_tokens_and_p95() {
let m = Metrics::new();
for i in 1..=100u64 {
m.record_turn(
Some(&Usage {
prompt_tokens: 10,
completion_tokens: 20,
total_tokens: 30,
..Default::default()
}),
i,
);
}
let s = m.snapshot();
assert_eq!(s.turns, 100);
assert_eq!(s.tokens_in, 1000);
assert_eq!(s.tokens_out, 2000);
assert!(s.turn_latency_p95_ms >= 95);
}
#[test]
fn records_per_tool_counts() {
let m = Metrics::new();
m.record_tool_call("bash", true);
m.record_tool_call("bash", true);
m.record_tool_call("read_file", false);
let s = m.snapshot();
assert_eq!(s.tool_calls, 3);
assert_eq!(s.per_tool.get("bash"), Some(&2));
assert_eq!(s.per_tool.get("read_file"), Some(&1));
}
#[test]
fn records_provider_tokens_and_cost() {
let m = Metrics::new();
m.record_provider("openai", "gpt-4o", Some(0.05), 120, false);
m.record_provider_tokens(
"openai",
&Usage {
prompt_tokens: 100,
completion_tokens: 50,
total_tokens: 150,
..Default::default()
},
);
let s = m.snapshot();
assert_eq!(s.cost, 0.05);
let p = s.providers.iter().find(|p| p.name == "openai").unwrap();
assert_eq!(p.tokens_in, 100);
assert_eq!(p.tokens_out, 50);
assert_eq!(p.latency_ms, 120);
assert_eq!(p.status, "ok");
}
#[test]
fn derives_provider_status() {
let m = Metrics::new();
for _ in 0..7 {
m.record_provider("openai", "gpt-4o", None, 100, false);
}
for _ in 0..3 {
m.record_provider("openai", "gpt-4o", None, 100, true);
}
let s = m.snapshot();
let p = s.providers.iter().find(|p| p.name == "openai").unwrap();
assert_eq!(p.status, "degraded");
}
#[test]
fn tool_call_count_accessor() {
let m = Metrics::new();
assert_eq!(m.tool_call_count("bash"), 0);
m.record_tool_call("bash", true);
assert_eq!(m.tool_call_count("bash"), 1);
}
#[test]
fn global_metrics_returns_same_instance() {
let a = global_metrics();
let b = global_metrics();
assert!(Arc::ptr_eq(&a, &b));
}
}

23
src/observability/mod.rs
View File

@ -3,8 +3,6 @@
//! This module provides an Observer pattern for emitting and collecting
//! telemetry events during agent execution.
pub mod metrics;
use std::time::Duration;
use crate::bus::MediaRef;
@ -62,9 +60,7 @@ pub struct ToolExecutionOutcome {
/// How long the tool took to execute.
pub duration: Duration,
/// Structured media returned by the tool for the next model iteration.
pub model_media_refs: Vec<MediaRef>,
/// Structured media that should be attached to the final user reply.
pub reply_media_refs: Vec<MediaRef>,
pub media_refs: Vec<MediaRef>,
}
impl ToolExecutionOutcome {
@ -75,24 +71,18 @@ impl ToolExecutionOutcome {
success: true,
error_reason: None,
duration: Duration::ZERO,
model_media_refs: Vec::new(),
reply_media_refs: Vec::new(),
media_refs: Vec::new(),
}
}
/// Create a successful outcome carrying processed structured artifacts.
pub fn success_with_output(
output: String,
model_media_refs: Vec<MediaRef>,
reply_media_refs: Vec<MediaRef>,
) -> Self {
/// Create a successful outcome carrying structured media artifacts.
pub fn success_with_media(output: String, media_refs: Vec<MediaRef>) -> Self {
Self {
output,
success: true,
error_reason: None,
duration: Duration::ZERO,
model_media_refs,
reply_media_refs,
media_refs,
}
}
@ -103,8 +93,7 @@ impl ToolExecutionOutcome {
success: false,
error_reason,
duration: Duration::ZERO,
model_media_refs: Vec::new(),
reply_media_refs: Vec::new(),
media_refs: Vec::new(),
}
}
}

512
src/protocol.rs
View File

@ -37,173 +37,6 @@ pub struct MessageAttachment {
pub mime_type: String,
}
/// Bounded client projection of a durable Agent run. Sensitive internals
/// (budget, signal contract, delivery context, provider state) never leave
/// the gateway; the full result is available through the HTTP API.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AgentRunView {
pub id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub parent_run_id: Option<String>,
pub agent_id: String,
pub provider_name: String,
pub model_id: String,
pub mode: String,
pub depth: u16,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub plan_item_id: Option<String>,
pub status: String,
/// Task prompt (bounded for client delivery).
pub task: String,
/// Bounded result excerpt; full content is loaded on demand.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub result: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
pub tool_calls_count: i64,
pub iterations: i64,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub deadline_at: Option<i64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub started_at: Option<i64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub finished_at: Option<i64>,
pub created_at: i64,
pub updated_at: i64,
}
/// Bounded client projection of a durable inbox event.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AgentEventView {
pub id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub run_id: Option<String>,
pub event_type: String,
pub delivery: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub severity: Option<String>,
/// Structured payload, bounded; signal summaries and completion status
/// only. The client must treat the payload as untrusted data.
pub payload_json: String,
pub status: String,
pub attempt_count: i64,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub last_error: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub consumed_at: Option<i64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub superseded_at: Option<i64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dead_lettered_at: Option<i64>,
pub created_at: i64,
}
/// Serialized transcript message for a single Agent run, exposed through the
/// HTTP detail endpoint. `reasoning_content` is client-visible here but
/// `provider_state` is never included.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AgentTranscriptMessage {
pub id: String,
pub run_id: String,
pub seq: i64,
pub role: String,
pub content: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_name: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<Vec<crate::providers::ToolCall>>,
pub created_at: i64,
}
impl From<crate::storage::agent_run::AgentRunMessageRecord> for AgentTranscriptMessage {
fn from(record: crate::storage::agent_run::AgentRunMessageRecord) -> Self {
let tool_calls = record
.tool_calls_json
.as_deref()
.and_then(|json| serde_json::from_str(json).ok());
Self {
id: record.id,
run_id: record.run_id,
seq: record.seq,
role: record.role,
content: record.content,
reasoning_content: record.reasoning_content,
tool_call_id: record.tool_call_id,
tool_name: record.tool_name,
tool_calls,
created_at: record.created_at,
}
}
}
impl AgentRunView {
pub fn from_record(
record: &crate::storage::agent_run::AgentRunRecord,
max_result_chars: usize,
) -> Self {
let task = truncate(&record.task, 2_000);
let result = record
.result
.as_deref()
.map(|result| truncate(result, max_result_chars));
Self {
id: record.id.clone(),
parent_run_id: record.parent_run_id.clone(),
agent_id: record.agent_id.clone(),
provider_name: record.provider_name.clone(),
model_id: record.model_id.clone(),
mode: record.mode.as_str().to_string(),
depth: u16::try_from(record.depth).unwrap_or(u16::MAX),
plan_item_id: record.plan_item_id.clone(),
status: record.status.as_str().to_string(),
task,
result,
error: record.error.clone(),
tool_calls_count: record.tool_calls_count,
iterations: record.iterations,
deadline_at: Some(record.deadline_at),
started_at: record.started_at,
finished_at: record.finished_at,
created_at: record.created_at,
updated_at: record.updated_at,
}
}
}
impl AgentEventView {
pub fn from_record(record: &crate::storage::agent_inbox::AgentInboxEventRecord) -> Self {
Self {
id: record.id.clone(),
run_id: record.run_id.clone(),
event_type: record.event_type.as_str().to_string(),
delivery: record.delivery.as_str().to_string(),
severity: record.severity.clone(),
payload_json: record.payload_json.clone(),
status: record.status.as_str().to_string(),
attempt_count: record.attempt_count,
last_error: record.last_error.clone(),
consumed_at: record.consumed_at,
superseded_at: record.superseded_at,
dead_lettered_at: record.dead_lettered_at,
created_at: record.created_at,
}
}
}
fn truncate(value: &str, max: usize) -> String {
if value.chars().count() <= max {
value.to_string()
} else {
let mut truncated: String = value.chars().take(max).collect();
truncated.push('…');
truncated
}
}
impl MessageAttachment {
pub fn from_media_ref(index: usize, media_ref: &crate::bus::MediaRef) -> Self {
let name = std::path::Path::new(&media_ref.path)
@ -229,10 +62,6 @@ pub struct HistoryMessage {
pub seq: i64,
pub role: String,
pub content: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<String>,
#[serde(default)]
pub completion_status: crate::bus::CompletionStatus,
pub created_at: i64,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
@ -242,63 +71,6 @@ pub struct HistoryMessage {
pub tool_calls: Option<Vec<crate::providers::ToolCall>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub attachments: Vec<MessageAttachment>,
#[serde(default)]
pub turn_origin: crate::bus::TurnOrigin,
}
impl From<crate::bus::CommittedMessage> for HistoryMessage {
fn from(message: crate::bus::CommittedMessage) -> Self {
let attachments = message
.media_refs
.iter()
.enumerate()
.map(|(index, media_ref)| MessageAttachment::from_media_ref(index, media_ref))
.collect();
Self {
id: message.id,
seq: message.seq,
role: message.role,
content: message.content,
reasoning_content: message.reasoning_content,
completion_status: message.completion_status,
created_at: message.created_at,
tool_call_id: message.tool_call_id,
tool_name: message.tool_name,
tool_calls: message.tool_calls,
attachments,
turn_origin: message.turn_origin,
}
}
}
impl HistoryMessage {
pub fn from_message_meta(message: crate::storage::message::MessageMeta) -> Self {
let attachments = message
.media_refs
.as_deref()
.and_then(|refs| serde_json::from_str::<Vec<crate::bus::MediaRef>>(refs).ok())
.unwrap_or_default()
.iter()
.enumerate()
.map(|(index, media_ref)| MessageAttachment::from_media_ref(index, media_ref))
.collect();
Self {
id: message.id,
seq: message.seq,
role: message.role,
content: message.content,
reasoning_content: message.reasoning_content,
completion_status: message.completion_status,
created_at: message.created_at,
tool_call_id: message.tool_call_id,
tool_name: message.tool_name,
tool_calls: message
.tool_calls
.and_then(|calls| serde_json::from_str(&calls).ok()),
attachments,
turn_origin: message.turn_origin,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@ -309,10 +81,6 @@ pub enum WsInbound {
content: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
upload_ids: Vec<String>,
/// Stable id generated by the client for optimistic-message
/// reconciliation. It is optional for older clients and channels.
#[serde(default, skip_serializing_if = "Option::is_none")]
client_message_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
channel: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
@ -347,8 +115,6 @@ pub enum WsInbound {
},
#[serde(rename = "get_session_plan")]
GetSessionPlan { session_id: String },
#[serde(rename = "get_session_stats")]
GetSessionStats { session_id: String },
#[serde(rename = "rename_session")]
RenameSession {
#[serde(default, skip_serializing_if = "Option::is_none")]
@ -367,16 +133,6 @@ pub enum WsInbound {
},
#[serde(rename = "get_slash_commands")]
GetSlashCommands,
#[serde(rename = "get_agent_runs")]
GetAgentRuns {
session_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
cursor: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
limit: Option<u32>,
},
#[serde(rename = "get_agent_run")]
GetAgentRun { session_id: String, run_id: String },
#[serde(rename = "ping")]
Ping,
}
@ -384,16 +140,6 @@ pub enum WsInbound {
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type")]
pub enum WsOutbound {
#[serde(rename = "turn_updated")]
TurnUpdated {
snapshot: crate::session::TurnSnapshot,
},
#[serde(rename = "turn_committed")]
TurnCommitted {
session_id: String,
history_revision: i64,
messages: Vec<HistoryMessage>,
},
#[serde(rename = "assistant_response")]
AssistantResponse {
id: String,
@ -436,8 +182,6 @@ pub enum WsOutbound {
session_id: String,
plan: Option<crate::work::TaskPlan>,
},
#[serde(rename = "session_stats")]
SessionStats { stats: crate::session::SessionStats },
#[serde(rename = "plan_updated")]
PlanUpdated {
session_id: String,
@ -455,26 +199,6 @@ pub enum WsOutbound {
HistoryCleared { session_id: String },
#[serde(rename = "slash_commands_list")]
SlashCommandsList { commands: Vec<SlashCommandInfo> },
#[serde(rename = "session_agent_runs")]
SessionAgentRuns {
session_id: String,
revision: i64,
runs: Vec<AgentRunView>,
#[serde(default, skip_serializing_if = "Option::is_none")]
next_cursor: Option<String>,
},
#[serde(rename = "agent_run_updated")]
AgentRunUpdated {
session_id: String,
revision: i64,
run: AgentRunView,
},
#[serde(rename = "agent_event_updated")]
AgentEventUpdated {
session_id: String,
revision: i64,
event: AgentEventView,
},
#[serde(rename = "pong")]
Pong,
#[serde(rename = "command_executed")]
@ -498,239 +222,3 @@ pub fn serialize_inbound(msg: &WsInbound) -> Result<String, serde_json::Error> {
pub fn serialize_outbound(msg: &WsOutbound) -> Result<String, serde_json::Error> {
serde_json::to_string(msg)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::session::{TurnId, TurnPhase, TurnState, TurnStatus};
#[test]
fn turn_updated_serializes_as_one_complete_snapshot_frame() {
let frame = WsOutbound::TurnUpdated {
snapshot: TurnState {
id: TurnId("turn-1".into()),
session_id: "cli_chat:client:dialog".into(),
message_id: "message-1".into(),
revision: 7,
status: TurnStatus::Running,
phase: TurnPhase::Responding,
blocks: Vec::new(),
usage: None,
error: None,
},
};
let json = serialize_outbound(&frame).unwrap();
let value: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(value["type"], "turn_updated");
assert_eq!(value["snapshot"]["revision"], 7);
assert_eq!(value["snapshot"]["status"], "running");
}
#[test]
fn turn_committed_serializes_revision_and_durable_delta() {
let frame = WsOutbound::TurnCommitted {
session_id: "session".to_string(),
history_revision: 7,
messages: vec![HistoryMessage {
id: "message".to_string(),
seq: 7,
role: "assistant".to_string(),
content: "done".to_string(),
reasoning_content: None,
completion_status: crate::bus::CompletionStatus::Completed,
created_at: 1,
tool_call_id: None,
tool_name: None,
tool_calls: None,
attachments: Vec::new(),
turn_origin: crate::bus::TurnOrigin::User,
}],
};
let value = serde_json::to_value(frame).unwrap();
assert_eq!(value["type"], "turn_committed");
assert_eq!(value["history_revision"], 7);
assert_eq!(value["messages"][0]["id"], "message");
}
#[test]
fn user_input_preserves_optional_client_message_id() {
let inbound = parse_inbound(
r#"{"type":"user_input","content":"hello","client_message_id":"client-1"}"#,
)
.unwrap();
match inbound {
WsInbound::UserInput {
client_message_id,
upload_ids,
..
} => {
assert_eq!(client_message_id.as_deref(), Some("client-1"));
assert!(upload_ids.is_empty());
}
other => panic!("unexpected frame: {other:?}"),
}
let serialized = serialize_inbound(&WsInbound::UserInput {
content: "hello".to_string(),
upload_ids: Vec::new(),
client_message_id: Some("client-1".to_string()),
channel: None,
chat_id: None,
sender_id: None,
})
.unwrap();
assert!(serialized.contains(r#""client_message_id":"client-1""#));
}
#[test]
fn history_defaults_new_reasoning_fields_for_old_frames() {
let message: HistoryMessage = serde_json::from_value(serde_json::json!({
"id": "message",
"seq": 1,
"role": "assistant",
"content": "answer",
"created_at": 1,
"attachments": []
}))
.unwrap();
assert_eq!(message.reasoning_content, None);
assert_eq!(
message.completion_status,
crate::bus::CompletionStatus::Completed
);
}
#[test]
fn session_stats_request_and_response_use_structured_frames() {
let inbound =
parse_inbound(r#"{"type":"get_session_stats","session_id":"cli_chat:client:dialog"}"#)
.unwrap();
assert!(matches!(inbound, WsInbound::GetSessionStats { .. }));
let stats = crate::session::SessionStats {
session_id: "cli_chat:client:dialog".into(),
title: "stats".into(),
provider: "provider".into(),
model: "model".into(),
user_message_count: 1,
history_message_count: 2,
lifetime_usage: crate::session::LifetimeUsage {
input_tokens: 100,
output_tokens: 20,
total_tokens: 120,
cached_input_tokens: Some(40),
request_count: 1,
turn_count: 1,
tracked_since: Some(1),
},
context: crate::session::ContextUsage {
configured_window_tokens: 128_000,
effective_window_tokens: 128_000,
configured_reserve_tokens: 16_384,
reserve_tokens: 16_384,
configured_keep_recent_tokens: 20_000,
effective_keep_recent_tokens: 20_000,
used_tokens: 100,
remaining_tokens: 127_900,
compression_threshold_tokens: 111_616,
source: crate::session::ContextUsageSource::Observed,
last_observed_prompt_tokens: Some(90),
observed_at: Some(1),
active_checkpoint_id: None,
checkpoint_generation: 0,
checkpoint_tokens_before: None,
checkpoint_tokens_after: None,
checkpoint_degraded: false,
},
created_at: 1,
last_active_at: 2,
updated_at: 3,
};
let value = serde_json::to_value(WsOutbound::SessionStats { stats }).unwrap();
assert_eq!(value["type"], "session_stats");
assert_eq!(value["stats"]["context"]["source"], "observed");
assert_eq!(value["stats"]["lifetime_usage"]["input_tokens"], 100);
}
#[test]
fn agent_run_and_event_views_serialize_without_sensitive_fields() {
let run = crate::storage::agent_run::AgentRunRecord {
id: "run-1".to_string(),
root_session_id: "cli:test:d1".to_string(),
root_turn_id: None,
parent_run_id: None,
caller_agent_id: "ROOT".to_string(),
caller_scope_id: "turn-1".to_string(),
idempotency_key: None,
agent_id: "researcher".to_string(),
definition_hash: "hash".to_string(),
provider_profile: "research".to_string(),
provider_name: "openai-test".to_string(),
model_id: "model-x".to_string(),
mode: crate::storage::agent_run::AgentRunMode::Background,
depth: 1,
plan_item_id: None,
execution_id: "run-1".to_string(),
task: "analyze the logs".to_string(),
context_json: Some("sensitive caller context".to_string()),
budget_json: r#"{"remaining_runs":3}"#.to_string(),
signal_contract_json: Some("secret contract".to_string()),
signal_delivery: None,
status: crate::storage::agent_run::AgentRunStatus::Completed,
result: Some("r".repeat(10_000)),
error: None,
prompt_tokens: None,
completion_tokens: None,
cost: None,
tool_calls_count: 3,
iterations: 2,
runtime_generation: 1,
attempt: 1,
completion_slot_reserved: false,
deadline_at: 1000,
revision: 7,
started_at: Some(10),
finished_at: Some(20),
created_at: 5,
updated_at: 20,
};
let view = AgentRunView::from_record(&run, 2_000);
let value = serde_json::to_value(&view).unwrap();
assert_eq!(value["id"], "run-1");
assert_eq!(value["status"], "completed");
assert!(!value.as_object().unwrap().contains_key("context_json"));
assert!(!value.as_object().unwrap().contains_key("budget_json"));
assert!(
!value
.as_object()
.unwrap()
.contains_key("signal_contract_json")
);
assert!(!value.as_object().unwrap().contains_key("execution_id"));
// The result is bounded for client delivery.
assert!(value["result"].as_str().unwrap().chars().count() <= 2_001);
let message = WsOutbound::AgentRunUpdated {
session_id: "cli:test:d1".to_string(),
revision: 7,
run: view,
};
let value = serde_json::to_value(message).unwrap();
assert_eq!(value["type"], "agent_run_updated");
assert_eq!(value["revision"], 7);
let inbound: WsInbound =
parse_inbound(r#"{"type":"get_agent_runs","session_id":"cli:test:d1","limit":10}"#)
.unwrap();
assert!(matches!(
inbound,
WsInbound::GetAgentRuns {
limit: Some(10),
..
}
));
}
}

710
src/providers/anthropic.rs
View File

@ -1,19 +1,12 @@
use async_trait::async_trait;
use futures_util::stream;
use reqwest::Client;
use serde::Serialize;
use serde_json::Value;
use std::collections::{BTreeMap, HashMap, VecDeque};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::time::Duration;
use thiserror::Error;
use super::stream::SseFramer;
use super::traits::Usage;
use super::{
ChatCompletionRequest, DynProviderError, FinishReason, LLMProvider, Message, ProviderChunk,
ProviderStream, Tool,
};
use crate::bus::{ProviderReasoningState, message::ContentBlock};
use super::{ChatCompletionRequest, ChatCompletionResponse, LLMProvider, Message, Tool, ToolCall};
use crate::bus::message::ContentBlock;
use crate::storage::Storage;
use std::sync::Arc;
@ -134,7 +127,6 @@ struct AnthropicRequest {
messages: Vec<AnthropicMessage>,
max_tokens: u32,
temperature: Option<f32>,
stream: bool,
#[serde(skip_serializing_if = "Option::is_none")]
tools: Option<Vec<AnthropicTool>>,
#[serde(flatten)]
@ -148,59 +140,20 @@ struct AnthropicMessage {
}
fn convert_messages(messages: &[Message]) -> Vec<AnthropicMessage> {
let mut converted = Vec::with_capacity(messages.len());
let mut index = 0;
while index < messages.len() {
let message = &messages[index];
// Anthropic requires all tool results for one assistant tool-use turn
// to be carried in a single `role: user` content array. Steering is
// represented as a normal user message in PicoBot history, so merge
// any immediately-following user messages into that same array at
// the provider boundary. Durable messages remain independent.
if message.role == "tool" && message.tool_call_id.is_some() {
let mut content = Vec::new();
while index < messages.len()
&& messages[index].role == "tool"
&& messages[index].tool_call_id.is_some()
{
let tool = &messages[index];
let tool_call_id = tool
.tool_call_id
.as_deref()
.expect("tool_call_id checked above");
content.push(serde_json::json!({
messages
.iter()
.map(|message| {
let role = if message.role == "tool" {
"user".to_string()
} else {
message.role.clone()
};
let content = if let Some(ref tool_call_id) = message.tool_call_id {
vec![serde_json::json!({
"type": "tool_result",
"tool_use_id": tool_call_id,
"content": convert_content_blocks(&tool.content, false),
}));
index += 1;
}
// One turn may receive more than one steering message before the
// next model request. Keep their order while emitting one native
// Anthropic user message alongside the tool_result blocks.
while index < messages.len() && messages[index].role == "user" {
let steering = &messages[index];
if let Some(native) = native_anthropic_content(steering) {
content.extend(native);
} else {
content.extend(convert_content_blocks(&steering.content, false));
}
index += 1;
}
converted.push(AnthropicMessage {
role: "user".to_string(),
content,
});
continue;
}
let role = message.role.clone();
let content = if let Some(native) = native_anthropic_content(message) {
native
"content": convert_content_blocks(&message.content, false),
})]
} else {
let mut blocks = convert_content_blocks(&message.content, message.role == "system");
if let Some(tool_calls) = message
@ -219,31 +172,9 @@ fn convert_messages(messages: &[Message]) -> Vec<AnthropicMessage> {
}
blocks
};
converted.push(AnthropicMessage { role, content });
index += 1;
}
converted
}
fn native_anthropic_content(message: &Message) -> Option<Vec<Value>> {
if message.role != "assistant" {
return None;
}
let state = message.provider_state.as_ref()?;
if state.provider != "anthropic" {
return None;
}
let blocks = state.payload.get("content")?.as_array()?;
blocks
.iter()
.all(|block| {
block
.as_object()
.and_then(|value| value.get("type"))
.is_some()
AnthropicMessage { role, content }
})
.then(|| blocks.clone())
.collect()
}
#[derive(Serialize)]
@ -255,298 +186,56 @@ struct AnthropicTool {
cache_control: Option<CacheControl>,
}
#[derive(Debug, Error)]
enum AnthropicStreamError {
#[error("invalid UTF-8 in Anthropic SSE event: {0}")]
Utf8(#[from] std::string::FromUtf8Error),
#[error("invalid Anthropic SSE payload: {0}")]
Json(#[from] serde_json::Error),
#[error("Anthropic stream error: {0}")]
Api(String),
#[error("Anthropic stream ended without message_stop")]
MissingFinish,
#[derive(Deserialize)]
struct AnthropicResponse {
id: Option<String>,
model: Option<String>,
#[serde(default)]
content: Vec<AnthropicContent>,
#[serde(default)]
usage: Option<AnthropicUsage>,
}
#[derive(Default)]
struct AnthropicSseDecoder {
framer: SseFramer,
blocks: BTreeMap<usize, Value>,
tool_json: HashMap<usize, String>,
usage: Usage,
finish_reason: Option<FinishReason>,
done_emitted: bool,
#[derive(Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
enum AnthropicContent {
Text {
#[serde(alias = "content")]
text: String,
},
Thinking {
#[serde(alias = "content")]
thinking: String,
},
#[serde(rename = "tool_use")]
ToolUse {
id: String,
name: String,
#[serde(alias = "arguments")]
input: serde_json::Value,
},
#[serde(other)]
Unknown,
}
impl AnthropicSseDecoder {
fn push(&mut self, bytes: &[u8]) -> Result<Vec<ProviderChunk>, AnthropicStreamError> {
let frames = self.framer.push(bytes)?;
self.decode_frames(frames)
}
fn finish(&mut self) -> Result<Vec<ProviderChunk>, AnthropicStreamError> {
let frames = self.framer.finish()?;
let chunks = self.decode_frames(frames)?;
if !self.done_emitted {
return Err(AnthropicStreamError::MissingFinish);
}
Ok(chunks)
}
fn decode_frames(
&mut self,
frames: Vec<String>,
) -> Result<Vec<ProviderChunk>, AnthropicStreamError> {
let mut chunks = Vec::new();
for data in frames {
let payload: Value = serde_json::from_str(&data)?;
match payload.get("type").and_then(Value::as_str) {
Some("message_start") => self.message_start(&payload, &mut chunks),
Some("content_block_start") => self.block_start(&payload, &mut chunks),
Some("content_block_delta") => self.block_delta(&payload, &mut chunks),
Some("content_block_stop") => self.block_stop(&payload),
Some("message_delta") => self.message_delta(&payload, &mut chunks),
Some("message_stop") => self.message_stop(&mut chunks),
Some("error") => {
let message = payload
.pointer("/error/message")
.and_then(Value::as_str)
.unwrap_or("unknown streaming error");
return Err(AnthropicStreamError::Api(message.to_string()));
}
Some("ping") | None | Some(_) => {}
}
}
Ok(chunks)
}
fn message_start(&mut self, payload: &Value, chunks: &mut Vec<ProviderChunk>) {
let message = payload.get("message").unwrap_or(&Value::Null);
let id = message
.get("id")
.and_then(Value::as_str)
.unwrap_or_default();
let model = message
.get("model")
.and_then(Value::as_str)
.unwrap_or_default();
chunks.push(ProviderChunk::Metadata {
id: id.to_string(),
model: model.to_string(),
});
if let Some(usage) = message.get("usage") {
update_anthropic_usage(&mut self.usage, usage);
chunks.push(ProviderChunk::Usage(self.usage.clone()));
}
}
fn block_start(&mut self, payload: &Value, chunks: &mut Vec<ProviderChunk>) {
let Some(index) = event_index(payload) else {
return;
};
let Some(block) = payload.get("content_block") else {
return;
};
self.blocks.insert(index, block.clone());
match block.get("type").and_then(Value::as_str) {
Some("text") => {
if let Some(text) = block.get("text").and_then(Value::as_str)
&& !text.is_empty()
{
chunks.push(ProviderChunk::Text(text.to_string()));
}
}
Some("thinking") => {
if let Some(thinking) = block.get("thinking").and_then(Value::as_str)
&& !thinking.is_empty()
{
chunks.push(ProviderChunk::Reasoning(thinking.to_string()));
}
}
Some("tool_use") => {
chunks.push(ProviderChunk::ToolCallStart {
index,
id: block.get("id").and_then(Value::as_str).map(str::to_string),
name: block
.get("name")
.and_then(Value::as_str)
.map(str::to_string),
});
let input = block.get("input").cloned().unwrap_or(Value::Null);
if !input.is_null() && input != serde_json::json!({}) {
chunks.push(ProviderChunk::ToolCallArguments {
index,
delta: input.to_string(),
});
}
}
_ => {}
}
}
fn block_delta(&mut self, payload: &Value, chunks: &mut Vec<ProviderChunk>) {
let Some(index) = event_index(payload) else {
return;
};
let Some(delta) = payload.get("delta") else {
return;
};
match delta.get("type").and_then(Value::as_str) {
Some("text_delta") => {
if let Some(text) = delta.get("text").and_then(Value::as_str) {
append_block_string(&mut self.blocks, index, "text", text);
if !text.is_empty() {
chunks.push(ProviderChunk::Text(text.to_string()));
}
}
}
Some("thinking_delta") => {
if let Some(thinking) = delta.get("thinking").and_then(Value::as_str) {
append_block_string(&mut self.blocks, index, "thinking", thinking);
if !thinking.is_empty() {
chunks.push(ProviderChunk::Reasoning(thinking.to_string()));
}
}
}
Some("signature_delta") => {
if let Some(signature) = delta.get("signature").and_then(Value::as_str) {
append_block_string(&mut self.blocks, index, "signature", signature);
}
}
Some("input_json_delta") => {
if let Some(partial) = delta.get("partial_json").and_then(Value::as_str) {
self.tool_json.entry(index).or_default().push_str(partial);
if !partial.is_empty() {
chunks.push(ProviderChunk::ToolCallArguments {
index,
delta: partial.to_string(),
});
}
}
}
_ => {}
}
}
fn block_stop(&mut self, payload: &Value) {
let Some(index) = event_index(payload) else {
return;
};
let Some(json) = self.tool_json.remove(&index) else {
return;
};
let input = serde_json::from_str(&json).unwrap_or(Value::Null);
if let Some(block) = self.blocks.get_mut(&index)
&& let Some(object) = block.as_object_mut()
{
object.insert("input".to_string(), input);
}
}
fn message_delta(&mut self, payload: &Value, chunks: &mut Vec<ProviderChunk>) {
if let Some(reason) = payload
.pointer("/delta/stop_reason")
.and_then(Value::as_str)
{
self.finish_reason = Some(FinishReason::from_provider(reason));
}
if let Some(usage) = payload.get("usage") {
update_anthropic_usage(&mut self.usage, usage);
chunks.push(ProviderChunk::Usage(self.usage.clone()));
}
}
fn message_stop(&mut self, chunks: &mut Vec<ProviderChunk>) {
if self.done_emitted {
return;
}
let content = self.blocks.values().cloned().collect::<Vec<_>>();
chunks.push(ProviderChunk::ProviderState(ProviderReasoningState {
provider: "anthropic".to_string(),
payload: serde_json::json!({ "version": 1, "content": content }),
}));
chunks.push(ProviderChunk::Done(
self.finish_reason.clone().unwrap_or(FinishReason::Stop),
));
self.done_emitted = true;
}
}
fn event_index(payload: &Value) -> Option<usize> {
payload
.get("index")
.and_then(Value::as_u64)
.and_then(|value| usize::try_from(value).ok())
}
fn append_block_string(blocks: &mut BTreeMap<usize, Value>, index: usize, key: &str, delta: &str) {
let Some(object) = blocks.get_mut(&index).and_then(Value::as_object_mut) else {
return;
};
let value = object
.entry(key.to_string())
.or_insert_with(|| Value::String(String::new()));
if let Some(current) = value.as_str() {
*value = Value::String(format!("{current}{delta}"));
}
}
fn update_anthropic_usage(usage: &mut Usage, value: &Value) {
if let Some(input) = json_u32(value, "input_tokens") {
usage.prompt_tokens = input;
}
if let Some(output) = json_u32(value, "output_tokens") {
usage.completion_tokens = output;
}
if let Some(cache_read) = json_u32(value, "cache_read_input_tokens") {
usage.cached_tokens = Some(cache_read);
usage.cache_read_input_tokens = Some(cache_read);
}
if let Some(cache_creation) = json_u32(value, "cache_creation_input_tokens") {
usage.cache_creation_input_tokens = Some(cache_creation);
}
usage.total_tokens = usage.prompt_tokens.saturating_add(usage.completion_tokens);
}
fn json_u32(value: &Value, key: &str) -> Option<u32> {
value
.get(key)
.and_then(Value::as_u64)
.and_then(|number| u32::try_from(number).ok())
}
struct AnthropicHttpStream {
response: reqwest::Response,
decoder: AnthropicSseDecoder,
pending: VecDeque<ProviderChunk>,
reached_eof: bool,
}
async fn next_anthropic_chunk(
mut state: AnthropicHttpStream,
) -> Result<Option<(ProviderChunk, AnthropicHttpStream)>, DynProviderError> {
loop {
if let Some(chunk) = state.pending.pop_front() {
return Ok(Some((chunk, state)));
}
if state.reached_eof {
return Ok(None);
}
match state.response.chunk().await? {
Some(bytes) => state.pending.extend(state.decoder.push(&bytes)?),
None => {
state.pending.extend(state.decoder.finish()?);
state.reached_eof = true;
}
}
}
#[derive(Deserialize)]
struct AnthropicUsage {
#[serde(default)]
input_tokens: u32,
#[serde(default)]
output_tokens: u32,
#[serde(default)]
cache_read_input_tokens: Option<u32>,
#[serde(default)]
cache_creation_input_tokens: Option<u32>,
}
#[async_trait]
impl LLMProvider for AnthropicProvider {
async fn stream(
async fn chat(
&self,
request: ChatCompletionRequest,
) -> Result<ProviderStream, DynProviderError> {
) -> Result<ChatCompletionResponse, Box<dyn std::error::Error + Send + Sync>> {
let start = std::time::Instant::now();
let url = format!("{}/v1/messages", self.base_url);
let max_tokens = request.max_tokens.or(self.max_tokens).unwrap_or(1024);
@ -568,7 +257,6 @@ impl LLMProvider for AnthropicProvider {
messages: convert_messages(&request.messages),
max_tokens,
temperature: request.temperature.or(self.temperature),
stream: true,
tools,
extra: self.model_extra.clone(),
};
@ -584,16 +272,8 @@ impl LLMProvider for AnthropicProvider {
req_builder = req_builder.header(key.as_str(), value.as_str());
}
let request_summary = super::stream::diagnostic_request_summary(
&self.model_id,
body.messages.len(),
body.tools.as_ref().map_or(0, Vec::len),
);
tracing::debug!(
message_count = body.messages.len(),
tool_count = body.tools.as_ref().map_or(0, Vec::len),
"Anthropic streaming request"
);
let req_body_str = serde_json::to_string_pretty(&body).unwrap_or_default();
tracing::debug!(req_body = %req_body_str, "LLM request");
let resp = req_builder.json(&body).send().await.inspect_err(|e| {
let is_timeout = e.is_timeout();
@ -609,8 +289,10 @@ impl LLMProvider for AnthropicProvider {
})?;
let status = resp.status();
if !status.is_success() {
let body_text = resp.text().await?;
tracing::debug!(status = %status, resp_body = %body_text, "LLM response");
if !status.is_success() {
let error_msg = serde_json::from_str::<serde_json::Value>(&body_text)
.ok()
.and_then(|v| {
@ -633,7 +315,7 @@ impl LLMProvider for AnthropicProvider {
.append_llm_call(
&self.name,
&self.model_id,
&request_summary,
&req_body_str,
Some(&body_text),
Some(&error_msg),
start.elapsed().as_millis() as u64,
@ -642,16 +324,110 @@ impl LLMProvider for AnthropicProvider {
}
return Err(format!("API error ({}): {}", status.as_u16(), error_msg).into());
}
tracing::debug!(status = %status, "Anthropic streaming response started");
Ok(Box::pin(stream::try_unfold(
AnthropicHttpStream {
response: resp,
decoder: AnthropicSseDecoder::default(),
pending: VecDeque::new(),
reached_eof: false,
let anthropic_resp: AnthropicResponse = match serde_json::from_str(&body_text) {
Ok(response) => response,
Err(e) => {
let err_msg = format!("decode error: {} | body: {}", e, &body_text);
if let Some(ref storage) = self.storage {
let dur = start.elapsed().as_millis() as u64;
if let Err(error) = storage
.append_llm_call(
&self.name,
&self.model_id,
&req_body_str,
Some(&body_text),
Some(&err_msg),
dur,
)
.await
{
tracing::warn!("failed to persist LLM call (decode error): {}", error);
}
}
return Err(err_msg.into());
}
};
let mut content = String::new();
let mut reasoning = None;
let mut tool_calls = Vec::new();
for c in &anthropic_resp.content {
match c {
AnthropicContent::Text { text } => {
if !text.is_empty() {
if !content.is_empty() {
content.push('\n');
}
content.push_str(text);
}
}
AnthropicContent::Thinking { thinking } => {
reasoning = Some(thinking.clone());
}
AnthropicContent::Unknown => {}
AnthropicContent::ToolUse { id, name, input } => {
tool_calls.push(ToolCall {
id: id.clone(),
name: name.clone(),
arguments: input.clone(),
});
}
}
}
let response = ChatCompletionResponse {
id: anthropic_resp.id.unwrap_or_default(),
model: anthropic_resp.model.unwrap_or_default(),
content,
reasoning_content: reasoning,
tool_calls,
usage: Usage {
prompt_tokens: anthropic_resp
.usage
.as_ref()
.map(|u| u.input_tokens)
.unwrap_or(0),
completion_tokens: anthropic_resp
.usage
.as_ref()
.map(|u| u.output_tokens)
.unwrap_or(0),
total_tokens: anthropic_resp
.usage
.as_ref()
.map(|u| u.input_tokens + u.output_tokens)
.unwrap_or(0),
cached_tokens: anthropic_resp
.usage
.as_ref()
.and_then(|u| u.cache_read_input_tokens),
cache_read_input_tokens: anthropic_resp
.usage
.as_ref()
.and_then(|u| u.cache_read_input_tokens),
cache_creation_input_tokens: anthropic_resp
.usage
.as_ref()
.and_then(|u| u.cache_creation_input_tokens),
},
next_anthropic_chunk,
)))
};
if let Some(ref storage) = self.storage {
let _ = storage
.append_llm_call(
&self.name,
&self.model_id,
&req_body_str,
Some(&body_text),
None,
start.elapsed().as_millis() as u64,
)
.await;
}
Ok(response)
}
fn ptype(&self) -> &str {
@ -670,7 +446,6 @@ impl LLMProvider for AnthropicProvider {
#[cfg(test)]
mod tests {
use super::*;
use crate::providers::ProviderResponseAccumulator;
use serde_json::json;
#[test]
@ -712,7 +487,6 @@ mod tests {
ContentBlock::image_url("data:image/png;base64,AAAA"),
],
reasoning_content: None,
provider_state: None,
tool_call_id: Some("call_1".to_string()),
name: Some("file_read".to_string()),
tool_calls: None,
@ -728,154 +502,4 @@ mod tests {
assert_eq!(result["content"][1]["source"]["media_type"], "image/png");
assert_eq!(result["content"][1]["source"]["data"], "AAAA");
}
#[test]
fn tool_results_and_following_steering_share_one_user_content_array() {
let messages = vec![
Message::tool("call_1", "lookup", "first result"),
Message::tool("call_2", "lookup", "second result"),
Message::user("用户补充指令"),
Message::user("再补充一条"),
Message::assistant("最终回答"),
];
let converted = convert_messages(&messages);
assert_eq!(converted.len(), 2);
assert_eq!(converted[0].role, "user");
assert_eq!(converted[0].content.len(), 4);
assert_eq!(converted[0].content[0]["type"], "tool_result");
assert_eq!(converted[0].content[0]["tool_use_id"], "call_1");
assert_eq!(converted[0].content[1]["tool_use_id"], "call_2");
assert_eq!(
converted[0].content[2],
json!({"type": "text", "text": "用户补充指令"})
);
assert_eq!(
converted[0].content[3],
json!({"type": "text", "text": "再补充一条"})
);
assert_eq!(converted[1].role, "assistant");
}
#[test]
fn native_stream_decodes_thinking_signature_tools_usage_and_replay_state() {
let events = [
json!({"type":"message_start","message":{"id":"msg_1","model":"claude-test","usage":{"input_tokens":11,"output_tokens":0,"cache_read_input_tokens":3}}}),
json!({"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"","signature":""}}),
json!({"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"check "}}),
json!({"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"facts"}}),
json!({"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"sig=="}}),
json!({"type":"content_block_stop","index":0}),
json!({"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"tool_1","name":"lookup","input":{}}}),
json!({"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"{\"q\":"}}),
json!({"type":"content_block_delta","index":1,"delta":{"type":"input_json_delta","partial_json":"\"rust\"}"}}),
json!({"type":"content_block_stop","index":1}),
json!({"type":"content_block_start","index":2,"content_block":{"type":"text","text":""}}),
json!({"type":"content_block_delta","index":2,"delta":{"type":"text_delta","text":"answer"}}),
json!({"type":"content_block_stop","index":2}),
json!({"type":"message_delta","delta":{"stop_reason":"tool_use","stop_sequence":null},"usage":{"output_tokens":7}}),
json!({"type":"message_stop"}),
];
let wire = events
.iter()
.map(|event| format!("event: ignored\ndata: {event}\n\n"))
.collect::<String>();
let mut decoder = AnthropicSseDecoder::default();
let mut accumulator = ProviderResponseAccumulator::default();
for bytes in wire.as_bytes().chunks(7) {
for chunk in decoder.push(bytes).unwrap() {
accumulator.push(chunk);
}
}
for chunk in decoder.finish().unwrap() {
accumulator.push(chunk);
}
let response = accumulator.finish();
assert_eq!(response.id, "msg_1");
assert_eq!(response.model, "claude-test");
assert_eq!(response.reasoning_content.as_deref(), Some("check facts"));
assert_eq!(response.content, "answer");
assert_eq!(response.tool_calls.len(), 1);
assert_eq!(response.tool_calls[0].id, "tool_1");
assert_eq!(response.tool_calls[0].arguments, json!({"q":"rust"}));
assert_eq!(response.usage.prompt_tokens, 11);
assert_eq!(response.usage.completion_tokens, 7);
assert_eq!(response.usage.total_tokens, 18);
assert_eq!(response.usage.cache_read_input_tokens, Some(3));
let state = response.provider_state.unwrap();
assert_eq!(state.provider, "anthropic");
assert_eq!(state.payload["content"][0]["thinking"], "check facts");
assert_eq!(state.payload["content"][0]["signature"], "sig==");
assert_eq!(state.payload["content"][1]["input"], json!({"q":"rust"}));
assert_eq!(state.payload["content"][2]["text"], "answer");
}
#[test]
fn matching_provider_state_replays_native_blocks_without_generic_duplicates() {
let native = json!([
{"type":"thinking","thinking":"signed thought","signature":"sig=="},
{"type":"tool_use","id":"tool_1","name":"lookup","input":{"q":"rust"}}
]);
let message = Message {
role: "assistant".into(),
content: vec![ContentBlock::text("generic text must not be appended")],
reasoning_content: Some("display copy".into()),
provider_state: Some(ProviderReasoningState {
provider: "anthropic".into(),
payload: json!({"version":1,"content":native}),
}),
tool_call_id: None,
name: None,
tool_calls: Some(vec![crate::providers::ToolCall {
id: "duplicate".into(),
name: "duplicate".into(),
arguments: json!({}),
}]),
};
let converted = convert_messages(&[message]);
assert_eq!(converted[0].content.len(), 2);
assert_eq!(converted[0].content[0]["signature"], "sig==");
assert_eq!(converted[0].content[1]["id"], "tool_1");
}
#[test]
fn foreign_provider_state_is_not_replayed_to_anthropic() {
let message = Message {
role: "assistant".into(),
content: vec![ContentBlock::text("answer")],
reasoning_content: Some("unsigned display reasoning".into()),
provider_state: Some(ProviderReasoningState {
provider: "openai".into(),
payload: json!({"private":"state"}),
}),
tool_call_id: None,
name: None,
tool_calls: None,
};
let converted = convert_messages(&[message]);
assert_eq!(
converted[0].content,
vec![json!({"type":"text","text":"answer"})]
);
}
#[test]
fn stream_requires_message_stop() {
let mut decoder = AnthropicSseDecoder::default();
decoder
.push(b"data: {\"type\":\"message_start\",\"message\":{}}\n\n")
.unwrap();
assert!(matches!(
decoder.finish(),
Err(AnthropicStreamError::MissingFinish)
));
}
}

7
src/providers/mod.rs
View File

@ -1,18 +1,11 @@
pub mod anthropic;
pub mod openai;
pub mod stream;
pub mod traits;
pub use self::anthropic::AnthropicProvider;
pub use self::openai::OpenAIProvider;
use crate::config::LLMProviderConfig;
#[cfg(test)]
pub use stream::provider_stream_for_test;
pub use stream::{
DynProviderError, FinishReason, ProviderChunk, ProviderResponseAccumulator, ProviderStream,
ProviderStreamItem, collect_provider_stream,
};
pub use traits::{
ChatCompletionRequest, ChatCompletionResponse, LLMProvider, Message, Tool, ToolCall,
ToolFunction, Usage,

720
src/providers/openai.rs
View File

@ -1,16 +1,12 @@
use async_trait::async_trait;
use futures_util::stream;
use reqwest::Client;
use serde::Deserialize;
use serde_json::{Value, json};
use std::collections::{HashMap, VecDeque};
use std::collections::HashMap;
use std::time::Duration;
use thiserror::Error;
use super::stream::SseFramer;
use super::{
ChatCompletionRequest, DynProviderError, FinishReason, LLMProvider, Message, ProviderChunk,
ProviderStream, Usage,
};
use super::traits::Usage;
use super::{ChatCompletionRequest, ChatCompletionResponse, LLMProvider, Message, ToolCall};
use crate::bus::message::ContentBlock;
use crate::storage::Storage;
use std::sync::Arc;
@ -47,18 +43,6 @@ fn text_content(blocks: &[ContentBlock]) -> String {
.join("\n")
}
fn role_safe_content(role: &str, blocks: &[ContentBlock]) -> Value {
if role == "user" {
convert_content_blocks(blocks)
} else {
// OpenAI-compatible endpoints commonly reject native image parts on
// system/assistant messages. AgentLoop already supplies a textual
// attachment manifest for persisted assistant media; keep this final
// provider boundary defensive for all callers.
Value::String(text_content(blocks))
}
}
fn regular_message_json(message: &Message) -> Value {
if message.role == "tool" {
json!({
@ -75,7 +59,7 @@ fn regular_message_json(message: &Message) -> Value {
{
let mut value = json!({
"role": message.role,
"content": role_safe_content(&message.role, &message.content),
"content": convert_content_blocks(&message.content),
"tool_calls": message.tool_calls.as_ref().map(|calls| {
calls.iter().map(|call| json!({
"id": call.id,
@ -94,7 +78,7 @@ fn regular_message_json(message: &Message) -> Value {
} else {
let mut value = json!({
"role": message.role,
"content": role_safe_content(&message.role, &message.content)
"content": convert_content_blocks(&message.content)
});
if message.role == "assistant"
&& let Some(ref reasoning_content) = message.reasoning_content
@ -224,348 +208,82 @@ impl OpenAIProvider {
body
}
fn build_stream_request_body(&self, request: &ChatCompletionRequest) -> Value {
let mut body = self.build_request_body(request);
body["stream"] = Value::Bool(true);
body["stream_options"] = json!({ "include_usage": true });
body
}
}
#[derive(Debug, Error)]
enum OpenAIStreamError {
#[error("invalid UTF-8 in SSE event: {0}")]
Utf8(#[from] std::string::FromUtf8Error),
#[error("invalid OpenAI-compatible SSE payload: {0}")]
Json(#[from] serde_json::Error),
#[error("OpenAI-compatible stream ended without a finish marker")]
MissingFinish,
#[derive(Deserialize)]
struct OpenAIResponse {
id: String,
model: String,
choices: Vec<OpenAIChoice>,
#[serde(default)]
usage: OpenAIUsage,
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum InlineMode {
Text,
Reasoning,
#[derive(Deserialize)]
struct OpenAIChoice {
message: OpenAIMessage,
}
struct InlineReasoningParser {
mode: InlineMode,
pending: String,
fn null_or_missing_tool_calls<'de, D>(deserializer: D) -> Result<Vec<OpenAIToolCall>, D::Error>
where
D: serde::Deserializer<'de>,
{
Ok(Option::<Vec<OpenAIToolCall>>::deserialize(deserializer)?.unwrap_or_default())
}
impl Default for InlineReasoningParser {
fn default() -> Self {
Self {
mode: InlineMode::Text,
pending: String::new(),
}
}
#[derive(Deserialize)]
struct OpenAIMessage {
#[serde(default)]
content: Option<String>,
#[serde(default)]
reasoning_content: Option<String>,
#[serde(default, deserialize_with = "null_or_missing_tool_calls")]
tool_calls: Vec<OpenAIToolCall>,
}
impl InlineReasoningParser {
const TAGS: [(&'static str, InlineMode); 4] = [
("<think>", InlineMode::Reasoning),
("<reasoning>", InlineMode::Reasoning),
("</think>", InlineMode::Text),
("</reasoning>", InlineMode::Text),
];
fn push(&mut self, delta: &str) -> Vec<ProviderChunk> {
self.pending.push_str(delta);
self.drain(false)
}
fn finish(&mut self) -> Vec<ProviderChunk> {
self.drain(true)
}
fn drain(&mut self, finish: bool) -> Vec<ProviderChunk> {
let mut chunks = Vec::new();
loop {
let next_tag = Self::TAGS
.iter()
.filter_map(|(tag, mode)| self.pending.find(tag).map(|index| (index, *tag, *mode)))
.min_by_key(|(index, _, _)| *index);
if let Some((index, tag, mode)) = next_tag {
let text = self.pending[..index].to_string();
self.emit_text(text, &mut chunks);
self.pending.drain(..index + tag.len());
self.mode = mode;
continue;
}
let retained = if finish {
0
} else {
longest_tag_prefix_suffix(&self.pending, &Self::TAGS)
};
let emit_len = self.pending.len() - retained;
if emit_len > 0 {
let text = self.pending[..emit_len].to_string();
self.pending.drain(..emit_len);
self.emit_text(text, &mut chunks);
}
break;
}
chunks
}
fn emit_text(&self, text: String, chunks: &mut Vec<ProviderChunk>) {
if text.is_empty() {
return;
}
chunks.push(match self.mode {
InlineMode::Text => ProviderChunk::Text(text),
InlineMode::Reasoning => ProviderChunk::Reasoning(text),
});
}
#[derive(Deserialize)]
struct OpenAIToolCall {
id: String,
#[serde(rename = "function")]
function: OAIFunction,
}
fn longest_tag_prefix_suffix(value: &str, tags: &[(&str, InlineMode)]) -> usize {
let mut best = 0;
for boundary in value
.char_indices()
.map(|(index, _)| index)
.chain([value.len()])
{
let suffix = &value[boundary..];
if tags.iter().any(|(tag, _)| tag.starts_with(suffix)) {
best = best.max(suffix.len());
}
}
best
#[derive(Deserialize)]
struct OAIFunction {
name: String,
arguments: String,
}
#[derive(Default)]
struct PartialStreamTool {
id: Option<String>,
name: Option<String>,
started: bool,
#[derive(Deserialize, Default)]
struct OpenAIUsage {
#[serde(default)]
prompt_tokens: u32,
#[serde(default)]
completion_tokens: u32,
#[serde(default)]
total_tokens: u32,
#[serde(default)]
cached_tokens: Option<u32>,
#[serde(default)]
prompt_tokens_details: Option<OpenAIPromptTokensDetails>,
}
#[derive(Default)]
struct OpenAISseDecoder {
framer: SseFramer,
inline_reasoning: InlineReasoningParser,
tools: HashMap<usize, PartialStreamTool>,
metadata_emitted: bool,
done_emitted: bool,
}
impl OpenAISseDecoder {
fn push(&mut self, bytes: &[u8]) -> Result<Vec<ProviderChunk>, OpenAIStreamError> {
let frames = self.framer.push(bytes)?;
self.decode_frames(frames)
}
fn finish(&mut self) -> Result<Vec<ProviderChunk>, OpenAIStreamError> {
let frames = self.framer.finish()?;
let mut chunks = self.decode_frames(frames)?;
chunks.extend(self.inline_reasoning.finish());
if !self.done_emitted {
return Err(OpenAIStreamError::MissingFinish);
}
Ok(chunks)
}
fn decode_frames(
&mut self,
frames: Vec<String>,
) -> Result<Vec<ProviderChunk>, OpenAIStreamError> {
let mut chunks = Vec::new();
for data in frames {
if data == "[DONE]" {
chunks.extend(self.inline_reasoning.finish());
if !self.done_emitted {
chunks.push(ProviderChunk::Done(FinishReason::Stop));
self.done_emitted = true;
}
continue;
}
let payload: Value = serde_json::from_str(&data)?;
if !self.metadata_emitted {
let id = payload
.get("id")
.and_then(Value::as_str)
.unwrap_or_default();
let model = payload
.get("model")
.and_then(Value::as_str)
.unwrap_or_default();
if !id.is_empty() || !model.is_empty() {
chunks.push(ProviderChunk::Metadata {
id: id.to_string(),
model: model.to_string(),
});
self.metadata_emitted = true;
}
}
if let Some(usage) = payload.get("usage").filter(|value| !value.is_null()) {
chunks.push(ProviderChunk::Usage(parse_openai_usage(usage)));
}
let Some(choice) = payload
.get("choices")
.and_then(Value::as_array)
.and_then(|choices| choices.first())
else {
continue;
};
if let Some(delta) = choice.get("delta") {
if let Some(reasoning) = delta
.get("reasoning_content")
.and_then(Value::as_str)
.or_else(|| delta.get("reasoning").and_then(Value::as_str))
.filter(|value| !value.is_empty())
{
chunks.push(ProviderChunk::Reasoning(reasoning.to_string()));
}
if let Some(content) = delta
.get("content")
.and_then(Value::as_str)
.filter(|value| !value.is_empty())
{
chunks.extend(self.inline_reasoning.push(content));
}
if let Some(tool_calls) = delta.get("tool_calls").and_then(Value::as_array) {
self.decode_tool_calls(tool_calls, &mut chunks);
}
}
if let Some(reason) = choice.get("finish_reason").and_then(Value::as_str) {
chunks.extend(self.inline_reasoning.finish());
self.flush_unstarted_tools(&mut chunks);
if !self.done_emitted {
chunks.push(ProviderChunk::Done(FinishReason::from_provider(reason)));
self.done_emitted = true;
}
}
}
Ok(chunks)
}
fn decode_tool_calls(&mut self, calls: &[Value], chunks: &mut Vec<ProviderChunk>) {
for (fallback_index, call) in calls.iter().enumerate() {
let index = call
.get("index")
.and_then(Value::as_u64)
.and_then(|value| usize::try_from(value).ok())
.unwrap_or(fallback_index);
let tool = self.tools.entry(index).or_default();
if let Some(id) = call.get("id").and_then(Value::as_str) {
tool.id = Some(id.to_string());
}
let function = call.get("function");
if let Some(name) = function
.and_then(|value| value.get("name"))
.and_then(Value::as_str)
.filter(|value| !value.is_empty())
{
tool.name = Some(name.to_string());
}
if !tool.started && tool.name.is_some() {
chunks.push(ProviderChunk::ToolCallStart {
index,
id: tool.id.clone(),
name: tool.name.clone(),
});
tool.started = true;
}
if let Some(arguments) = function
.and_then(|value| value.get("arguments"))
.and_then(Value::as_str)
.filter(|value| !value.is_empty())
{
chunks.push(ProviderChunk::ToolCallArguments {
index,
delta: arguments.to_string(),
});
}
}
}
fn flush_unstarted_tools(&mut self, chunks: &mut Vec<ProviderChunk>) {
let mut indexes = self.tools.keys().copied().collect::<Vec<_>>();
indexes.sort_unstable();
for index in indexes {
let tool = self
.tools
.get_mut(&index)
.expect("tool index came from map");
if !tool.started {
chunks.push(ProviderChunk::ToolCallStart {
index,
id: tool.id.clone(),
name: tool.name.clone(),
});
tool.started = true;
}
}
}
}
fn parse_openai_usage(value: &Value) -> Usage {
let direct_cached = value.get("cached_tokens").and_then(Value::as_u64);
let nested_cached = value
.get("prompt_tokens_details")
.and_then(|details| details.get("cached_tokens"))
.and_then(Value::as_u64);
Usage {
prompt_tokens: json_u32(value, "prompt_tokens"),
completion_tokens: json_u32(value, "completion_tokens"),
total_tokens: json_u32(value, "total_tokens"),
cached_tokens: nested_cached
.or(direct_cached)
.and_then(|tokens| u32::try_from(tokens).ok()),
cache_read_input_tokens: None,
cache_creation_input_tokens: None,
}
}
fn json_u32(value: &Value, key: &str) -> u32 {
value
.get(key)
.and_then(Value::as_u64)
.and_then(|number| u32::try_from(number).ok())
.unwrap_or_default()
}
struct OpenAIHttpStream {
response: reqwest::Response,
decoder: OpenAISseDecoder,
pending: VecDeque<ProviderChunk>,
reached_eof: bool,
}
async fn next_openai_chunk(
mut state: OpenAIHttpStream,
) -> Result<Option<(ProviderChunk, OpenAIHttpStream)>, DynProviderError> {
loop {
if let Some(chunk) = state.pending.pop_front() {
return Ok(Some((chunk, state)));
}
if state.reached_eof {
return Ok(None);
}
match state.response.chunk().await? {
Some(bytes) => state.pending.extend(state.decoder.push(&bytes)?),
None => {
state.pending.extend(state.decoder.finish()?);
state.reached_eof = true;
}
}
}
#[derive(Deserialize, Default)]
struct OpenAIPromptTokensDetails {
#[serde(default)]
cached_tokens: Option<u32>,
}
#[async_trait]
impl LLMProvider for OpenAIProvider {
async fn stream(
async fn chat(
&self,
request: ChatCompletionRequest,
) -> Result<ProviderStream, DynProviderError> {
) -> Result<ChatCompletionResponse, Box<dyn std::error::Error + Send + Sync>> {
let start = std::time::Instant::now();
let url = format!("{}/chat/completions", self.base_url);
let body = self.build_stream_request_body(&request);
let body = self.build_request_body(&request);
// Debug: Log LLM request summary (only in debug builds)
#[cfg(debug_assertions)]
@ -604,11 +322,8 @@ impl LLMProvider for OpenAIProvider {
req_builder = req_builder.header(key.as_str(), value.as_str());
}
let request_summary = super::stream::diagnostic_request_summary(
&self.model_id,
body["messages"].as_array().map_or(0, Vec::len),
body["tools"].as_array().map_or(0, Vec::len),
);
let req_body_str = serde_json::to_string_pretty(&body).unwrap_or_default();
tracing::debug!(req_body = %req_body_str, "LLM request");
let resp = req_builder.json(&body).send().await.inspect_err(|e| {
let is_timeout = e.is_timeout();
@ -624,8 +339,10 @@ impl LLMProvider for OpenAIProvider {
})?;
let status = resp.status();
if !status.is_success() {
let text = resp.text().await?;
tracing::debug!(status = %status, resp_body = %text, "LLM response");
if !status.is_success() {
let error = format!("API error {}: {}", status, text);
tracing::error!(
provider = %self.name,
@ -640,7 +357,7 @@ impl LLMProvider for OpenAIProvider {
.append_llm_call(
&self.name,
&self.model_id,
&request_summary,
&req_body_str,
Some(&text),
Some(&error),
start.elapsed().as_millis() as u64,
@ -652,13 +369,93 @@ impl LLMProvider for OpenAIProvider {
return Err(error.into());
}
let state = OpenAIHttpStream {
response: resp,
decoder: OpenAISseDecoder::default(),
pending: VecDeque::new(),
reached_eof: false,
let openai_resp: OpenAIResponse = match serde_json::from_str(&text) {
Ok(response) => response,
Err(e) => {
let err_msg = format!("decode error: {} | body: {}", e, &text);
if let Some(ref storage) = self.storage {
let dur = start.elapsed().as_millis() as u64;
if let Err(error) = storage
.append_llm_call(
&self.name,
&self.model_id,
&req_body_str,
Some(&text),
Some(&err_msg),
dur,
)
.await
{
tracing::warn!("failed to persist LLM call (decode error): {}", error);
}
}
return Err(err_msg.into());
}
};
Ok(Box::pin(stream::try_unfold(state, next_openai_chunk)))
let first_choice = openai_resp
.choices
.into_iter()
.next()
.ok_or("no choices in response")?;
let content = first_choice
.message
.content
.as_ref()
.unwrap_or(&String::new())
.clone();
let tool_calls: Vec<ToolCall> = first_choice
.message
.tool_calls
.iter()
.map(|tc| ToolCall {
id: tc.id.clone(),
name: tc.function.name.clone(),
arguments: serde_json::from_str(&tc.function.arguments)
.unwrap_or(serde_json::Value::Null),
})
.collect();
let usage = openai_resp.usage;
let nested_cached_tokens = usage
.prompt_tokens_details
.as_ref()
.and_then(|d| d.cached_tokens);
let cached_tokens = nested_cached_tokens.or(usage.cached_tokens);
let response = ChatCompletionResponse {
id: openai_resp.id,
model: openai_resp.model,
content,
reasoning_content: first_choice.message.reasoning_content,
tool_calls,
usage: Usage {
prompt_tokens: usage.prompt_tokens,
completion_tokens: usage.completion_tokens,
total_tokens: usage.total_tokens,
cached_tokens,
cache_read_input_tokens: None,
cache_creation_input_tokens: None,
},
};
if let Some(ref storage) = self.storage
&& let Err(e) = storage
.append_llm_call(
&self.name,
&self.model_id,
&req_body_str,
Some(&text),
None,
start.elapsed().as_millis() as u64,
)
.await
{
tracing::warn!("failed to persist LLM call: {}", e);
}
Ok(response)
}
fn ptype(&self) -> &str {
@ -677,7 +474,7 @@ impl LLMProvider for OpenAIProvider {
#[cfg(test)]
mod tests {
use super::*;
use crate::providers::{Message, ToolCall};
use crate::providers::Message;
#[test]
fn test_build_request_body_includes_assistant_tool_calls() {
@ -697,7 +494,6 @@ mod tests {
role: "assistant".to_string(),
content: vec![ContentBlock::text("calling tool")],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: Some(vec![ToolCall {
@ -723,10 +519,6 @@ mod tests {
tool_calls[0]["function"]["arguments"],
"{\"expression\":\"1+1\"}"
);
let stream_body = provider.build_stream_request_body(&request);
assert_eq!(stream_body["stream"], true);
assert_eq!(stream_body["stream_options"]["include_usage"], true);
}
#[test]
@ -740,7 +532,6 @@ mod tests {
ContentBlock::image_url("data:image/png;base64,AAAA"),
],
reasoning_content: None,
provider_state: None,
tool_call_id: Some("call_2".to_string()),
name: Some("file_read".to_string()),
tool_calls: None,
@ -762,172 +553,75 @@ mod tests {
}
#[test]
fn assistant_tools_precede_all_tool_results_and_following_steering() {
let messages = vec![
Message {
role: "assistant".to_string(),
content: vec![ContentBlock::text("calling tools")],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: Some(vec![
ToolCall {
id: "call_1".to_string(),
name: "lookup".to_string(),
arguments: json!({"q": "one"}),
},
ToolCall {
id: "call_2".to_string(),
name: "lookup".to_string(),
arguments: json!({"q": "two"}),
},
]),
},
Message::tool("call_1", "lookup", "result"),
Message::tool("call_2", "lookup", "second result"),
Message::user("用户补充指令"),
];
let converted = convert_messages(&messages);
assert_eq!(converted.len(), 4);
assert_eq!(converted[0]["role"], "assistant");
assert_eq!(converted[1]["role"], "tool");
assert_eq!(converted[1]["tool_call_id"], "call_1");
assert_eq!(converted[2]["role"], "tool");
assert_eq!(converted[2]["tool_call_id"], "call_2");
assert_eq!(converted[3]["role"], "user");
assert_eq!(converted[3]["content"], "用户补充指令");
fn test_decode_response_accepts_null_tool_calls() {
let text = r#"{
"id": "d21abaa6552741949e2aba76bde59359",
"choices": [{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "你好！",
"role": "assistant",
"tool_calls": null,
"reasoning_content": "The user sent a greeting."
}
#[test]
fn assistant_images_are_never_serialized_as_native_content_parts() {
let converted = convert_messages(&[Message {
role: "assistant".to_string(),
content: vec![
ContentBlock::text("screenshot delivered"),
ContentBlock::image_url("data:image/png;base64,AAAA"),
],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: None,
}]);
assert_eq!(converted.len(), 1);
assert_eq!(converted[0]["role"], "assistant");
assert_eq!(converted[0]["content"], "screenshot delivered");
}],
"created": 1781622889,
"model": "mimo-v2.5",
"object": "chat.completion",
"usage": {
"completion_tokens": 65,
"prompt_tokens": 11741,
"total_tokens": 11806,
"completion_tokens_details": {"reasoning_tokens": 40},
"prompt_tokens_details": {}
}
}"#;
#[tokio::test]
async fn sse_decoder_handles_byte_boundaries_reasoning_content_and_usage() {
let input = concat!(
"data: {\"id\":\"r1\",\"model\":\"m1\",\"choices\":[{\"delta\":{\"reasoning_content\":\"why\",\"content\":\"你好\"},\"finish_reason\":null}]}\r\n\r\n",
"data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
"data: {\"choices\":[],\"usage\":{\"prompt_tokens\":10,\"completion_tokens\":5,\"total_tokens\":15,\"prompt_tokens_details\":{\"cached_tokens\":3}}}\n\n",
"data: [DONE]\n\n"
);
let mut decoder = OpenAISseDecoder::default();
let mut chunks = Vec::new();
for byte in input.as_bytes() {
chunks.extend(decoder.push(std::slice::from_ref(byte)).unwrap());
}
chunks.extend(decoder.finish().unwrap());
let response: OpenAIResponse = serde_json::from_str(text).unwrap();
let message = &response.choices[0].message;
let response = crate::providers::collect_provider_stream(Box::pin(
futures_util::stream::iter(chunks.into_iter().map(Ok)),
))
.await
.unwrap();
assert_eq!(response.id, "r1");
assert_eq!(response.model, "m1");
assert_eq!(response.reasoning_content.as_deref(), Some("why"));
assert_eq!(response.content, "你好");
assert_eq!(response.usage.total_tokens, 15);
assert_eq!(response.usage.cached_tokens, Some(3));
}
#[test]
fn inline_reasoning_tags_may_span_sse_chunks() {
let input = concat!(
"data: {\"choices\":[{\"delta\":{\"content\":\"<thi\"},\"finish_reason\":null}]}\n\n",
"data: {\"choices\":[{\"delta\":{\"content\":\"nk>secret</th\"},\"finish_reason\":null}]}\n\n",
"data: {\"choices\":[{\"delta\":{\"content\":\"ink>answer\"},\"finish_reason\":\"stop\"}]}\n\n",
"data: [DONE]\n\n"
);
let mut decoder = OpenAISseDecoder::default();
let chunks = decoder.push(input.as_bytes()).unwrap();
assert_eq!(message.content.as_deref(), Some("你好！"));
assert_eq!(
chunks
.iter()
.filter_map(|chunk| match chunk {
ProviderChunk::Reasoning(value) => Some(value.as_str()),
_ => None,
})
.collect::<String>(),
"secret"
);
assert_eq!(
chunks
.iter()
.filter_map(|chunk| match chunk {
ProviderChunk::Text(value) => Some(value.as_str()),
_ => None,
})
.collect::<String>(),
"answer"
message.reasoning_content.as_deref(),
Some("The user sent a greeting.")
);
assert!(message.tool_calls.is_empty());
assert_eq!(response.usage.total_tokens, 11806);
}
#[test]
fn reasoning_alias_is_used_when_reasoning_content_is_null() {
let input = concat!(
"data: {\"choices\":[{\"delta\":{\"reasoning_content\":null,\"reasoning\":\"alias\"},\"finish_reason\":\"stop\"}]}\n\n",
"data: [DONE]\n\n"
);
let mut decoder = OpenAISseDecoder::default();
let chunks = decoder.push(input.as_bytes()).unwrap();
assert!(
chunks
.iter()
.any(|chunk| matches!(chunk, ProviderChunk::Reasoning(value) if value == "alias"))
);
fn test_decode_response_exposes_cached_tokens() {
let text = r#"{
"id": "d21abaa6552741949e2aba76bde59359",
"choices": [{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "你好！",
"role": "assistant",
"tool_calls": null
}
}],
"created": 1781622889,
"model": "mimo-v2.5",
"object": "chat.completion",
"usage": {
"completion_tokens": 65,
"prompt_tokens": 11741,
"total_tokens": 11806,
"prompt_tokens_details": {"cached_tokens": 1200}
}
}"#;
#[tokio::test]
async fn tool_call_arguments_are_assembled_across_sse_events() {
let input = concat!(
"data: {\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":0,\"id\":\"call_1\",\"function\":{\"name\":\"calculator\",\"arguments\":\"{\\\"expression\\\":\"}}]},\"finish_reason\":null}]}\n\n",
"data: {\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":0,\"function\":{\"arguments\":\"\\\"1+1\\\"}\"}}]},\"finish_reason\":\"tool_calls\"}]}\n\n",
"data: [DONE]\n\n"
);
let mut decoder = OpenAISseDecoder::default();
let chunks = decoder.push(input.as_bytes()).unwrap();
let response = crate::providers::collect_provider_stream(Box::pin(
futures_util::stream::iter(chunks.into_iter().map(Ok)),
))
.await
.unwrap();
assert_eq!(response.tool_calls.len(), 1);
assert_eq!(response.tool_calls[0].id, "call_1");
assert_eq!(response.tool_calls[0].name, "calculator");
let response: OpenAIResponse = serde_json::from_str(text).unwrap();
assert_eq!(
response.tool_calls[0].arguments,
serde_json::json!({"expression":"1+1"})
response
.usage
.prompt_tokens_details
.as_ref()
.and_then(|d| d.cached_tokens),
Some(1200)
);
}
#[test]
fn missing_finish_marker_is_an_error() {
let mut decoder = OpenAISseDecoder::default();
decoder
.push(b"data: {\"choices\":[{\"delta\":{\"content\":\"partial\"}}]}\n\n")
.unwrap();
assert!(matches!(
decoder.finish(),
Err(OpenAIStreamError::MissingFinish)
));
}
}

331
src/providers/stream.rs
View File

@ -1,331 +0,0 @@
use std::collections::BTreeMap;
use std::error::Error;
use std::pin::Pin;
use futures_util::{Stream, StreamExt};
use serde::{Deserialize, Serialize};
use crate::bus::ProviderReasoningState;
use super::{ChatCompletionResponse, ToolCall, Usage};
pub type DynProviderError = Box<dyn Error + Send + Sync>;
pub type ProviderStreamItem = Result<ProviderChunk, DynProviderError>;
pub type ProviderStream = Pin<Box<dyn Stream<Item = ProviderStreamItem> + Send>>;
pub(crate) fn diagnostic_request_summary(
model: &str,
message_count: usize,
tool_count: usize,
) -> String {
serde_json::json!({
"model": model,
"message_count": message_count,
"tool_count": tool_count,
"stream": true,
})
.to_string()
}
/// Incremental framing shared by SSE-based providers. It accepts arbitrary
/// byte/UTF-8 boundaries and returns only joined `data:` payloads.
#[derive(Default)]
pub(crate) struct SseFramer {
buffer: Vec<u8>,
}
impl SseFramer {
pub(crate) fn push(&mut self, bytes: &[u8]) -> Result<Vec<String>, std::string::FromUtf8Error> {
self.buffer.extend_from_slice(bytes);
self.drain_frames(false)
}
pub(crate) fn finish(&mut self) -> Result<Vec<String>, std::string::FromUtf8Error> {
self.drain_frames(true)
}
fn drain_frames(&mut self, finish: bool) -> Result<Vec<String>, std::string::FromUtf8Error> {
let mut frames = Vec::new();
while let Some((position, delimiter_len)) = find_sse_delimiter(&self.buffer) {
let frame = self.buffer.drain(..position).collect::<Vec<_>>();
self.buffer.drain(..delimiter_len);
if let Some(data) = sse_data(frame)? {
frames.push(data);
}
}
if finish && !self.buffer.is_empty() {
let frame = std::mem::take(&mut self.buffer);
if let Some(data) = sse_data(frame)? {
frames.push(data);
}
}
Ok(frames)
}
}
fn find_sse_delimiter(buffer: &[u8]) -> Option<(usize, usize)> {
let lf = buffer.windows(2).position(|window| window == b"\n\n");
let crlf = buffer.windows(4).position(|window| window == b"\r\n\r\n");
match (lf, crlf) {
(Some(left), Some(right)) if left <= right => Some((left, 2)),
(Some(_), Some(right)) => Some((right, 4)),
(Some(position), None) => Some((position, 2)),
(None, Some(position)) => Some((position, 4)),
(None, None) => None,
}
}
fn sse_data(frame: Vec<u8>) -> Result<Option<String>, std::string::FromUtf8Error> {
let frame = String::from_utf8(frame)?;
let data = frame
.lines()
.filter_map(|line| {
line.strip_prefix("data:")
.map(|value| value.strip_prefix(' ').unwrap_or(value))
})
.collect::<Vec<_>>()
.join("\n");
Ok((!data.is_empty()).then_some(data))
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum FinishReason {
Stop,
ToolCalls,
Length,
ContentFilter,
Other(String),
}
impl FinishReason {
pub fn from_provider(value: &str) -> Self {
match value {
"stop" | "end_turn" | "stop_sequence" => Self::Stop,
"tool_calls" | "tool_use" => Self::ToolCalls,
"length" | "max_tokens" => Self::Length,
"content_filter" => Self::ContentFilter,
other => Self::Other(other.to_string()),
}
}
}
#[derive(Debug, Clone, PartialEq)]
pub enum ProviderChunk {
Metadata {
id: String,
model: String,
},
Text(String),
Reasoning(String),
ToolCallStart {
index: usize,
id: Option<String>,
name: Option<String>,
},
ToolCallArguments {
index: usize,
delta: String,
},
ProviderState(ProviderReasoningState),
Usage(Usage),
Done(FinishReason),
}
#[derive(Default)]
struct PartialToolCall {
id: Option<String>,
name: Option<String>,
arguments: String,
}
#[derive(Default)]
pub struct ProviderResponseAccumulator {
id: String,
model: String,
content: String,
reasoning_content: String,
provider_state: Option<ProviderReasoningState>,
usage: Usage,
tool_calls: BTreeMap<usize, PartialToolCall>,
}
impl ProviderResponseAccumulator {
pub fn push(&mut self, chunk: ProviderChunk) {
match chunk {
ProviderChunk::Metadata {
id: response_id,
model: response_model,
} => {
self.id = response_id;
self.model = response_model;
}
ProviderChunk::Text(delta) => self.content.push_str(&delta),
ProviderChunk::Reasoning(delta) => self.reasoning_content.push_str(&delta),
ProviderChunk::ToolCallStart {
index,
id: call_id,
name,
} => {
let partial = self.tool_calls.entry(index).or_default();
if call_id.is_some() {
partial.id = call_id;
}
if name.is_some() {
partial.name = name;
}
}
ProviderChunk::ToolCallArguments { index, delta } => {
self.tool_calls
.entry(index)
.or_default()
.arguments
.push_str(&delta);
}
ProviderChunk::ProviderState(state) => self.provider_state = Some(state),
ProviderChunk::Usage(value) => self.usage = value,
ProviderChunk::Done(_) => {}
}
}
pub fn finish(self) -> ChatCompletionResponse {
let tool_calls = self
.tool_calls
.into_iter()
.map(|(index, partial)| ToolCall {
id: partial.id.unwrap_or_else(|| format!("tool_call_{index}")),
name: partial.name.unwrap_or_default(),
arguments: serde_json::from_str(&partial.arguments)
.unwrap_or(serde_json::Value::Null),
})
.collect();
ChatCompletionResponse {
id: self.id,
model: self.model,
content: self.content,
reasoning_content: (!self.reasoning_content.is_empty())
.then_some(self.reasoning_content),
provider_state: self.provider_state,
tool_calls,
usage: self.usage,
}
}
}
pub async fn collect_provider_stream(
mut provider_stream: ProviderStream,
) -> Result<ChatCompletionResponse, DynProviderError> {
let mut accumulator = ProviderResponseAccumulator::default();
while let Some(chunk) = provider_stream.next().await {
accumulator.push(chunk?);
}
Ok(accumulator.finish())
}
#[cfg(test)]
pub fn provider_stream_for_test(response: ChatCompletionResponse) -> ProviderStream {
let finish_reason = if response.tool_calls.is_empty() {
FinishReason::Stop
} else {
FinishReason::ToolCalls
};
let mut chunks = vec![ProviderChunk::Metadata {
id: response.id,
model: response.model,
}];
if let Some(reasoning) = response.reasoning_content {
chunks.push(ProviderChunk::Reasoning(reasoning));
}
if !response.content.is_empty() {
chunks.push(ProviderChunk::Text(response.content));
}
for (index, call) in response.tool_calls.into_iter().enumerate() {
chunks.push(ProviderChunk::ToolCallStart {
index,
id: Some(call.id),
name: Some(call.name),
});
chunks.push(ProviderChunk::ToolCallArguments {
index,
delta: serde_json::to_string(&call.arguments).unwrap_or_else(|_| "null".to_string()),
});
}
if let Some(state) = response.provider_state {
chunks.push(ProviderChunk::ProviderState(state));
}
chunks.push(ProviderChunk::Usage(response.usage));
chunks.push(ProviderChunk::Done(finish_reason));
Box::pin(futures_util::stream::iter(chunks.into_iter().map(Ok)))
}
#[cfg(test)]
mod tests {
use super::*;
use futures_util::stream;
#[test]
fn diagnostic_summary_contains_counts_without_message_content() {
let summary = diagnostic_request_summary("model", 3, 2);
assert_eq!(
serde_json::from_str::<serde_json::Value>(&summary).unwrap(),
serde_json::json!({
"model": "model",
"message_count": 3,
"tool_count": 2,
"stream": true
})
);
assert!(!summary.contains("reasoning"));
assert!(!summary.contains("signature"));
}
#[tokio::test]
async fn collector_assembles_interleaved_tool_argument_fragments() {
let chunks = vec![
ProviderChunk::Metadata {
id: "response".into(),
model: "model".into(),
},
ProviderChunk::Reasoning("why".into()),
ProviderChunk::Text("answer".into()),
ProviderChunk::ToolCallStart {
index: 1,
id: Some("second".into()),
name: Some("b".into()),
},
ProviderChunk::ToolCallArguments {
index: 1,
delta: "{\"n\":".into(),
},
ProviderChunk::ToolCallStart {
index: 0,
id: Some("first".into()),
name: Some("a".into()),
},
ProviderChunk::ToolCallArguments {
index: 0,
delta: "{}".into(),
},
ProviderChunk::ToolCallArguments {
index: 1,
delta: "2}".into(),
},
ProviderChunk::Done(FinishReason::ToolCalls),
];
let response = collect_provider_stream(Box::pin(stream::iter(chunks.into_iter().map(Ok))))
.await
.unwrap();
assert_eq!(response.content, "answer");
assert_eq!(response.reasoning_content.as_deref(), Some("why"));
assert_eq!(response.tool_calls.len(), 2);
assert_eq!(response.tool_calls[0].id, "first");
assert_eq!(
response.tool_calls[1].arguments,
serde_json::json!({"n": 2})
);
}
}

21
src/providers/traits.rs
View File

@ -2,17 +2,12 @@ use crate::bus::message::ContentBlock;
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use super::stream::{DynProviderError, ProviderStream, collect_provider_stream};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Message {
pub role: String,
pub content: Vec<ContentBlock>,
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_content: Option<String>,
/// Opaque state replayed only by the provider that produced it.
#[serde(skip)]
pub provider_state: Option<crate::bus::ProviderReasoningState>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
@ -27,7 +22,6 @@ impl Message {
role: "user".to_string(),
content: vec![ContentBlock::text(content)],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: None,
@ -39,7 +33,6 @@ impl Message {
role: "assistant".to_string(),
content: vec![ContentBlock::text(content)],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: None,
@ -51,7 +44,6 @@ impl Message {
role: "system".to_string(),
content: vec![ContentBlock::text(content)],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: None,
@ -67,7 +59,6 @@ impl Message {
role: "tool".to_string(),
content: vec![ContentBlock::text(content)],
reasoning_content: None,
provider_state: None,
tool_call_id: Some(tool_call_id.into()),
name: Some(tool_name.into()),
tool_calls: None,
@ -110,12 +101,11 @@ pub struct ChatCompletionResponse {
pub model: String,
pub content: String,
pub reasoning_content: Option<String>,
pub provider_state: Option<crate::bus::ProviderReasoningState>,
pub tool_calls: Vec<ToolCall>,
pub usage: Usage,
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Usage {
pub prompt_tokens: u32,
pub completion_tokens: u32,
@ -130,17 +120,10 @@ pub struct Usage {
#[async_trait]
pub trait LLMProvider: Send + Sync {
async fn stream(
&self,
request: ChatCompletionRequest,
) -> Result<ProviderStream, DynProviderError>;
async fn chat(
&self,
request: ChatCompletionRequest,
) -> Result<ChatCompletionResponse, DynProviderError> {
collect_provider_stream(self.stream(request).await?).await
}
) -> Result<ChatCompletionResponse, Box<dyn std::error::Error + Send + Sync>>;
fn ptype(&self) -> &str;

673
src/scheduler/mod.rs
View File

@ -3,41 +3,81 @@ pub mod types;
use std::sync::Arc;
use std::time::Instant;
use tokio::task::JoinSet;
use futures_util::stream::{self, StreamExt};
use tokio::time;
use crate::config::SchedulerConfig;
use crate::session::{ScheduledDeliveryError, SessionManager};
use crate::storage::{
ClaimedScheduledRun, JobRun, ScheduledOutcomeKind, ScheduledRunCompletion, ScheduledRunStatus,
Storage,
};
use crate::session::SessionManager;
use crate::session::session::HandleResult;
use crate::storage::ScheduledJob;
use crate::storage::Storage;
use crate::storage::{DeliveryPolicy, JobKind, JobRun};
pub use types::Schedule;
#[derive(Debug, Clone, PartialEq, Eq)]
enum ScheduledDisposition {
Content(String),
Quiet(String),
ReportedFailure(String),
Refused(String),
}
fn parse_scheduled_disposition(output: &str) -> ScheduledDisposition {
let trimmed = output.trim();
if trimmed.eq_ignore_ascii_case("NO_REPLY") {
return ScheduledDisposition::Quiet(String::new());
}
let upper = trimmed.to_ascii_uppercase();
for (prefix, kind) in [
("NO_REPLY[INFO]", "info"),
("NO_REPLY[FAIL]", "fail"),
("NO_REPLY[REFUSE]", "refuse"),
] {
if upper.starts_with(prefix) {
let suffix = &trimmed[prefix.len()..];
if !suffix.is_empty() && !suffix.trim_start().starts_with(':') {
continue;
}
let reason = suffix.trim().trim_start_matches(':').trim().to_string();
return match kind {
"info" => ScheduledDisposition::Quiet(reason),
"fail" => ScheduledDisposition::ReportedFailure(reason),
_ => ScheduledDisposition::Refused(reason),
};
}
}
if trimmed.is_empty() {
ScheduledDisposition::ReportedFailure("scheduled agent returned empty output".into())
} else {
ScheduledDisposition::Content(trimmed.to_string())
}
}
/// Compute the next execution time (Unix ms) for a schedule, given `from` (Unix ms).
/// Returns `None` if no next time can be determined (e.g. an invalid cron expression).
/// Returns `None` if no next time can be determined (e.g., invalid cron expression).
pub fn next_run_for_schedule(schedule: &Schedule, from: i64) -> Option<i64> {
use chrono::{TimeZone, Utc};
use std::str::FromStr;
match schedule {
Schedule::At { at } => Some(*at),
Schedule::Every { every_ms } => Some(from.saturating_add(i64::try_from(*every_ms).ok()?)),
Schedule::Every { every_ms } => Some(from + *every_ms as i64),
Schedule::Cron { expr, tz } => {
let cron_schedule = cron::Schedule::from_str(expr.as_str()).ok()?;
let from_secs = from / 1000;
let from_nanos = ((from % 1000) * 1_000_000) as u32;
let from_dt = Utc.timestamp_opt(from_secs, from_nanos).single()?;
let next_utc = if let Some(tz_str) = tz {
let tz: chrono_tz::Tz = tz_str.parse().ok()?;
cron_schedule
.after(&from_dt.with_timezone(&tz))
.next()?
.with_timezone(&Utc)
let from_local = from_dt.with_timezone(&tz);
let next_local = cron_schedule.after(&from_local).next()?;
next_local.with_timezone(&Utc)
} else {
cron_schedule.after(&from_dt).next()?
};
Some(next_utc.timestamp_millis())
}
}
@ -50,12 +90,13 @@ fn now_ms() -> i64 {
.as_millis() as i64
}
/// The scheduler runs as a background tokio task, periodically checking for due jobs
/// and executing them via `SessionManager::handle_cron_message`.
pub struct Scheduler {
storage: Arc<Storage>,
session_manager: Arc<SessionManager>,
config: SchedulerConfig,
owner: String,
admission: crate::gateway::reload::RuntimeAdmission,
}
impl Scheduler {
@ -63,40 +104,24 @@ impl Scheduler {
storage: Arc<Storage>,
session_manager: Arc<SessionManager>,
config: SchedulerConfig,
) -> Self {
Self::with_admission(
storage,
session_manager,
config,
crate::gateway::reload::RuntimeAdmission::open(),
)
}
pub(crate) fn with_admission(
storage: Arc<Storage>,
session_manager: Arc<SessionManager>,
config: SchedulerConfig,
admission: crate::gateway::reload::RuntimeAdmission,
) -> Self {
Self {
storage,
session_manager,
config,
owner: uuid::Uuid::new_v4().to_string(),
admission,
}
}
/// Non-blocking event loop. Execution and delivery use separate bounded
/// JoinSets so one long Agent run cannot delay other claims or outbox work.
/// Claim due jobs with a durable lease, then execute the claimed batch with
/// bounded concurrency.
pub async fn run(self: Arc<Self>) {
let poll_duration = time::Duration::from_secs(self.config.poll_interval_secs.max(1));
let mut interval = time::interval(poll_duration);
interval.set_missed_tick_behavior(time::MissedTickBehavior::Skip);
// Keep accidental configuration values from claiming an unbounded
// batch and overwhelming the runtime or SQLite parameter conversion.
let max_concurrent = self.config.max_concurrent.clamp(1, 256);
let max_delivery = max_concurrent.clamp(1, 16);
let mut runs = JoinSet::new();
let mut deliveries = JoinSet::new();
tracing::info!(
poll_interval_secs = self.config.poll_interval_secs,
@ -106,306 +131,259 @@ impl Scheduler {
);
loop {
tokio::select! {
_ = interval.tick() => {}
Some(result) = runs.join_next(), if !runs.is_empty() => {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled run task panicked");
}
}
Some(result) = deliveries.join_next(), if !deliveries.is_empty() => {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled delivery task panicked");
}
}
}
while let Some(result) = runs.try_join_next() {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled run task panicked");
}
}
while let Some(result) = deliveries.try_join_next() {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled delivery task panicked");
}
}
if !self.admission.is_accepting() {
continue;
}
interval.tick().await;
let now = now_ms();
let delivery_slots = max_delivery.saturating_sub(deliveries.len());
if delivery_slots > 0 {
let lease_until = now.saturating_add(180_000);
let delivery_owner = format!("{}:delivery:{}", self.owner, uuid::Uuid::new_v4());
match self
.storage
.claim_scheduled_deliveries(now, lease_until, &delivery_owner, delivery_slots)
.await
{
Ok(claimed) => {
for run in claimed {
let scheduler = self.clone();
deliveries.spawn(async move {
scheduler.deliver_claimed_run(run).await;
});
}
}
Err(error) => {
tracing::error!(error = %error, "scheduler: failed to claim deliveries");
}
}
}
let run_slots = max_concurrent.saturating_sub(runs.len());
if run_slots == 0 {
continue;
}
let lease_ms = self
.config
.execution_timeout_secs
.saturating_add(150)
.saturating_mul(1000)
.min(i64::MAX as u64) as i64;
let run_owner = format!("{}:run:{}", self.owner, uuid::Uuid::new_v4());
match self
let lease_until = now.saturating_add(lease_ms);
let jobs = match self
.storage
.claim_due_scheduled_runs(now, now.saturating_add(lease_ms), &run_owner, run_slots)
.claim_due_scheduled_jobs(now, lease_until, &self.owner, max_concurrent)
.await
{
Ok(claimed) => {
for run in claimed {
let scheduler = self.clone();
runs.spawn(async move {
scheduler.execute_claimed_run(run).await;
});
}
}
Ok(jobs) => jobs,
Err(error) => {
tracing::error!(error = %error, "scheduler: failed to claim due runs");
tracing::error!(error = %error, "scheduler: failed to claim due jobs");
continue;
}
};
if jobs.is_empty() {
continue;
}
tracing::info!(count = jobs.len(), "scheduler: claimed due jobs");
stream::iter(jobs)
.for_each_concurrent(max_concurrent, |job| {
let scheduler = self.clone();
async move { scheduler.execute_claimed_job(job).await }
})
.await;
}
}
async fn execute_claimed_run(self: Arc<Self>, claimed: ClaimedScheduledRun) {
async fn execute_claimed_job(self: Arc<Self>, job: ScheduledJob) {
let start = Instant::now();
let job = &claimed.job;
let (completion, agent_execution) = if let Some(_activity) = self.admission.try_enter() {
match self.session_manager.agent_coordinator() {
Some(coordinator) => match coordinator
.execute_scheduled(
claimed.run_id,
&claimed.owner,
let started_at = now_ms();
tracing::info!(job_id = %job.id, job_name = %job.name, "scheduler: executing claimed job");
let managed = job.delivery_policy != DeliveryPolicy::Direct;
let execution = async {
if managed {
self.session_manager
.handle_managed_scheduled_message(
&job.prompt,
&job.id,
&job.name,
job.agent_id.as_deref(),
job.job_kind == JobKind::Monitor,
)
.await
.map(HandleResult::AgentResponse)
} else {
self.session_manager
.handle_cron_message(
&job.channel,
&job.chat_id,
&job.prompt,
self.config.execution_timeout_secs.max(1),
)
.await
{
Ok(execution) => {
if execution.status == ScheduledRunStatus::Completed
&& let Some(outcome) = execution.outcome.clone()
{
(
ScheduledRunCompletion {
status: ScheduledRunStatus::Completed,
outcome: outcome.kind,
message: outcome.message,
diagnostic: execution.error.clone(),
duration_ms: start.elapsed().as_millis() as i64,
},
Some(execution),
)
} else {
let diagnostic = execution.error.clone().unwrap_or_else(|| {
"scheduled Agent returned ordinary text without calling complete_scheduled_run"
.to_string()
});
let status = match execution.status {
ScheduledRunStatus::TimedOut => ScheduledRunStatus::TimedOut,
ScheduledRunStatus::Interrupted | ScheduledRunStatus::Cancelled => {
ScheduledRunStatus::Interrupted
}
_ => ScheduledRunStatus::Failed,
};
(
ScheduledRunCompletion {
status,
outcome: ScheduledOutcomeKind::Failed,
message: if status == ScheduledRunStatus::TimedOut {
format!("定时任务「{}」执行超时。", job.name)
} else if status == ScheduledRunStatus::Interrupted {
format!("定时任务「{}」在系统关停时被中断。", job.name)
} else {
format!(
"定时任务「{}」未可靠完成：Agent 未提交结构化运行结果。",
job.name
)
},
diagnostic: Some(diagnostic),
duration_ms: start.elapsed().as_millis() as i64,
},
Some(execution),
)
}
}
Err(error) => (
ScheduledRunCompletion {
status: ScheduledRunStatus::Failed,
outcome: ScheduledOutcomeKind::Failed,
message: job.agent_id.as_deref().map_or_else(
|| format!("定时任务「{}」未能启动 Root Agent 执行。", job.name),
|agent_id| {
format!(
"定时任务「{}」无法使用 Agent「{}」执行。请恢复该 Agent 定义，或更新任务的 agent_id。",
job.name, agent_id
)
},
),
diagnostic: Some(error.to_string()),
duration_ms: start.elapsed().as_millis() as i64,
},
None,
),
},
None => (
ScheduledRunCompletion {
status: ScheduledRunStatus::Failed,
outcome: ScheduledOutcomeKind::Failed,
message: format!("定时任务「{}」未能启动执行。", job.name),
diagnostic: Some("AgentCoordinator is unavailable".to_string()),
duration_ms: start.elapsed().as_millis() as i64,
},
None,
),
}
} else {
(
ScheduledRunCompletion {
status: ScheduledRunStatus::Interrupted,
outcome: ScheduledOutcomeKind::Failed,
message: format!("定时任务「{}」因 Gateway 重载而中断。", job.name),
diagnostic: Some("runtime admission closed".to_string()),
duration_ms: start.elapsed().as_millis() as i64,
},
None,
)
};
let finished_at = now_ms();
let mut commit_attempt = 0_u64;
let commit = loop {
let result = match agent_execution.as_ref() {
Some(execution) => {
self.storage
.finish_scheduled_run_with_agent(
claimed.run_id,
&claimed.owner,
&completion,
&execution.agent_run_id,
&execution.agent_run_id,
execution.runtime_generation,
&execution.agent_terminal,
finished_at,
)
.await
}
None => {
self.storage
.finish_scheduled_run(
claimed.run_id,
&claimed.owner,
&completion,
finished_at,
&job.id,
&job.name,
)
.await
}
};
match result {
Err(error) if error.is_transient() && commit_attempt < 2 => {
commit_attempt += 1;
tracing::warn!(
job_id = %job.id,
run_id = claimed.run_id,
attempt = commit_attempt + 1,
error = %error,
"scheduler: retrying transient run completion commit"
);
time::sleep(time::Duration::from_millis(50 * commit_attempt)).await;
}
result => break result,
}
};
match commit {
Ok(true) => tracing::info!(
job_id = %job.id,
run_id = claimed.run_id,
status = completion.status.as_str(),
outcome = completion.outcome.as_str(),
duration_ms = completion.duration_ms,
"scheduler: run completed"
),
Ok(false) => tracing::warn!(
job_id = %job.id,
run_id = claimed.run_id,
"scheduler: late run result discarded"
),
Err(error) => tracing::error!(
job_id = %job.id,
run_id = claimed.run_id,
error = %error,
"scheduler: failed to commit run completion"
),
}
}
async fn deliver_claimed_run(self: Arc<Self>, run: JobRun) {
let Some(delivery_owner) = run.delivery_lease_owner.clone() else {
tracing::error!(
run_id = run.id,
"scheduler: claimed delivery has no lease owner"
);
return;
};
let result = self
.session_manager
.deliver_scheduled_run(&run, &delivery_owner)
let result = time::timeout(
time::Duration::from_secs(self.config.execution_timeout_secs.max(1)),
execution,
)
.await;
let (delivered, permanent, error) = match result {
Ok(()) => (true, false, None),
Err(ScheduledDeliveryError::Transient(error)) => {
(false, false, Some(sanitize_error(&error)))
let finished_at = now_ms();
let duration_ms = start.elapsed().as_millis() as i64;
let (mut status, output, error, result_kind, mut delivery_status, mut delivery_error) =
match result {
Ok(Ok(
HandleResult::AgentResponse(output) | HandleResult::CommandOutput(output),
)) => {
let output = if output.len() > 8000 {
format!(
"{}...[truncated]",
&output[..output.ceil_char_boundary(8000)]
)
} else {
output
};
if !managed {
(
"ok".into(),
Some(output),
None,
None,
Some("direct".into()),
None,
)
} else {
let disposition = parse_scheduled_disposition(&output);
let (kind, content, alert) = match &disposition {
ScheduledDisposition::Content(value) => {
("content", Some(value.as_str()), true)
}
Err(ScheduledDeliveryError::Permanent(error)) => {
(false, true, Some(sanitize_error(&error)))
ScheduledDisposition::Quiet(_) => ("quiet", None, false),
ScheduledDisposition::ReportedFailure(value) => {
("reported_failure", Some(value.as_str()), true)
}
ScheduledDisposition::Refused(value) => {
("refused", Some(value.as_str()), true)
}
};
if let Err(commit_error) = self
.storage
.complete_scheduled_delivery(
run.id,
&delivery_owner,
delivered,
permanent,
error.as_deref(),
now_ms(),
let should_deliver = match job.delivery_policy {
DeliveryPolicy::Always => true,
DeliveryPolicy::OnAlert => alert,
DeliveryPolicy::Never => false,
DeliveryPolicy::Direct => false,
};
if should_deliver {
let message = content.unwrap_or("巡检完成，未发现需要关注的问题。");
match self
.session_manager
.deliver_scheduled_message(
&job.channel,
&job.chat_id,
&job.id,
&job.name,
message,
)
.await
{
tracing::error!(
run_id = run.id,
error = %commit_error,
"scheduler: failed to commit delivery receipt"
Ok(()) => (
"ok".into(),
Some(output),
None,
Some(kind.into()),
Some("delivered".into()),
None,
),
Err(delivery_error) => (
"delivery_error".into(),
Some(output),
None,
Some(kind.into()),
Some("failed".into()),
Some(delivery_error),
),
}
} else {
let delivery = if job.delivery_policy == DeliveryPolicy::Never {
"skipped"
} else {
"suppressed"
};
(
"ok".into(),
Some(output),
None,
Some(kind.into()),
Some(delivery.into()),
None,
)
}
}
}
Ok(Ok(HandleResult::AgentProcessing)) => (
"error".to_string(),
None,
Some("cron execution returned asynchronous processing".to_string()),
None,
None,
None,
),
Ok(Err(error)) => (
"error".to_string(),
None,
Some(error.to_string()),
None,
None,
None,
),
Err(_) => (
"timeout".to_string(),
None,
Some(format!(
"execution exceeded {} seconds",
self.config.execution_timeout_secs.max(1)
)),
None,
None,
None,
),
};
if managed
&& delivery_status.is_none()
&& job.delivery_policy != DeliveryPolicy::Never
&& let Some(message) = error.as_deref()
{
let notice = format!("定时任务「{}」执行失败：{}", job.name, message);
match self
.session_manager
.deliver_scheduled_message(&job.channel, &job.chat_id, &job.id, &job.name, &notice)
.await
{
Ok(()) => delivery_status = Some("delivered".into()),
Err(error) => {
status = "delivery_error".into();
delivery_status = Some("failed".into());
delivery_error = Some(error);
}
}
}
let (next_run_at, disable, delete) = match &job.schedule {
Schedule::At { .. } => (None, !job.delete_after_run, job.delete_after_run),
Schedule::Every { .. } | Schedule::Cron { .. } => {
match next_run_for_schedule(&job.schedule, finished_at) {
Some(next) => (Some(next), false, false),
None => (None, true, false),
}
}
};
let run = JobRun {
id: 0,
job_id: job.id.clone(),
started_at,
finished_at,
status,
output,
error,
duration_ms,
result_kind,
delivery_status,
delivery_error,
};
if let Err(error) = self
.storage
.complete_scheduled_job(&run, &self.owner, next_run_at, disable, delete)
.await
{
tracing::error!(job_id = %job.id, error = %error, "scheduler: failed to commit job completion");
let _ = self
.storage
.release_scheduled_job_lease(&job.id, &self.owner)
.await;
return;
}
tracing::info!(
job_id = %job.id,
status = %run.status,
duration_ms,
"scheduler: job completed"
);
}
}
}
fn sanitize_error(error: &str) -> String {
error.chars().take(1_024).collect()
}
#[cfg(test)]
@ -413,33 +391,102 @@ mod tests {
use super::*;
#[test]
fn next_run_for_every_uses_claim_time() {
assert_eq!(
next_run_for_schedule(&Schedule::Every { every_ms: 5_000 }, 1_000),
Some(6_000)
);
fn test_next_run_at_schedule() {
let now = 1000000;
let next = next_run_for_schedule(&Schedule::At { at: 2000000 }, now);
assert_eq!(next, Some(2000000));
}
#[test]
fn next_run_for_at_keeps_absolute_timestamp() {
assert_eq!(
next_run_for_schedule(&Schedule::At { at: 2_000 }, 1_000),
Some(2_000)
);
fn test_next_run_every_schedule() {
let now = 1000000;
let next = next_run_for_schedule(&Schedule::Every { every_ms: 5000 }, now);
assert_eq!(next, Some(1005000));
}
#[test]
fn cron_timezone_uses_from_argument() {
fn test_next_run_cron_every_minute() {
let expr = "0 * * * * *".to_string();
let schedule = Schedule::Cron { expr, tz: None };
let now = 1000000;
let next = next_run_for_schedule(&schedule, now);
assert!(next.is_some());
assert!(next.unwrap() > now);
}
#[test]
fn test_next_run_cron_every_day_at_9am() {
let expr = "0 0 9 * * *".to_string();
let schedule = Schedule::Cron { expr, tz: None };
let now = 1000000;
let next = next_run_for_schedule(&schedule, now);
assert!(next.is_some());
let next_ms = next.unwrap();
assert!(next_ms > now);
}
#[test]
fn test_next_run_cron_uses_from_argument() {
let expr = "0 * * * * *".to_string();
let schedule = Schedule::Cron { expr, tz: None };
let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T12:34:20Z")
.unwrap()
.timestamp_millis();
let next = next_run_for_schedule(&schedule, from).unwrap();
let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T12:35:00Z")
.unwrap()
.timestamp_millis();
assert_eq!(next, expected);
}
#[test]
fn scheduled_disposition_is_fail_safe() {
assert!(matches!(
parse_scheduled_disposition("NO_REPLY"),
ScheduledDisposition::Quiet(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[INFO]: healthy"),
ScheduledDisposition::Quiet(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[FAIL]: timeout"),
ScheduledDisposition::ReportedFailure(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[REFUSE]: denied"),
ScheduledDisposition::Refused(_)
));
assert!(matches!(
parse_scheduled_disposition("text mentioning NO_REPLY"),
ScheduledDisposition::Content(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[INFO] but this is content"),
ScheduledDisposition::Content(_)
));
assert!(matches!(
parse_scheduled_disposition(""),
ScheduledDisposition::ReportedFailure(_)
));
}
#[test]
fn test_next_run_cron_timezone_uses_from_argument() {
let expr = "0 0 9 * * *".to_string();
let schedule = Schedule::Cron {
expr: "0 0 9 * * *".to_string(),
expr,
tz: Some("Asia/Shanghai".to_string()),
};
let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T00:30:00Z")
.unwrap()
.timestamp_millis();
let next = next_run_for_schedule(&schedule, from).unwrap();
let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T01:00:00Z")
.unwrap()
.timestamp_millis();
assert_eq!(next_run_for_schedule(&schedule, from), Some(expected));
assert_eq!(next, expected);
}
}

13
src/session/commands.rs
View File

@ -28,19 +28,6 @@ pub enum SessionCommand {
},
/// Load the active task plan for a dialog.
GetTaskPlan { session_id: UnifiedSessionId },
/// Load token totals and context-window state for a dialog.
GetSessionStats { session_id: UnifiedSessionId },
/// Load the durable Agent run projection for a dialog.
GetAgentRuns {
session_id: UnifiedSessionId,
cursor: Option<String>,
limit: u32,
},
/// Load one durable Agent run projection.
GetAgentRun {
session_id: UnifiedSessionId,
run_id: String,
},
/// Get the current dialog for a chat
GetCurrentDialog { channel: String, chat_id: String },
/// Rename a dialog

15
src/session/events.rs
View File

@ -41,21 +41,6 @@ pub enum SessionEvent {
session_id: UnifiedSessionId,
plan: Option<crate::work::TaskPlan>,
},
/// Provider usage totals and current context-window state.
SessionStats { stats: crate::session::SessionStats },
/// Durable Agent run projection page for a dialog.
AgentRuns {
session_id: UnifiedSessionId,
revision: i64,
runs: Vec<crate::protocol::AgentRunView>,
next_cursor: Option<String>,
},
/// One durable Agent run projection.
AgentRun {
session_id: UnifiedSessionId,
revision: i64,
run: Option<crate::protocol::AgentRunView>,
},
/// Dialog renamed
DialogRenamed {
session_id: UnifiedSessionId,

167
src/session/messenger.rs
View File

@ -2,15 +2,10 @@ use std::collections::HashMap;
use crate::bus::{ChatMessage, MediaItem, MessageSource, OutboundMessage, SourceKind};
use crate::session::UnifiedSessionId;
use crate::tools::{OutboundDelivery, OutboundMessenger};
use crate::tools::OutboundMessenger;
use super::persistence::{
append_active_turn_message, append_persisted_message_if_absent, append_persisted_messages,
};
use super::session::{
CURRENT_SOURCE_SESSION, CURRENT_TURN_DELIVERIES, CURRENT_TURN_ID, PendingTurnDelivery,
SessionManager,
};
use super::persistence::append_persisted_messages;
use super::session::{CURRENT_SOURCE_SESSION, SessionManager};
#[async_trait::async_trait]
impl OutboundMessenger for SessionManager {
@ -22,7 +17,7 @@ impl OutboundMessenger for SessionManager {
content: &str,
mut source: MessageSource,
media: Vec<MediaItem>,
) -> Result<OutboundDelivery, String> {
) -> Result<(), String> {
if source.from_session.is_none() {
source.from_session = CURRENT_SOURCE_SESSION
.try_with(|value| value.clone())
@ -52,32 +47,6 @@ impl OutboundMessenger for SessionManager {
let origin = source.from_session.as_deref().unwrap_or("unknown");
let origin_id = source.from_session.clone();
let same_session = source.from_session.as_deref() == Some(target_sid.to_string().as_str());
let current_turn_id = CURRENT_TURN_ID
.try_with(|value| value.clone())
.ok()
.flatten();
let current_turn_deliveries = CURRENT_TURN_DELIVERIES
.try_with(|value| value.clone())
.ok()
.flatten();
if same_session
&& channel == "cli_chat"
&& !media.is_empty()
&& let (Some(turn_id), Some(deliveries)) =
(current_turn_id.clone(), current_turn_deliveries)
{
let guard = session.lock().await;
if guard.owns_active_turn(&turn_id) {
deliveries
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.push(PendingTurnDelivery {
content: content.to_string(),
media,
});
return Ok(OutboundDelivery::AttachedToCurrentTurn);
}
}
let marked_content = if content.trim().is_empty() && !media.is_empty() && same_session {
String::new()
} else {
@ -86,13 +55,8 @@ impl OutboundMessenger for SessionManager {
let message = outbound_history_message(marked_content.clone(), source, &media);
let message_id = message.id.clone();
if same_session && let Some(turn_id) = current_turn_id {
// Ownership is revalidated atomically with the in-memory append;
// a concurrent /stop therefore falls back to a versioned write.
append_active_turn_message(&session, message, turn_id).await
} else {
append_persisted_messages(&session, vec![message]).await
}
append_persisted_messages(&session, vec![message])
.await
.map_err(|error| error.to_string())?;
if let Some(origin_id) = origin_id {
@ -114,112 +78,37 @@ impl OutboundMessenger for SessionManager {
delivery: None,
})
.await
.map_err(|error| error.to_string())?;
Ok(OutboundDelivery::Delivered)
.map_err(|error| error.to_string())
}
}
impl SessionManager {
pub async fn deliver_scheduled_run(
pub async fn deliver_scheduled_message(
&self,
run: &crate::storage::JobRun,
delivery_owner: &str,
) -> Result<(), ScheduledDeliveryError> {
let content = run
.message
.as_deref()
.unwrap_or("定时任务已结束，但没有生成可投递的结果。请在任务运行记录中查看诊断信息。");
let target_sid = if let Some(session_id) = run.target_session_id.as_deref() {
UnifiedSessionId::parse(session_id).ok_or_else(|| {
ScheduledDeliveryError::Permanent("stored target session is invalid".to_string())
})?
} else {
let resolved = self
.resolve_dialog_id(&run.target_channel, &run.target_chat_id)
.await
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
let fixed = self
.storage
.set_scheduled_delivery_target_session(
run.id,
delivery_owner,
&resolved.to_string(),
chrono::Utc::now().timestamp_millis(),
)
.await
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?
.ok_or_else(|| {
ScheduledDeliveryError::Transient(
"scheduled delivery lost its claim before fixing target session"
.to_string(),
)
})?;
UnifiedSessionId::parse(&fixed).ok_or_else(|| {
ScheduledDeliveryError::Permanent("fixed target session is invalid".to_string())
})?
};
if target_sid.channel != run.target_channel || target_sid.chat_id != run.target_chat_id {
return Err(ScheduledDeliveryError::Permanent(
"fixed target session does not belong to the scheduled destination".to_string(),
));
}
let session = self
.get_or_activate_session(&target_sid)
.await
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
let source = MessageSource {
channel: &str,
chat_id: &str,
job_id: &str,
job_name: &str,
content: &str,
) -> Result<(), String> {
<Self as OutboundMessenger>::send_message(
self,
channel,
chat_id,
None,
content,
MessageSource {
kind: SourceKind::ExternalTrigger,
from_channel: Some("scheduler".to_string()),
from_session: Some(format!("scheduled-run:{}", run.id)),
from_session: Some(format!("cron:{job_id}")),
from_user_id: None,
system_name: Some("scheduled task".to_string()),
task_id: Some(run.job_id.clone()),
from_run_id: run.agent_run_id.clone(),
from_agent_id: run.agent_id.clone(),
};
let mut message = outbound_history_message(content, source, &[]);
message.id = format!("scheduled:{}", run.id);
let message_id = message.id.clone();
append_persisted_message_if_absent(&session, message)
system_name: Some(job_name.to_string()),
task_id: Some(job_id.to_string()),
},
Vec::new(),
)
.await
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
let metadata = HashMap::from([
("_session_id".to_string(), target_sid.to_string()),
("_message_id".to_string(), message_id),
("scheduled_delivery_id".to_string(), run.id.to_string()),
]);
self.bus
.deliver_outbound(OutboundMessage {
channel: run.target_channel.clone(),
chat_id: run.target_chat_id.clone(),
content: content.to_string(),
reply_to: None,
media: Vec::new(),
metadata,
delivery: None,
})
.await
.map_err(|error| match error {
crate::bus::BusError::Closed
| crate::bus::BusError::DeliveryTimedOut
| crate::bus::BusError::DeliveryTransient(_) => {
ScheduledDeliveryError::Transient(error.to_string())
}
crate::bus::BusError::DeliveryPermanent(summary) => {
ScheduledDeliveryError::Permanent(summary)
}
})?;
Ok(())
}
}
#[derive(Debug, thiserror::Error)]
pub enum ScheduledDeliveryError {
#[error("transient scheduled delivery failure: {0}")]
Transient(String),
#[error("permanent scheduled delivery failure: {0}")]
Permanent(String),
}
fn outbound_history_message(
@ -246,8 +135,6 @@ mod tests {
from_user_id: None,
system_name: None,
task_id: None,
from_run_id: None,
from_agent_id: None,
};
let media = vec![MediaItem::new("/tmp/report.pdf", "file")];

14
src/session/mod.rs
View File

@ -3,25 +3,13 @@ pub mod error;
pub mod events;
mod messenger;
mod persistence;
mod turn_input;
// The public `session::session` path is retained for API compatibility.
#[allow(clippy::module_inception)]
pub mod session;
pub mod session_id;
pub mod stats;
pub mod turn;
pub use commands::SessionCommand;
pub use error::SessionError;
pub use events::{DialogInfo, SessionEvent};
pub use messenger::ScheduledDeliveryError;
pub use session::{
AgentCatalogPreparation, SLASH_COMMANDS, Session, SessionManager, SessionManagerServices,
SlashCommand,
};
pub use session::{SLASH_COMMANDS, Session, SessionManager, SlashCommand};
pub use session_id::UnifiedSessionId;
pub use stats::{ContextUsage, ContextUsageSource, LifetimeUsage, SessionStats};
pub use turn::{
BlockId, ToolStatus, TurnBlock, TurnController, TurnId, TurnPhase, TurnSnapshot, TurnState,
TurnStatus,
};

Some files were not shown because too many files have changed in this diff Show More