xiaoxixi
5501c539fc
- drop agent_run_groups table and group_id/scope_kind/scope_id columns (schema v8) - remove group_id from AgentExecutionContext and recovery group counters - flatten TasksPage background tab into a per-run list - add WebUI Agents page with definition CRUD and inline provider/model - bump version to 1.11.0
159 lines
18 KiB
Markdown
159 lines
18 KiB
Markdown
# PicoBot
|
||
|
||
This file is the operational contract for coding agents working in this repository. Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before making architectural or lifecycle changes. Code and tests are the final source of truth; update the document when an architectural invariant changes.
|
||
|
||
## Build & Run
|
||
|
||
- `cargo build` — build the binary
|
||
- `cargo run -- gateway` — start gateway server (binds `127.0.0.1:19876` by default)
|
||
- `cargo run -- chat` — connect to gateway as CLI client (default `ws://127.0.0.1:19876/ws`)
|
||
- `cargo run -- run "prompt"` — send one prompt through Gateway, print the terminal Turn, and exit; stdin, JSON, verbose progress, and timeout modes are available
|
||
- `cargo run -- reload` — validate and gracefully reload a running Gateway's configuration
|
||
- `cargo run -- health [--json]` — check core, configuration-dependent, and optional runtime dependencies without starting Gateway
|
||
- `docker compose up -d` — start the container with Gateway bound/published on `0.0.0.0:19876`; override `PICOBOT_GATEWAY_HOST`, `PICOBOT_PUBLISH_HOST`, or `PICOBOT_GATEWAY_PORT` as needed
|
||
- WebUI — start Gateway, then open `http://127.0.0.1:19876/`; no separate frontend build is required
|
||
- `cd webui && npm ci && npm run check && npm run build` — validate the Svelte WebUI independently (Node.js 20+); its local `dist/` is ignored
|
||
- `cargo build` automatically runs an incremental WebUI production build into Cargo `OUT_DIR`; it runs `npm ci` only when `package-lock.json` is not represented by the installed dependency stamp
|
||
- `picobot service install|start|stop|status|restart|uninstall` — manage the Linux systemd user service (`picobot.service`)
|
||
|
||
## Config
|
||
|
||
- Config load order: `~/.picobot/config.json` then fallback to `./config.json` (`Config::load_default` in `src/config/mod.rs`)
|
||
- `.env` files use a custom parser, not dotenv: load `<config-dir>/.env`, then `<workspace_dir>/.env`, while pre-existing process variables remain highest priority; config placeholders `<VAR_NAME>` use the merged values
|
||
- Config example: `resources/templates/config.example.json` (released to `~/.picobot/` on first run)
|
||
- CLI TUI identity is stored in `~/.picobot/tui_client_id`; it is a non-secret stable chat scope used to restore dialogs across reconnects
|
||
- One-shot `run` uses a unique chat scope per invocation; for loopback Gateway URLs it authenticates `/ws` with `~/.picobot/web_admin_token`, while remote URLs use the existing paired CLI token
|
||
|
||
## Tests
|
||
|
||
- `cargo test --lib` — run unit tests (runs all `#[test]` in `src/`)
|
||
- `cargo clippy --all-targets --all-features -- -D warnings` — required for Rust changes
|
||
- `cargo test --test test_scheduler` and `cargo test --test test_request_format` — offline integration/protocol tests
|
||
- `cargo test --test test_integration -- --ignored` and `cargo test --test test_tool_calling -- --ignored` — model API integration tests
|
||
- API-backed tests require `tests/test.env` with real API keys; copy from `tests/test.env.example` and fill in keys
|
||
- API-backed tests are `#[ignore]` by default; use `-- --ignored` to run them
|
||
|
||
## Reference
|
||
|
||
- `reference/` — third-party reference implementations; not part of this project; do not modify
|
||
|
||
## Architecture
|
||
|
||
### Modes
|
||
|
||
- **Gateway mode** (`cargo run -- gateway`): HTTP/WebSocket server; owns `GatewayState` which holds all services
|
||
- **Client mode** (`cargo run -- chat`): TUI chat client; connects to gateway via WebSocket, purely for user interaction
|
||
- **One-shot client mode** (`cargo run -- run "prompt"`): isolated CLI chat scope; connects to Gateway, waits for a terminal Turn, prints it, and exits
|
||
|
||
### Core Data Flow
|
||
|
||
```
|
||
Channel → MessageBus.inbound → Gateway inbound router/lane → SessionManager → per-session worker → AgentLoop
|
||
↑ │
|
||
└── Channel ← OutboundDispatcher ← per-conversation lane ← MessageBus.outbound
|
||
|
||
AgentLoop → TurnEvent → Session TurnController → latest TurnSnapshot → DeliveryCoordinator → per-turn TurnSink → Channel
|
||
|
||
WebSocket/Channel → MessageBus.control → Gateway control router → SessionManager (dialog operations)
|
||
Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler delivery policy → SessionManager/MessageBus
|
||
```
|
||
|
||
### Modules
|
||
|
||
| Module | Responsibility | Key Types |
|
||
|--------|---------------|-----------|
|
||
| `gateway` | Server lifecycle, HTTP/WS/WebUI endpoints, owns `GatewayState` | `GatewayState`, `run()` |
|
||
| `client` | TUI rendering, WebSocket client for CLI chat | `App`, `run()` |
|
||
| `channels` | External integrations (Feishu, CLI chat) | `ChannelManager`, `Channel` trait |
|
||
| `bus` | Bounded async queues and ordered outbound delivery lanes | `MessageBus`, `OutboundDispatcher`, `InboundMessage`, `OutboundMessage`, `ControlMessage` |
|
||
| `session` | Conversation lifecycle, dialog operations, per-session serialization, Turn state, persistence coordination | `SessionManager`, `Session`, `TurnController` |
|
||
| `agent` | LLM call loop, tool execution, context compression, semantic Turn events | `AgentLoop`, `TurnEvent` |
|
||
| `providers` | Native LLM streams normalized into text/reasoning/tool/usage chunks | `LLMProvider`, `ProviderChunk`, `create_provider()` |
|
||
| `delivery` | Snapshot projection, latest-wins throttling, terminal retry, per-turn sink lifecycle | `DeliveryCoordinator`, `TurnDeliveryService`, `PresentationPolicy` |
|
||
| `tools` | Agent tools and external adapters (bash, files, HTTP, browser, health) | `ToolRegistry`, `Tool`, `ToolExecutionContext` |
|
||
| `health` | Shared read-only dependency diagnostics for CLI/tool/slash entry points | `HealthService`, `HealthReport` |
|
||
| `skills` | Skills loading, management, and prompt building | `SkillsLoader`, `Skill` |
|
||
| `storage` | SQLite persistence for sessions and messages | `Storage`, `SessionMeta`, `MessageMeta` |
|
||
| `scheduler` | Cron-based job scheduling, next-run computation | `Scheduler`, `Schedule`, `next_run_for_schedule()` |
|
||
| `work` | Session-scoped active plan and concurrently executable checklist items | `WorkManager`, `TaskPlan`, `TaskItem` |
|
||
| `observability` | Observer pattern for agent/tool telemetry events | `Observer` trait, `ObserverEvent`, `MultiObserver` |
|
||
| `protocol` | WebSocket protocol message types | `WsInbound`, `WsOutbound`, `SessionSummary`, `HistoryMessage` |
|
||
| `config` | Config loading, env substitution, path resolution | `Config`, `LLMProviderConfig` |
|
||
| `logging` | Tracing initialization with file rotation | `init_logging()`, `init_logging_console_only()` |
|
||
| `task_supervisor` | Owns, cancels, and boundedly joins gateway background tasks | `TaskSupervisor` |
|
||
|
||
### Functional Boundaries
|
||
|
||
- **Channels** publish inbound messages through `MessageBus`; outbound writes arrive through `OutboundDispatcher` or a per-turn `TurnSink`. They know nothing about sessions or LLM
|
||
- **Inbound contract** carries normalized sender/time/media plus `ChannelContext`; core routing may interpret `reply_to` but must treat platform-private context as opaque reply data
|
||
- **MessageBus** owns bounded inbound/outbound/control queues; outbound routing and retries belong to `OutboundDispatcher`, not the queue
|
||
- **SessionManager** owns session state, dialog operations, context construction, per-session work queues, active-Turn steering admission, and persistence coordination
|
||
- **TurnController** is the only owner of active Turn state; snapshots are complete latest-wins values, not token queues, and `Completed` is published only after atomic persistence succeeds
|
||
- **DeliveryCoordinator** projects active Turn snapshots without mutating history; it owns `TurnSink` lifecycle but no platform message IDs, which remain private to each sink
|
||
- **WorkManager** owns the single active plan per session, item state transitions, plan versions, and plan-change events; plans are optional and absent from ordinary chat context
|
||
- **Scheduler** supports legacy direct-delivery jobs and managed `task`/`monitor` jobs; managed agents cannot call `send_message`, and `on_alert` suppresses only healthy informational results
|
||
- **AgentLoop** is stateless across turns; it receives prepared history, drains same-Turn steering only at safe model boundaries, calls LLM providers, executes tools, and returns one result
|
||
- **AgentCatalog** is immutable per runtime generation; when orchestration is enabled, candidate preparation strictly validates trusted Markdown definitions, Provider profiles, tool/Skill allowlists, and delegation edges before activation. Named Agents support foreground (single/batch) and Root-initiated single background execution with durable run/inbox delivery; a built-in general-purpose definition is released to `~/.picobot/agents/` on first run. Background batches and nested background remain restricted
|
||
- **WebUI management APIs** only expose allowlisted config/profile/log/storage operations; keep response limits, secret redaction, atomic config writes, and profile path allowlists intact
|
||
- **WebUI styling** uses the local Fluent 2 semantic tokens in `webui/src/styles.css`; page and component styles should consume the aliases instead of introducing independent hard-coded palettes, and must preserve selectable light/dark and brand-color themes, keyboard focus, responsive layout, and reduced-motion behavior. Browser-only appearance settings belong in `lib/theme.js`/`localStorage`, and `public/theme-init.js` must restore them before Svelte mounts
|
||
- **Gateway config reload** validates and prepares a complete next runtime generation before retiring the old one; close runtime admission before draining inbound lanes, Sessions, Scheduler jobs, and background sub-agents; MCP connects only during activation; host/port, workspace, and effective storage path remain restart-only, and runtime reload must never mutate process environment variables
|
||
- **WebUI slash completion** must consume the existing `get_slash_commands` WebSocket response; do not duplicate the backend command list in frontend source
|
||
- **Session token statistics** persist Provider-reported usage atomically with each completed Turn; WebUI `session_stats` and `/info [--json]` must consume the same SessionStats projection, and context occupancy must use the final request's prompt usage rather than accumulated Turn totals
|
||
- **WebUI chat rendering** sanitizes Markdown before inserting HTML; durable `turn_committed` deltas calibrate normal terminal Turns without a full history reload, and history must preserve structured tool-call metadata so calls and results remain independently collapsible
|
||
- **WebUI/TUI file transfer** streams bytes over authenticated HTTP and sends only short-lived upload IDs/attachment metadata over WebSocket; messages persist local media paths without guaranteeing later availability, and client responses must never expose those paths
|
||
- **WebUI/TUI same-turn media delivery** stages same-session `send_message(files=...)` media on the active Turn and commits it on the final assistant message, after durable tool-call history; safe raster formats should render as an inline preview with download fallback
|
||
- **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token. The same conjunction may authenticate only `/ws` for local one-shot `run`; it must never authorize management APIs. Never put bearer or admin tokens in URLs or logs
|
||
- **Providers** are pure HTTP clients; no bus/session/channel awareness
|
||
- **Provider reasoning state** is private replay data: persist it, replay it only to the matching provider, and never expose it to clients, channels, or logs
|
||
- **Tools** are executed by `AgentLoop`; every invocation is normalized to `ToolOutput` and passes through `ToolOutputProcessor`. Plain `ToolResult` implementations use the default conversion, while artifact-producing tools declare model/user audience explicitly; model capability checks, final-reply attachment, channel delivery, and Provider serialization stay outside tools
|
||
- **Delegated tool access**: a named Agent's tool set is decided solely by its definition file (admin-authored). `delegate`, `emit_signal`, `get_skill` and `agent_task` are runtime-injected and must never be declared in `tools` (`get_skill` is the scoped-skill switch); `allowed_tools` can only narrow the definition, never expand it
|
||
- **Sleep tool** is a cancellable foreground wait bounded to 24 hours; longer or durable delays belong to Scheduler/background work, and cancelling a Turn must normalize active tool blocks to `Cancelled`
|
||
- **Stateful tools** receive `ToolExecutionContext`; browser calls without `persistent_id` map each PicoBot dialog to an opaque transient agent-browser session. For long-running work an Agent may autonomously create a persistent identity and must pass its validated ID on every related action; the same ID shares one agent-browser session and serialization gate across dialogs, while different IDs have independent sessions/gates. `browser_profiles` may create IDs, persist bounded semantic labels, list, or delete only validated IDs beneath the configured profile root. Do not introduce a global persistence mode switch, a default persistent ID, automatic per-dialog persistent Profiles, Fantoccini, ChromeDriver, WebDriver, model-controlled raw agent-browser session IDs, or arbitrary Profile paths
|
||
- **Health diagnostics** are read-only and share `HealthService` across `picobot health`, the `health` tool, and `/health`; checks must not install/fix dependencies, call Provider APIs, or expose secrets
|
||
|
||
### Concurrency and Lifecycle Invariants
|
||
|
||
- One session runs at most one Turn; ordinary input steers its active Turn by default, `/queue` explicitly waits for the next Turn, and different sessions may run concurrently
|
||
- Steering admission, final close, fallback, and `/stop` must be lossless and mutually exclusive: an input belongs to exactly the active Turn or the next-Turn FIFO, while `/stop` intentionally discards both
|
||
- Outbound messages are ordered per `(channel, chat_id)`; a slow destination must not block unrelated destinations
|
||
- Active Turn delivery and ordinary outbound delivery share the same per-`(channel, chat_id)` write lock; never enqueue token deltas into MessageBus
|
||
- Slow Turn consumers may skip intermediate snapshots but must receive an explicit bounded terminal delivery; shutdown must call sink abort so platform cleanup remains possible
|
||
- Never hold a Session mutex across model, network, or database I/O unless a documented invariant requires it
|
||
- Slow work derived from session state must validate `worker_generation`/`state_version` before committing results
|
||
- A same-session `send_message` write may avoid advancing `state_version` only when its task-local Turn ID still owns that session's active Turn; all other writes remain versioned
|
||
- Related durable mutations use Storage transaction APIs; persistence failure must not leave silent memory/database divergence
|
||
- Long-lived gateway tasks must be owned by `TaskSupervisor`; connection-local tasks must be explicitly joined or aborted by their owner
|
||
- Connection, retry sleep, queue wait, and shutdown join paths must observe cancellation and have hard time bounds
|
||
- Retry only errors explicitly classified as transient; permanent failures must surface immediately
|
||
- Never log secrets, authorization headers, or full connection URLs containing temporary credentials
|
||
- The workspace cwd is not a filesystem sandbox: default file tools accept absolute paths and Bash inherits process permissions; add explicit canonical-path/process isolation when a hard boundary is required
|
||
|
||
### Key Constraints
|
||
|
||
- Gateway **changes working directory** to workspace in `GatewayState::new` (`src/gateway/mod.rs`)
|
||
- Session/message persistence uses SQLite via `sqlx`; DB stored in workspace as `picobot.db` by default
|
||
- `ChannelManager` owns the `MessageBus` and all channel instances
|
||
- `OutboundDispatcher` routes outbound messages to the correct channel via `ChannelManager`
|
||
- Layered config/workspace `.env` loading uses `unsafe { env::set_var(...) }` during single-threaded startup — don't move it after Gateway tasks are spawned or refactor it without understanding process-wide side effects
|
||
|
||
## Change Workflow
|
||
|
||
1. Inspect the relevant implementation, tests, and `docs/ARCHITECTURE.md`; do not rely on names or old reports alone.
|
||
2. Search `reference/` only for comparison. Never edit it or copy behavior without checking PicoBot's boundaries.
|
||
3. Preserve unrelated user changes in a dirty worktree. Use `rg` for search and `apply_patch` for edits.
|
||
4. Add regression tests for bugs, especially cancellation, timeout, queue saturation, stale state, persistence failure, and retry classification.
|
||
5. For WebUI changes run `npm run check` and `npm run build` in `webui/`, then run `cargo build` to verify the `OUT_DIR` embedding path. Do not commit generated `dist/`; verify there is no external runtime dependency and keep browser chat on the existing `/ws` protocol.
|
||
6. For Rust changes run targeted tests, `cargo test --lib`, Clippy with warnings denied, and `cargo build`. Integration tests require real credentials.
|
||
7. For documentation-only changes verify links, commands, paths, and `git diff --check`.
|
||
8. Update README, this file, and the architecture document together when public behavior or an architectural invariant changes.
|
||
|
||
## Documentation Roles
|
||
|
||
- `README.md` — user-facing overview, setup, capabilities, and navigation
|
||
- `docs/ARCHITECTURE.md` — maintainer-facing runtime design, invariants, lifecycle, and extension guidance
|
||
- `AGENTS.md` — concise operational rules for repository agents
|
||
- `resources/skills/about-picobot/references/` — runtime knowledge shipped to PicoBot; update it only when the assistant's built-in product knowledge must change
|
||
|
||
## Version Management
|
||
- 在每次功能变化、架构变化后，适当地更新整个产品的版本号。功能变化增加中段数字，ui变化、bug修改增加末端数字。 注意版本号变更是提交时和仓库中的版本比较，不要在长时间的工程中，不断变化版本号。
|