# PicoBot

This file is the operational contract for coding agents working in this repository. Read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before making architectural or lifecycle changes. Code and tests are the final source of truth; update the document when an architectural invariant changes.

## Build & Run

- `cargo build` — build the binary
- `cargo run -- gateway` — start gateway server (binds `127.0.0.1:19876` by default)
- `cargo run -- chat` — connect to gateway as CLI client (default `ws://127.0.0.1:19876/ws`)
- `cargo run -- run "prompt"` — send one prompt through Gateway, print the terminal Turn, and exit; stdin, JSON, verbose progress, and timeout modes are available
- `cargo run -- reload` — validate and gracefully reload a running Gateway's configuration
- `cargo run -- health [--json]` — check core, configuration-dependent, and optional runtime dependencies without starting Gateway
- `docker compose up -d` — start the container with Gateway bound/published on `0.0.0.0:19876`; override `PICOBOT_GATEWAY_HOST`, `PICOBOT_PUBLISH_HOST`, or `PICOBOT_GATEWAY_PORT` as needed
- WebUI — start Gateway, then open `http://127.0.0.1:19876/`; no separate frontend build is required
- `cd webui && npm ci && npm run check && npm run build` — validate the Svelte WebUI independently (Node.js 20+); its local `dist/` is ignored
- `cargo build` automatically runs an incremental WebUI production build into Cargo `OUT_DIR`; it runs `npm ci` only when `package-lock.json` is not represented by the installed dependency stamp
- `picobot service install|start|stop|status|restart|uninstall` — manage the Linux systemd user service (`picobot.service`)

## Config

- Config load order: `~/.picobot/config.json` then fallback to `./config.json` (`Config::load_default` in `src/config/mod.rs`)
- `.env` files use a custom parser, not dotenv: load `<config-dir>/.env`, then `<workspace_dir>/.env`, while pre-existing process variables remain highest priority; config placeholders `<VAR_NAME>` use the merged values
- Config example: `resources/templates/config.example.json` (released to `~/.picobot/` on first run)
- Runtime config loading ignores recoverable unknown/type-mismatched fields and invalid non-core named entries while retaining diagnostics and the raw file revision; malformed JSON, an unusable `default` Agent chain, and unsafe runtime construction remain fatal. WebUI writes are strict, and backend cleanup may remove only diagnosed paths from the same revision
- CLI TUI identity is stored in `~/.picobot/tui_client_id`; it is a non-secret stable chat scope used to restore dialogs across reconnects
- One-shot `run` uses a unique chat scope per invocation; for loopback Gateway URLs it authenticates `/ws` with `~/.picobot/web_admin_token`, while remote URLs use the existing paired CLI token

## Tests

- `cargo test --lib` — run unit tests (runs all `#[test]` in `src/`)
- `cargo clippy --all-targets --all-features -- -D warnings` — required for Rust changes
- `cargo test --test test_scheduler` and `cargo test --test test_request_format` — offline integration/protocol tests
- `cargo test --test test_integration -- --ignored` and `cargo test --test test_tool_calling -- --ignored` — model API integration tests
- API-backed tests require `tests/test.env` with real API keys; copy from `tests/test.env.example` and fill in keys
- API-backed tests are `#[ignore]` by default; use `-- --ignored` to run them

## Reference

- `reference/` — third-party reference implementations; not part of this project; do not modify

## Architecture

### Modes

- **Gateway mode** (`cargo run -- gateway`): HTTP/WebSocket server; owns `GatewayState` which holds all services
- **Client mode** (`cargo run -- chat`): TUI chat client; connects to gateway via WebSocket, purely for user interaction
- **One-shot client mode** (`cargo run -- run "prompt"`): isolated CLI chat scope; connects to Gateway, waits for a terminal Turn, prints it, and exits

### Core Data Flow

```
Channel → MessageBus.inbound → Gateway inbound router/lane → SessionManager → per-session worker → AgentLoop
   ↑                                                                           │
   └── Channel ← OutboundDispatcher ← per-conversation lane ← MessageBus.outbound

AgentLoop → TurnEvent → Session TurnController → latest TurnSnapshot → DeliveryCoordinator → per-turn TurnSink → Channel

WebSocket/Channel → MessageBus.control → Gateway control router → SessionManager (dialog operations)
Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler delivery policy → SessionManager/MessageBus
```

### Modules

| Module | Responsibility | Key Types |
|--------|---------------|-----------|
| `gateway` | Server lifecycle, HTTP/WS/WebUI endpoints, owns `GatewayState` | `GatewayState`, `run()` |
| `client` | TUI rendering, WebSocket client for CLI chat | `App`, `run()` |
| `channels` | External integrations (Feishu, CLI chat) | `ChannelManager`, `Channel` trait |
| `bus` | Bounded async queues and ordered outbound delivery lanes | `MessageBus`, `OutboundDispatcher`, `InboundMessage`, `OutboundMessage`, `ControlMessage` |
| `session` | Conversation lifecycle, dialog operations, per-session serialization, Turn state, persistence coordination | `SessionManager`, `Session`, `TurnController` |
| `agent` | LLM call loop, tool execution, context compression, semantic Turn events | `AgentLoop`, `TurnEvent` |
| `providers` | Native LLM streams normalized into text/reasoning/tool/usage chunks | `LLMProvider`, `ProviderChunk`, `create_provider()` |
| `delivery` | Snapshot projection, latest-wins throttling, terminal retry, per-turn sink lifecycle | `DeliveryCoordinator`, `TurnDeliveryService`, `PresentationPolicy` |
| `tools` | Agent tools and external adapters (bash, files, HTTP, browser, health) | `ToolRegistry`, `Tool`, `ToolExecutionContext` |
| `health` | Shared read-only dependency diagnostics for CLI/tool/slash entry points | `HealthService`, `HealthReport` |
| `skills` | Skills loading, management, and prompt building | `SkillsLoader`, `Skill` |
| `storage` | SQLite persistence for sessions and messages | `Storage`, `SessionMeta`, `MessageMeta` |
| `scheduler` | Cron-based job scheduling, next-run computation | `Scheduler`, `Schedule`, `next_run_for_schedule()` |
| `work` | Session-scoped active plan and concurrently executable checklist items | `WorkManager`, `TaskPlan`, `TaskItem` |
| `observability` | Observer pattern for agent/tool telemetry events | `Observer` trait, `ObserverEvent`, `MultiObserver` |
| `protocol` | WebSocket protocol message types | `WsInbound`, `WsOutbound`, `SessionSummary`, `HistoryMessage` |
| `config` | Config loading, env substitution, path resolution | `Config`, `LLMProviderConfig` |
| `logging` | Tracing initialization with file rotation | `init_logging()`, `init_logging_console_only()` |
| `task_supervisor` | Owns, cancels, and boundedly joins gateway background tasks | `TaskSupervisor` |

### Functional Boundaries

- **Channels** publish inbound messages through `MessageBus`; outbound writes arrive through `OutboundDispatcher` or a per-turn `TurnSink`. They know nothing about sessions or LLM
- **Inbound contract** carries normalized sender/time/media plus `ChannelContext`; core routing may interpret `reply_to` but must treat platform-private context as opaque reply data
- **MessageBus** owns bounded inbound/outbound/control queues; outbound routing and retries belong to `OutboundDispatcher`, not the queue
- **SessionManager** owns session state, dialog operations, context construction, per-session work queues, active-Turn steering admission, and persistence coordination
- **TurnController** is the only owner of active Turn state; snapshots are complete latest-wins values, not token queues, and `Completed` is published only after atomic persistence succeeds
- **DeliveryCoordinator** projects active Turn snapshots without mutating history; it owns `TurnSink` lifecycle but no platform message IDs, which remain private to each sink
- **WorkManager** owns the single active plan per session, item state transitions, plan versions, and plan-change events; plans are optional and absent from ordinary chat context
- **Scheduler** owns one unified Scheduled Run path: claim-time JobRun snapshots, isolated Root/named Agent execution, exactly-once `complete_scheduled_run`, structured outcome, and policy-driven outbox delivery. Scheduled origin propagates to descendants, forces background delegation to foreground, and disables direct messaging, signals, Inbox completion slots, and cron/config management tools; `on_alert` suppresses only structured `ok`
- **AgentLoop** is stateless across turns; it receives prepared history, drains same-Turn steering only at safe model boundaries, calls LLM providers, executes tools, and returns one result
- **Context overflow recovery** is type-driven: before tool progress Session may commit one checkpoint and retry once; after any tool batch AgentLoop may retry the current Provider step once from its in-memory transcript, preserving current tool calls/results, and Session must never restart that Turn from durable history
- **Context compaction** keeps `messages` append-only and uses one active checkpoint per Session (`summary + first_retained_seq`) for deterministic Provider projection; `/compact`, Turn-boundary auto compaction, and overflow share the same compactor/CAS commit path, Session restoration never derives context from Timeline or calls a Provider, the Model `token_limit` (default 128K) is the hard window ceiling and an optional Agent `token_limit` can only narrow it via `min(agent, model)`, summary input is bounded from that effective window rather than a fixed cap, and the only automatic threshold is `context_tokens > context_window - effective_reserve`
- **AgentCatalog** is immutable per runtime generation; candidate preparation strictly validates trusted Markdown definitions, Provider profiles, tool/Skill allowlists, and delegation edges before activation. A definition that fails per-file validation (bad YAML, unknown provider/profile/model/tool/skill, or an explicit delegate edge to an absent target) is disabled for that generation only and reported via `load_errors` (exposed by `GET /api/agents`), never blocking startup or reload; config- and directory-trust-level failures remain fatal. Sub-Agent orchestration is an intrinsic, always-on mechanism (no feature switch). Named Agents support foreground (single/batch) and Root-initiated single background execution with durable run/inbox delivery; a built-in general-purpose definition is released to `~/.picobot/agents/` on first run. Background batches and nested background remain restricted
- **WebUI management APIs** only expose allowlisted config/profile/log/storage operations; keep response limits, secret redaction, atomic config writes, and profile path allowlists intact
- **Configuration recovery** builds an effective typed `Config` from a request-local copy and never rewrites the source automatically; diagnostics use raw RFC 6901 paths, array recovery preserves original indexes, ordinary writes reject ignored fields, and cleanup requires the exact source SHA-256 revision before atomically deleting diagnosed paths
- **WebUI styling** uses the local Fluent 2 semantic tokens in `webui/src/styles.css`; page and component styles should consume the aliases instead of introducing independent hard-coded palettes, and must preserve selectable light/dark and brand-color themes, keyboard focus, responsive layout, and reduced-motion behavior. Browser-only appearance settings belong in `lib/theme.js`/`localStorage`, and `public/theme-init.js` must restore them before Svelte mounts
- **Gateway config reload** validates and prepares a complete next runtime generation before retiring the old one; close runtime admission before draining inbound lanes, Sessions, Scheduler jobs, and background sub-agents; MCP connects only during activation; host/port, workspace, and effective storage path remain restart-only, and runtime reload must never mutate process environment variables
- **WebUI slash completion** must consume the existing `get_slash_commands` WebSocket response; do not duplicate the backend command list in frontend source
- **Session token statistics** persist Provider-reported usage atomically with each completed Turn; WebUI `session_stats` and `/info [--json]` must consume the same SessionStats projection, and context occupancy must use the final request's prompt usage rather than accumulated Turn totals
- **WebUI chat rendering** sanitizes Markdown before inserting HTML; durable `turn_committed` deltas calibrate normal terminal Turns without a full history reload, and history must preserve structured tool-call metadata so calls and results remain independently collapsible
- **WebUI/TUI file transfer** streams bytes over authenticated HTTP and sends only short-lived upload IDs/attachment metadata over WebSocket; messages persist local media paths without guaranteeing later availability, and client responses must never expose those paths
- **WebUI/TUI same-turn media delivery** stages same-session `send_message(files=...)` media on the active Turn and commits it on the final assistant message, after durable tool-call history; safe raster formats should render as an inline preview with download fallback
- **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token. The same conjunction may authenticate only `/ws` for local one-shot `run`; it must never authorize management APIs. Never put bearer or admin tokens in URLs or logs
- **Providers** are pure HTTP clients; no bus/session/channel awareness
- **Provider reasoning state** is private replay data: persist it, replay it only to the matching provider, and never expose it to clients, channels, or logs
- **Tools** are executed by `AgentLoop`; every invocation is normalized to `ToolOutput` and passes through `ToolOutputProcessor`. Plain `ToolResult` implementations use the default conversion, while artifact-producing tools declare model/user audience explicitly; model capability checks, final-reply attachment, channel delivery, and Provider serialization stay outside tools
- **Delegated tool access**: a named Agent's tool set is decided solely by its definition file (admin-authored). `delegate`, `emit_signal`, `get_skill` and `agent_task` are runtime-injected and must never be declared in `tools` (`get_skill` is the scoped-skill switch); `allowed_tools` can only narrow the definition, never expand it
- **No foreground wait tool**: Agents wait for asynchronous work by ending the Turn and letting queued completions/signals open a continuation Turn, or by polling status tools; there is no model-callable `sleep`/wait tool. Cancelling a Turn must still normalize active tool blocks to `Cancelled`
- **Skill enable/disable**: skills default to enabled; user-disabled skill names are persisted in `<config_dir>/skills_state.json` (skill files are never modified), and disabled skills are excluded from prompts, listings, and `get_skill` at load time
- **MCP enable/disable**: `mcp.servers[].enabled` defaults to true; disabled servers are skipped at activation (no connection attempt) and by health checks
- **Stateful tools** receive `ToolExecutionContext`; browser calls without `persistent_id` map each PicoBot dialog to an opaque transient agent-browser session, whose idle daemon timeout is controlled by `browser.idle_timeout_secs` (one hour by default). For long-running work an Agent may autonomously create a persistent identity and must pass its validated ID on every related action; persistent browser daemons disable idle auto-close, the same ID shares one agent-browser session and serialization gate across dialogs, and different IDs have independent sessions/gates. `browser_profiles` may create IDs, persist bounded semantic labels, list, or delete only validated IDs beneath the configured profile root. Do not introduce a global persistence mode switch, a default persistent ID, automatic per-dialog persistent Profiles, Fantoccini, ChromeDriver, WebDriver, model-controlled raw agent-browser session IDs, or arbitrary Profile paths
- **Health diagnostics** are read-only and share `HealthService` across `picobot health`, the `health` tool, `/health`, and authenticated WebUI `GET /api/health`; the public `GET /health` remains a lightweight liveness/version probe, while full checks run only on demand and must not install/fix dependencies, call Provider APIs, or expose secrets. Treat `fd`/`fdfind` as equivalent preferred backends; browser launch diagnostics must isolate their socket namespace from active agent-browser sessions and must not use `--quick`, which skips the live launch probe

### Concurrency and Lifecycle Invariants

- One session runs at most one Turn; ordinary input steers its active Turn by default, `/queue` explicitly waits for the next Turn, and different sessions may run concurrently
- Steering admission, final close, fallback, and `/stop` must be lossless and mutually exclusive: an input belongs to exactly the active Turn or the next-Turn FIFO, while `/stop` intentionally discards both
- Outbound messages are ordered per `(channel, chat_id)`; a slow destination must not block unrelated destinations
- Active Turn delivery and ordinary outbound delivery share the same per-`(channel, chat_id)` write lock; never enqueue token deltas into MessageBus
- Slow Turn consumers may skip intermediate snapshots but must receive an explicit bounded terminal delivery; shutdown must call sink abort so platform cleanup remains possible
- Never hold a Session mutex across model, network, or database I/O unless a documented invariant requires it
- Slow work derived from session state must validate `worker_generation`/`state_version` before committing results
- A same-session `send_message` write may avoid advancing `state_version` only when its task-local Turn ID still owns that session's active Turn; all other writes remain versioned
- Related durable mutations use Storage transaction APIs; persistence failure must not leave silent memory/database divergence
- Long-lived gateway tasks must be owned by `TaskSupervisor`; connection-local tasks must be explicitly joined or aborted by their owner
- Connection, retry sleep, queue wait, and shutdown join paths must observe cancellation and have hard time bounds
- Retry only errors explicitly classified as transient; permanent failures must surface immediately
- Never log secrets, authorization headers, or full connection URLs containing temporary credentials
- The workspace cwd is not a filesystem sandbox: default file tools accept absolute paths and Bash inherits process permissions; add explicit canonical-path/process isolation when a hard boundary is required

### Key Constraints

- Gateway **changes working directory** to workspace in `GatewayState::new` (`src/gateway/mod.rs`)
- Session/message persistence uses SQLite via `sqlx`; DB stored in `<config_dir>/data/picobot.db` by default (`config_dir` is `~/.picobot`), independent of the workspace
- `ChannelManager` owns the `MessageBus` and all channel instances
- `OutboundDispatcher` routes outbound messages to the correct channel via `ChannelManager`
- Layered config/workspace `.env` loading uses `unsafe { env::set_var(...) }` during single-threaded startup — don't move it after Gateway tasks are spawned or refactor it without understanding process-wide side effects

## Change Workflow

1. Inspect the relevant implementation, tests, and `docs/ARCHITECTURE.md`; do not rely on names or old reports alone.
2. Search `reference/` only for comparison. Never edit it or copy behavior without checking PicoBot's boundaries.
3. Preserve unrelated user changes in a dirty worktree. Use `rg` for search and `apply_patch` for edits.
4. Add regression tests for bugs, especially cancellation, timeout, queue saturation, stale state, persistence failure, and retry classification.
5. For WebUI changes run `npm run check` and `npm run build` in `webui/`, then run `cargo build` to verify the `OUT_DIR` embedding path. Do not commit generated `dist/`; verify there is no external runtime dependency and keep browser chat on the existing `/ws` protocol.
6. For Rust changes run targeted tests, `cargo test --lib`, Clippy with warnings denied, and `cargo build`. Integration tests require real credentials.
7. For documentation-only changes verify links, commands, paths, and `git diff --check`.
8. Update README, this file, and the architecture document together when public behavior or an architectural invariant changes.

## Documentation Roles

- `README.md` — user-facing overview, setup, capabilities, and navigation
- `docs/ARCHITECTURE.md` — maintainer-facing runtime design, invariants, lifecycle, and extension guidance
- `AGENTS.md` — concise operational rules for repository agents
- `resources/skills/about-picobot/references/` — runtime knowledge shipped to PicoBot; update it only when the assistant's built-in product knowledge must change

## Version Management
- 在每次功能变化、架构变化后，适当地更新整个产品的版本号。功能变化增加中段数字，ui变化、bug修改增加末端数字。 注意版本号变更是提交时和仓库中的版本比较，不要在长时间的工程中，不断变化版本号。
