xiaoski/PicoBot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(scheduler): unify scheduled task execution and delivery

Browse Source 
Replace the dual task/monitor model, NO_REPLY string protocol, and Agent
self-delivery with a single Scheduled Run path: claim-time JobRun snapshots,
isolated Root/named Agent execution, exactly-once complete_scheduled_run
termination, and Scheduler-owned policy delivery through a persistent outbox.

- SQLite v11: drop job_kind/model/delete_after_run, add job_runs with
  status/outcome joint constraints and delivery lease columns; one-shot
  BEGIN IMMEDIATE migration with atomic rollback.
- Non-blocking JoinSet event loop with bounded run/delivery concurrency;
  terminal commit before any channel I/O; recover unfinished runs as unknown.
- ExecutionOrigin::Scheduled propagates to descendants, completion sink is
  top-level only, background delegation downgrades to foreground.
- Typed delivery receipts, fixed target_session_id, idempotent
  scheduled:<job_run_id> history insert.
- New cron_runs read-only tool; cron_add/update drop kind/model; WebUI and
  Health consume the same JobRun projection.
- Bump version to 1.22.0.
This commit is contained in:
xiaoxixi 2026-08-21 14:59:02 +08:00
parent ddd5efc11b
commit d9ad58b84b
39 changed files with 5132 additions and 2213 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
AGENTS.md
View File

@ -92,7 +92,7 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
- **TurnController** is the only owner of active Turn state; snapshots are complete latest-wins values, not token queues, and `Completed` is published only after atomic persistence succeeds - **TurnController** is the only owner of active Turn state; snapshots are complete latest-wins values, not token queues, and `Completed` is published only after atomic persistence succeeds
- **DeliveryCoordinator** projects active Turn snapshots without mutating history; it owns `TurnSink` lifecycle but no platform message IDs, which remain private to each sink - **DeliveryCoordinator** projects active Turn snapshots without mutating history; it owns `TurnSink` lifecycle but no platform message IDs, which remain private to each sink
- **WorkManager** owns the single active plan per session, item state transitions, plan versions, and plan-change events; plans are optional and absent from ordinary chat context - **WorkManager** owns the single active plan per session, item state transitions, plan versions, and plan-change events; plans are optional and absent from ordinary chat context
- **Scheduler** supports legacy direct-delivery jobs and managed `task`/`monitor` jobs; managed agents cannot call `send_message`, and `on_alert` suppresses only healthy informational results - **Scheduler** owns one unified Scheduled Run path: claim-time JobRun snapshots, isolated Root/named Agent execution, exactly-once `complete_scheduled_run`, structured outcome, and policy-driven outbox delivery. Scheduled origin propagates to descendants, forces background delegation to foreground, and disables direct messaging, signals, Inbox completion slots, and cron/config management tools; `on_alert` suppresses only structured `ok`
- **AgentLoop** is stateless across turns; it receives prepared history, drains same-Turn steering only at safe model boundaries, calls LLM providers, executes tools, and returns one result - **AgentLoop** is stateless across turns; it receives prepared history, drains same-Turn steering only at safe model boundaries, calls LLM providers, executes tools, and returns one result
- **Context overflow recovery** is type-driven: before tool progress Session may commit one checkpoint and retry once; after any tool batch AgentLoop may retry the current Provider step once from its in-memory transcript, preserving current tool calls/results, and Session must never restart that Turn from durable history - **Context overflow recovery** is type-driven: before tool progress Session may commit one checkpoint and retry once; after any tool batch AgentLoop may retry the current Provider step once from its in-memory transcript, preserving current tool calls/results, and Session must never restart that Turn from durable history
- **Context compaction** keeps `messages` append-only and uses one active checkpoint per Session (`summary + first_retained_seq`) for deterministic Provider projection; `/compact`, Turn-boundary auto compaction, and overflow share the same compactor/CAS commit path, Session restoration never derives context from Timeline or calls a Provider, the Model `token_limit` (default 128K) is the hard window ceiling and an optional Agent `token_limit` can only narrow it via `min(agent, model)`, summary input is bounded from that effective window rather than a fixed cap, and the only automatic threshold is `context_tokens > context_window - effective_reserve` - **Context compaction** keeps `messages` append-only and uses one active checkpoint per Session (`summary + first_retained_seq`) for deterministic Provider projection; `/compact`, Turn-boundary auto compaction, and overflow share the same compactor/CAS commit path, Session restoration never derives context from Timeline or calls a Provider, the Model `token_limit` (default 128K) is the hard window ceiling and an optional Agent `token_limit` can only narrow it via `min(agent, model)`, summary input is bounded from that effective window rather than a fixed cap, and the only automatic threshold is `context_tokens > context_window - effective_reserve`

2
Cargo.toml
View File

@ -1,6 +1,6 @@
[package] [package]
name = "picobot" name = "picobot"
version = "1.21.0" version = "1.22.0"
edition = "2024" edition = "2024"
[dependencies] [dependencies]

11
README.md
View File

@ -17,7 +17,7 @@ PicoBot 是一个用 Rust 编写的个人 AI 助手运行时。它在本地启
- 将同一套 Agent 能力接入飞书/Lark，并可选用单张卡片实时更新回复。 - 将同一套 Agent 能力接入飞书/Lark，并可选用单张卡片实时更新回复。
- 让 Agent 使用本地文件、Shell、搜索、HTTP、浏览器、MCP 工具完成任务。 - 让 Agent 使用本地文件、Shell、搜索、HTTP、浏览器、MCP 工具完成任务。
- 把长期偏好、事实和历史摘要存成可检索记忆。 - 把长期偏好、事实和历史摘要存成可检索记忆。
- 用 Cron 定时执行任务，并把结果发回目标渠道。 - 用 Cron 运行隔离的 Root 或命名 Agent，以结构化结果决定始终通知、异常通知或静默记录。
- 通过 Skills 为 Agent 注入项目知识和专用操作指南。 - 通过 Skills 为 Agent 注入项目知识和专用操作指南。
## 快速开始 ## 快速开始
@ -152,7 +152,7 @@ picobot health --json
缺少核心或当前配置要求的依赖时退出码为 `1`；`rg` / `fd` 等有回退实现的加速项只会标记为 `DEGRADED`。运行中的 Gateway 也提供 `/health` 斜杠命令，Agent 可调用同名 `health` 工具，WebUI 的“配置 → 健康检查”可显示相同的结构化结果并手动复查；这些入口共享同一套只读检查逻辑。 缺少核心或当前配置要求的依赖时退出码为 `1`；`rg` / `fd` 等有回退实现的加速项只会标记为 `DEGRADED`。运行中的 Gateway 也提供 `/health` 斜杠命令，Agent 可调用同名 `health` 工具，WebUI 的“配置 → 健康检查”可显示相同的结构化结果并手动复查；这些入口共享同一套只读检查逻辑。
Debian/Ubuntu 将同一个 fd 程序安装为 `fdfind`，两者都视为首选文件搜索后端；只有退回传统 `find` 时才提示性能警告。启用浏览器工具后，Health 除了检查 agent-browser 版本和浏览器路径，还会在隔离的临时 socket namespace 中执行完整离线 doctor，分别报告浏览器安装、真实 headless 启动和运行环境，因此可发现“文件存在但 Chrome 无法启动”或缺少 Linux 共享库等问题。 Debian/Ubuntu 将同一个 fd 程序安装为 `fdfind`，两者都视为首选文件搜索后端；只有退回传统 `find` 时才提示性能警告。启用浏览器工具后，Health 除了检查 agent-browser 版本和浏览器路径，还会在隔离的临时 socket namespace 中执行完整离线 doctor，分别报告浏览器安装、真实 headless 启动和运行环境，因此可发现“文件存在但 Chrome 无法启动”或缺少 Linux 共享库等问题。Gateway 内按需检查还会报告定时任务的无效 Agent/渠道引用、投递积压、最近失败/超时/unknown、静默 unknown 和执行周期覆盖；Health 不会触发任务或连接 Provider。
### 5.3 使用 WebUI ### 5.3 使用 WebUI
@ -273,7 +273,7 @@ WebUI/TUI 上传文件默认保存到 `~/.picobot/media/cli_chat`，单文件上
| `delivery` | 活动 Turn 的展示过滤、latest-wins 节流、终态投递与 TurnSink 生命周期 | | `delivery` | 活动 Turn 的展示过滤、latest-wins 节流、终态投递与 TurnSink 生命周期 |
| `tools` | Agent 可调用工具集合 | | `tools` | Agent 可调用工具集合 |
| `storage` | SQLite schema、CRUD、消息和任务持久化 | | `storage` | SQLite schema、CRUD、消息和任务持久化 |
| `scheduler` | 领取定时任务，执行普通/巡检 Agent，并按投递策略记录或发送结果 | | `scheduler` | 原子领取 occurrence，运行隔离的 Scheduled Agent，并通过持久化 outbox 按策略投递结构化结果 |
| `work` | 管理 session 级单 active plan、并行子项状态和 WebSocket 变更事件 | | `work` | 管理 session 级单 active plan、并行子项状态和 WebSocket 变更事件 |
| `skills` | 加载 Skill，并把 Skill 指南注入系统提示 | | `skills` | 加载 Skill，并把 Skill 指南注入系统提示 |
| `mcp` | 连接 MCP Server，将远端工具包装成普通 Tool | | `mcp` | 连接 MCP Server，将远端工具包装成普通 Tool |
@ -329,7 +329,7 @@ PicoBot 有两类记忆：
| Knowledge | 偏好、事实、项目规则、长期可复用信息 | 长期保留，手动删除 | | Knowledge | 偏好、事实、项目规则、长期可复用信息 | 长期保留，手动删除 |
| Timeline | 长对话压缩后的历史摘要 | 默认保留 90 天 | | Timeline | 长对话压缩后的历史摘要 | 默认保留 90 天 |
每轮处理用户消息时，MemoryManager 会按用户输入召回 Knowledge，并作为运行时上下文附加到本轮用户消息。当前召回上限固定为 5；`memory.recall_limit` 已支持解析但尚未接入 worker。长会话使用一个活动 checkpoint：累计摘要加 `first_retained_seq` 之后的原始消息尾部构成模型上下文，原始消息、工具调用结果、ID 和 seq 均不会被压缩改写。旧工具结果会保留在原始历史中，但 checkpoint 边界推进后不再永久占用 Provider 上下文。成功的语义摘要还会 best-effort 保存为 Timeline，供 `timeline_recall` 检索；Timeline 不参与会话恢复正确性。Scheduler 默认创建一个每日维护巡检，按 `memory.timeline_retention_days` 清理过期 Timeline；Knowledge 不会被自动删除。 每轮处理用户消息时，MemoryManager 会按用户输入召回 Knowledge，并作为运行时上下文附加到本轮用户消息。当前召回上限固定为 5；`memory.recall_limit` 已支持解析但尚未接入 worker。长会话使用一个活动 checkpoint：累计摘要加 `first_retained_seq` 之后的原始消息尾部构成模型上下文，原始消息、工具调用结果、ID 和 seq 均不会被压缩改写。旧工具结果会保留在原始历史中，但 checkpoint 边界推进后不再永久占用 Provider 上下文。成功的语义摘要还会 best-effort 保存为 Timeline，供 `timeline_recall` 检索；Timeline 不参与会话恢复正确性。Scheduler 默认创建一个每日维护任务，按 `memory.timeline_retention_days` 清理过期 Timeline；结果通过 `complete_scheduled_run` 结构化提交，Knowledge 不会被自动删除。
模型的 `models.<name>.token_limit` 给出上下文窗口上限，未配置时默认为 128,000；Agent 的 `agents.<name>.token_limit` 是可选的收紧上限，两者都有配置时有效窗口取二者最小值，因此 Agent 不能扩大模型窗口。自动压缩使用保留量阈值 `context_tokens > context_window - effective_reserve`，默认 reserve 为 16,384 tokens，并尽量原样保留最近 20,000 tokens。小窗口会自动把 reserve 限制为窗口的一半、把近期保留量限制为有效阈值的一半。摘要请求不使用固定 32K 输入上限，而是按有效窗口扣除摘要输出、提示词和安全余量；超大历史只在摘要请求副本中按“已有 checkpoint + 最新消息优先”生成有界 head/tail 转录，SQLite 原文不变。手动 `/compact` 跳过自动阈值；换成小模型后若发送前预检已发现硬超限，或首次请求返回真实 context overflow，语义摘要不可用时才使用明确标记的确定性降级裁剪，正式请求最多重试一次。若 overflow 发生在工具已经执行之后，AgentLoop 只在当前内存转录上裁掉旧完整 Turn 并重试当前模型步骤一次，不会从数据库历史重跑工具。 模型的 `models.<name>.token_limit` 给出上下文窗口上限，未配置时默认为 128,000；Agent 的 `agents.<name>.token_limit` 是可选的收紧上限，两者都有配置时有效窗口取二者最小值，因此 Agent 不能扩大模型窗口。自动压缩使用保留量阈值 `context_tokens > context_window - effective_reserve`，默认 reserve 为 16,384 tokens，并尽量原样保留最近 20,000 tokens。小窗口会自动把 reserve 限制为窗口的一半、把近期保留量限制为有效阈值的一半。摘要请求不使用固定 32K 输入上限，而是按有效窗口扣除摘要输出、提示词和安全余量；超大历史只在摘要请求副本中按“已有 checkpoint + 最新消息优先”生成有界 head/tail 转录，SQLite 原文不变。手动 `/compact` 跳过自动阈值；换成小模型后若发送前预检已发现硬超限，或首次请求返回真实 context overflow，语义摘要不可用时才使用明确标记的确定性降级裁剪，正式请求最多重试一次。若 overflow 发生在工具已经执行之后，AgentLoop 只在当前内存转录上裁掉旧完整 Turn 并重试当前模型步骤一次，不会从数据库历史重跑工具。
@ -353,7 +353,8 @@ PicoBot 有两类记忆：
| `todo` | 为复杂、多轮任务创建并更新当前 session 的持久化计划 | | `todo` | 为复杂、多轮任务创建并更新当前 session 的持久化计划 |
| `send_message` | 向指定渠道或当前会话发送消息，可附带文件/截图；WebUI/TUI 当前 Turn 的附件并入最终回复 | | `send_message` | 向指定渠道或当前会话发送消息，可附带文件/截图；WebUI/TUI 当前 Turn 的附件并入最终回复 |
| `chat_manager` | 查看渠道、会话和历史消息 | | `chat_manager` | 查看渠道、会话和历史消息 |
| `cron_add/list/remove/enable/disable/update` | 管理定时任务 | | `cron_add/list/remove/enable/disable/update` | 管理定时任务；`agent_id` 选择 Root/命名 Agent，`delivery_policy` 支持 `always/on_alert/never` |
| `cron_runs` | 查询定时任务的结构化执行结果、诊断和投递状态，包括静默任务 |
| `routine_maintenance` | 安全清理超过保留期的 Timeline，不删除 Knowledge | | `routine_maintenance` | 安全清理超过保留期的 Timeline，不删除 Knowledge |
| `health` | 检查核心、配置相关和可选运行依赖 | | `health` | 检查核心、配置相关和可选运行依赖 |
| `browser` | 可选 agent-browser 浏览器自动化；默认按 dialog 临时使用，长期任务可用 `persistent_id` 复用个人 Profile | | `browser` | 可选 agent-browser 浏览器自动化；默认按 dialog 临时使用，长期任务可用 `persistent_id` 复用个人 Profile |

16
docs/ARCHITECTURE.md
View File

@ -2,7 +2,7 @@
本文档描述 PicoBot 当前实现的运行时边界、数据流、并发模型和演进约束。它面向维护者和后续参与改进的 Agent，是代码架构的主入口；行为细节仍以代码和测试为最终依据。 本文档描述 PicoBot 当前实现的运行时边界、数据流、并发模型和演进约束。它面向维护者和后续参与改进的 Agent，是代码架构的主入口；行为细节仍以代码和测试为最终依据。
流式模型输出、reasoning 展示、活动 Turn 快照和 Channel 实时投递的详细设计与取舍见 [STREAMING_TURN_DESIGN.md](STREAMING_TURN_DESIGN.md)。用户输入路由、Session 执行拆分、终态投递确认和历史增量校准的重构方案见 [MESSAGE_FLOW_REFACTOR_DESIGN.md](MESSAGE_FLOW_REFACTOR_DESIGN.md)。已实施的 checkpoint 上下文压缩、pi 风格 reserve 阈值、统一编排和 overflow 失败语义见 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md)。配置运行代、重载边界和失败语义见 [CONFIG_HOT_RELOAD_DESIGN.md](CONFIG_HOT_RELOAD_DESIGN.md)。具名子 Agent、委托图、后台收件箱、结果传递机制与 `queue`/`steer` 信号的设计见 [SUB_AGENT_DESIGN.md](SUB_AGENT_DESIGN.md)。计划中的统一 Scheduled Run、结构化终结协议、中央投递和 v11 数据库迁移见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。 流式模型输出、reasoning 展示、活动 Turn 快照和 Channel 实时投递的详细设计与取舍见 [STREAMING_TURN_DESIGN.md](STREAMING_TURN_DESIGN.md)。用户输入路由、Session 执行拆分、终态投递确认和历史增量校准的重构方案见 [MESSAGE_FLOW_REFACTOR_DESIGN.md](MESSAGE_FLOW_REFACTOR_DESIGN.md)。已实施的 checkpoint 上下文压缩、pi 风格 reserve 阈值、统一编排和 overflow 失败语义见 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md)。配置运行代、重载边界和失败语义见 [CONFIG_HOT_RELOAD_DESIGN.md](CONFIG_HOT_RELOAD_DESIGN.md)。具名子 Agent、委托图、后台收件箱、结果传递机制与 `queue`/`steer` 信号的设计见 [SUB_AGENT_DESIGN.md](SUB_AGENT_DESIGN.md)。已实施的统一 Scheduled Run、结构化终结协议、中央投递和 v11 数据库迁移见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。
## 1. 设计目标 ## 1. 设计目标
@ -78,7 +78,7 @@ flowchart LR
| `health` | 聚合只读依赖检查，供 CLI、Tool 与 slash command 复用 | 安装、修复或连接 Provider | | `health` | 聚合只读依赖检查，供 CLI、Tool 与 slash command 复用 | 安装、修复或连接 Provider |
| `storage` | SQLite schema、迁移、原子 CRUD | 运行时调度策略 | | `storage` | SQLite schema、迁移、原子 CRUD | 运行时调度策略 |
| `memory` | Knowledge/Timeline 的存取与召回 | 直接驱动消息发送 | | `memory` | Knowledge/Timeline 的存取与召回 | 直接驱动消息发送 |
| `scheduler` | 领取到期任务、执行普通/巡检 Agent、应用投递策略、原子记录结果 | 复用聊天会话历史、直接感知 Channel | | `scheduler` | 原子领取 occurrence、运行隔离的 Root/命名 Agent、提交结构化结果并 drain 持久化投递 outbox | 解析模型自然语言、绕过 Bus 直接调用 Channel |
| `work` | session 级单 active plan、并行子项状态机、版本和变更事件 | 执行模型调用、持有 Channel/WebSocket | | `work` | session 级单 active plan、并行子项状态机、版本和变更事件 | 执行模型调用、持有 Channel/WebSocket |
| `task_supervisor` | 后台任务注册、取消、限时回收 | 业务级重试和结果语义 | | `task_supervisor` | 后台任务注册、取消、限时回收 | 业务级重试和结果语义 |
@ -171,6 +171,18 @@ sequenceDiagram
不要把“已进入 Bus”误认为“外部渠道已收到”。需要确认语义时必须使用 `deliver_outbound`。 不要把“已进入 Bus”误认为“外部渠道已收到”。需要确认语义时必须使用 `deliver_outbound`。
### Scheduled Run
Scheduler 不复用聊天历史，也不根据模型正文猜测是否通知。一次到期状态在同一 SQLite 事务中取得 Job 租约、插入 `job_runs(status=claimed)`、快照 Agent/投递目标/策略，并提前推进 recurring `next_run_at`；`At` 在 claim 时立即禁用。事件循环以独立有界 JoinSet 执行 Run 和 drain delivery，长任务不阻塞其他领取或通知。
每次 Run 通过 `AgentCoordinator` 建立顶层 `agent_runs` 审计记录并执行隔离的 Root 或命名 Agent。Scheduled origin 贯穿所有后代，但只有顶层获得 exactly-once `complete_scheduled_run` sink；后代不继承 sink。Scheduled Agent 不获得 `send_message`、cron/config 管理或 `emit_signal`，不创建 Inbox completion slot，任何 background 委托都收敛为 foreground。普通最终文本不代表成功；没有提交 `ok/alert/failed/refused` 之一即 fail-closed。
顶层 AgentRun 终态、JobRun 的 lifecycle/outcome/message/diagnostic、Job 最近摘要、初始 delivery status 和租约释放在一个事务中提交。`always` 投递所有 outcome，`on_alert` 只抑制结构化 `ok`，`never` 始终只记录。`job_runs` 同时作为轻量 outbox：`pending → delivering → delivered/failed`，瞬态错误最多进行三次持久化尝试；OutboundDispatcher 返回清洗后的类型化回执，Scheduler 不解析错误字符串。
首次投递把目标 dialog 固定到 `target_session_id`，并先用稳定消息 ID `scheduled:<job_run_id>` 幂等写入本地历史，再调用 `MessageBus::deliver_outbound`。发送成功但 ack 提交前崩溃允许带稳定 metadata 的重复通知，不能为避免重复而丢失告警。启动恢复把遗留 claimed/running JobRun 原子收敛为 `unknown+unknown`、关联非终态 AgentRun 收敛为 interrupted，并按 claim-time 策略决定是否进入 outbox；已推进的 occurrence 不自动重跑。完整状态矩阵、v11 schema 和迁移规则见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。
按需 Health 检查只读查询任务引用、投递积压、最近失败/超时/unknown、`never+unknown`、Every 周期被执行时长覆盖以及不可计算的 next run；它不执行任务、不连接 Provider，也不修改数据。
### Control 消息 ### Control 消息
WebSocket dialog 操作通过 `ControlMessage` 携带一次性回复通道。Gateway control router 以最多 64 个并发受监督任务调用 `SessionManager`，再将 `SessionEvent` 回传给发起者；慢 control 不阻塞其他聊天的 inbound 路由。Bus 只承载消息，不解释操作。 WebSocket dialog 操作通过 `ControlMessage` 携带一次性回复通道。Gateway control router 以最多 64 个并发受监督任务调用 `SessionManager`，再将 `SessionEvent` 回传给发起者；慢 control 不阻塞其他聊天的 inbound 路由。Bus 只承载消息，不解释操作。

369
docs/SCHEDULED_RUN_DESIGN.md
View File

@ -1,6 +1,6 @@
# 统一定时任务执行与投递设计 # 统一定时任务执行与投递设计
状态：设计完成，尚未实施 状态：已实施
目标数据库版本：v11 目标数据库版本：v11
@ -30,6 +30,7 @@
7. 执行结果先持久化，再投递；进程重启后可以恢复未完成投递。 7. 执行结果先持久化，再投递；进程重启后可以恢复未完成投递。
8. 无法确认是否完成的执行标记为 `unknown`，不盲目重跑同一 occurrence。 8. 无法确认是否完成的执行标记为 `unknown`，不盲目重跑同一 occurrence。
9. 使用一次性、原子、可失败回滚的 SQLite v11 自动迁移；迁移后运行时代码只认识新 schema 和新枚举。 9. 使用一次性、原子、可失败回滚的 SQLite v11 自动迁移；迁移后运行时代码只认识新 schema 和新枚举。
10. `never` 等静默任务的每次运行仍可通过管理 API、WebUI 和只读工具审计。
## 3. 非目标 ## 3. 非目标
@ -39,6 +40,7 @@
- 不自动把上一次执行的工具结果带到下一次执行。 - 不自动把上一次执行的工具结果带到下一次执行。
- 不增加 `shell`、Webhook 等第二种 Job 执行类型；本期所有任务仍是 Agent 任务。 - 不增加 `shell`、Webhook 等第二种 Job 执行类型；本期所有任务仍是 Agent 任务。
- 不保留旧字段、旧枚举、旧工具参数或旧字符串协议的运行时兼容分支。 - 不保留旧字段、旧枚举、旧工具参数或旧字符串协议的运行时兼容分支。
- `never` 不提供“失败时例外通知”；选择该策略意味着 `failed`、`refused`、`timed_out` 和 `unknown` 也只进入审计、WebUI 与 Health。
## 4. 参考项目取舍 ## 4. 参考项目取舍
@ -99,7 +101,7 @@ pub struct ScheduledJob {
- `job_kind`：与 `delivery_policy` 重复； - `job_kind`：与 `delivery_policy` 重复；
- `model`：当前并未实际应用，且会绕开命名 Agent 的 Provider/Model 定义； - `model`：当前并未实际应用，且会绕开命名 Agent 的 Provider/Model 定义；
- `delete_after_run`：当前工具不开放且始终写 `false`；`Schedule::At` 完成领取后统一禁用，用户显式删除即可。 - `delete_after_run`：当前工具不开放且始终写 `false`；`Schedule::At` 在 claim 事务内立即禁用，用户显式删除即可。
### 5.2 DeliveryPolicy ### 5.2 DeliveryPolicy
@ -172,21 +174,36 @@ pub enum ScheduledRunStatus {
成功调用终结工具后，生命周期状态是 `completed`，业务结果由 `outcome` 表达；例如 `completed + failed` 表示 Agent 正常结束并明确报告检查失败。Provider 错误或结果协议缺失则是 `failed + failed`。 成功调用终结工具后，生命周期状态是 `completed`，业务结果由 `outcome` 表达；例如 `completed + failed` 表示 Agent 正常结束并明确报告检查失败。Provider 错误或结果协议缺失则是 `failed + failed`。
两者必须满足以下组合约束，不能独立随意取值：
| Run status | 合法 Outcome |
|---|---|
| `claimed` / `running` | `NULL` |
| `completed` | `ok` / `alert` / `failed` / `refused` |
| `failed` / `timed_out` / `cancelled` / `interrupted` | `failed` |
| `unknown` | `unknown` |
因此 `unknown` status 与 `unknown` outcome 恒同现，且只能由平台恢复路径生成：包括启动时恢复未完成运行，以及迁移无法映射的旧状态；模型不能提交它。历史迁移必须先按生命周期状态归一化，再决定 Outcome，不能产生 `unknown + ok` 等非法组合。
## 6. 总体架构 ## 6. 总体架构
```mermaid ```mermaid
flowchart TD flowchart TD
Tick[Scheduler tick] --> Claim[Storage claim occurrence] EventLoop[Scheduler event loop] -->|tick + free slots| Claim[Storage claim occurrence]
EventLoop -->|run completion| Reap[reap JoinSet]
EventLoop -->|pending due| Delivery[bounded delivery drain]
Claim --> RunRow[(job_runs: claimed)] Claim --> RunRow[(job_runs: claimed)]
Claim --> Advance[提前推进 next_run / 禁用 At] Claim --> Advance[提前推进 next_run / 禁用 At]
RunRow --> Resolve[ScheduledAgentRunner] RunRow --> JoinSet[bounded JoinSet]
JoinSet --> Resolve[ScheduledAgentRunner]
Resolve --> Catalog[Root config / AgentCatalog] Resolve --> Catalog[Root config / AgentCatalog]
Resolve --> Agent[AgentLoop] Resolve --> Agent[AgentLoop]
Agent --> Tools[受限工具 + complete_scheduled_run] Agent --> Tools[受限工具 + complete_scheduled_run]
Tools --> Outcome[ScheduledOutcome] Tools --> Outcome[ScheduledOutcome]
Outcome --> Commit[原子提交 Run 终态与 delivery_status] Outcome --> Reap
Reap --> Commit[原子提交 Run 终态与 delivery_status]
Commit -->|suppressed / not_requested| Done[完成] Commit -->|suppressed / not_requested| Done[完成]
Commit -->|pending| Delivery[Scheduler delivery drain] Commit -->|pending| Delivery
Delivery --> Bus[MessageBus / OutboundDispatcher] Delivery --> Bus[MessageBus / OutboundDispatcher]
Bus --> Channel[Channel] Bus --> Channel[Channel]
Channel --> Ack[delivery_status = delivered] Channel --> Ack[delivery_status = delivered]
@ -196,13 +213,23 @@ flowchart TD
| 组件 | 职责 | 明确不负责 | | 组件 | 职责 | 明确不负责 |
|---|---|---| |---|---|---|
| `Scheduler` | tick、领取、并发上限、执行超时、恢复、投递 drain | 解释模型自然语言 | | `Scheduler` | 非阻塞事件循环、领取、并发上限、执行超时、恢复、投递 drain | 解释模型自然语言 |
| `ScheduledAgentRunner` | 解析 Root/命名 Agent、构造隔离上下文、运行 Agent | 渠道发送、next-run 计算 | | `ScheduledAgentRunner` | 解析 Root/命名 Agent、构造隔离上下文、运行 Agent | 渠道发送、next-run 计算 |
| `AgentLoop` | 模型/工具循环、识别终结工具已提交 | Job 状态或渠道策略 | | `AgentLoop` | 模型/工具循环、识别终结工具已提交 | Job 状态或渠道策略 |
| `complete_scheduled_run` | Schema 校验并提交一次结构化 Outcome | 消息发送、数据库写入 | | `complete_scheduled_run` | Schema 校验并提交一次结构化 Outcome | 消息发送、数据库写入 |
| `Storage` | occurrence、租约、终态、投递状态的原子转换 | Provider 和 Channel I/O | | `Storage` | occurrence、租约、终态、投递状态的原子转换 | Provider 和 Channel I/O |
| `OutboundDispatcher` | 目标有序、瞬态错误重试、渠道调用、类型化投递回执 | Scheduled Run 业务分类 | | `OutboundDispatcher` | 目标有序、瞬态错误重试、渠道调用、类型化投递回执 | Scheduled Run 业务分类 |
Scheduler 主循环维护一个 `JoinSet` 和固定并发上限，不再像旧实现一样等待整批 Job 全部结束后才进入下一次 poll：
1. tick 到达时先回收已完成 Run，并按空闲槽位领取新 occurrence；
2. 每个 Run 终态提交后立即尝试认领并投递自己的 pending 结果；
3. 启动和后续 tick 另行有界回收遗留 pending delivery；
4. 长任务只占用一个执行槽，不阻塞其他到期 Job 的领取或已完成结果的投递；
5. Run task、delivery task 和主循环都由 TaskSupervisor 拥有，关停时停止 admission、取消并有界 join。
这仍是一个 Scheduler、一种 Job 和一条投递路径；`JoinSet` 只解决生命周期阻塞，不引入新的任务类型或第二套执行器。
## 7. 结构化终结协议 ## 7. 结构化终结协议
### 7.1 工具定义 ### 7.1 工具定义
@ -242,7 +269,25 @@ flowchart TD
### 7.2 AgentLoop 终结行为 ### 7.2 AgentLoop 终结行为
`ToolExecutionContext` 增加可选的 `ScheduledCompletionSink`。终结工具用一次性 compare-and-set 写入 Outcome。AgentLoop 在每个工具调用之后检查 sink： 运行上下文增加两个用途不同的字段：
```rust
pub enum ExecutionOrigin {
Interactive,
Scheduled { job_run_id: i64 },
}
pub struct ToolExecutionContext {
// existing fields...
pub execution_origin: ExecutionOrigin,
pub scheduled_completion: Option<Arc<ScheduledCompletionSink>>,
}
```
- `execution_origin` 是事实标记，普通构造器缺省为 `Interactive`，`execute_scheduled()` 显式设为 Scheduled；每次 Coordinator 构造子 Agent 的 ToolExecutionContext 时从父 context 复制，因此所有子孙都继承。它用来统一禁止 signal/inbox、把 background 委托降级为 foreground；不得通过字符串形式的 session ID 或 Agent ID 推断来源。
- `scheduled_completion` 是顶层独占能力，只存在于 Scheduled 顶层 Agent，不得传给任何子 Agent。
终结工具用一次性 compare-and-set 写入 `ScheduledCompletionSink`。AgentLoop 在每个工具调用之后检查 sink：
1. 未提交：继续普通工具循环； 1. 未提交：继续普通工具循环；
2. 已提交：把本次终结工具调用和结果写入 Agent transcript； 2. 已提交：把本次终结工具调用和结果写入 Agent transcript；
@ -276,11 +321,11 @@ flowchart TD
`agent_id = NULL` 表示使用当前 Root Provider、模型、上下文窗口和 Skills。工具从 Root registry 派生，但移除： `agent_id = NULL` 表示使用当前 Root Provider、模型、上下文窗口和 Skills。工具从 Root registry 派生，但移除：
- `send_message`； - `send_message`；
- `cron_add`、`cron_update`、`cron_remove`、`cron_enable`、`cron_disable`； - `cron_add`、`cron_update`、`cron_remove`、`cron_enable`、`cron_disable`、`cron_list`、`cron_runs`；
- `reload_config`； - `reload_config`；
- 只服务于交互会话或后台收件箱控制的工具。 - 只服务于交互会话或后台收件箱控制的工具。
再注入 `complete_scheduled_run`。Root Scheduled Run 不加载用户聊天历史。 再注入 `complete_scheduled_run` 和 `delegate`。`delegate` 的目标仍受 AgentCatalog `root_can_delegate()` 约束，其 background 参数在 Scheduled origin 下统一降级为 foreground。Root Scheduled Run 不加载用户聊天历史。
### 8.2 命名 Agent 任务 ### 8.2 命名 Agent 任务
@ -302,11 +347,16 @@ Scheduled 顶层执行仍复用 `agent_runs` 审计和 transcript，但不增加
- `caller_agent_id = SCHEDULER`； - `caller_agent_id = SCHEDULER`；
- `caller_scope_id = scheduled:<job_id>`； - `caller_scope_id = scheduled:<job_id>`；
- `root_session_id = scheduled-run:<job_run_id>`，它是审计 scope，不是可接收 Inbox 的 Session； - `root_session_id = scheduled-run:<job_run_id>`，它是审计 scope，不是可接收 Inbox 的 Session；
- 顶层 `ToolExecutionContext.session_id = scheduled-run:<job_run_id>`，供 browser 等 session-scoped 状态工具隔离使用；
- 顶层 `ToolExecutionContext.agent` 必须指向该 AgentRun 的完整 AgentExecutionContext；Agent 访问授权以其中的 `root_session_id` 和 ancestry 为准，不以 `session_id` 字符串代替；
- `execution_origin = Scheduled { job_run_id }`；
- `completion_slot_reserved = false`； - `completion_slot_reserved = false`；
- `job_runs.agent_run_id` 关联顶层 AgentRun。 - `job_runs.agent_run_id` 关联顶层 AgentRun。
这是执行来源的区别，不是第三种并发模式，因此不扩展 `foreground/background` 枚举。 这是执行来源的区别，不是第三种并发模式，因此不扩展 `foreground/background` 枚举。
Scheduled 顶层及其后代不得创建 `agent_session_state`，不得调用 `reserve_completion_slots`，也不得插入 signal/completion inbox event。`recover_agent_state()` 的通用容量对账不应看到合成 Scheduled scope。
## 9. 子 Agent 语义 ## 9. 子 Agent 语义
Scheduled Agent 可以调用 `delegate`，但所有子任务必须在本次 Scheduled Run 内收敛： Scheduled Agent 可以调用 `delegate`，但所有子任务必须在本次 Scheduled Run 内收敛：
@ -323,9 +373,9 @@ delegate(mode=background) → 自动改为 foreground，并在工具结果中说
- 避免创建无法投递的 `cron:<id>` completion； - 避免创建无法投递的 `cron:<id>` completion；
- 前台子 Agent 仍可并行批量执行并由父 Agent 汇总。 - 前台子 Agent 仍可并行批量执行并由父 Agent 汇总。
Scheduled Agent 不能使用 `emit_signal` 向用户 Session 建立旁路。子 Agent 的最终结果作为普通工具结果返回父 Scheduled Agent，只有父 Agent 可以调用 `complete_scheduled_run`。 Scheduled Agent 不能使用 `emit_signal` 向用户 Session 建立旁路。即使命名 Agent definition 声明了 signal contract，Scheduled origin 也不得注入 `emit_signal`，不得预留 completion slot。子 Agent 的最终结果作为普通工具结果返回父 Scheduled Agent，只有父 Agent 可以调用 `complete_scheduled_run`。
`ScheduledCompletionSink` 是顶层运行能力，不随 `ToolExecutionContext` 克隆给子 Agent。Coordinator 构造子 Agent context 时必须显式清空该能力，子 Agent registry 也不得注入终结工具；否则子 Agent 会越过父 Agent 汇总直接结束整次任务。 `ExecutionOrigin::Scheduled` 必须随每一层子 Agent 继承，使嵌套委托也保持同步收敛语义；`ScheduledCompletionSink` 则是顶层运行能力，Coordinator 构造子 Agent context 时必须显式清空，子 Agent registry 也不得注入终结工具。二者不能合并为同一个可选字段，否则清空 sink 后嵌套子 Agent 会丢失 Scheduled 限制。
Root Scheduled Agent 的第一次委托必须走 AgentCatalog 的 `root_can_delegate()` 规则，不能把合成的 `SCHEDULER` 或 `ROOT` 身份误当作命名 Agent 传给 `can_delegate()`。命名 Scheduled Agent 的后续委托仍走 `can_delegate(caller, target)`。这一区分只影响委托边校验，不创建第二条 Scheduled 执行路径。 Root Scheduled Agent 的第一次委托必须走 AgentCatalog 的 `root_can_delegate()` 规则，不能把合成的 `SCHEDULER` 或 `ROOT` 身份误当作命名 Agent 传给 `can_delegate()`。命名 Scheduled Agent 的后续委托仍走 `can_delegate(caller, target)`。这一区分只影响委托边校验，不创建第二条 Scheduled 执行路径。
@ -338,11 +388,15 @@ Root Scheduled Agent 的第一次委托必须走 AgentCatalog 的 `root_can_dele
1. 选择 `enabled=1 AND next_run_at<=now` 且租约为空/过期的 Job； 1. 选择 `enabled=1 AND next_run_at<=now` 且租约为空/过期的 Job；
2. 条件更新租约； 2. 条件更新租约；
3. 插入唯一的 `job_runs(status=claimed, delivery_status=awaiting_result)` 行，并把其自增 `id` 作为本次 occurrence 的稳定 ID，同时快照 Agent、投递策略和目标； 3. 插入唯一的 `job_runs(status=claimed, delivery_status=awaiting_result)` 行，并把其自增 `id` 作为本次 occurrence 的稳定 ID，同时快照 Agent、投递策略和目标；
5. 对 `Every/Cron` 把 `next_run_at` 推进到领取时刻之后的第一个未来时间； 4. 对 `Every/Cron` 把 `next_run_at` 推进到领取时刻之后的第一个未来时间；
6. 对 `At` 立即设置 `enabled=0`； 5. 对 `At` 在本次 claim 事务内立即设置 `enabled=0`；
7. 提交后返回 `ClaimedScheduledRun { job_snapshot, run_id, owner }`。 6. 提交后返回 `ClaimedScheduledRun { job_snapshot, run_id, owner }`。
领取条件、Run 插入和 Job 推进处于同一写事务；SQLite 的写串行化和条件租约保证一次到期状态只能提交一个 Run。无需额外维护 occurrence key 或 schedule revision。推进下次时间发生在执行之前，因此进程崩溃不会使本次 occurrence 被自动重放；用户之后重新启用一次性任务会建立一个新的 Run，不会与历史运行冲突。错过的多个历史 tick 不逐个补跑，只执行当前到期 occurrence，并计算下一个未来时间。 领取条件、Run 插入和 Job 推进处于同一写事务；SQLite 的写串行化和条件租约保证一次到期状态只能提交一个 Run。无需额外维护 occurrence key 或 schedule revision。推进下次时间发生在执行之前，因此进程崩溃不会使本次 occurrence 被自动重放。错过的多个历史 tick 不逐个补跑，只执行当前到期 occurrence，并计算下一个未来时间。
`Every` 的间隔从领取/开始时刻计算，而不是从完成时刻计算；例如每小时任务执行 55 分钟，下一次约 5 分钟后到期。`Cron` 始终表达绝对日历时间。任务仍受 Job 租约约束，不允许重叠；如果执行时间持续超过调度间隔，Health 应报告“执行时长覆盖调度间隔”，由管理员调整周期或拆分任务。
过去时间的 `At` 不能通过创建、更新或重新启用隐式重跑：`cron_add` / `cron_update` 拒绝 `at <= now`，`cron_enable` 拒绝启用已经过期的 At，并要求先把 schedule 更新到未来时间。迁移前已经 enabled 且到期的 At 仍保留为一次合法待执行 occurrence。
Job 执行租约固定为 `execution_timeout + shutdown_grace`，执行本身必须在 `execution_timeout` 内终止，因此不增加心跳续租任务。终态提交同时校验 `job_run_id + lock_owner + 非终态 status`；租约过期后迟到的旧执行不能修改新 Run 或 Job 摘要。 Job 执行租约固定为 `execution_timeout + shutdown_grace`，执行本身必须在 `execution_timeout` 内终止，因此不增加心跳续租任务。终态提交同时校验 `job_run_id + lock_owner + 非终态 status`；租约过期后迟到的旧执行不能修改新 Run 或 Job 摘要。
@ -362,14 +416,17 @@ claimed/running --process restart--> unknown
### 10.3 启动恢复 ### 10.3 启动恢复
Gateway 启动、Scheduler admission 尚未开放前： Gateway 启动、Scheduler admission 尚未开放前，Storage 先执行一次 `recover_scheduled_runs(active_generation, now)` 原子事务：
1. 找到所有 `job_runs.status IN ('claimed','running')`； 1. 找到所有 `job_runs.status IN ('claimed','running')`；
2. 标记为 `unknown`，写入固定诊断； 2. 把 JobRun 标记为 `status=unknown/outcome=unknown`，写入固定诊断；
3. 根据该 run 快照的 delivery policy 设置 `pending` 或 `not_requested`； 3. 若已关联非终态 AgentRun，把 AgentRun 标记为 `interrupted`，说明其执行生命周期随旧进程结束；
4. 清除关联 Job 的旧租约； 4. 根据 JobRun 快照的 delivery policy 设置 `pending` 或 `not_requested`；
5. 不回退已推进的 `next_run_at`，也不重跑 occurrence； 5. 清除关联 Job 的旧租约；
6. 开放 Scheduler admission 后先 drain pending delivery，再领取新任务。 6. 不回退已推进的 `next_run_at`，也不重跑 occurrence；
7. 在同一事务内提交以上跨表变化。
随后再执行通用 `AgentCoordinator::recover_on_activation()`：已由 Scheduled 恢复事务终结的 AgentRun 不会再次处理。`job_runs.status/outcome` 是业务结果和投递决策的唯一权威；`agent_runs.status=interrupted` 只表示编排执行被进程切断，不得反向覆盖 JobRun 的 `unknown`。开放 Scheduler admission 后先 drain pending delivery，再领取新任务。
优雅关停由 TaskSupervisor 先停止新领取，再取消/限时等待运行；能够得到明确取消结果时记录 `interrupted`，只有硬崩溃才在下次启动归为 `unknown`。 优雅关停由 TaskSupervisor 先停止新领取，再取消/限时等待运行；能够得到明确取消结果时记录 `interrupted`，只有硬崩溃才在下次启动归为 `unknown`。
@ -401,7 +458,7 @@ pending → delivering → delivered
└→ failed（永久失败或次数耗尽） └→ failed（永久失败或次数耗尽）
``` ```
Scheduler 在每次 tick 前后以及启动恢复后有界 drain： Scheduler 在 Run 终态提交后立即尝试该 Run，并在每次 tick 和启动恢复后有界 drain 遗留项：
- 原子认领 `pending` 或租约过期的 `delivering` 行； - 原子认领 `pending` 或租约过期的 `delivering` 行；
- 最多 3 次持久化尝试； - 最多 3 次持久化尝试；
@ -420,11 +477,36 @@ Scheduler 在每次 tick 前后以及启动恢复后有界 drain：
OutboundDispatcher 保留现有单次调用内的短暂、内存级瞬态重试；该调用最终返回的回执算一次持久化 delivery attempt。`TransientFailure`、`TimedOut` 和 `DispatcherClosed` 在未达到持久化尝试上限时回到 `pending`，`PermanentFailure` 直接进入 `failed`。Scheduler 不根据错误字符串猜测是否可重试，也不把“成功写入 outbound 队列”当作渠道送达。 OutboundDispatcher 保留现有单次调用内的短暂、内存级瞬态重试；该调用最终返回的回执算一次持久化 delivery attempt。`TransientFailure`、`TimedOut` 和 `DispatcherClosed` 在未达到持久化尝试上限时回到 `pending`，`PermanentFailure` 直接进入 `failed`。Scheduler 不根据错误字符串猜测是否可重试，也不把“成功写入 outbound 队列”当作渠道送达。
类型映射固定如下，Channel adapter 必须先把平台错误归一化为正确的 ChannelError：
| 来源 | Delivery receipt | 行为 |
|---|---|---|
| Channel 成功返回 | `Delivered` | 标记 delivered |
| `ConnectionError` / `SendError` 经 Dispatcher 内部重试耗尽 | `TransientFailure` | 持久化退避后重试 |
| Dispatcher 单次发送最终超时 | `TimedOut` | 持久化退避后重试 |
| Bus/Dispatcher 关停、lane 暂时饱和 | `DispatcherClosed` / `TransientFailure` | 保留 pending，等待当前或下一运行代 |
| channel 不存在、`ConfigError`、明确的永久平台错误 | `PermanentFailure` | 立即标记 failed |
未知 `Other` 默认按永久失败处理，除非产生它的调用点明确证明可以安全重试。回执只持久化经过清洗和长度限制的 summary，不保存平台响应正文、凭据或临时 URL。Dispatcher 内部的秒级重试与 Scheduler 的持久化分钟级退避职责不同，不得互相递归调用或把每次内部尝试计入 `delivery_attempts`。
外部发送成功但本地 `delivered` 提交前崩溃时可能重复发送。渠道支持幂等键时传递稳定 ID；不支持时允许“至少一次”并在重试消息 metadata 中标出可能重复。不得为了避免重复而丢失告警。 外部发送成功但本地 `delivered` 提交前崩溃时可能重复发送。渠道支持幂等键时传递稳定 ID；不支持时允许“至少一次”并在重试消息 metadata 中标出可能重复。不得为了避免重复而丢失告警。
### 11.3 会话历史 ### 11.3 会话历史
通知使用确定性的本地 message ID，例如 `scheduled:<job_run_id>`。向目标会话写历史时执行幂等插入，同一个 JobRun 的重试不会产生多条本地历史。消息来源为： 需要投递时先解析一次目标会话，并用 `UPDATE ... WHERE target_session_id IS NULL` 把结果固定到 JobRun。后续重试不得重新选择“当前最近 dialog”，避免用户切换会话后同一个通知写入不同历史。目标会话 ID 同时用于本地历史和 OutboundMessage `_session_id` metadata。
会话解析或幂等历史事务失败时不得绕过历史直接发送：明确的无效目标进入永久 delivery failure；SQLite busy、运行代关停等瞬态错误回到 pending，并计为本次持久化 delivery attempt。
通知使用确定性的本地 message ID `scheduled:<job_run_id>`。现有 `messages.id` 已是主键，不增加第二个唯一索引；SessionManager 新增专用 `append_scheduled_notification_if_absent()`，在现有 session persistence lock 下协调内存，Storage 提供对应的原子事务：
1. 使用固定 message ID 执行 `INSERT ... ON CONFLICT(id) DO NOTHING`；
2. 只有实际插入时才更新持久化 Session metadata，并由 SessionManager 把同一消息加入内存、推进 state_version；
3. 已存在时 SessionManager 直接复用该消息和固定 target_session_id，不重复修改内存或 Session metadata；
4. 历史写入成功后才把 OutboundMessage 交给 `deliver_outbound()`。
SessionManager 可以使用专用 persistence lock 串行化该流程，但不得在 SQLite I/O 期间持有 Session mutex；数据库提交后重新取得 Session mutex，并以固定 message ID 检查内存后再应用一次变更。
同一个 JobRun 的重试因此不会产生多条本地历史。消息来源为：
```text ```text
SourceKind::ExternalTrigger SourceKind::ExternalTrigger
@ -475,15 +557,36 @@ from_run_id = agent_run_id
- 删除 `model`； - 删除 `model`；
- 不接受 `direct`。 - 不接受 `direct`。
`cron_update` 支持更新 prompt、schedule、channel、chat_id、agent_id 和 delivery_policy。`agent_id: null` 明确切回 Root；字段缺失表示不修改。 `cron_add` 和 `cron_update` 共用一个 ScheduledJob validator：channel 必须来自当前 ChannelManager allowlist，命名 `agent_id` 必须存在于当前不可变 AgentCatalog；Storage 只接受已验证的类型，不自行读取运行代配置。重载后 Agent 消失时保留 Job 并在运行时明确失败，见 §8.2。
`cron_update` 支持更新 prompt、schedule、channel、chat_id、agent_id 和 delivery_policy。`agent_id: null` 明确切回 Root；字段缺失表示不修改。新建或更新 `At` 时要求 `at > now`；`cron_enable` 对已过期 At 返回失败并要求先更新 schedule。
`cron_disable` 只阻止后续 occurrence，不取消已经 claimed/running 的 Run。`cron_remove` 在存在非终态 JobRun 或 Job 租约时返回 conflict，要求先 disable 并等待当前 Run 到达终态；不得依赖 `ON DELETE CASCADE` 静默删除正在执行的 occurrence。
`cron_list` 展示： `cron_list` 展示：
```text ```text
enabled · agent=web-monitor · delivery=on_alert · next=... · last=ok enabled · agent=web-monitor · delivery=on_alert · next=... · last=alert
``` ```
不再展示 kind 或 model。 不再展示 kind、model 或截断的 `last_error`；`last` 只显示 `last_outcome`，详细信息由 `cron_runs` 查询。
新增只读 `cron_runs` 工具并注册到普通全局 registry；Root 交互 Agent 可用，命名交互 Agent 仍须由 definition allowlist 授权，Scheduled Agent 的收窄规则一律移除它：
```json
{
"job_id": "job-id",
"limit": 20,
"run_id": 123
}
```
- `job_id` 必填；
- 不传 `run_id` 时列出最近记录，`limit` 缺省 20、范围 1–100，每条返回 run_id、时间、status、outcome、delivery status/attempts、duration 及有界 message/diagnostic 摘要；
- 传 `run_id` 时校验它属于该 Job，并返回该次运行的完整结构化字段；message 上限沿用 16 KiB，diagnostic 经过清洗并执行独立上限；
- `read_only() == true`，不修改 Job/Run、不触发重投；
- `never`、`suppressed` 和投递失败记录与其他策略一样可查询；
- 输出不包含 Provider 私有状态、reasoning、凭据或 Agent transcript。
### 13.2 WebUI ### 13.2 WebUI
@ -493,7 +596,10 @@ enabled · agent=web-monitor · delivery=on_alert · next=... · last=ok
- 投递：始终通知 / 异常通知 / 从不通知； - 投递：始终通知 / 异常通知 / 从不通知；
- 最近 Outcome； - 最近 Outcome；
- 最近运行生命周期状态； - 最近运行生命周期状态；
- 投递状态及失败原因。 - 投递状态、尝试次数及失败原因；
- 最近运行的 message/diagnostic 摘要，并可展开单次详情。
`GET /api/jobs/{id}/runs` 明确切换到 v11 shape：`id/job_id/scheduled_for/started_at/finished_at/status/outcome/message/diagnostic/duration_ms/delivery_status/delivery_attempts/delivery_error`；删除 `output/error/result_kind`。API 继续受管理认证保护并保持 limit 上限。
创建表单可以提供三个用户友好模板，但模板不进入后端模型： 创建表单可以提供三个用户友好模板，但模板不进入后端模型：
@ -509,6 +615,8 @@ HealthService 增加只读 Scheduler 配置检查：
- 非法 channel； - 非法 channel；
- 长时间停留在 pending/delivering 的投递； - 长时间停留在 pending/delivering 的投递；
- 最近一次 `unknown`、`failed` 或 `timed_out`； - 最近一次 `unknown`、`failed` 或 `timed_out`；
- `never` Job 最近的 unknown（明确提示其不会自动通知）；
- 最近执行时长持续覆盖 `Every` 周期；
- enabled Job 无法计算 next run。 - enabled Job 无法计算 next run。
Health 不执行 Job、不连接 Provider、不修复数据。 Health 不执行 Job、不连接 Provider、不修复数据。
@ -573,6 +681,8 @@ CREATE TABLE job_runs (
CHECK (delivery_policy IN ('always','on_alert','never')), CHECK (delivery_policy IN ('always','on_alert','never')),
target_channel TEXT NOT NULL, target_channel TEXT NOT NULL,
target_chat_id TEXT NOT NULL, target_chat_id TEXT NOT NULL,
-- resolved once when delivery is first prepared; stable across retries
target_session_id TEXT,
started_at INTEGER, started_at INTEGER,
finished_at INTEGER, finished_at INTEGER,
@ -603,8 +713,21 @@ CREATE TABLE job_runs (
CHECK (agent_id IS NULL OR length(trim(agent_id)) > 0), CHECK (agent_id IS NULL OR length(trim(agent_id)) > 0),
CHECK (length(trim(target_channel)) > 0), CHECK (length(trim(target_channel)) > 0),
CHECK (length(trim(target_chat_id)) > 0), CHECK (length(trim(target_chat_id)) > 0),
CHECK (target_session_id IS NULL OR length(trim(target_session_id)) > 0),
CHECK (delivery_attempts >= 0), CHECK (delivery_attempts >= 0),
CHECK (duration_ms IS NULL OR duration_ms >= 0), CHECK (duration_ms IS NULL OR duration_ms >= 0),
CHECK (
(status IN ('claimed','running') AND outcome IS NULL
AND finished_at IS NULL AND delivery_status = 'awaiting_result') OR
(status = 'completed' AND outcome IS NOT NULL
AND outcome IN ('ok','alert','failed','refused')
AND finished_at IS NOT NULL AND delivery_status != 'awaiting_result') OR
(status IN ('failed','timed_out','cancelled','interrupted')
AND outcome IS NOT NULL AND outcome = 'failed'
AND finished_at IS NOT NULL AND delivery_status != 'awaiting_result') OR
(status = 'unknown' AND outcome IS NOT NULL AND outcome = 'unknown'
AND finished_at IS NOT NULL AND delivery_status != 'awaiting_result')
),
CHECK ( CHECK (
(delivery_lease_owner IS NULL AND delivery_lease_until IS NULL) OR (delivery_lease_owner IS NULL AND delivery_lease_until IS NULL) OR
(delivery_lease_owner IS NOT NULL AND delivery_lease_until IS NOT NULL) (delivery_lease_owner IS NOT NULL AND delivery_lease_until IS NOT NULL)
@ -621,7 +744,7 @@ CREATE INDEX idx_job_runs_delivery
ON job_runs(delivery_status, delivery_next_attempt_at, delivery_lease_until); ON job_runs(delivery_status, delivery_next_attempt_at, delivery_lease_until);
``` ```
不为 AgentCatalog 建数据库外键；Agent 定义是配置运行代资源，不是 SQLite 行。`agent_id` 和 definition hash 的最终审计值保存在关联 AgentRun 中。 不为 AgentCatalog 建数据库外键；Agent 定义是配置运行代资源，不是 SQLite 行。`agent_id` 和 definition hash 的最终审计值保存在关联 AgentRun 中。`target_session_id` 也不建 Session 外键：它是投递时固定的路由快照，Session 使用软删除且重试不能因当前 dialog 变化而自动改投；写入历史时仍由 SessionManager 验证目标行和作用域。
## 15. 自动迁移设计 ## 15. 自动迁移设计
@ -632,7 +755,7 @@ ON job_runs(delivery_status, delivery_next_attempt_at, delivery_lease_until);
- migration 使用一个专用连接和 `BEGIN IMMEDIATE`，在单个 SQLite 事务中完成； - migration 使用一个专用连接和 `BEGIN IMMEDIATE`，在单个 SQLite 事务中完成；
- 任一步失败则整体回滚，`user_version` 保持原值，Gateway 拒绝启动； - 任一步失败则整体回滚，`user_version` 保持原值，Gateway 拒绝启动；
- v11 运行时代码不读取旧列、不解析旧枚举、不调用旧函数； - v11 运行时代码不读取旧列、不解析旧枚举、不调用旧函数；
- 旧 schema 知识只存在于 `storage/migrations/v11_scheduled.rs`； - 旧版本形状探测只存在于冻结的 `storage/migrations/legacy_to_v10.rs` 和 `storage/migrations/v11_scheduled.rs`；
- v11 数据库由旧版本二进制打开时，沿用现有“数据库版本过新”拒绝策略； - v11 数据库由旧版本二进制打开时，沿用现有“数据库版本过新”拒绝策略；
- 不支持自动降级。 - 不支持自动降级。
@ -640,12 +763,19 @@ ON job_runs(delivery_status, delivery_next_attempt_at, delivery_lease_until);
### 15.2 初始化顺序调整 ### 15.2 初始化顺序调整
当前 `init_scheduler_schema()` 在 `migrate_schema()` 之前执行，会让新旧 DDL 的所有权混乱。实施后： 当前 `migrate_schema()` 是一段按实际 table shape 累积修补的迁移，并不存在现成的逐版本 `v10::up()`；同时 `init_scheduler_schema()` 在 migration 前创建旧 Scheduler DDL。实施不重写全部 v1–v10 历史，而采用最小版本化：
1. `migrate_schema()` 成为 Scheduler 表 DDL 的唯一入口； 1. 删除 migration 前的 `init_scheduler_schema()` 调用，`migrate_schema()` 成为 Scheduler 表 DDL 的唯一入口；
2. 全新数据库直接创建 v11 表； 2. 把当前累积逻辑冻结、抽取为 `normalize_legacy_to_v10(conn, detected_shape)`，只供 `current < 10` 的启动迁移调用；
3. `user_version=0` 但已有旧表的数据库先执行现有 legacy→v10 规范化，再执行 v10→v11； 3. 新增 `migrate_v10_to_v11(conn)`，只接受经过预检的 canonical v10 Scheduler 表；
4. 测试不再直接调用可绕过 migration 的 `init_scheduler_schema()`，统一通过 `Storage::new` 或专用 latest-schema fixture。 4. 全新数据库直接创建 latest base/Agent/Scheduler v11 schema，不先建立再重建 v10 Scheduler 表；
5. `user_version=0` 且已有历史应用表时先运行 legacy normalizer，再运行 v11；若旧数据库从未有 Scheduler 表，normalizer 只处理其他历史表，随后直接创建 v11 Scheduler 表；
6. `current=10` 只执行 v11，`current=11` 不执行迁移，`current>11` 拒绝；
7. Agent schema 必须在创建带 `agent_runs` 外键的 v11 `job_runs` 之前就绪；
8. 整个所需步骤共享同一个专用连接和 `BEGIN IMMEDIATE` 事务，只有最后写 `user_version=11`；
9. 测试不再直接调用可绕过 migration 的旧初始化函数，统一通过 `Storage::new`、fresh-schema fixture 或明确的 v10 fixture。
这只是启动 migration 的版本兼容，不是运行时双路径。无需把历史逻辑拆成 `v1→v2→...→v10`，也不得在 v11 Scheduler 查询中保留 shape probing。
### 15.3 v10 预检 ### 15.3 v10 预检
@ -666,29 +796,42 @@ ON job_runs(delivery_status, delivery_next_attempt_at, delivery_lease_until);
| `delivery_policy=direct` | `always` | 新代码统一中央投递 | | `delivery_policy=direct` | `always` | 新代码统一中央投递 |
| `always/on_alert/never` | 原值 | 保留 | | `always/on_alert/never` | 原值 | 保留 |
| `model` | 删除 | 当前执行路径未应用该值；迁移日志报告被删除的非空数量 | | `model` | 删除 | 当前执行路径未应用该值；迁移日志报告被删除的非空数量 |
| `delete_after_run` | 删除 | `At` 任务改为完成领取后禁用 | | `delete_after_run` | 删除 | `At` 任务改为 claim 事务内立即禁用 |
| 无 `agent_id` | `NULL` | 使用 Root Scheduled Agent | | 无 `agent_id` | `NULL` | 使用 Root Scheduled Agent |
| `last_status=ok` | `last_outcome=ok` | 仅作列表摘要 | | `last_status` | `last_outcome` | 优先取迁移后最新 JobRun.outcome；没有历史 Run 时 `ok→ok`、其他非空值→failed |
| 其他非空 `last_status` | `last_outcome=failed` | 详细旧信息仍在历史 run | | `last_error` | 删除 | 作为最新旧 JobRun diagnostic 的 fallback 后删除，不保留 Job 级兼容列 |
内置 `picobot-routine-maintenance` 的 prompt 在迁移中按固定 Job ID 改成不含 `NO_REPLY` 的任务描述。用户自定义 prompt 不做不安全的字符串替换；新的系统级 Scheduled 契约明确忽略其中关于 `NO_REPLY`、`send_message` 和旧输出格式的指令。 内置 `picobot-routine-maintenance` 的 prompt 在迁移中按固定 Job ID 改成不含 `NO_REPLY` 的任务描述。用户自定义 prompt 不做不安全的字符串替换；新的系统级 Scheduled 契约明确忽略其中关于 `NO_REPLY`、`send_message` 和旧输出格式的指令。
### 15.5 历史 JobRun 映射 ### 15.5 历史 JobRun 映射
历史行只用于审计，绝不因升级重新投递： 历史行只用于审计，绝不因升级重新投递。status 与 outcome 必须联合映射：
| v10 值 | v11 值 | | v10 条件 | v11 status | v11 outcome |
|---|---|---|
| `status=ok` | `completed` | 按下表 result_kind 映射 |
| `status=delivery_error` 且已有模型结果/result_kind | `completed` | 按下表 result_kind 映射 |
| `status=delivery_error` 且执行 error 非空、无结果 | `failed` | `failed` |
| `status=timeout` | `timed_out` | `failed` |
| `status=error` | `failed` | `failed` |
| 其他状态 | `unknown` | `unknown` |
仅在新 status 为 `completed` 时读取 result_kind：
| v10 result_kind | v11 outcome |
|---|---|
| `quiet` | `ok` |
| `content` 且 Job 为 `on_alert` | `alert` |
| `content` 其他情况 | `ok` |
| `reported_failure` | `failed` |
| `refused` | `refused` |
| `NULL`（旧 Direct） | `ok` |
| 其他无法映射值 | migration error，不猜测 |
delivery 单独映射，不反向改变执行 status/outcome：
| v10 delivery_status | v11 delivery_status |
|---|---| |---|---|
| `status=ok` | `status=completed` |
| `status=timeout` | `status=timed_out` |
| `status=error/delivery_error` | `status=failed` |
| 其他状态 | `status=unknown` |
| `result_kind=quiet` | `outcome=ok` |
| `result_kind=content` 且 Job 为 `on_alert` | `outcome=alert` |
| `result_kind=content` 其他情况 | `outcome=ok` |
| `result_kind=reported_failure` | `outcome=failed` |
| `result_kind=refused` | `outcome=refused` |
| 无法映射 | `outcome=unknown` |
| `delivery_status=direct/delivered` | `delivered` | | `delivery_status=direct/delivered` | `delivered` |
| `suppressed/skipped` | `suppressed` | | `suppressed/skipped` | `suppressed` |
| `failed` 或存在 `delivery_error` | `failed` | | `failed` 或存在 `delivery_error` | `failed` |
@ -698,11 +841,14 @@ ON job_runs(delivery_status, delivery_next_attempt_at, delivery_lease_until);
- `scheduled_for = started_at`； - `scheduled_for = started_at`；
- `message = output`； - `message = output`；
- `diagnostic = error`； - `diagnostic = error`；最新历史 Run 的 error 为空时可以用同 Job 的 `last_error` 补全；
- `target_*`、`delivery_policy` 从迁移时的 Job 行快照； - `target_*`、`delivery_policy` 从迁移时的 Job 行快照；
- `target_session_id = NULL`，历史行不重新投递，因此不解析会话；
- `agent_run_id = NULL`； - `agent_run_id = NULL`；
- 所有 delivery lease 字段为空。 - 所有 delivery lease 字段为空。
若 Job 存在非空 `last_error` 但没有任何可承载它的历史 Run，migration 记录 Job ID 和计数后丢弃该冗余摘要，不为兼容旧投影伪造一条执行记录。
### 15.6 迁移时发现旧执行锁 ### 15.6 迁移时发现旧执行锁
v10 只有 Job lease，没有预先建立的 JobRun。若迁移时发现 `lock_owner` 非空或 lease 尚未清理，说明旧进程可能在终态提交前退出： v10 只有 Job lease，没有预先建立的 JobRun。若迁移时发现 `lock_owner` 非空或 lease 尚未清理，说明旧进程可能在终态提交前退出：
@ -720,16 +866,17 @@ v10 只有 Job lease，没有预先建立的 JobRun。若迁移时发现 `lock_o
SQLite 删除列和修改 CHECK 约束采用 canonical table rebuild： SQLite 删除列和修改 CHECK 约束采用 canonical table rebuild：
1. `CREATE TABLE scheduled_jobs_v11 ...`； 1. 确保 latest `agent_runs` schema 已创建；
2. 写入转换后的 Job； 2. 把旧 `job_runs`、`scheduled_jobs` 依次 rename 为内部 legacy 临时表；
3. `CREATE TABLE job_runs_v11 ... REFERENCES scheduled_jobs_v11`； 3. 使用 fresh database 相同的 v11 DDL 常量创建 canonical `scheduled_jobs` 和 `job_runs`；
4. 写入转换后的历史 Run 和旧锁恢复 Run； 4. 写入转换后的 Job、历史 Run 和旧锁恢复 Run；
5. 删除旧 `job_runs`； 5. 删除 legacy `job_runs`；
6. 删除旧 `scheduled_jobs`； 6. 删除 legacy `scheduled_jobs`；
7. rename `scheduled_jobs_v11`、`job_runs_v11`； 7. 重建索引；
8. 重建索引； 8. 执行 `PRAGMA foreign_key_check` 并要求零行；
9. 执行 `PRAGMA foreign_key_check` 并要求零行； 9. 最后设置 `PRAGMA user_version = 11` 并提交。
10. 最后设置 `PRAGMA user_version = 11` 并提交。
全新数据库走 latest schema creation，不经过临时 v10 表和上述数据搬迁；但使用同一组 v11 DDL 常量，避免 fresh schema 与 migrated schema 漂移。
表重建必须在同一连接、同一事务中完成。不得用多个 pool connection 分散 DDL，也不得在事务提交前启动 Scheduler。 表重建必须在同一连接、同一事务中完成。不得用多个 pool connection 分散 DDL，也不得在事务提交前启动 Scheduler。
@ -766,7 +913,7 @@ SQLite 删除列和修改 CHECK 约束采用 canonical table rebuild：
- Agent 返回普通文本后再分类的代码； - Agent 返回普通文本后再分类的代码；
- 先发送、后写 JobRun 的顺序。 - 先发送、后写 JobRun 的顺序。
替换为：claim occurrence → `ScheduledAgentRunner` → typed outcome → terminal commit → delivery drain。 替换为：非阻塞 tick/JoinSet 事件循环 → claim occurrence → `ScheduledAgentRunner` → typed outcome → terminal commit → immediate/bounded delivery drain。删除等待整批 Run 完成后才继续 poll 的 `for_each_concurrent(...).await` 结构。
### `src/session/session.rs` ### `src/session/session.rs`
@ -790,15 +937,15 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
- `set_scheduled_job_behavior()`； - `set_scheduled_job_behavior()`；
- 完成时才插入 JobRun 的旧事务。 - 完成时才插入 JobRun 的旧事务。
新增严格 typed parser、occurrence claim、终态提交、未知恢复和持久化 delivery claim。 新增严格 typed parser、occurrence claim、status/outcome 联合约束、跨 JobRun/AgentRun 的原子未知恢复、终态提交、固定 target_session_id 和持久化 delivery claim。
### `src/storage/mod.rs` 与 `src/storage/migrations/` ### `src/storage/mod.rs` 与 `src/storage/migrations/`
把 `SCHEMA_VERSION` 一次提升到 11，删除 migration 之前调用 Scheduler 旧 DDL 初始化函数的顺序。新增唯一的 v11 Scheduler migration 模块和跨表终态事务 API；迁移完成后通用 Storage 查询不得包含任何 v10 列名。 把 `SCHEMA_VERSION` 一次提升到 11，删除 migration 之前调用 Scheduler 旧 DDL 初始化函数的顺序。把现有累积迁移冻结为 `legacy_to_v10` normalizer，新增唯一的 v11 Scheduler migration、fresh v11 DDL 和跨表终态/恢复事务 API；迁移完成后通用 Storage 查询不得包含任何 v10 列名。
### `src/tools/cron.rs` ### `src/tools/cron.rs`
删除 `kind`、`model` 参数与所有默认联动；默认 `delivery_policy=always`。新增可选 `agent_id`，更新 list 输出和测试。 删除 `kind`、`model` 参数与所有默认联动；默认 `delivery_policy=always`。新增可选 `agent_id` 和共享 Agent/Channel validator，拒绝过去的 At；更新 list 输出，并新增只读 `CronRunsTool`。
### `src/tools` ### `src/tools`
@ -806,22 +953,23 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
### `src/agent` ### `src/agent`
新增 `ScheduledCompletionSink` 和 `AgentCoordinator::execute_scheduled()`；复用 foreground AgentRun 持久化，不创建 inbox completion。AgentLoop 在终结工具成功后停止。 新增可继承的 `ExecutionOrigin::Scheduled`、顶层独占的 `ScheduledCompletionSink` 和 `AgentCoordinator::execute_scheduled()`；复用 foreground AgentRun 持久化，不创建 agent_session_state、signal 或 inbox completion。AgentLoop 在终结工具成功后停止。
### `src/session/messenger.rs` ### `src/session/messenger.rs`
把 Scheduled 通知改成稳定 message ID 的幂等写入；删除任何依赖 `cron:<id>` 作为可消费会话的行为。 第一次投递时固定 target_session_id；把 Scheduled 通知改成利用现有 `messages.id` 主键的稳定 message ID/原子 insert-if-absent 写入；只有实际插入时更新 Session 内存和 metadata。删除任何依赖 `cron:<id>` 作为可消费会话的行为。
### `src/bus` ### `src/bus`
保留 `MessageBus::deliver_outbound()` 的等待回执入口，把 `OutboundMessage.delivery` 和 `BusError` 的字符串结果改为类型化、可判定 retry class 的安全回执。OutboundDispatcher 仍是唯一 Channel 调用方；普通无回执消息继续使用 `publish_outbound()`。 保留 `MessageBus::deliver_outbound()` 的等待回执入口，把 `OutboundMessage.delivery` 和 `BusError` 的字符串结果改为类型化、可判定 retry class 的安全回执，并补齐 ChannelError→receipt 映射。OutboundDispatcher 仍是唯一 Channel 调用方；普通无回执消息继续使用 `publish_outbound()`。
### Gateway / WebUI / Protocol ### Gateway / WebUI / Protocol
- API JSON 删除 `job_kind`、`model`、`delete_after_run`、`result_kind`； - Gateway 激活时先执行原子 Scheduled 恢复，再执行通用 Agent recovery，最后开放 Scheduler admission；
- 增加 `agent_id`、`outcome`、delivery attempts/状态； - API JSON 删除 `job_kind`、`model`、`delete_after_run`、`output/error/result_kind`；
- 增加 `agent_id`、`outcome/message/diagnostic` 和 delivery attempts/状态；JobRun 内部另存固定 target Session 路由快照，不向非管理客户端暴露；
- `webui/src/pages/TasksPage.svelte` 删除“巡检/任务”判断； - `webui/src/pages/TasksPage.svelte` 删除“巡检/任务”判断；
- Health 与任务详情展示新的结构化状态。 - `/api/jobs/{id}/runs`、TasksPage、Health 与 `cron_runs` 消费同一 JobRun projection。
### 文档与内置知识 ### 文档与内置知识
@ -843,19 +991,26 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
1. 一个 Job 同时最多有一个持租约 occurrence；短周期任务不会重叠执行。 1. 一个 Job 同时最多有一个持租约 occurrence；短周期任务不会重叠执行。
2. occurrence 的 JobRun 在 Agent 启动前持久化；同一个到期状态只能提交一个 Run。 2. occurrence 的 JobRun 在 Agent 启动前持久化；同一个到期状态只能提交一个 Run。
3. `next_run_at` 在 claim 事务中推进；执行失败和进程崩溃不会重放同一 occurrence。 3. `next_run_at` 在 claim 事务中推进；执行失败和进程崩溃不会重放同一 occurrence。
4. 只有 lease owner 可以把 run 从非终态推进到终态。 4. `At` 在 claim 事务内禁用；过期 At 不能通过 enable 隐式重跑。
5. 所有终态不可重写；迟到 Agent 结果必须丢弃并记录日志。 5. disable 只影响未来 occurrence；存在非终态 Run 时禁止删除 Job。
6. Outcome 和投递策略都使用 claim-time snapshot；运行过程中编辑 Job 只影响后续 occurrence。 6. Scheduler 主循环不等待整批执行结束；长任务只占并发槽，不能阻塞其他 Job 或 pending delivery。
7. Outcome、Job 摘要和初始 delivery status 在同一事务提交。 7. 只有 `job_run_id + lease owner + 非终态 status` 同时匹配才可以提交终态。
8. 渠道 I/O 不发生在 SQLite 事务或 Session mutex 内。 8. 所有终态不可重写；迟到 Agent 结果必须丢弃并记录日志。
9. Scheduled Run 不预留 Agent Inbox slot，也不向合成 session 投递 completion。 9. status/outcome 必须满足 §5.4 联合约束；`unknown` 只能由运行时生成。
10. 后台委托在模型可见参数不变，但 Scheduled context 强制变为 foreground。 10. Outcome 和投递策略都使用 claim-time snapshot；运行过程中编辑 Job 只影响后续 occurrence。
11. `never` 严格不通知，包括失败、拒绝、超时和 unknown；这些状态仍可通过管理页面和 Health 查看。 11. Outcome、Job 摘要和初始 delivery status 在同一事务提交。
12. 静默只来源于结构化 `ok` 与策略矩阵，不能来源于普通文本。 12. 渠道 I/O 不发生在 SQLite 事务或 Session mutex 内。
13. Scheduled origin 传递给所有后代；ScheduledCompletionSink 只属于顶层。
14. Scheduled Run 及其后代不创建 agent_session_state、不预留 Inbox slot、不发 signal/completion event。
15. 后台委托在模型可见参数不变，但 Scheduled origin 强制变为 foreground。
16. JobRun 是 Outcome/投递的唯一权威；关联 AgentRun 只提供编排审计，不能反向改写 JobRun。
17. 第一次投递解析出的 target_session_id、message ID 和目标 metadata 在所有重试中保持不变。
18. `never` 严格不通知，包括失败、拒绝、超时和 unknown；这些状态仍可通过 `cron_runs`、管理页面和 Health 查看。
19. 静默只来源于结构化 `ok` 与策略矩阵，不能来源于普通文本。
## 18. 错误处理矩阵 ## 18. 错误处理矩阵
| 场景 | Run status | Outcome | 投递内容来源 | | 场景 | Scheduled JobRun status | Outcome | 投递内容来源 |
|---|---|---|---| |---|---|---|---|
| Agent 提交 `ok` | completed | ok | Agent message | | Agent 提交 `ok` | completed | ok | Agent message |
| Agent 提交 `alert` | completed | alert | Agent message | | Agent 提交 `alert` | completed | alert | Agent message |
@ -865,7 +1020,7 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
| Provider 失败 | failed | failed | 安全归一化错误，不含响应正文 | | Provider 失败 | failed | failed | 安全归一化错误，不含响应正文 |
| 运行超时 | timed_out | failed | Scheduler 固定超时说明 | | 运行超时 | timed_out | failed | Scheduler 固定超时说明 |
| Gateway 优雅取消 | interrupted | failed | Scheduler 固定中断说明 | | Gateway 优雅取消 | interrupted | failed | Scheduler 固定中断说明 |
| 硬崩溃恢复 | unknown | unknown | Scheduler 固定 unknown 说明 | | 硬崩溃恢复 | unknown | unknown | Scheduler 固定 unknown 说明；关联 AgentRun 审计为 interrupted |
| Agent 定义不存在 | failed | failed | Agent ID 和修复建议 | | Agent 定义不存在 | failed | failed | Agent ID 和修复建议 |
| 渠道发送永久失败 | 原 run 不变 | 原 outcome | delivery_status=failed | | 渠道发送永久失败 | 原 run 不变 | 原 outcome | delivery_status=failed |
@ -875,14 +1030,15 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
本设计应在一个功能版本中完成，不能长期保留两套路径： 本设计应在一个功能版本中完成，不能长期保留两套路径：
1. 增加 v11 migration 和新 Storage 类型、测试； 1. 抽取 legacy→v10 normalizer，增加 fresh v11/v10→v11 migration、新 Storage 类型与迁移测试；
2. 增加 Scheduled completion sink、终结工具和 AgentLoop 终止语义； 2. 增加 `ExecutionOrigin::Scheduled`、Scheduled completion sink、终结工具和 AgentLoop 终止语义；
3. 增加 `AgentCoordinator::execute_scheduled()`； 3. 增加 `AgentCoordinator::execute_scheduled()`，落实同步委托、signal/inbox 禁止和顶层 AgentRun；
4. 改写 Scheduler claim、执行、恢复和 delivery drain； 4. 改写 Scheduler 为非阻塞 JoinSet 事件循环，实现 occurrence claim、跨表恢复、终态提交和持久化 delivery drain；
5. 切换 Cron tools/API/WebUI； 5. 实现固定 target_session_id、幂等历史插入和类型化 Bus 回执；
6. 更新内置维护任务和文档； 6. 切换 Cron tools（含 `cron_runs`）、管理 API、WebUI 与 Health；
7. 删除所有旧类型、旧函数和魔法字符串； 7. 更新内置维护任务和文档；
8. 运行全量验证后再提交，并按功能变化增加产品中段版本号一次。 8. 删除所有旧类型、旧函数、旧字段读取和魔法字符串；
9. 运行全量验证后再提交，并按功能变化增加产品中段版本号一次。
代码合并点只允许新路径。迁移代码可以先写，但最终提交中不允许 Scheduler 通过 feature flag 或 schema 判断走旧路径。 代码合并点只允许新路径。迁移代码可以先写，但最终提交中不允许 Scheduler 通过 feature flag 或 schema 判断走旧路径。
@ -895,25 +1051,40 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
- 终结工具之后的同批工具归约为 Cancelled； - 终结工具之后的同批工具归约为 Cancelled；
- 普通文本、所有 `NO_REPLY` 变体都不能产生 `ok`； - 普通文本、所有 `NO_REPLY` 变体都不能产生 `ok`；
- named/root Agent 解析和工具收窄； - named/root Agent 解析和工具收窄；
- Scheduled background delegate 自动前台化； - Scheduled origin 继承到多层子 Agent，而 completion sink 只在顶层；
- Scheduled background delegate 在任意嵌套深度自动前台化；
- Scheduled Agent/子 Agent 不注入 emit_signal、不预留 slot、不创建 agent_session_state；
- 两个并发 Scheduler 对同一到期 Job 只有一个 claim 成功； - 两个并发 Scheduler 对同一到期 Job 只有一个 claim 成功；
- At claim 后禁用； - At claim 后禁用；
- 新建/更新过去 At 被拒绝，过期 At 无法直接 enable；
- disable 不影响活动 Run，remove 在活动 Run/租约存在时返回 conflict；
- recurring claim 时推进到未来； - recurring claim 时推进到未来；
- 长 Run 不阻塞其他到期 Job 的 claim 或已完成 Run 的投递；
- lease owner 条件提交； - lease owner 条件提交；
- 迟到结果不能覆盖 unknown/timeout； - 迟到结果不能覆盖 unknown/timeout；
- status/outcome 非法组合被数据库约束拒绝；
- Scheduled 恢复在一个事务中完成 JobRun unknown、AgentRun interrupted、delivery 决策和租约清理；
- delivery claim、瞬态重试、永久失败和尝试上限； - delivery claim、瞬态重试、永久失败和尝试上限；
- ChannelError 到类型化 delivery receipt 的完整映射；
- `deliver_outbound` 只有收到 Channel 成功回执才返回 Delivered，入队成功不算送达； - `deliver_outbound` 只有收到 Channel 成功回执才返回 Delivered，入队成功不算送达；
- 回执保留 transient/permanent 分类且不泄露渠道敏感响应； - 回执保留 transient/permanent 分类且不泄露渠道敏感响应；
- 本地历史稳定 message ID 去重。 - 第一次投递后 target_session_id 固定，切换当前 dialog 不改变重试目标；
- 本地历史稳定 message ID 去重，重复调用不重复推进 Session metadata；
- `cron_runs` 摘要/单条详情、limit 边界、Job 所属校验和 read_only 声明。
### Migration 测试 ### Migration 测试
- 空数据库直接得到 v11； - 空数据库直接得到 v11；
- fresh v11 与迁移所得 v11 的 `sqlite_master` schema 等价；
- `user_version=0` 的历史数据库先规范化再升级，缺少旧 Scheduler 表时直接创建 v11 Scheduler；
- 真实 v10 fixture 保留所有 Job 和历史 Run； - 真实 v10 fixture 保留所有 Job 和历史 Run；
- `task/monitor` 列被物理删除； - `task/monitor` 列被物理删除；
- `direct` 全部转为 `always`，运行时无法解析 direct； - `direct` 全部转为 `always`，运行时无法解析 direct；
- 非空 model 计数被记录且列被删除； - 非空 model 计数被记录且列被删除；
- delete_after_run 列被删除； - delete_after_run 列被删除；
- `last_error` 只补入最近历史 Run diagnostic，v11 Job 表不保留该列；
- 旧 delivery_error 按“执行是否已有结果”分别映射 completed/failed，且 delivery 始终为 failed；
- 迁移不会生成非法的 status/outcome 组合；
- 历史 Run 不产生 pending delivery； - 历史 Run 不产生 pending delivery；
- 旧锁生成 unknown run 并按策略决定 pending/not_requested； - 旧锁生成 unknown run 并按策略决定 pending/not_requested；
- 内置维护 prompt 不含 `NO_REPLY`； - 内置维护 prompt 不含 `NO_REPLY`；
@ -928,12 +1099,16 @@ Scheduled Agent 构造迁移到 Agent/Coordinator 边界，SessionManager 不再
- `on_alert + ok` 静默但 JobRun 可查询； - `on_alert + ok` 静默但 JobRun 可查询；
- `on_alert + alert/failed/refused` 实际投递； - `on_alert + alert/failed/refused` 实际投递；
- `never` 在所有 Outcome 下均不投递； - `never` 在所有 Outcome 下均不投递；
- `never` 的完整结果可以通过 `cron_runs` 和管理 API 查询；
- Provider 普通文本完成被转为协议失败通知； - Provider 普通文本完成被转为协议失败通知；
- Gateway 在结果提交后、发送前重启，pending 在启动后恢复； - Gateway 在结果提交后、发送前重启，pending 在启动后恢复；
- Gateway 在发送后、ack 前重启，允许有标识的重复但不丢失； - Gateway 在发送后、ack 前重启，允许有标识的重复但不丢失；
- Gateway 在 Agent 执行中退出，恢复为 unknown 且不重跑同一 occurrence； - Gateway 在 Agent 执行中退出，恢复为 unknown 且不重跑同一 occurrence；
- 恢复后关联 AgentRun 为 interrupted，但任何消费者都不能用它覆盖 JobRun unknown；
- 命名 Agent 被删除后 Job 明确失败且 Gateway 仍能启动； - 命名 Agent 被删除后 Job 明确失败且 Gateway 仍能启动；
- Cron 内 foreground 子 Agent 汇总成功，background 参数不会产生 Inbox 事件。 - Cron 内 foreground 子 Agent 汇总成功，嵌套 background 参数不会产生 Inbox/agent_session_state；
- 投递失败后切换当前 dialog，重试仍写入并指向首次固定的 target_session_id；
- 一个执行到超时上限的 Job 不延迟其他 Job 和 pending delivery。
### 必跑验证 ### 必跑验证
@ -958,4 +1133,10 @@ git diff --check
7. `on_alert + ok` 不投递，其余矩阵行为与本设计一致。 7. `on_alert + ok` 不投递，其余矩阵行为与本设计一致。
8. 结果提交后崩溃不会丢失待投递消息。 8. 结果提交后崩溃不会丢失待投递消息。
9. 执行中崩溃产生 `unknown`，同一 occurrence 不自动重跑。 9. 执行中崩溃产生 `unknown`，同一 occurrence 不自动重跑。
10. v10 数据库首次启动自动迁移到 v11，失败时完整回滚；迁移后没有运行时兼容代码。 10. Scheduled 恢复原子地产生 JobRun unknown 和 AgentRun interrupted，JobRun 始终是投递权威。
11. Scheduled origin 贯穿所有后代，但 completion sink 只属于顶层；数据库中不产生合成 agent_session_state 或 Inbox event。
12. Scheduler 的长 Run 不阻塞其他到期 Job 或 pending delivery。
13. 过期 At 不能直接新建、更新或重新启用。
14. 每次投递重试复用固定 target_session_id 和 `scheduled:<job_run_id>` message ID，本地历史和 Session metadata 均不重复。
15. `cron_runs`、管理 API 和 WebUI 都能查询 `never` 等静默任务的结构化结果。
16. v10 数据库首次启动自动迁移到 v11，失败时完整回滚；fresh/migrated v11 schema 一致，迁移后没有运行时兼容代码。

8
resources/skills/about-picobot/assets/config.example.json
View File

@ -54,7 +54,13 @@
"gateway": { "gateway": {
"host": "127.0.0.1", "host": "127.0.0.1",
"port": 19876, "port": 19876,
"require_pairing": true "require_pairing": true,
"scheduler": {
"enabled": true,
"poll_interval_secs": 60,
"max_concurrent": 1,
"execution_timeout_secs": 900
}
}, },
"client": { "client": {
"gateway_url": "ws://127.0.0.1:19876/ws" "gateway_url": "ws://127.0.0.1:19876/ws"

2
resources/skills/about-picobot/references/architecture.md
View File

@ -10,7 +10,7 @@ Channel → MessageBus.inbound → Gateway processor → SessionManager → per-
AgentLoop → TurnEvent → TurnController → latest TurnSnapshot → DeliveryCoordinator → TurnSink → Channel AgentLoop → TurnEvent → TurnController → latest TurnSnapshot → DeliveryCoordinator → TurnSink → Channel
WebSocket/Channel → MessageBus.control → Gateway processor → SessionManager (dialog 操作) WebSocket/Channel → MessageBus.control → Gateway processor → SessionManager (dialog 操作)
Scheduler → SessionManager.handle_cron_message → AgentLoop → send_message Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Scheduled Agent → complete_scheduled_run → JobRun outbox → SessionManager/MessageBus
``` ```
## 模块职责 ## 模块职责

6
resources/skills/about-picobot/references/config.md
View File

@ -102,8 +102,8 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
|------|------|------|------| |------|------|------|------|
| `enabled` | bool | true | 是否启动调度器并注册 cron 工具 | | `enabled` | bool | true | 是否启动调度器并注册 cron 工具 |
| `poll_interval_secs` | int | 60 | 检查到期任务的轮询间隔 | | `poll_interval_secs` | int | 60 | 检查到期任务的轮询间隔 |
| `max_concurrent` | int | 1 | 每批到期任务的最大并发数，运行时限制在 1–256 | | `max_concurrent` | int | 1 | 同时执行的 Scheduled Run 上限，运行时限制在 1–256；投递使用独立有界并发 |
| `execution_timeout_secs` | int | 900 | 单个定时任务 Agent 执行的硬超时；租约会覆盖执行和托管投递等待 | | `execution_timeout_secs` | int | 900 | 单个定时任务 Agent 执行的硬超时；Job 执行租约额外覆盖关停宽限，投递由持久化 outbox 独立恢复 |
## memory 字段 ## memory 字段
@ -116,7 +116,7 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
| `timeline_retention_days` | int | 90 | 默认日常维护巡检删除超过该期限的 Timeline；Knowledge 不受影响 | | `timeline_retention_days` | int | 90 | 默认日常维护巡检删除超过该期限的 Timeline；Knowledge 不受影响 |
| `max_failures_before_degrade` | int | 3 | 预留的归并失败阈值；当前无失败降级循环 | | `max_failures_before_degrade` | int | 3 | 预留的归并失败阈值；当前无失败降级循环 |
注意：当前 worker 的 Knowledge 召回数量仍固定为 5；idle consolidation 和失败降级循环尚未接入。Timeline 清理由默认启用的 `picobot-routine-maintenance` 定时巡检执行。 注意：当前 worker 的 Knowledge 召回数量仍固定为 5；idle consolidation 和失败降级循环尚未接入。Timeline 清理由默认启用的 `picobot-routine-maintenance` Scheduled Run 执行；该任务使用 `never` 策略，结构化结果只进入运行审计和 Health。
## channels.feishu 字段 ## channels.feishu 字段

37
resources/skills/about-picobot/references/db-schema.md
View File

@ -2,7 +2,7 @@
数据库为 SQLite，默认位于配置目录（`~/.picobot`）`data/` 下的 `picobot.db`，与 workspace 相互独立。 数据库为 SQLite，默认位于配置目录（`~/.picobot`）`data/` 下的 `picobot.db`，与 workspace 相互独立。
连接启用 WAL、`synchronous=NORMAL`、foreign keys、5 秒 busy timeout，连接池最多 8 个连接。当前 `PRAGMA user_version=10`；启动时会在事务内补齐旧库字段和索引，遇到比程序更新的 schema version 会拒绝启动。 连接启用 WAL、`synchronous=NORMAL`、foreign keys、5 秒 busy timeout，连接池最多 8 个连接。当前 `PRAGMA user_version=11`；启动时会在单个事务内迁移旧库，遇到比程序更新的 schema version 会拒绝启动。
## sessions 表 ## sessions 表
@ -176,22 +176,21 @@ background 完成/信号投递的唯一事实源：`pending → leased → admit
| `name` | TEXT | 任务名称 | | `name` | TEXT | 任务名称 |
| `schedule` | TEXT | 调度规则 JSON（at/every/cron） | | `schedule` | TEXT | 调度规则 JSON（at/every/cron） |
| `prompt` | TEXT | 任务提示词 | | `prompt` | TEXT | 任务提示词 |
| `channel` | TEXT | 执行渠道 | | `agent_id` | TEXT | 可选命名 Agent；NULL 表示 Root |
| `channel` | TEXT | 目标渠道 |
| `chat_id` | TEXT | 目标对话 | | `chat_id` | TEXT | 目标对话 |
| `model` | TEXT | 可选模型标记；当前会存储/展示，但 Scheduler 执行仍使用默认 Agent 模型 | | `delivery_policy` | TEXT | `always` / `on_alert` / `never` |
| `enabled` | INTEGER | 是否启用 (1/0) | | `enabled` | INTEGER | 是否启用 (1/0) |
| `delete_after_run` | INTEGER | 执行后自动删除 (1/0) |
| `next_run_at` | INTEGER | 下次执行时间 | | `next_run_at` | INTEGER | 下次执行时间 |
| `last_run_at` | INTEGER | 上次执行时间 | | `last_run_at` | INTEGER | 上次执行时间 |
| `last_status` | TEXT | 上次执行状态 | | `last_outcome` | TEXT | 最近结构化结果：ok/alert/failed/refused/unknown |
| `last_error` | TEXT | 上次错误信息 |
| `locked_at` | INTEGER | 本次领取时间 | | `locked_at` | INTEGER | 本次领取时间 |
| `lock_owner` | TEXT | 领取任务的 Scheduler owner UUID | | `lock_owner` | TEXT | 本次 occurrence 的唯一 owner token |
| `lease_until` | INTEGER | 租约到期时间；进程崩溃后允许其他实例重新领取 | | `lease_until` | INTEGER | 租约到期时间 |
| `created_at` | INTEGER | 创建时间（Unix 毫秒） | | `created_at` | INTEGER | 创建时间（Unix 毫秒） |
| `updated_at` | INTEGER | 更新时间（Unix 毫秒） | | `updated_at` | INTEGER | 更新时间（Unix 毫秒） |
Scheduler 使用原子 `UPDATE ... RETURNING` 领取到期任务。任务结果、下次运行时间和租约释放在同一事务中提交，并校验 owner，防止过期 worker 覆盖已恢复的任务。 Scheduler 在领取事务中插入 JobRun、快照执行/投递字段并推进下次时间。`At` 在领取时立即禁用；执行失败或崩溃不重放同一个 occurrence。
## job_runs 表 ## job_runs 表
@ -199,12 +198,26 @@ Scheduler 使用原子 `UPDATE ... RETURNING` 领取到期任务。任务结果
|------|------|------| |------|------|------|
| `id` | INTEGER PK | 自增 ID | | `id` | INTEGER PK | 自增 ID |
| `job_id` | TEXT FK | 关联任务，外键关联 scheduled_jobs(id) | | `job_id` | TEXT FK | 关联任务，外键关联 scheduled_jobs(id) |
| `scheduled_for` | INTEGER | 本 occurrence 原计划时间 |
| `agent_run_id` | TEXT FK | 顶层 AgentRun 审计记录 |
| `agent_id` | TEXT | claim-time Agent 快照 |
| `delivery_policy` | TEXT | claim-time 投递策略快照 |
| `target_channel` / `target_chat_id` | TEXT | claim-time 目标快照 |
| `target_session_id` | TEXT | 首次投递时固定的目标 dialog |
| `started_at` | INTEGER | 开始时间 | | `started_at` | INTEGER | 开始时间 |
| `finished_at` | INTEGER | 结束时间 | | `finished_at` | INTEGER | 结束时间 |
| `status` | TEXT | 执行状态 | | `status` | TEXT | claimed/running/completed/failed/timed_out/cancelled/interrupted/unknown |
| `output` | TEXT | 执行输出 | | `outcome` | TEXT | ok/alert/failed/refused/unknown；与 status 有联合约束 |
| `error` | TEXT | 错误信息 | | `message` | TEXT | 面向用户的结构化结果 |
| `diagnostic` | TEXT | 有界内部诊断 |
| `duration_ms` | INTEGER | 耗时（毫秒） | | `duration_ms` | INTEGER | 耗时（毫秒） |
| `delivery_status` | TEXT | awaiting_result/not_requested/suppressed/pending/delivering/delivered/failed |
| `delivery_attempts` | INTEGER | 持久化投递尝试次数，最多 3 次 |
| `delivery_next_attempt_at` | INTEGER | 瞬态失败后的退避时间 |
| `delivery_lease_owner` / `delivery_lease_until` | TEXT / INTEGER | outbox 领取租约 |
| `delivery_error` | TEXT | 清洗后的投递失败摘要 |
JobRun 是执行结果和投递状态的唯一权威。顶层 AgentRun 与 JobRun 终态、Job 最近摘要和租约释放原子提交；启动恢复将遗留运行归为 `unknown`，不会自动重跑。
## llm_calls 表 ## llm_calls 表

13
resources/skills/about-picobot/references/tools.md
View File

@ -50,14 +50,15 @@
## Cron 定时任务工具 ## Cron 定时任务工具
Cron 不是一个带 `action` 的统一工具，而是六个独立工具；仅在 `gateway.scheduler.enabled=true` 时注册。 Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅在 `gateway.scheduler.enabled=true` 时注册。
| 工具 | 主要参数 | 说明 | | 工具 | 主要参数 | 说明 |
|------|----------|------| |------|----------|------|
| `cron_add` | `schedule`, `prompt`, `channel`, `chat_id`; 可选 `name`, `model` | 创建任务 | | `cron_add` | `schedule`, `prompt`, `channel`, `chat_id`; 可选 `name`, `agent_id`, `delivery_policy` | 创建任务 |
| `cron_list` | 可选 `status=all|enabled|disabled` | 列出任务 | | `cron_list` | 可选 `status=all|enabled|disabled` | 列出任务 |
| `cron_update` | `job_id`; 可选 `prompt`, `schedule`, `channel`, `chat_id`, `model` | 更新指定字段 | | `cron_runs` | `job_id`; 可选 `run_id`, `limit` | 查询结构化运行和投递记录，包括静默结果 |
| `cron_remove` | `job_id` | 永久删除任务和关联 job runs | | `cron_update` | `job_id`; 可选 `name`, `prompt`, `schedule`, `channel`, `chat_id`, `agent_id`, `delivery_policy` | 更新指定字段；`agent_id:null` 切回 Root |
| `cron_remove` | `job_id` | 无活动 Run 或 pending delivery 时永久删除任务 |
| `cron_enable` | `job_id` | 启用并重新计算下次运行时间 | | `cron_enable` | `job_id` | 启用并重新计算下次运行时间 |
| `cron_disable` | `job_id` | 禁用但保留任务 | | `cron_disable` | `job_id` | 禁用但保留任务 |
@ -69,7 +70,9 @@ Cron 不是一个带 `action` 的统一工具，而是六个独立工具；仅
{"type":"cron","expr":"0 0 9 * * *","tz":"Asia/Shanghai"} {"type":"cron","expr":"0 0 9 * * *","tz":"Asia/Shanghai"}
``` ```
时间戳和间隔单位为毫秒；Cron 表达式为 6 段（秒、分、时、日、月、周）。定时 Agent 不复用聊天历史，`prompt` 必须包含完整上下文。`kind` 可为 `task` 或 `monitor`；`delivery_policy` 可为 `always`、`on_alert` 或 `never`。托管任务由 Scheduler 投递，巡检返回 `NO_REPLY[INFO]` 时静默，`NO_REPLY[FAIL]`/`NO_REPLY[REFUSE]` 仍视为需关注结果。升级前创建的任务保留 Agent 直接调用 `send_message` 的兼容行为。`model` 当前会持久化和展示，但执行仍使用默认 Agent Provider/Model，不能依赖它实现模型覆盖。 时间戳和间隔单位为毫秒；Cron 表达式为 6 段（秒、分、时、日、月、周）。过去时间的 At 不能创建、更新或直接重新启用。定时 Agent 不复用聊天历史，`prompt` 必须包含完整上下文；`agent_id` 省略时使用 Root，否则使用当前 AgentCatalog 中的命名 Agent。
每次运行必须恰好一次调用 `complete_scheduled_run(outcome,message)`，outcome 只能是 `ok`、`alert`、`failed`、`refused`。普通最终文本不会被解释为结果，缺少结构化终结会 fail-closed。投递完全由 Scheduler 决定：`always` 投递所有结果，`on_alert` 只抑制 `ok`，`never` 只保留记录。Scheduled Agent 不能自行发送最终通知；子 Agent 委托会同步完成，也不会产生后台 Inbox/Signal。
--- ---

6
resources/templates/config.example.json
View File

@ -78,6 +78,12 @@
"max_files_per_message": 8, "max_files_per_message": 8,
"max_message_bytes": 67108864, "max_message_bytes": 67108864,
"pending_ttl_seconds": 3600 "pending_ttl_seconds": 3600
},
"scheduler": {
"enabled": true,
"poll_interval_secs": 60,
"max_concurrent": 1,
"execution_timeout_secs": 900
} }
}, },
"client": { "client": {

135
src/agent/agent_loop.rs
View File

@ -1240,6 +1240,31 @@ impl AgentLoop {
} }
completed_tool_batches = completed_tool_batches.saturating_add(1); completed_tool_batches = completed_tool_batches.saturating_add(1);
if let Some(outcome) = tool_context
.scheduled_completion
.as_ref()
.and_then(|sink| sink.outcome())
{
Self::close_steering(turn.as_ref());
let mut final_message = ChatMessage::assistant(outcome.message);
attach_reply_media(&mut final_message, &reply_media_refs);
Self::annotate_message(&mut final_message, turn.as_ref(), iteration, true);
emitted_messages.push(final_message.clone());
self.forward_to_transcript_sink(&final_message);
crate::observability::metrics::global_metrics().record_turn(
Some(&accumulated_usage),
turn_start.elapsed().as_millis() as u64,
);
return Ok(AgentProcessResult {
final_response: final_message,
emitted_messages,
total_tokens: Some(accumulated_tokens),
usage: Some(accumulated_usage),
last_request_usage,
last_request_digest,
});
}
// A complete tool batch is the first safe steering boundary. Do // A complete tool batch is the first safe steering boundary. Do
// not drain at the final available iteration: those inputs must // not drain at the final available iteration: those inputs must
// remain in the closed mailbox for Session to queue after this // remain in the closed mailbox for Session to queue after this
@ -1517,6 +1542,17 @@ impl AgentLoop {
let mut outcomes = Vec::with_capacity(tool_calls.len()); let mut outcomes = Vec::with_capacity(tool_calls.len());
for tool_call in tool_calls { for tool_call in tool_calls {
if context
.scheduled_completion
.as_ref()
.is_some_and(|sink| sink.is_completed())
{
outcomes.push(ToolExecutionOutcome::failure(
"Cancelled: scheduled run was already completed".to_string(),
Some("scheduled run was already completed".to_string()),
));
continue;
}
if context.cancellation.is_cancelled() { if context.cancellation.is_cancelled() {
return Err(AgentError::Cancelled); return Err(AgentError::Cancelled);
} }
@ -1666,6 +1702,58 @@ mod tests {
requests: std::sync::atomic::AtomicUsize, requests: std::sync::atomic::AtomicUsize,
} }
struct ScheduledCompletionProvider {
requests: std::sync::atomic::AtomicUsize,
}
#[async_trait::async_trait]
impl LLMProvider for ScheduledCompletionProvider {
async fn stream(
&self,
_request: ChatCompletionRequest,
) -> Result<ProviderStream, crate::providers::DynProviderError> {
self.requests
.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Ok(crate::providers::provider_stream_for_test(
ChatCompletionResponse {
id: "scheduled-complete".to_string(),
model: "scheduled-complete".to_string(),
content: String::new(),
reasoning_content: None,
provider_state: None,
tool_calls: vec![
ToolCall {
id: "complete".to_string(),
name: "complete_scheduled_run".to_string(),
arguments: serde_json::json!({
"outcome": "ok",
"message": "healthy"
}),
},
ToolCall {
id: "late-side-effect".to_string(),
name: "side_effect".to_string(),
arguments: serde_json::json!({}),
},
],
usage: Usage::default(),
},
))
}
fn ptype(&self) -> &str {
"test"
}
fn name(&self) -> &str {
"scheduled-complete"
}
fn model_id(&self) -> &str {
"scheduled-complete"
}
}
#[async_trait::async_trait] #[async_trait::async_trait]
impl LLMProvider for AlwaysOverflowProvider { impl LLMProvider for AlwaysOverflowProvider {
async fn stream( async fn stream(
@ -1725,6 +1813,53 @@ mod tests {
} }
} }
#[tokio::test]
async fn scheduled_completion_ends_the_loop_and_cancels_later_batch_calls() {
let provider = Arc::new(ScheduledCompletionProvider {
requests: std::sync::atomic::AtomicUsize::new(0),
});
let executions = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let tools = Arc::new(ToolRegistry::new());
tools.register(crate::tools::CompleteScheduledRunTool::new());
tools.register(CountingSideEffectTool {
executions: executions.clone(),
});
let agent = AgentLoop::with_provider_and_tools(
provider.clone(),
tools,
3,
"scheduled-complete".to_string(),
PathBuf::from("."),
Vec::new(),
);
let sink = Arc::new(crate::tools::ScheduledCompletionSink::default());
let context = ToolExecutionContext::for_session("scheduled-run:1")
.with_execution_origin(crate::tools::ExecutionOrigin::Scheduled { job_run_id: 1 })
.with_scheduled_completion(sink.clone());
let result = agent
.process_with_context(vec![ChatMessage::user("check")], context)
.await
.unwrap();
assert_eq!(result.final_response.content, "healthy");
assert_eq!(
sink.outcome().unwrap().kind,
crate::storage::ScheduledOutcomeKind::Ok
);
assert_eq!(executions.load(std::sync::atomic::Ordering::SeqCst), 0);
assert_eq!(
provider.requests.load(std::sync::atomic::Ordering::SeqCst),
1
);
assert!(result.emitted_messages.iter().any(|message| {
message.role == "tool"
&& message
.content
.contains("scheduled run was already completed")
}));
}
#[async_trait::async_trait] #[async_trait::async_trait]
impl LLMProvider for OverflowAfterToolProvider { impl LLMProvider for OverflowAfterToolProvider {
async fn stream( async fn stream(

272
src/agent/coordinator.rs
View File

@ -32,6 +32,16 @@ pub struct BackgroundAdmission {
pub run_ids: Vec<String>, pub run_ids: Vec<String>,
} }
#[derive(Debug, Clone)]
pub struct ScheduledAgentExecution {
pub agent_run_id: String,
pub status: crate::storage::ScheduledRunStatus,
pub outcome: Option<crate::tools::ScheduledOutcome>,
pub error: Option<String>,
pub agent_terminal: AgentTerminalOutcome,
pub runtime_generation: i64,
}
pub struct AgentCoordinator { pub struct AgentCoordinator {
storage: Arc<Storage>, storage: Arc<Storage>,
manager: Arc<SubAgentManager>, manager: Arc<SubAgentManager>,
@ -88,6 +98,263 @@ impl AgentCoordinator {
}) })
} }
#[allow(clippy::too_many_arguments)]
pub async fn execute_scheduled(
self: &Arc<Self>,
job_run_id: i64,
lease_owner: &str,
job_id: &str,
job_name: &str,
agent_id: Option<&str>,
prompt: &str,
timeout_secs: u64,
) -> Result<ScheduledAgentExecution, CoordinatorError> {
let run_id = Uuid::new_v4().to_string();
let root_session_id = format!("scheduled-run:{job_run_id}");
let sink = Arc::new(crate::tools::ScheduledCompletionSink::default());
let caller = ToolExecutionContext::for_session(root_session_id.clone())
.with_turn_id(format!("scheduled:{job_run_id}"))
.with_execution_origin(crate::tools::ExecutionOrigin::Scheduled { job_run_id });
let contract = format!(
"## Unattended Scheduled Run\n\nYou are executing scheduled task “{job_name}” ({job_id}). The user will not see ordinary final text. After completing all necessary work, you must call complete_scheduled_run exactly once. Use ok only when the task completed and found nothing requiring attention; use alert for actionable findings; use failed when the task did not complete reliably; use refused for a permission or safety refusal. Legacy textual suppression and direct-messaging instructions are obsolete."
);
let config = SubAgentConfig {
target: agent_id.map(str::to_string),
prompt: prompt.to_string(),
context: Some(contract.clone()),
mode: ExecutionMode::Foreground,
allowed_tools: None,
max_iterations: None,
timeout_secs: Some(timeout_secs),
plan_item_id: None,
session_id: Some(root_session_id.clone()),
};
let mut resolution = if agent_id.is_some() {
let mut resolution = self.manager.resolve_agent(&config, &caller, &run_id)?;
resolution
.tools
.register(crate::tools::CompleteScheduledRunTool::new());
resolution.tool_context.session_id = Some(root_session_id.clone());
resolution.tool_context.scheduled_completion = Some(sink.clone());
resolution.timeout_secs = resolution.timeout_secs.min(timeout_secs);
resolution.signal_contract = None;
resolution
} else {
self.manager
.resolve_scheduled_root(&caller, &run_id, timeout_secs, sink.clone())?
};
resolution.tool_context.execution_origin =
crate::tools::ExecutionOrigin::Scheduled { job_run_id };
let now = chrono::Utc::now().timestamp_millis();
let deadline_at = now.saturating_add((resolution.timeout_secs * 1000) as i64);
let new_run = NewAgentRun {
id: run_id.clone(),
root_session_id,
root_turn_id: None,
parent_run_id: None,
caller_agent_id: "SCHEDULER".to_string(),
caller_scope_id: format!("scheduled:{job_id}"),
idempotency_key: Some(format!("scheduled:{job_run_id}")),
agent_id: agent_id.unwrap_or("ROOT").to_string(),
definition_hash: resolution.definition_hash.clone().unwrap_or_default(),
provider_profile: resolution.llm_profile.clone().unwrap_or_default(),
provider_name: resolution.provider_config.name.clone(),
model_id: resolution.provider_config.model_id.clone(),
mode: AgentRunMode::Foreground,
depth: 1,
plan_item_id: None,
execution_id: run_id.clone(),
task: prompt.to_string(),
context_json: None,
budget_json: serde_json::json!({
"remaining_runs": self.manager.catalog().max_runs_per_tree().saturating_sub(1),
"remaining_depth": self.manager.catalog().max_tree_depth().saturating_sub(1),
})
.to_string(),
signal_contract_json: None,
signal_delivery: None,
deadline_at,
runtime_generation: self.runtime_generation,
completion_slot_reserved: false,
};
match self
.storage
.accept_agent_runs(AcceptAgentRequest {
runs: vec![new_run],
now,
})
.await?
{
AcceptedAgentRuns::Accepted { .. } => {}
AcceptedAgentRuns::Existing { .. } => {
return Err(CoordinatorError::Rejected(format!(
"scheduled occurrence {job_run_id} already has an Agent run"
)));
}
}
let mut mark_attempt = 0_u64;
let marked = match loop {
match self
.storage
.mark_scheduled_run_running(job_run_id, lease_owner, Some(&run_id), now)
.await
{
Err(error) if error.is_transient() && mark_attempt < 2 => {
mark_attempt += 1;
tokio::time::sleep(std::time::Duration::from_millis(50 * mark_attempt)).await;
}
result => break result,
}
} {
Ok(marked) => marked,
Err(error) => {
let _ = self
.storage
.cancel_agent_run_with_completion(
&run_id,
"scheduled occurrence could not enter running state",
true,
now,
)
.await;
return Err(error.into());
}
};
if !marked {
let _ = self
.storage
.cancel_agent_run_with_completion(
&run_id,
"scheduled occurrence was no longer active",
true,
now,
)
.await;
return Err(CoordinatorError::Rejected(format!(
"scheduled occurrence {job_run_id} lost its lease"
)));
}
let (execution, agent_terminal) = match self
.execute_scheduled_agent_run(&run_id, &config, resolution)
.await
{
Ok(execution) => execution,
Err(error) => (
Err(error),
AgentTerminalOutcome::Failed {
error: "scheduled Agent could not enter its execution lifecycle".to_string(),
prompt_tokens: None,
completion_tokens: None,
cost: None,
signal_ids: Vec::new(),
},
),
};
let status = match &execution {
Ok(result) => match &result.status {
TaskStatus::Completed => crate::storage::ScheduledRunStatus::Completed,
TaskStatus::Failed(_) => crate::storage::ScheduledRunStatus::Failed,
TaskStatus::TimedOut => crate::storage::ScheduledRunStatus::TimedOut,
TaskStatus::Cancelled => crate::storage::ScheduledRunStatus::Interrupted,
},
Err(_) => crate::storage::ScheduledRunStatus::Failed,
};
let error = match &execution {
Ok(result) => match &result.status {
TaskStatus::Completed => None,
TaskStatus::Failed(_) => Some(
"scheduled Agent execution failed; inspect Gateway logs for details"
.to_string(),
),
TaskStatus::TimedOut => Some("scheduled Agent timed out".to_string()),
TaskStatus::Cancelled => Some("scheduled Agent was cancelled".to_string()),
},
Err(_) => Some(
"scheduled Agent execution failed; inspect Gateway logs for details".to_string(),
),
};
Ok(ScheduledAgentExecution {
agent_run_id: run_id,
status,
outcome: sink.outcome(),
error,
agent_terminal,
runtime_generation: self.runtime_generation,
})
}
async fn execute_scheduled_agent_run(
self: &Arc<Self>,
run_id: &str,
config: &SubAgentConfig,
resolution: ResolvedAgentRun,
) -> Result<
(
Result<SubAgentResult, CoordinatorError>,
AgentTerminalOutcome,
),
CoordinatorError,
> {
let execution_id = run_id.to_string();
let token = resolution.tool_context.cancellation.clone();
self.active_tokens.insert(run_id.to_string(), token);
let started = self
.storage
.mark_agent_run_running(run_id, &execution_id, chrono::Utc::now().timestamp_millis())
.await?;
if !started {
self.active_tokens.remove(run_id);
return Err(CoordinatorError::Rejected(format!(
"scheduled Agent run {run_id} was closed before execution started"
)));
}
let result = self
.manager
.execute_resolved(config, resolution, run_id)
.await
.map_err(CoordinatorError::SubAgent);
self.active_tokens.remove(run_id);
let terminal = match &result {
Ok(result) => match &result.status {
TaskStatus::Completed => AgentTerminalOutcome::Completed {
result: result.full_content.clone(),
prompt_tokens: None,
completion_tokens: None,
cost: None,
tool_calls: result.tool_calls_count as i64,
iterations: result.iterations as i64,
signal_ids: Vec::new(),
},
TaskStatus::Failed(error) => AgentTerminalOutcome::Failed {
error: error.clone(),
prompt_tokens: None,
completion_tokens: None,
cost: None,
signal_ids: Vec::new(),
},
TaskStatus::TimedOut => AgentTerminalOutcome::TimedOut {
deadline_at: chrono::Utc::now().timestamp_millis(),
signal_ids: Vec::new(),
},
TaskStatus::Cancelled => AgentTerminalOutcome::Interrupted {
reason: "scheduled Agent interrupted by shutdown".to_string(),
signal_ids: Vec::new(),
},
},
Err(error) => AgentTerminalOutcome::Failed {
error: error.to_string(),
prompt_tokens: None,
completion_tokens: None,
cost: None,
signal_ids: Vec::new(),
},
};
Ok((result, terminal))
}
/// Admit a named background run for the root caller and spawn its runner. /// Admit a named background run for the root caller and spawn its runner.
/// Completion is guaranteed by the reserved inbox slot; the returned ID /// Completion is guaranteed by the reserved inbox slot; the returned ID
/// is only valid when every durable step succeeded. /// is only valid when every durable step succeeded.
@ -100,6 +367,11 @@ impl AgentCoordinator {
caller: &ToolExecutionContext, caller: &ToolExecutionContext,
configs: Vec<SubAgentConfig>, configs: Vec<SubAgentConfig>,
) -> Result<BackgroundAdmission, CoordinatorError> { ) -> Result<BackgroundAdmission, CoordinatorError> {
if caller.execution_origin.is_scheduled() {
return Err(CoordinatorError::Rejected(
"scheduled Agents cannot create background runs".to_string(),
));
}
if caller.agent.is_some() { if caller.agent.is_some() {
return Err(CoordinatorError::Rejected( return Err(CoordinatorError::Rejected(
"nested background runs are not available yet; only the root Agent may delegate background work".to_string(), "nested background runs are not available yet; only the root Agent may delegate background work".to_string(),

2
src/agent/mod.rs
View File

@ -21,7 +21,7 @@ pub use context_compaction::{
ContextRequestKey, ContextUsageTracker, PreviousCheckpoint, SequencedMessage, ContextRequestKey, ContextUsageTracker, PreviousCheckpoint, SequencedMessage,
context_request_digest, estimate_tokens, context_request_digest, estimate_tokens,
}; };
pub use coordinator::{AgentCoordinator, CoordinatorError}; pub use coordinator::{AgentCoordinator, CoordinatorError, ScheduledAgentExecution};
pub use definition::{AgentDefinition, AgentLimits}; pub use definition::{AgentDefinition, AgentLimits};
pub use gate::ExecutionGate; pub use gate::ExecutionGate;
pub use inbox::{AgentInboxNotifier, AgentInboxWakeTarget}; pub use inbox::{AgentInboxNotifier, AgentInboxWakeTarget};

170
src/agent/sub_agent.rs
View File

@ -10,6 +10,22 @@ use crate::providers::{LLMProvider, create_provider};
use crate::skills::SkillsLoader; use crate::skills::SkillsLoader;
use crate::tools::{ToolExecutionContext, ToolRegistry}; use crate::tools::{ToolExecutionContext, ToolRegistry};
const SCHEDULED_DISABLED_TOOLS: &[&str] = &[
"send_message",
"cron_add",
"cron_update",
"cron_remove",
"cron_enable",
"cron_disable",
"cron_list",
"cron_runs",
"reload_config",
"agent_task",
"chat_manager",
"todo",
"emit_signal",
];
const DEFAULT_MAX_ITERATIONS: usize = 99; const DEFAULT_MAX_ITERATIONS: usize = 99;
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@ -191,7 +207,13 @@ impl SubAgentManager {
})?; })?;
let cancellation = caller.cancellation.child_token(); let cancellation = caller.cancellation.child_token();
let execution = if let Some(parent) = caller.agent.as_ref() { let execution = if let Some(parent) = caller.agent.as_ref() {
if !self.catalog.can_delegate(&parent.current_agent_id, target) { let allowed =
if caller.execution_origin.is_scheduled() && parent.current_agent_id == "ROOT" {
self.catalog.root_can_delegate(target)
} else {
self.catalog.can_delegate(&parent.current_agent_id, target)
};
if !allowed {
return Err(SubAgentError::Other(format!( return Err(SubAgentError::Other(format!(
"Agent '{}' is not allowed to delegate to '{target}'", "Agent '{}' is not allowed to delegate to '{target}'",
parent.current_agent_id parent.current_agent_id
@ -235,10 +257,14 @@ impl SubAgentManager {
.budget .budget
.remaining_depth .remaining_depth
.min(definition.limits.max_depth); .min(definition.limits.max_depth);
child.signal_contract = definition child.signal_contract = if caller.execution_origin.is_scheduled() {
.signal_contract None
.as_ref() } else {
.map(|contract| Arc::new(contract.clone())); definition
.signal_contract
.as_ref()
.map(|contract| Arc::new(contract.clone()))
};
Arc::new(child) Arc::new(child)
} else { } else {
if !self.catalog.root_can_delegate(target) { if !self.catalog.root_can_delegate(target) {
@ -252,7 +278,11 @@ impl SubAgentManager {
run_id: task_id.to_string(), run_id: task_id.to_string(),
execution_id: task_id.to_string(), execution_id: task_id.to_string(),
parent_run_id: None, parent_run_id: None,
caller_agent_id: "ROOT".to_string(), caller_agent_id: if caller.execution_origin.is_scheduled() {
"SCHEDULER".to_string()
} else {
"ROOT".to_string()
},
current_agent_id: target.to_string(), current_agent_id: target.to_string(),
ancestry: vec![target.to_string()], ancestry: vec![target.to_string()],
depth: 1, depth: 1,
@ -267,10 +297,14 @@ impl SubAgentManager {
.min(definition.limits.max_depth), .min(definition.limits.max_depth),
}, },
tree_runs: Arc::new(std::sync::atomic::AtomicUsize::new(1)), tree_runs: Arc::new(std::sync::atomic::AtomicUsize::new(1)),
signal_contract: definition signal_contract: if caller.execution_origin.is_scheduled() {
.signal_contract None
.as_ref() } else {
.map(|contract| Arc::new(contract.clone())), definition
.signal_contract
.as_ref()
.map(|contract| Arc::new(contract.clone()))
},
emitted_signals: Arc::new(std::sync::Mutex::new(Vec::new())), emitted_signals: Arc::new(std::sync::Mutex::new(Vec::new())),
}) })
}; };
@ -279,6 +313,9 @@ impl SubAgentManager {
if let Some(allowed) = config.allowed_tools.as_ref() { if let Some(allowed) = config.allowed_tools.as_ref() {
effective_names.retain(|name| allowed.iter().any(|allowed| allowed == name)); effective_names.retain(|name| allowed.iter().any(|allowed| allowed == name));
} }
if caller.execution_origin.is_scheduled() {
effective_names.retain(|name| !SCHEDULED_DISABLED_TOOLS.contains(&name.as_str()));
}
let has_get_skill = effective_names.iter().any(|name| name == "get_skill"); let has_get_skill = effective_names.iter().any(|name| name == "get_skill");
let mut names = effective_names; let mut names = effective_names;
names.retain(|name| name != "get_skill"); names.retain(|name| name != "get_skill");
@ -310,7 +347,7 @@ impl SubAgentManager {
// The signal tool is contract-bound: it exists only when the // The signal tool is contract-bound: it exists only when the
// definition declares a signal block and the durable Coordinator is // definition declares a signal block and the durable Coordinator is
// live. If either is missing the run cannot emit signals. // live. If either is missing the run cannot emit signals.
if definition.signal_contract.is_some() { if definition.signal_contract.is_some() && !caller.execution_origin.is_scheduled() {
match self.coordinator() { match self.coordinator() {
Some(coordinator) => { Some(coordinator) => {
let contract = definition.signal_contract.clone().unwrap(); let contract = definition.signal_contract.clone().unwrap();
@ -343,17 +380,91 @@ impl SubAgentManager {
agent_id: Some(target.to_string()), agent_id: Some(target.to_string()),
definition_hash: Some(definition.definition_hash.clone()), definition_hash: Some(definition.definition_hash.clone()),
llm_profile: definition.llm_profile.clone(), llm_profile: definition.llm_profile.clone(),
signal_contract: definition.signal_contract.clone(), signal_contract: (!caller.execution_origin.is_scheduled())
tool_context: ToolExecutionContext::for_session(format!("agent-run:{task_id}")) .then(|| definition.signal_contract.clone())
.with_turn_id( .flatten(),
caller tool_context: ToolExecutionContext::for_session(
.turn_id if caller.execution_origin.is_scheduled() {
.clone() root_session_id
.unwrap_or_else(|| task_id.to_string()), } else {
) format!("agent-run:{task_id}")
},
)
.with_turn_id(
caller
.turn_id
.clone()
.unwrap_or_else(|| task_id.to_string()),
)
.with_agent(execution)
.with_cancellation(cancellation)
.with_execution_gate(self.execution_gate.clone())
.with_execution_origin(caller.execution_origin),
})
}
pub(crate) fn resolve_scheduled_root(
&self,
caller: &ToolExecutionContext,
task_id: &str,
timeout_secs: u64,
sink: Arc<crate::tools::ScheduledCompletionSink>,
) -> Result<ResolvedAgentRun, SubAgentError> {
if !caller.execution_origin.is_scheduled() {
return Err(SubAgentError::Other(
"scheduled root resolution requires Scheduled execution origin".to_string(),
));
}
let root_session_id = caller.session_id.clone().ok_or_else(|| {
SubAgentError::Other("scheduled root requires an execution scope".to_string())
})?;
let run_id = task_id.to_string();
let cancellation = caller.cancellation.child_token();
let execution = Arc::new(crate::agent::AgentExecutionContext {
root_session_id: root_session_id.clone(),
root_turn_id: None,
run_id: run_id.clone(),
execution_id: run_id,
parent_run_id: None,
caller_agent_id: "SCHEDULER".to_string(),
current_agent_id: "ROOT".to_string(),
ancestry: vec!["ROOT".to_string()],
depth: 1,
plan_item_id: None,
cancellation: cancellation.clone(),
budget: crate::agent::AgentBudget {
remaining_runs: self.catalog.max_runs_per_tree().saturating_sub(1),
remaining_depth: self.catalog.max_tree_depth().saturating_sub(1),
},
tree_runs: Arc::new(std::sync::atomic::AtomicUsize::new(1)),
signal_contract: None,
emitted_signals: Arc::new(std::sync::Mutex::new(Vec::new())),
});
let tools = self.full_tools.without(SCHEDULED_DISABLED_TOOLS);
tools.register(crate::tools::CompleteScheduledRunTool::new());
let skills_prompt = self
.skills_loader
.as_ref()
.map(|loader| loader.build_skills_prompt())
.filter(|prompt| !prompt.is_empty());
Ok(ResolvedAgentRun {
provider_config: Arc::new(self.provider_config.clone()),
tools,
timeout_secs,
max_iterations: self.provider_config.max_tool_iterations,
max_result_chars: 16_384,
role_prompt: None,
skills_prompt,
tool_context: ToolExecutionContext::for_session(root_session_id)
.with_agent(execution) .with_agent(execution)
.with_cancellation(cancellation) .with_cancellation(cancellation)
.with_execution_gate(self.execution_gate.clone()), .with_execution_gate(self.execution_gate.clone())
.with_execution_origin(caller.execution_origin)
.with_scheduled_completion(sink),
agent_id: None,
definition_hash: None,
llm_profile: None,
signal_contract: None,
}) })
} }
@ -664,6 +775,25 @@ mod tests {
assert!(!crate::tools::Tool::runtime_injected(&reload)); assert!(!crate::tools::Tool::runtime_injected(&reload));
} }
#[test]
fn scheduled_runs_remove_direct_and_interactive_control_tools() {
for name in [
"send_message",
"cron_add",
"cron_runs",
"reload_config",
"agent_task",
"chat_manager",
"todo",
"emit_signal",
] {
assert!(SCHEDULED_DISABLED_TOOLS.contains(&name));
}
assert!(!SCHEDULED_DISABLED_TOOLS.contains(&"bash"));
assert!(!SCHEDULED_DISABLED_TOOLS.contains(&"delegate"));
assert!(!SCHEDULED_DISABLED_TOOLS.contains(&"complete_scheduled_run"));
}
#[test] #[test]
fn resolve_agent_rejects_missing_target() { fn resolve_agent_rejects_missing_target() {
let manager = manager(); let manager = manager();

112
src/bus/dispatcher.rs
View File

@ -5,7 +5,7 @@ use std::time::Duration;
use tokio::sync::mpsc; use tokio::sync::mpsc;
use crate::bus::{MessageBus, OutboundMessage}; use crate::bus::{DeliveryReceipt, MessageBus, OutboundMessage};
use crate::channels::ChannelManager; use crate::channels::ChannelManager;
use crate::channels::base::{Channel, ChannelError}; use crate::channels::base::{Channel, ChannelError};
use crate::delivery::ConversationWriteLocks; use crate::delivery::ConversationWriteLocks;
@ -64,12 +64,14 @@ impl OutboundDispatcher {
if sender.as_ref().is_none_or(mpsc::Sender::is_closed) { if sender.as_ref().is_none_or(mpsc::Sender::is_closed) {
let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else { let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else {
tracing::warn!(channel = %msg.channel, "No channel found for message"); tracing::warn!(channel = %msg.channel, "No channel found for message");
msg.complete_delivery(Err(format!("channel not found: {}", msg.channel))); msg.complete_delivery(DeliveryReceipt::PermanentFailure {
summary: format!("channel not found: {}", msg.channel),
});
continue; continue;
}; };
let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY); let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY);
if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone()) { if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone()) {
msg.complete_delivery(Err("dispatcher is shutting down".to_string())); msg.complete_delivery(DeliveryReceipt::DispatcherClosed);
continue; continue;
} }
lanes.insert(lane_key.clone(), new_sender.clone()); lanes.insert(lane_key.clone(), new_sender.clone());
@ -89,7 +91,9 @@ impl OutboundDispatcher {
capacity = LANE_CAPACITY, capacity = LANE_CAPACITY,
"Outbound lane full; rejecting message instead of blocking other destinations" "Outbound lane full; rejecting message instead of blocking other destinations"
); );
msg.complete_delivery(Err("outbound lane is full".to_string())); msg.complete_delivery(DeliveryReceipt::TransientFailure {
summary: "outbound lane is full".to_string(),
});
} }
Err(mpsc::error::TrySendError::Closed(msg)) => { Err(mpsc::error::TrySendError::Closed(msg)) => {
// The lane may have expired between the closed check and // The lane may have expired between the closed check and
@ -97,13 +101,15 @@ impl OutboundDispatcher {
lanes.remove(&lane_key); lanes.remove(&lane_key);
let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else { let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else {
tracing::warn!(channel = %msg.channel, "No channel found for message"); tracing::warn!(channel = %msg.channel, "No channel found for message");
msg.complete_delivery(Err(format!("channel not found: {}", msg.channel))); msg.complete_delivery(DeliveryReceipt::PermanentFailure {
summary: format!("channel not found: {}", msg.channel),
});
continue; continue;
}; };
let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY); let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY);
if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone()) if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone())
{ {
msg.complete_delivery(Err("dispatcher is shutting down".to_string())); msg.complete_delivery(DeliveryReceipt::DispatcherClosed);
continue; continue;
} }
match new_sender.try_send(msg) { match new_sender.try_send(msg) {
@ -111,9 +117,9 @@ impl OutboundDispatcher {
lanes.insert(lane_key, new_sender); lanes.insert(lane_key, new_sender);
} }
Err(error) => { Err(error) => {
error.into_inner().complete_delivery(Err( error
"outbound lane could not be restarted during shutdown".to_string(), .into_inner()
)); .complete_delivery(DeliveryReceipt::DispatcherClosed);
} }
} }
} }
@ -143,15 +149,15 @@ impl OutboundDispatcher {
Ok(None) | Err(_) => break, Ok(None) | Err(_) => break,
}; };
let result = Self::send_with_retry(&*channel, &msg, &target_lock).await; let result = Self::send_with_retry(&*channel, &msg, &target_lock).await;
if let Err(error) = &result { if result != DeliveryReceipt::Delivered {
tracing::error!( tracing::error!(
channel = %channel_name, channel = %channel_name,
chat_id = %chat_id, chat_id = %chat_id,
error = %error, result = ?result,
"Failed to send message after retries" "Failed to send message after retries"
); );
} }
msg.complete_delivery(result.map_err(|error| error.to_string())); msg.complete_delivery(result);
} }
}, },
) )
@ -161,26 +167,28 @@ impl OutboundDispatcher {
channel: &dyn Channel, channel: &dyn Channel,
msg: &OutboundMessage, msg: &OutboundMessage,
target_lock: &tokio::sync::Mutex<()>, target_lock: &tokio::sync::Mutex<()>,
) -> Result<(), ChannelError> { ) -> DeliveryReceipt {
let _guard = target_lock.lock().await; let _guard = target_lock.lock().await;
const DELAYS: &[u64] = &[1, 2, 4]; const DELAYS: &[u64] = &[1, 2, 4];
for (attempt, &delay) in DELAYS.iter().enumerate() { for (attempt, &delay) in DELAYS.iter().enumerate() {
let result = tokio::time::timeout(SEND_TIMEOUT, channel.send(msg.clone())).await; let result = tokio::time::timeout(SEND_TIMEOUT, channel.send(msg.clone())).await;
match result { match result {
Ok(Ok(())) => return Ok(()), Ok(Ok(())) => return DeliveryReceipt::Delivered,
Ok(Err(error)) if attempt < DELAYS.len() - 1 && error.is_transient() => { Ok(Err(error)) if attempt < DELAYS.len() - 1 && error.is_transient() => {
tracing::warn!(attempt = attempt + 1, delay, error = %error, "Send failed, retrying"); tracing::warn!(
attempt = attempt + 1,
delay,
error_class = channel_error_class(&error),
"Send failed, retrying"
);
} }
Ok(Err(error)) => return Err(error), Ok(Err(error)) => return receipt_from_channel_error(error),
Err(_) if attempt < DELAYS.len() - 1 => { Err(_) if attempt < DELAYS.len() - 1 => {
tracing::warn!(attempt = attempt + 1, delay, "Send timed out, retrying"); tracing::warn!(attempt = attempt + 1, delay, "Send timed out, retrying");
} }
Err(_) => { Err(_) => {
return Err(ChannelError::Other(format!( return DeliveryReceipt::TimedOut;
"send timed out after {} seconds",
SEND_TIMEOUT.as_secs()
)));
} }
} }
tokio::time::sleep(Duration::from_secs(delay)).await; tokio::time::sleep(Duration::from_secs(delay)).await;
@ -189,6 +197,36 @@ impl OutboundDispatcher {
} }
} }
fn channel_error_class(error: &ChannelError) -> &'static str {
match error {
ChannelError::ConnectionError(_) => "connection",
ChannelError::SendError(_) => "send",
ChannelError::BusError(_) => "bus",
ChannelError::ConfigError(_) => "config",
ChannelError::Other(_) => "other",
}
}
fn receipt_from_channel_error(error: ChannelError) -> DeliveryReceipt {
match error {
ChannelError::ConnectionError(_) => DeliveryReceipt::TransientFailure {
summary: "channel connection failed after retries".to_string(),
},
ChannelError::SendError(_) => DeliveryReceipt::TransientFailure {
summary: "channel send failed after retries".to_string(),
},
ChannelError::BusError(_) => DeliveryReceipt::TransientFailure {
summary: "channel bus was unavailable".to_string(),
},
ChannelError::ConfigError(_) => DeliveryReceipt::PermanentFailure {
summary: "channel configuration rejected delivery".to_string(),
},
ChannelError::Other(_) => DeliveryReceipt::PermanentFailure {
summary: "channel rejected delivery".to_string(),
},
}
}
/// Decrements the active-lane counter exactly once when a lane task ends, /// Decrements the active-lane counter exactly once when a lane task ends,
/// whether it exits normally, is cancelled, or is aborted. /// whether it exits normally, is cancelled, or is aborted.
struct LaneGuard { struct LaneGuard {
@ -348,7 +386,7 @@ mod tests {
message.channel = "missing".to_string(); message.channel = "missing".to_string();
let error = bus.deliver_outbound(message).await.unwrap_err(); let error = bus.deliver_outbound(message).await.unwrap_err();
assert!(matches!(error, crate::bus::BusError::DeliveryFailed(_))); assert!(matches!(error, crate::bus::BusError::DeliveryPermanent(_)));
task.abort(); task.abort();
supervisor.shutdown(Duration::from_secs(1)).await; supervisor.shutdown(Duration::from_secs(1)).await;
} }
@ -435,18 +473,40 @@ mod tests {
}; };
let target_lock = tokio::sync::Mutex::new(()); let target_lock = tokio::sync::Mutex::new(());
let error = OutboundDispatcher::send_with_retry( let receipt = OutboundDispatcher::send_with_retry(
&channel, &channel,
&outbound("invalid", "message"), &outbound("invalid", "message"),
&target_lock, &target_lock,
) )
.await .await;
.unwrap_err();
assert!(matches!(error, ChannelError::Other(_))); assert!(matches!(receipt, DeliveryReceipt::PermanentFailure { .. }));
assert_eq!(channel.attempts.load(Ordering::SeqCst), 1); assert_eq!(channel.attempts.load(Ordering::SeqCst), 1);
} }
#[test]
fn channel_errors_map_to_typed_sanitized_receipts() {
for error in [
ChannelError::ConnectionError("https://secret.example/?token=x".to_string()),
ChannelError::SendError("private response body".to_string()),
ChannelError::BusError("private queue detail".to_string()),
] {
let receipt = receipt_from_channel_error(error);
assert!(matches!(receipt, DeliveryReceipt::TransientFailure { .. }));
assert!(!format!("{receipt:?}").contains("private"));
assert!(!format!("{receipt:?}").contains("secret"));
}
for error in [
ChannelError::ConfigError("api_key=x".to_string()),
ChannelError::Other("private platform payload".to_string()),
] {
let receipt = receipt_from_channel_error(error);
assert!(matches!(receipt, DeliveryReceipt::PermanentFailure { .. }));
assert!(!format!("{receipt:?}").contains("private"));
assert!(!format!("{receipt:?}").contains("api_key"));
}
}
#[tokio::test] #[tokio::test]
async fn shared_write_lock_orders_dispatcher_with_live_turn_writes() { async fn shared_write_lock_orders_dispatcher_with_live_turn_writes() {
let channel = RecordingChannel { let channel = RecordingChannel {
@ -468,7 +528,7 @@ mod tests {
assert!(channel.sent.lock().await.is_empty()); assert!(channel.sent.lock().await.is_empty());
drop(live_write); drop(live_write);
send.await.unwrap(); assert_eq!(send.await, DeliveryReceipt::Delivered);
assert_eq!(channel.sent.lock().await.as_slice(), &["after-live-update"]); assert_eq!(channel.sent.lock().await.as_slice(), &["after-live-update"]);
} }
} }

13
src/bus/message.rs
View File

@ -509,17 +509,26 @@ pub struct OutboundMessage {
pub reply_to: Option<String>, pub reply_to: Option<String>,
pub media: Vec<MediaItem>, pub media: Vec<MediaItem>,
pub metadata: HashMap<String, String>, pub metadata: HashMap<String, String>,
pub(crate) delivery: Option<tokio::sync::watch::Sender<Option<Result<(), String>>>>, pub(crate) delivery: Option<tokio::sync::watch::Sender<Option<DeliveryReceipt>>>,
} }
impl OutboundMessage { impl OutboundMessage {
pub(crate) fn complete_delivery(&self, result: Result<(), String>) { pub(crate) fn complete_delivery(&self, result: DeliveryReceipt) {
if let Some(delivery) = &self.delivery { if let Some(delivery) = &self.delivery {
delivery.send_replace(Some(result)); delivery.send_replace(Some(result));
} }
} }
} }
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum DeliveryReceipt {
Delivered,
TransientFailure { summary: String },
PermanentFailure { summary: String },
TimedOut,
DispatcherClosed,
}
// ============================================================================ // ============================================================================
// ControlMessage - Message for control channel (session management) // ControlMessage - Message for control channel (session management)
// Uses SessionCommand from session module // Uses SessionCommand from session module

26
src/bus/mod.rs
View File

@ -4,8 +4,8 @@ pub mod message;
pub use dispatcher::OutboundDispatcher; pub use dispatcher::OutboundDispatcher;
pub use message::{ pub use message::{
ChannelContext, ChatMessage, ClientVisibility, CommittedMessage, CommittedTurnDelta, ChannelContext, ChatMessage, ClientVisibility, CommittedMessage, CommittedTurnDelta,
CompletionStatus, ContentBlock, ControlMessage, InboundMessage, MediaItem, MediaRef, CompletionStatus, ContentBlock, ControlMessage, DeliveryReceipt, InboundMessage, MediaItem,
MessageSource, OutboundMessage, ProviderReasoningState, SourceKind, TurnOrigin, MediaRef, MessageSource, OutboundMessage, ProviderReasoningState, SourceKind, TurnOrigin,
}; };
use std::sync::Arc; use std::sync::Arc;
@ -80,7 +80,17 @@ impl MessageBus {
loop { loop {
delivery_rx.changed().await.map_err(|_| BusError::Closed)?; delivery_rx.changed().await.map_err(|_| BusError::Closed)?;
if let Some(result) = delivery_rx.borrow().clone() { if let Some(result) = delivery_rx.borrow().clone() {
return result.map_err(BusError::DeliveryFailed); return match result {
DeliveryReceipt::Delivered => Ok(()),
DeliveryReceipt::TransientFailure { summary } => {
Err(BusError::DeliveryTransient(summary))
}
DeliveryReceipt::PermanentFailure { summary } => {
Err(BusError::DeliveryPermanent(summary))
}
DeliveryReceipt::TimedOut => Err(BusError::DeliveryTimedOut),
DeliveryReceipt::DispatcherClosed => Err(BusError::Closed),
};
} }
} }
}) })
@ -138,7 +148,8 @@ pub struct QueueDepths {
#[derive(Debug)] #[derive(Debug)]
pub enum BusError { pub enum BusError {
Closed, Closed,
DeliveryFailed(String), DeliveryTransient(String),
DeliveryPermanent(String),
DeliveryTimedOut, DeliveryTimedOut,
} }
@ -146,7 +157,12 @@ impl std::fmt::Display for BusError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self { match self {
BusError::Closed => write!(f, "Bus channel closed"), BusError::Closed => write!(f, "Bus channel closed"),
BusError::DeliveryFailed(error) => write!(f, "Outbound delivery failed: {error}"), BusError::DeliveryTransient(error) => {
write!(f, "Transient outbound delivery failure: {error}")
}
BusError::DeliveryPermanent(error) => {
write!(f, "Permanent outbound delivery failure: {error}")
}
BusError::DeliveryTimedOut => write!(f, "Outbound delivery confirmation timed out"), BusError::DeliveryTimedOut => write!(f, "Outbound delivery confirmation timed out"),
} }
} }

70
src/gateway/http.rs
View File

@ -812,8 +812,12 @@ fn scheduler_snapshot(jobs: &[crate::storage::ScheduledJob]) -> Value {
.iter() .iter()
.filter(|job| { .filter(|job| {
matches!( matches!(
job.last_status.as_deref(), job.last_outcome,
Some("error" | "timeout" | "delivery_error") Some(
crate::storage::ScheduledOutcomeKind::Failed
| crate::storage::ScheduledOutcomeKind::Refused
| crate::storage::ScheduledOutcomeKind::Unknown
)
) )
}) })
.count(); .count();
@ -1465,6 +1469,26 @@ pub async fn get_job_runs(
.list_scheduled_job_runs(&id, limit) .list_scheduled_job_runs(&id, limit)
.await .await
.map_err(ApiError::internal)?; .map_err(ApiError::internal)?;
let runs = runs
.into_iter()
.map(|run| {
json!({
"id": run.id,
"job_id": run.job_id,
"scheduled_for": run.scheduled_for,
"started_at": run.started_at,
"finished_at": run.finished_at,
"status": run.status,
"outcome": run.outcome,
"message": run.message,
"diagnostic": run.diagnostic,
"duration_ms": run.duration_ms,
"delivery_status": run.delivery_status,
"delivery_attempts": run.delivery_attempts,
"delivery_error": run.delivery_error,
})
})
.collect::<Vec<_>>();
Ok(Json(json!({ "runs": runs }))) Ok(Json(json!({ "runs": runs })))
} }
@ -1564,37 +1588,57 @@ mod tests {
id: &str, id: &str,
enabled: bool, enabled: bool,
next_run_at: i64, next_run_at: i64,
last_status: Option<&str>, last_outcome: Option<crate::storage::ScheduledOutcomeKind>,
) -> crate::storage::ScheduledJob { ) -> crate::storage::ScheduledJob {
crate::storage::ScheduledJob { crate::storage::ScheduledJob {
id: id.to_string(), id: id.to_string(),
name: id.to_string(), name: id.to_string(),
schedule: crate::scheduler::Schedule::Every { every_ms: 60_000 }, schedule: crate::scheduler::Schedule::Every { every_ms: 60_000 },
prompt: String::new(), prompt: String::new(),
agent_id: None,
channel: "cli_chat".to_string(), channel: "cli_chat".to_string(),
chat_id: "test".to_string(), chat_id: "test".to_string(),
model: None,
job_kind: crate::storage::JobKind::Task,
delivery_policy: crate::storage::DeliveryPolicy::Never, delivery_policy: crate::storage::DeliveryPolicy::Never,
enabled, enabled,
delete_after_run: false,
next_run_at, next_run_at,
last_run_at: None, last_run_at: None,
last_status: last_status.map(str::to_string), last_outcome,
last_error: None,
created_at: 0, created_at: 0,
updated_at: 0, updated_at: 0,
locked_at: None,
lock_owner: None,
lease_until: None,
} }
} }
#[test] #[test]
fn scheduler_snapshot_classifies_failures_and_next_enabled_run() { fn scheduler_snapshot_classifies_failures_and_next_enabled_run() {
let jobs = vec![ let jobs = vec![
scheduled_job("healthy", true, 300, Some("ok")), scheduled_job(
scheduled_job("error", true, 200, Some("error")), "healthy",
scheduled_job("timeout", false, 100, Some("timeout")), true,
scheduled_job("delivery", true, 400, Some("delivery_error")), 300,
scheduled_job("other", false, 50, Some("cancelled")), Some(crate::storage::ScheduledOutcomeKind::Ok),
),
scheduled_job(
"error",
true,
200,
Some(crate::storage::ScheduledOutcomeKind::Failed),
),
scheduled_job(
"refused",
false,
100,
Some(crate::storage::ScheduledOutcomeKind::Refused),
),
scheduled_job(
"unknown",
true,
400,
Some(crate::storage::ScheduledOutcomeKind::Unknown),
),
scheduled_job("other", false, 50, None),
]; ];
assert_eq!( assert_eq!(

47
src/gateway/mod.rs
View File

@ -178,6 +178,7 @@ impl GatewayState {
.init(&config, workspace_path.clone()) .init(&config, workspace_path.clone())
.await .await
.map_err(|e| format!("Failed to init channels: {}", e))?; .map_err(|e| format!("Failed to init channels: {}", e))?;
let available_channels = channel_manager.list_channel_names().await;
let turn_delivery = TurnDeliveryService::new( let turn_delivery = TurnDeliveryService::new(
delivery_coordinator.clone(), delivery_coordinator.clone(),
channel_manager.clone(), channel_manager.clone(),
@ -189,7 +190,10 @@ impl GatewayState {
} else { } else {
None None
}; };
let health = Arc::new(crate::health::HealthService::new(config.clone())); let health = Arc::new(
crate::health::HealthService::new(config.clone())
.with_scheduler_runtime(storage.clone(), available_channels.clone()),
);
let provider_profiles: std::collections::HashMap<String, _> = config let provider_profiles: std::collections::HashMap<String, _> = config
.agents .agents
.keys() .keys()
@ -245,9 +249,9 @@ impl GatewayState {
let session_manager = Arc::new(session_manager); let session_manager = Arc::new(session_manager);
session_manager.bind_inbox_wake(); session_manager.bind_inbox_wake();
let agent_catalog = session_manager.agent_catalog(); let agent_catalog = session_manager.agent_catalog();
health.bind_agent_catalog(agent_catalog.clone());
// Register send_message tool with available channel names // Register send_message tool with available channel names
let available_channels = channel_manager.list_channel_names().await;
let valid_channels = available_channels.clone(); let valid_channels = available_channels.clone();
session_manager.register_outbound_tool(available_channels); session_manager.register_outbound_tool(available_channels);
@ -277,11 +281,15 @@ impl GatewayState {
.tools() .tools()
.register(crate::tools::cron::CronAddTool::new( .register(crate::tools::cron::CronAddTool::new(
storage.clone(), storage.clone(),
valid_channels, valid_channels.clone(),
agent_catalog.clone(),
)); ));
session_manager session_manager
.tools() .tools()
.register(crate::tools::cron::CronListTool::new(storage.clone())); .register(crate::tools::cron::CronListTool::new(storage.clone()));
session_manager
.tools()
.register(crate::tools::cron::CronRunsTool::new(storage.clone()));
session_manager session_manager
.tools() .tools()
.register(crate::tools::cron::CronRemoveTool::new(storage.clone())); .register(crate::tools::cron::CronRemoveTool::new(storage.clone()));
@ -293,7 +301,11 @@ impl GatewayState {
.register(crate::tools::cron::CronDisableTool::new(storage.clone())); .register(crate::tools::cron::CronDisableTool::new(storage.clone()));
session_manager session_manager
.tools() .tools()
.register(crate::tools::cron::CronUpdateTool::new(storage.clone())); .register(crate::tools::cron::CronUpdateTool::new(
storage.clone(),
valid_channels,
agent_catalog.clone(),
));
tracing::info!("Cron tools registered"); tracing::info!("Cron tools registered");
} }
@ -332,7 +344,25 @@ impl GatewayState {
} }
/// Start the message processing loops /// Start the message processing loops
pub async fn start_message_processing(&self) { pub async fn start_message_processing(&self) -> Result<(), String> {
match self
.storage
.recover_scheduled_runs(chrono::Utc::now().timestamp_millis())
.await
{
Ok(recovered) if recovered > 0 => {
tracing::warn!(
recovered,
"Scheduled runs recovered as unknown on activation"
);
}
Ok(_) => {}
Err(error) => {
return Err(format!(
"Scheduled run recovery failed on activation: {error}"
));
}
}
// Recover durable Agent state for this runtime generation: interrupt // Recover durable Agent state for this runtime generation: interrupt
// runs of older generations, expire stale inbox leases and reconcile // runs of older generations, expire stale inbox leases and reconcile
// capacity rows. Runs never recover while the generation is still a candidate. // capacity rows. Runs never recover while the generation is still a candidate.
@ -351,7 +381,9 @@ impl GatewayState {
} }
} }
Err(error) => { Err(error) => {
tracing::error!(error = %error, "Agent state recovery failed on activation"); return Err(format!(
"Agent state recovery failed on activation: {error}"
));
} }
} }
} }
@ -461,6 +493,7 @@ impl GatewayState {
}); });
tracing::info!("Scheduler background task spawned"); tracing::info!("Scheduler background task spawned");
} }
Ok(())
} }
} }
@ -525,7 +558,7 @@ pub async fn run(
reload_controller.set_failed(current_generation, error.to_string()); reload_controller.set_failed(current_generation, error.to_string());
return Err(error.into()); return Err(error.into());
} }
state.start_message_processing().await; state.start_message_processing().await?;
reload_controller.set_phase(current_generation, reload::ReloadPhase::Active); reload_controller.set_phase(current_generation, reload::ReloadPhase::Active);
let app = build_router(state.clone()); let app = build_router(state.clone());
let generation_listener = TcpListener::from_std(listener.try_clone()?)?; let generation_listener = TcpListener::from_std(listener.try_clone()?)?;

2
src/gateway/router.rs
View File

@ -459,7 +459,7 @@ mod tests {
Some("command") Some("command")
); );
assert!(!publish_task.is_finished()); assert!(!publish_task.is_finished());
output.complete_delivery(Ok(())); output.complete_delivery(crate::bus::DeliveryReceipt::Delivered);
publish_task.await.unwrap(); publish_task.await.unwrap();
} }

221
src/health.rs
View File

@ -1,6 +1,7 @@
use std::collections::HashSet; use std::collections::HashSet;
use std::path::Path; use std::path::Path;
use std::process::{Output, Stdio}; use std::process::{Output, Stdio};
use std::sync::{Arc, RwLock};
use std::time::Duration; use std::time::Duration;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@ -117,11 +118,41 @@ impl HealthReport {
#[derive(Clone)] #[derive(Clone)]
pub struct HealthService { pub struct HealthService {
config: Config, config: Config,
scheduler_runtime: Arc<RwLock<Option<SchedulerHealthRuntime>>>,
}
#[derive(Clone)]
struct SchedulerHealthRuntime {
storage: Arc<crate::storage::Storage>,
channels: HashSet<String>,
catalog: Option<Arc<crate::agent::AgentCatalog>>,
} }
impl HealthService { impl HealthService {
pub fn new(config: Config) -> Self { pub fn new(config: Config) -> Self {
Self { config } Self {
config,
scheduler_runtime: Arc::new(RwLock::new(None)),
}
}
pub fn with_scheduler_runtime(
self,
storage: Arc<crate::storage::Storage>,
channels: Vec<String>,
) -> Self {
*self.scheduler_runtime.write().unwrap() = Some(SchedulerHealthRuntime {
storage,
channels: channels.into_iter().collect(),
catalog: None,
});
self
}
pub fn bind_agent_catalog(&self, catalog: Arc<crate::agent::AgentCatalog>) {
if let Some(runtime) = self.scheduler_runtime.write().unwrap().as_mut() {
runtime.catalog = Some(catalog);
}
} }
pub async fn check(&self) -> HealthReport { pub async fn check(&self) -> HealthReport {
@ -139,9 +170,165 @@ impl HealthService {
]; ];
checks.extend(self.check_mcp_commands()); checks.extend(self.check_mcp_commands());
checks.extend(self.check_browser().await); checks.extend(self.check_browser().await);
checks.extend(self.check_scheduler().await);
HealthReport::from_checks(checks) HealthReport::from_checks(checks)
} }
async fn check_scheduler(&self) -> Vec<HealthCheck> {
let scheduler_config = self.config.gateway.scheduler.clone().unwrap_or_default();
if !scheduler_config.enabled {
return vec![HealthCheck {
name: "scheduled runs".to_string(),
category: "configured".to_string(),
required: false,
status: HealthStatus::Pass,
detail: "scheduler disabled; persisted jobs are not executed".to_string(),
remediation: None,
}];
}
let runtime = self.scheduler_runtime.read().unwrap().clone();
let Some(runtime) = runtime else {
return vec![HealthCheck {
name: "scheduled runs".to_string(),
category: "runtime".to_string(),
required: false,
status: HealthStatus::Pass,
detail: "runtime scheduler state is checked by the running Gateway".to_string(),
remediation: None,
}];
};
let jobs = match runtime.storage.list_scheduled_jobs().await {
Ok(jobs) => jobs,
Err(error) => {
return vec![HealthCheck {
name: "scheduled run storage".to_string(),
category: "runtime".to_string(),
required: true,
status: HealthStatus::Fail,
detail: format!("cannot read scheduled jobs: {error}"),
remediation: Some(
"Check the configured SQLite database and Gateway logs.".to_string(),
),
}];
}
};
let now = chrono::Utc::now().timestamp_millis();
let stale_after = scheduler_config
.poll_interval_secs
.max(60)
.saturating_mul(10_000)
.min(i64::MAX as u64) as i64;
let mut bad_references = Vec::new();
let mut stale_deliveries = Vec::new();
let mut unhealthy_latest = Vec::new();
let mut silent_unknown = Vec::new();
let mut covered_intervals = Vec::new();
let mut invalid_next = Vec::new();
match runtime
.storage
.list_stale_scheduled_deliveries(now.saturating_sub(stale_after), 100)
.await
{
Ok(runs) => {
stale_deliveries.extend(
runs.into_iter()
.map(|run| format!("{}#{}", run.job_id, run.id)),
);
}
Err(error) => bad_references.push(format!("delivery backlog unavailable ({error})")),
}
for job in &jobs {
if !runtime.channels.contains(&job.channel) {
bad_references.push(format!("{}: channel {}", job.id, job.channel));
}
if let Some(agent_id) = job.agent_id.as_deref()
&& runtime
.catalog
.as_ref()
.is_none_or(|catalog| catalog.get(agent_id).is_none())
{
bad_references.push(format!("{}: agent {}", job.id, agent_id));
}
if job.enabled && crate::scheduler::next_run_for_schedule(&job.schedule, now).is_none()
{
invalid_next.push(job.id.clone());
}
let runs = match runtime.storage.list_scheduled_job_runs(&job.id, 20).await {
Ok(runs) => runs,
Err(error) => {
bad_references.push(format!("{}: run history unavailable ({error})", job.id));
continue;
}
};
if let Some(latest) = runs.first() {
if matches!(
latest.status,
crate::storage::ScheduledRunStatus::Unknown
| crate::storage::ScheduledRunStatus::Failed
| crate::storage::ScheduledRunStatus::TimedOut
) {
unhealthy_latest.push(format!("{}#{}", job.id, latest.id));
}
if job.delivery_policy == crate::storage::DeliveryPolicy::Never
&& latest.outcome == Some(crate::storage::ScheduledOutcomeKind::Unknown)
{
silent_unknown.push(format!("{}#{}", job.id, latest.id));
}
if let crate::scheduler::Schedule::Every { every_ms } = job.schedule
&& i64::try_from(every_ms).ok().is_some_and(|interval| {
latest
.duration_ms
.is_some_and(|duration| duration >= interval)
})
{
covered_intervals.push(format!("{}#{}", job.id, latest.id));
}
}
}
vec![
scheduler_health_check(
"scheduled references",
bad_references,
"all Agent and channel references are available",
"Update or disable jobs that reference missing Agents or channels.",
),
scheduler_health_check(
"scheduled delivery backlog",
stale_deliveries,
"no stale pending or delivering notifications",
"Inspect cron_runs and the target channel configuration.",
),
scheduler_health_check(
"scheduled latest outcomes",
unhealthy_latest,
"no latest run is failed, timed out, or unknown",
"Inspect cron_runs for the diagnostic and assess external side effects before retrying.",
),
scheduler_health_check(
"silent unknown scheduled runs",
silent_unknown,
"no never-delivery job has an unknown latest outcome",
"Inspect the run manually; delivery_policy=never prevents automatic notification.",
),
scheduler_health_check(
"scheduled execution intervals",
covered_intervals,
"recent execution durations fit their Every intervals",
"Increase the interval or split long-running jobs.",
),
scheduler_health_check(
"scheduled next runs",
invalid_next,
"all enabled schedules can compute a next run",
"Correct the schedule expression or disable the job.",
),
]
}
fn check_mcp_commands(&self) -> Vec<HealthCheck> { fn check_mcp_commands(&self) -> Vec<HealthCheck> {
let mut seen = HashSet::new(); let mut seen = HashSet::new();
let mut checks = Vec::new(); let mut checks = Vec::new();
@ -365,6 +552,38 @@ impl HealthService {
} }
} }
fn scheduler_health_check(
name: &str,
findings: Vec<String>,
healthy_detail: &str,
remediation: &str,
) -> HealthCheck {
if findings.is_empty() {
HealthCheck {
name: name.to_string(),
category: "runtime".to_string(),
required: false,
status: HealthStatus::Pass,
detail: healthy_detail.to_string(),
remediation: None,
}
} else {
let total = findings.len();
let mut sample = findings.into_iter().take(5).collect::<Vec<_>>().join(", ");
if total > 5 {
sample.push_str(&format!(", and {} more", total - 5));
}
HealthCheck {
name: name.to_string(),
category: "runtime".to_string(),
required: false,
status: HealthStatus::Warning,
detail: format!("{total} finding(s): {sample}"),
remediation: Some(remediation.to_string()),
}
}
}
fn check_configuration_recovery(config: &Config) -> HealthCheck { fn check_configuration_recovery(config: &Config) -> HealthCheck {
if config.diagnostics.is_empty() { if config.diagnostics.is_empty() {
return HealthCheck { return HealthCheck {

650
src/scheduler/mod.rs
View File

@ -3,81 +3,41 @@ pub mod types;
use std::sync::Arc; use std::sync::Arc;
use std::time::Instant; use std::time::Instant;
use futures_util::stream::{self, StreamExt}; use tokio::task::JoinSet;
use tokio::time; use tokio::time;
use crate::config::SchedulerConfig; use crate::config::SchedulerConfig;
use crate::session::SessionManager; use crate::session::{ScheduledDeliveryError, SessionManager};
use crate::session::session::HandleResult; use crate::storage::{
use crate::storage::ScheduledJob; ClaimedScheduledRun, JobRun, ScheduledOutcomeKind, ScheduledRunCompletion, ScheduledRunStatus,
use crate::storage::Storage; Storage,
use crate::storage::{DeliveryPolicy, JobKind, JobRun}; };
pub use types::Schedule; pub use types::Schedule;
#[derive(Debug, Clone, PartialEq, Eq)]
enum ScheduledDisposition {
Content(String),
Quiet(String),
ReportedFailure(String),
Refused(String),
}
fn parse_scheduled_disposition(output: &str) -> ScheduledDisposition {
let trimmed = output.trim();
if trimmed.eq_ignore_ascii_case("NO_REPLY") {
return ScheduledDisposition::Quiet(String::new());
}
let upper = trimmed.to_ascii_uppercase();
for (prefix, kind) in [
("NO_REPLY[INFO]", "info"),
("NO_REPLY[FAIL]", "fail"),
("NO_REPLY[REFUSE]", "refuse"),
] {
if upper.starts_with(prefix) {
let suffix = &trimmed[prefix.len()..];
if !suffix.is_empty() && !suffix.trim_start().starts_with(':') {
continue;
}
let reason = suffix.trim().trim_start_matches(':').trim().to_string();
return match kind {
"info" => ScheduledDisposition::Quiet(reason),
"fail" => ScheduledDisposition::ReportedFailure(reason),
_ => ScheduledDisposition::Refused(reason),
};
}
}
if trimmed.is_empty() {
ScheduledDisposition::ReportedFailure("scheduled agent returned empty output".into())
} else {
ScheduledDisposition::Content(trimmed.to_string())
}
}
/// Compute the next execution time (Unix ms) for a schedule, given `from` (Unix ms). /// Compute the next execution time (Unix ms) for a schedule, given `from` (Unix ms).
/// Returns `None` if no next time can be determined (e.g., invalid cron expression). /// Returns `None` if no next time can be determined (e.g. an invalid cron expression).
pub fn next_run_for_schedule(schedule: &Schedule, from: i64) -> Option<i64> { pub fn next_run_for_schedule(schedule: &Schedule, from: i64) -> Option<i64> {
use chrono::{TimeZone, Utc}; use chrono::{TimeZone, Utc};
use std::str::FromStr; use std::str::FromStr;
match schedule { match schedule {
Schedule::At { at } => Some(*at), Schedule::At { at } => Some(*at),
Schedule::Every { every_ms } => Some(from + *every_ms as i64), Schedule::Every { every_ms } => Some(from.saturating_add(i64::try_from(*every_ms).ok()?)),
Schedule::Cron { expr, tz } => { Schedule::Cron { expr, tz } => {
let cron_schedule = cron::Schedule::from_str(expr.as_str()).ok()?; let cron_schedule = cron::Schedule::from_str(expr.as_str()).ok()?;
let from_secs = from / 1000; let from_secs = from / 1000;
let from_nanos = ((from % 1000) * 1_000_000) as u32; let from_nanos = ((from % 1000) * 1_000_000) as u32;
let from_dt = Utc.timestamp_opt(from_secs, from_nanos).single()?; let from_dt = Utc.timestamp_opt(from_secs, from_nanos).single()?;
let next_utc = if let Some(tz_str) = tz { let next_utc = if let Some(tz_str) = tz {
let tz: chrono_tz::Tz = tz_str.parse().ok()?; let tz: chrono_tz::Tz = tz_str.parse().ok()?;
let from_local = from_dt.with_timezone(&tz); cron_schedule
let next_local = cron_schedule.after(&from_local).next()?; .after(&from_dt.with_timezone(&tz))
next_local.with_timezone(&Utc) .next()?
.with_timezone(&Utc)
} else { } else {
cron_schedule.after(&from_dt).next()? cron_schedule.after(&from_dt).next()?
}; };
Some(next_utc.timestamp_millis()) Some(next_utc.timestamp_millis())
} }
} }
@ -90,8 +50,6 @@ fn now_ms() -> i64 {
.as_millis() as i64 .as_millis() as i64
} }
/// The scheduler runs as a background tokio task, periodically checking for due jobs
/// and executing them via `SessionManager::handle_cron_message`.
pub struct Scheduler { pub struct Scheduler {
storage: Arc<Storage>, storage: Arc<Storage>,
session_manager: Arc<SessionManager>, session_manager: Arc<SessionManager>,
@ -129,15 +87,16 @@ impl Scheduler {
} }
} }
/// Claim due jobs with a durable lease, then execute the claimed batch with /// Non-blocking event loop. Execution and delivery use separate bounded
/// bounded concurrency. /// JoinSets so one long Agent run cannot delay other claims or outbox work.
pub async fn run(self: Arc<Self>) { pub async fn run(self: Arc<Self>) {
let poll_duration = time::Duration::from_secs(self.config.poll_interval_secs.max(1)); let poll_duration = time::Duration::from_secs(self.config.poll_interval_secs.max(1));
let mut interval = time::interval(poll_duration); let mut interval = time::interval(poll_duration);
interval.set_missed_tick_behavior(time::MissedTickBehavior::Skip); interval.set_missed_tick_behavior(time::MissedTickBehavior::Skip);
// Keep accidental configuration values from claiming an unbounded
// batch and overwhelming the runtime or SQLite parameter conversion.
let max_concurrent = self.config.max_concurrent.clamp(1, 256); let max_concurrent = self.config.max_concurrent.clamp(1, 256);
let max_delivery = max_concurrent.clamp(1, 16);
let mut runs = JoinSet::new();
let mut deliveries = JoinSet::new();
tracing::info!( tracing::info!(
poll_interval_secs = self.config.poll_interval_secs, poll_interval_secs = self.config.poll_interval_secs,
@ -147,375 +106,340 @@ impl Scheduler {
); );
loop { loop {
interval.tick().await; tokio::select! {
_ = interval.tick() => {}
Some(result) = runs.join_next(), if !runs.is_empty() => {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled run task panicked");
}
}
Some(result) = deliveries.join_next(), if !deliveries.is_empty() => {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled delivery task panicked");
}
}
}
while let Some(result) = runs.try_join_next() {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled run task panicked");
}
}
while let Some(result) = deliveries.try_join_next() {
if let Err(error) = result {
tracing::error!(error = %error, "scheduled delivery task panicked");
}
}
if !self.admission.is_accepting() { if !self.admission.is_accepting() {
continue; continue;
} }
let now = now_ms(); let now = now_ms();
let delivery_slots = max_delivery.saturating_sub(deliveries.len());
if delivery_slots > 0 {
let lease_until = now.saturating_add(180_000);
let delivery_owner = format!("{}:delivery:{}", self.owner, uuid::Uuid::new_v4());
match self
.storage
.claim_scheduled_deliveries(now, lease_until, &delivery_owner, delivery_slots)
.await
{
Ok(claimed) => {
for run in claimed {
let scheduler = self.clone();
deliveries.spawn(async move {
scheduler.deliver_claimed_run(run).await;
});
}
}
Err(error) => {
tracing::error!(error = %error, "scheduler: failed to claim deliveries");
}
}
}
let run_slots = max_concurrent.saturating_sub(runs.len());
if run_slots == 0 {
continue;
}
let lease_ms = self let lease_ms = self
.config .config
.execution_timeout_secs .execution_timeout_secs
.saturating_add(150) .saturating_add(150)
.saturating_mul(1000) .saturating_mul(1000)
.min(i64::MAX as u64) as i64; .min(i64::MAX as u64) as i64;
let lease_until = now.saturating_add(lease_ms); let run_owner = format!("{}:run:{}", self.owner, uuid::Uuid::new_v4());
let jobs = match self match self
.storage .storage
.claim_due_scheduled_jobs(now, lease_until, &self.owner, max_concurrent) .claim_due_scheduled_runs(now, now.saturating_add(lease_ms), &run_owner, run_slots)
.await .await
{ {
Ok(jobs) => jobs, Ok(claimed) => {
Err(error) => { for run in claimed {
tracing::error!(error = %error, "scheduler: failed to claim due jobs"); let scheduler = self.clone();
continue; runs.spawn(async move {
scheduler.execute_claimed_run(run).await;
});
}
}
Err(error) => {
tracing::error!(error = %error, "scheduler: failed to claim due runs");
} }
};
if jobs.is_empty() {
continue;
} }
tracing::info!(count = jobs.len(), "scheduler: claimed due jobs");
stream::iter(jobs)
.for_each_concurrent(max_concurrent, |job| {
let scheduler = self.clone();
async move { scheduler.execute_claimed_job(job).await }
})
.await;
} }
} }
async fn execute_claimed_job(self: Arc<Self>, job: ScheduledJob) { async fn execute_claimed_run(self: Arc<Self>, claimed: ClaimedScheduledRun) {
let Some(_activity) = self.admission.try_enter() else {
if let Err(error) = self
.storage
.release_scheduled_job_lease(&job.id, &self.owner)
.await
{
tracing::error!(job_id = %job.id, error = %error, "scheduler: failed to release job claimed during reload drain");
}
return;
};
let start = Instant::now(); let start = Instant::now();
let started_at = now_ms(); let job = &claimed.job;
tracing::info!(job_id = %job.id, job_name = %job.name, "scheduler: executing claimed job"); let (completion, agent_execution) = if let Some(_activity) = self.admission.try_enter() {
match self.session_manager.agent_coordinator() {
let managed = job.delivery_policy != DeliveryPolicy::Direct; Some(coordinator) => match coordinator
let execution = async { .execute_scheduled(
if managed { claimed.run_id,
self.session_manager &claimed.owner,
.handle_managed_scheduled_message(
&job.prompt,
&job.id, &job.id,
&job.name, &job.name,
job.job_kind == JobKind::Monitor, job.agent_id.as_deref(),
)
.await
.map(HandleResult::AgentResponse)
} else {
self.session_manager
.handle_cron_message(
&job.channel,
&job.chat_id,
&job.prompt, &job.prompt,
&job.id, self.config.execution_timeout_secs.max(1),
&job.name,
) )
.await .await
} {
}; Ok(execution) => {
let result = time::timeout( if execution.status == ScheduledRunStatus::Completed
time::Duration::from_secs(self.config.execution_timeout_secs.max(1)), && let Some(outcome) = execution.outcome.clone()
execution, {
) (
.await; ScheduledRunCompletion {
let finished_at = now_ms(); status: ScheduledRunStatus::Completed,
let duration_ms = start.elapsed().as_millis() as i64; outcome: outcome.kind,
message: outcome.message,
let (mut status, output, error, result_kind, mut delivery_status, mut delivery_error) = diagnostic: execution.error.clone(),
match result { duration_ms: start.elapsed().as_millis() as i64,
Ok(Ok( },
HandleResult::AgentResponse(output) | HandleResult::CommandOutput(output), Some(execution),
)) => { )
let output = if output.len() > 8000 {
format!(
"{}...[truncated]",
&output[..output.ceil_char_boundary(8000)]
)
} else {
output
};
if !managed {
(
"ok".into(),
Some(output),
None,
None,
Some("direct".into()),
None,
)
} else {
let disposition = parse_scheduled_disposition(&output);
let (kind, content, alert) = match &disposition {
ScheduledDisposition::Content(value) => {
("content", Some(value.as_str()), true)
}
ScheduledDisposition::Quiet(_) => ("quiet", None, false),
ScheduledDisposition::ReportedFailure(value) => {
("reported_failure", Some(value.as_str()), true)
}
ScheduledDisposition::Refused(value) => {
("refused", Some(value.as_str()), true)
}
};
let should_deliver = match job.delivery_policy {
DeliveryPolicy::Always => true,
DeliveryPolicy::OnAlert => alert,
DeliveryPolicy::Never => false,
DeliveryPolicy::Direct => false,
};
if should_deliver {
let message = content.unwrap_or("巡检完成，未发现需要关注的问题。");
match self
.session_manager
.deliver_scheduled_message(
&job.channel,
&job.chat_id,
&job.id,
&job.name,
message,
)
.await
{
Ok(()) => (
"ok".into(),
Some(output),
None,
Some(kind.into()),
Some("delivered".into()),
None,
),
Err(delivery_error) => (
"delivery_error".into(),
Some(output),
None,
Some(kind.into()),
Some("failed".into()),
Some(delivery_error),
),
}
} else { } else {
let delivery = if job.delivery_policy == DeliveryPolicy::Never { let diagnostic = execution.error.clone().unwrap_or_else(|| {
"skipped" "scheduled Agent returned ordinary text without calling complete_scheduled_run"
} else { .to_string()
"suppressed" });
let status = match execution.status {
ScheduledRunStatus::TimedOut => ScheduledRunStatus::TimedOut,
ScheduledRunStatus::Interrupted | ScheduledRunStatus::Cancelled => {
ScheduledRunStatus::Interrupted
}
_ => ScheduledRunStatus::Failed,
}; };
( (
"ok".into(), ScheduledRunCompletion {
Some(output), status,
None, outcome: ScheduledOutcomeKind::Failed,
Some(kind.into()), message: if status == ScheduledRunStatus::TimedOut {
Some(delivery.into()), format!("定时任务「{}」执行超时。", job.name)
None, } else if status == ScheduledRunStatus::Interrupted {
format!("定时任务「{}」在系统关停时被中断。", job.name)
} else {
format!(
"定时任务「{}」未可靠完成：Agent 未提交结构化运行结果。",
job.name
)
},
diagnostic: Some(diagnostic),
duration_ms: start.elapsed().as_millis() as i64,
},
Some(execution),
) )
} }
} }
Err(error) => (
ScheduledRunCompletion {
status: ScheduledRunStatus::Failed,
outcome: ScheduledOutcomeKind::Failed,
message: job.agent_id.as_deref().map_or_else(
|| format!("定时任务「{}」未能启动 Root Agent 执行。", job.name),
|agent_id| {
format!(
"定时任务「{}」无法使用 Agent「{}」执行。请恢复该 Agent 定义，或更新任务的 agent_id。",
job.name, agent_id
)
},
),
diagnostic: Some(error.to_string()),
duration_ms: start.elapsed().as_millis() as i64,
},
None,
),
},
None => (
ScheduledRunCompletion {
status: ScheduledRunStatus::Failed,
outcome: ScheduledOutcomeKind::Failed,
message: format!("定时任务「{}」未能启动执行。", job.name),
diagnostic: Some("AgentCoordinator is unavailable".to_string()),
duration_ms: start.elapsed().as_millis() as i64,
},
None,
),
}
} else {
(
ScheduledRunCompletion {
status: ScheduledRunStatus::Interrupted,
outcome: ScheduledOutcomeKind::Failed,
message: format!("定时任务「{}」因 Gateway 重载而中断。", job.name),
diagnostic: Some("runtime admission closed".to_string()),
duration_ms: start.elapsed().as_millis() as i64,
},
None,
)
};
let finished_at = now_ms();
let mut commit_attempt = 0_u64;
let commit = loop {
let result = match agent_execution.as_ref() {
Some(execution) => {
self.storage
.finish_scheduled_run_with_agent(
claimed.run_id,
&claimed.owner,
&completion,
&execution.agent_run_id,
&execution.agent_run_id,
execution.runtime_generation,
&execution.agent_terminal,
finished_at,
)
.await
}
None => {
self.storage
.finish_scheduled_run(
claimed.run_id,
&claimed.owner,
&completion,
finished_at,
)
.await
} }
Ok(Ok(HandleResult::AgentProcessing)) => (
"error".to_string(),
None,
Some("cron execution returned asynchronous processing".to_string()),
None,
None,
None,
),
Ok(Err(error)) => (
"error".to_string(),
None,
Some(error.to_string()),
None,
None,
None,
),
Err(_) => (
"timeout".to_string(),
None,
Some(format!(
"execution exceeded {} seconds",
self.config.execution_timeout_secs.max(1)
)),
None,
None,
None,
),
}; };
match result {
if managed Err(error) if error.is_transient() && commit_attempt < 2 => {
&& delivery_status.is_none() commit_attempt += 1;
&& job.delivery_policy != DeliveryPolicy::Never tracing::warn!(
&& let Some(message) = error.as_deref() job_id = %job.id,
{ run_id = claimed.run_id,
let notice = format!("定时任务「{}」执行失败：{}", job.name, message); attempt = commit_attempt + 1,
match self error = %error,
.session_manager "scheduler: retrying transient run completion commit"
.deliver_scheduled_message(&job.channel, &job.chat_id, &job.id, &job.name, &notice) );
.await time::sleep(time::Duration::from_millis(50 * commit_attempt)).await;
{
Ok(()) => delivery_status = Some("delivered".into()),
Err(error) => {
status = "delivery_error".into();
delivery_status = Some("failed".into());
delivery_error = Some(error);
} }
result => break result,
} }
};
match commit {
Ok(true) => tracing::info!(
job_id = %job.id,
run_id = claimed.run_id,
status = completion.status.as_str(),
outcome = completion.outcome.as_str(),
duration_ms = completion.duration_ms,
"scheduler: run completed"
),
Ok(false) => tracing::warn!(
job_id = %job.id,
run_id = claimed.run_id,
"scheduler: late run result discarded"
),
Err(error) => tracing::error!(
job_id = %job.id,
run_id = claimed.run_id,
error = %error,
"scheduler: failed to commit run completion"
),
} }
}
let (next_run_at, disable, delete) = match &job.schedule { async fn deliver_claimed_run(self: Arc<Self>, run: JobRun) {
Schedule::At { .. } => (None, !job.delete_after_run, job.delete_after_run), let Some(delivery_owner) = run.delivery_lease_owner.clone() else {
Schedule::Every { .. } | Schedule::Cron { .. } => { tracing::error!(
match next_run_for_schedule(&job.schedule, finished_at) { run_id = run.id,
Some(next) => (Some(next), false, false), "scheduler: claimed delivery has no lease owner"
None => (None, true, false), );
} return;
};
let result = self
.session_manager
.deliver_scheduled_run(&run, &delivery_owner)
.await;
let (delivered, permanent, error) = match result {
Ok(()) => (true, false, None),
Err(ScheduledDeliveryError::Transient(error)) => {
(false, false, Some(sanitize_error(&error)))
}
Err(ScheduledDeliveryError::Permanent(error)) => {
(false, true, Some(sanitize_error(&error)))
} }
}; };
let run = JobRun { if let Err(commit_error) = self
id: 0,
job_id: job.id.clone(),
started_at,
finished_at,
status,
output,
error,
duration_ms,
result_kind,
delivery_status,
delivery_error,
};
if let Err(error) = self
.storage .storage
.complete_scheduled_job(&run, &self.owner, next_run_at, disable, delete) .complete_scheduled_delivery(
run.id,
&delivery_owner,
delivered,
permanent,
error.as_deref(),
now_ms(),
)
.await .await
{ {
tracing::error!(job_id = %job.id, error = %error, "scheduler: failed to commit job completion"); tracing::error!(
let _ = self run_id = run.id,
.storage error = %commit_error,
.release_scheduled_job_lease(&job.id, &self.owner) "scheduler: failed to commit delivery receipt"
.await; );
return;
} }
tracing::info!(
job_id = %job.id,
status = %run.status,
duration_ms,
"scheduler: job completed"
);
} }
} }
fn sanitize_error(error: &str) -> String {
error.chars().take(1_024).collect()
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
#[test] #[test]
fn test_next_run_at_schedule() { fn next_run_for_every_uses_claim_time() {
let now = 1000000; assert_eq!(
let next = next_run_for_schedule(&Schedule::At { at: 2000000 }, now); next_run_for_schedule(&Schedule::Every { every_ms: 5_000 }, 1_000),
assert_eq!(next, Some(2000000)); Some(6_000)
);
} }
#[test] #[test]
fn test_next_run_every_schedule() { fn next_run_for_at_keeps_absolute_timestamp() {
let now = 1000000; assert_eq!(
let next = next_run_for_schedule(&Schedule::Every { every_ms: 5000 }, now); next_run_for_schedule(&Schedule::At { at: 2_000 }, 1_000),
assert_eq!(next, Some(1005000)); Some(2_000)
);
} }
#[test] #[test]
fn test_next_run_cron_every_minute() { fn cron_timezone_uses_from_argument() {
let expr = "0 * * * * *".to_string();
let schedule = Schedule::Cron { expr, tz: None };
let now = 1000000;
let next = next_run_for_schedule(&schedule, now);
assert!(next.is_some());
assert!(next.unwrap() > now);
}
#[test]
fn test_next_run_cron_every_day_at_9am() {
let expr = "0 0 9 * * *".to_string();
let schedule = Schedule::Cron { expr, tz: None };
let now = 1000000;
let next = next_run_for_schedule(&schedule, now);
assert!(next.is_some());
let next_ms = next.unwrap();
assert!(next_ms > now);
}
#[test]
fn test_next_run_cron_uses_from_argument() {
let expr = "0 * * * * *".to_string();
let schedule = Schedule::Cron { expr, tz: None };
let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T12:34:20Z")
.unwrap()
.timestamp_millis();
let next = next_run_for_schedule(&schedule, from).unwrap();
let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T12:35:00Z")
.unwrap()
.timestamp_millis();
assert_eq!(next, expected);
}
#[test]
fn scheduled_disposition_is_fail_safe() {
assert!(matches!(
parse_scheduled_disposition("NO_REPLY"),
ScheduledDisposition::Quiet(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[INFO]: healthy"),
ScheduledDisposition::Quiet(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[FAIL]: timeout"),
ScheduledDisposition::ReportedFailure(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[REFUSE]: denied"),
ScheduledDisposition::Refused(_)
));
assert!(matches!(
parse_scheduled_disposition("text mentioning NO_REPLY"),
ScheduledDisposition::Content(_)
));
assert!(matches!(
parse_scheduled_disposition("NO_REPLY[INFO] but this is content"),
ScheduledDisposition::Content(_)
));
assert!(matches!(
parse_scheduled_disposition(""),
ScheduledDisposition::ReportedFailure(_)
));
}
#[test]
fn test_next_run_cron_timezone_uses_from_argument() {
let expr = "0 0 9 * * *".to_string();
let schedule = Schedule::Cron { let schedule = Schedule::Cron {
expr, expr: "0 0 9 * * *".to_string(),
tz: Some("Asia/Shanghai".to_string()), tz: Some("Asia/Shanghai".to_string()),
}; };
let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T00:30:00Z") let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T00:30:00Z")
.unwrap() .unwrap()
.timestamp_millis(); .timestamp_millis();
let next = next_run_for_schedule(&schedule, from).unwrap();
let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T01:00:00Z") let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T01:00:00Z")
.unwrap() .unwrap()
.timestamp_millis(); .timestamp_millis();
assert_eq!(next, expected); assert_eq!(next_run_for_schedule(&schedule, from), Some(expected));
} }
} }

129
src/session/messenger.rs
View File

@ -4,7 +4,9 @@ use crate::bus::{ChatMessage, MediaItem, MessageSource, OutboundMessage, SourceK
use crate::session::UnifiedSessionId; use crate::session::UnifiedSessionId;
use crate::tools::{OutboundDelivery, OutboundMessenger}; use crate::tools::{OutboundDelivery, OutboundMessenger};
use super::persistence::{append_active_turn_message, append_persisted_messages}; use super::persistence::{
append_active_turn_message, append_persisted_message_if_absent, append_persisted_messages,
};
use super::session::{ use super::session::{
CURRENT_SOURCE_SESSION, CURRENT_TURN_DELIVERIES, CURRENT_TURN_ID, PendingTurnDelivery, CURRENT_SOURCE_SESSION, CURRENT_TURN_DELIVERIES, CURRENT_TURN_ID, PendingTurnDelivery,
SessionManager, SessionManager,
@ -118,37 +120,108 @@ impl OutboundMessenger for SessionManager {
} }
impl SessionManager { impl SessionManager {
pub async fn deliver_scheduled_message( pub async fn deliver_scheduled_run(
&self, &self,
channel: &str, run: &crate::storage::JobRun,
chat_id: &str, delivery_owner: &str,
job_id: &str, ) -> Result<(), ScheduledDeliveryError> {
job_name: &str, let content = run
content: &str, .message
) -> Result<(), String> { .as_deref()
<Self as OutboundMessenger>::send_message( .unwrap_or("定时任务已结束，但没有生成可投递的结果。请在任务运行记录中查看诊断信息。");
self, let target_sid = if let Some(session_id) = run.target_session_id.as_deref() {
channel, UnifiedSessionId::parse(session_id).ok_or_else(|| {
chat_id, ScheduledDeliveryError::Permanent("stored target session is invalid".to_string())
None, })?
content, } else {
MessageSource { let resolved = self
kind: SourceKind::ExternalTrigger, .resolve_dialog_id(&run.target_channel, &run.target_chat_id)
from_channel: Some("scheduler".to_string()), .await
from_session: Some(format!("cron:{job_id}")), .map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
from_user_id: None, let fixed = self
system_name: Some(job_name.to_string()), .storage
task_id: Some(job_id.to_string()), .set_scheduled_delivery_target_session(
from_run_id: None, run.id,
from_agent_id: None, delivery_owner,
}, &resolved.to_string(),
Vec::new(), chrono::Utc::now().timestamp_millis(),
) )
.await .await
.map(|_| ()) .map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?
.ok_or_else(|| {
ScheduledDeliveryError::Transient(
"scheduled delivery lost its claim before fixing target session"
.to_string(),
)
})?;
UnifiedSessionId::parse(&fixed).ok_or_else(|| {
ScheduledDeliveryError::Permanent("fixed target session is invalid".to_string())
})?
};
if target_sid.channel != run.target_channel || target_sid.chat_id != run.target_chat_id {
return Err(ScheduledDeliveryError::Permanent(
"fixed target session does not belong to the scheduled destination".to_string(),
));
}
let session = self
.get_or_activate_session(&target_sid)
.await
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
let source = MessageSource {
kind: SourceKind::ExternalTrigger,
from_channel: Some("scheduler".to_string()),
from_session: Some(format!("scheduled-run:{}", run.id)),
from_user_id: None,
system_name: Some("scheduled task".to_string()),
task_id: Some(run.job_id.clone()),
from_run_id: run.agent_run_id.clone(),
from_agent_id: run.agent_id.clone(),
};
let mut message = outbound_history_message(content, source, &[]);
message.id = format!("scheduled:{}", run.id);
let message_id = message.id.clone();
append_persisted_message_if_absent(&session, message)
.await
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
let metadata = HashMap::from([
("_session_id".to_string(), target_sid.to_string()),
("_message_id".to_string(), message_id),
("scheduled_delivery_id".to_string(), run.id.to_string()),
]);
self.bus
.deliver_outbound(OutboundMessage {
channel: run.target_channel.clone(),
chat_id: run.target_chat_id.clone(),
content: content.to_string(),
reply_to: None,
media: Vec::new(),
metadata,
delivery: None,
})
.await
.map_err(|error| match error {
crate::bus::BusError::Closed
| crate::bus::BusError::DeliveryTimedOut
| crate::bus::BusError::DeliveryTransient(_) => {
ScheduledDeliveryError::Transient(error.to_string())
}
crate::bus::BusError::DeliveryPermanent(summary) => {
ScheduledDeliveryError::Permanent(summary)
}
})?;
Ok(())
} }
} }
#[derive(Debug, thiserror::Error)]
pub enum ScheduledDeliveryError {
#[error("transient scheduled delivery failure: {0}")]
Transient(String),
#[error("permanent scheduled delivery failure: {0}")]
Permanent(String),
}
fn outbound_history_message( fn outbound_history_message(
content: impl Into<String>, content: impl Into<String>,
source: MessageSource, source: MessageSource,

1
src/session/mod.rs
View File

@ -14,6 +14,7 @@ pub mod turn;
pub use commands::SessionCommand; pub use commands::SessionCommand;
pub use error::SessionError; pub use error::SessionError;
pub use events::{DialogInfo, SessionEvent}; pub use events::{DialogInfo, SessionEvent};
pub use messenger::ScheduledDeliveryError;
pub use session::{ pub use session::{
AgentCatalogPreparation, SLASH_COMMANDS, Session, SessionManager, SessionManagerServices, AgentCatalogPreparation, SLASH_COMMANDS, Session, SessionManager, SessionManagerServices,
SlashCommand, SlashCommand,

95
src/session/persistence.rs
View File

@ -101,6 +101,41 @@ pub(super) async fn append_persisted_messages_with_meta(
append_persisted_messages_inner(session, messages, VersionPolicy::Advance, None, None).await append_persisted_messages_inner(session, messages, VersionPolicy::Advance, None, None).await
} }
pub(super) async fn append_persisted_message_if_absent(
session: &Arc<Mutex<Session>>,
message: ChatMessage,
) -> Result<bool, StorageError> {
let persistence_lock = { session.lock().await.persistence_lock.clone() };
let _persistence_guard = persistence_lock.lock().await;
let message_id = message.id.clone();
let snapshot = {
let guard = session.lock().await;
if guard.contains_message_id(&message_id) {
return Ok(false);
}
guard.prepare_message_persist_snapshot(&message)
};
let Some((storage, session_id, persisted, meta)) = snapshot else {
return Ok(false);
};
let inserted = storage
.persist_message_if_absent(&session_id, &persisted, &meta)
.await?;
if !inserted {
return Ok(false);
}
let mut guard = session.lock().await;
if guard.contains_message_id(&message_id) {
return Ok(true);
}
if !guard.apply_prepared_message_in_memory(message, persisted.seq, persisted.created_at, true) {
return Err(StorageError::Conflict(format!(
"session changed while committing idempotent message {message_id}"
)));
}
Ok(true)
}
pub(super) async fn append_persisted_turn_messages( pub(super) async fn append_persisted_turn_messages(
session: &Arc<Mutex<Session>>, session: &Arc<Mutex<Session>>,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
@ -247,7 +282,7 @@ mod tests {
} }
#[tokio::test] #[tokio::test]
async fn active_turn_side_effect_does_not_invalidate_its_session_version() { async fn side_effect_messages_apply_the_expected_session_version_once() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let storage = Arc::new( let storage = Arc::new(
crate::storage::Storage::new(&dir.path().join("memory.db")) crate::storage::Storage::new(&dir.path().join("memory.db"))
@ -255,7 +290,7 @@ mod tests {
.unwrap(), .unwrap(),
); );
let memory_manager = Arc::new(MemoryManager::new( let memory_manager = Arc::new(MemoryManager::new(
storage, storage.clone(),
"test".to_string(), "test".to_string(),
"test".to_string(), "test".to_string(),
)); ));
@ -276,12 +311,32 @@ mod tests {
price_input_per_million: None, price_input_per_million: None,
price_output_per_million: None, price_output_per_million: None,
}; };
let unified_id = crate::session::UnifiedSessionId::new("cli_chat", "chat", "dialog");
let now = chrono::Utc::now().timestamp_millis();
storage
.upsert_session(&crate::storage::session::SessionMeta {
id: unified_id.to_string(),
channel: "cli_chat".to_string(),
chat_id: "chat".to_string(),
dialog_id: "dialog".to_string(),
title: "test".to_string(),
created_at: now,
last_active_at: now,
message_count: 0,
routing_info: None,
archived_at: None,
deleted_at: None,
last_consolidated_at: None,
last_compressed_message_at: None,
})
.await
.unwrap();
let session = Arc::new(Mutex::new( let session = Arc::new(Mutex::new(
Session::new( Session::new(
crate::session::UnifiedSessionId::new("cli_chat", "chat", "dialog"), unified_id,
config, config,
Arc::new(ToolRegistry::new()), Arc::new(ToolRegistry::new()),
None, Some(storage.clone()),
String::new(), String::new(),
"test".to_string(), "test".to_string(),
super::super::session::SessionContextServices { super::super::session::SessionContextServices {
@ -320,5 +375,37 @@ mod tests {
let guard = session.lock().await; let guard = session.lock().await;
assert_eq!(guard.state_version_for_test(), base_version + 1); assert_eq!(guard.state_version_for_test(), base_version + 1);
assert_eq!(guard.get_history().len(), 2); assert_eq!(guard.get_history().len(), 2);
drop(guard);
let idempotent_base_version = session.lock().await.state_version_for_test();
let mut scheduled = ChatMessage::assistant("scheduled result");
scheduled.id = "scheduled:42".to_string();
assert!(
append_persisted_message_if_absent(&session, scheduled.clone())
.await
.unwrap()
);
assert!(
!append_persisted_message_if_absent(&session, scheduled)
.await
.unwrap()
);
let guard = session.lock().await;
assert_eq!(guard.state_version_for_test(), idempotent_base_version + 1);
assert_eq!(
guard
.get_history()
.iter()
.filter(|message| message.id == "scheduled:42")
.count(),
1
);
drop(guard);
let persisted: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM messages WHERE id = ?")
.bind("scheduled:42")
.fetch_one(storage.pool())
.await
.unwrap();
assert_eq!(persisted, 1);
} }
} }

295
src/session/session.rs
View File

@ -1083,59 +1083,101 @@ impl Session {
persist: bool, persist: bool,
advance_state_version: bool, advance_state_version: bool,
) -> Option<MessagePersistSnapshot> { ) -> Option<MessagePersistSnapshot> {
let persist_snapshot = persist
.then(|| self.prepare_message_persist_snapshot(&message))
.flatten();
let (seq, now) = persist_snapshot
.as_ref()
.map(|(_, _, message, _)| (message.seq, message.created_at))
.unwrap_or_else(|| (self.seq_counter, chrono::Utc::now().timestamp_millis()));
let applied =
self.apply_prepared_message_in_memory(message, seq, now, advance_state_version);
debug_assert!(applied, "fresh in-memory message snapshot must apply");
persist_snapshot
}
pub(super) fn prepare_message_persist_snapshot(
&self,
message: &ChatMessage,
) -> Option<MessagePersistSnapshot> {
let storage = self.storage.clone()?;
let is_user = message.role == "user"; let is_user = message.role == "user";
let counts_as_user_input = let counts_as_user_input =
is_user && message.client_visibility == crate::bus::ClientVisibility::Visible; is_user && message.client_visibility == crate::bus::ClientVisibility::Visible;
let now = chrono::Utc::now().timestamp_millis(); let now = chrono::Utc::now().timestamp_millis();
// Assign seq
let seq = self.seq_counter; let seq = self.seq_counter;
self.seq_counter += 1; let msg_meta = crate::storage::message::MessageMeta {
id: message.id.clone(),
let persist_snapshot = if persist { session_id: self.id.to_string(),
self.storage.clone().map(|storage| { seq,
let msg_meta = crate::storage::message::MessageMeta { role: message.role.clone(),
id: message.id.clone(), content: message.content.clone(),
session_id: self.id.to_string(), reasoning_content: message.reasoning_content.clone(),
seq, provider_state: message
role: message.role.clone(), .provider_state
content: message.content.clone(), .as_ref()
reasoning_content: message.reasoning_content.clone(), .and_then(|state| serde_json::to_string(state).ok()),
provider_state: message turn_id: message.turn_id.clone(),
.provider_state iteration: message.iteration.map(i64::from),
.as_ref() completion_status: message.completion_status,
.and_then(|state| serde_json::to_string(state).ok()), client_visibility: message.client_visibility,
turn_id: message.turn_id.clone(), turn_origin: message.turn_origin,
iteration: message.iteration.map(i64::from), media_refs: if message.media_refs.is_empty() {
completion_status: message.completion_status, None
client_visibility: message.client_visibility, } else {
turn_origin: message.turn_origin, Some(serde_json::to_string(&message.media_refs).unwrap_or_default())
media_refs: if message.media_refs.is_empty() { },
None tool_call_id: message.tool_call_id.clone(),
} else { tool_name: message.tool_name.clone(),
Some(serde_json::to_string(&message.media_refs).unwrap_or_default()) tool_calls: message
}, .tool_calls
tool_call_id: message.tool_call_id.clone(), .as_ref()
tool_name: message.tool_name.clone(), .and_then(|tc| serde_json::to_string(tc).ok()),
tool_calls: message source: message
.tool_calls .source
.as_ref() .as_ref()
.and_then(|tc| serde_json::to_string(tc).ok()), .map(|s| serde_json::to_string(s).unwrap_or_default()),
source: message created_at: now,
.source
.as_ref()
.map(|s| serde_json::to_string(s).unwrap_or_default()),
created_at: now,
};
(storage, self.id.to_string(), msg_meta)
})
} else {
None
}; };
let session_id = self.id.to_string();
let session_meta = crate::storage::session::SessionMeta {
id: session_id.clone(),
channel: self.id.channel.clone(),
chat_id: self.id.chat_id.clone(),
dialog_id: self.id.dialog_id.clone(),
title: self.title.clone(),
created_at: self.created_at,
last_active_at: now,
message_count: self.message_count + i64::from(counts_as_user_input),
routing_info: if self.routing_info.is_empty() {
None
} else {
Some(self.routing_info.clone())
},
archived_at: self.archived_at,
deleted_at: None,
last_consolidated_at: self.last_consolidated_at,
last_compressed_message_at: self.last_compressed_message_at,
};
Some((storage, session_id, msg_meta, session_meta))
}
// Update in-memory state pub(super) fn apply_prepared_message_in_memory(
&mut self,
message: ChatMessage,
seq: i64,
now: i64,
advance_state_version: bool,
) -> bool {
if self.seq_counter != seq || self.contains_message_id(&message.id) {
return false;
}
let is_user = message.role == "user";
let counts_as_user_input =
is_user && message.client_visibility == crate::bus::ClientVisibility::Visible;
self.message_seqs.insert(message.id.clone(), seq); self.message_seqs.insert(message.id.clone(), seq);
self.messages.push(message); self.messages.push(message);
self.seq_counter += 1;
self.total_message_count += 1; self.total_message_count += 1;
if counts_as_user_input { if counts_as_user_input {
self.message_count += 1; self.message_count += 1;
@ -1144,29 +1186,11 @@ impl Session {
if advance_state_version { if advance_state_version {
self.state_version = self.state_version.wrapping_add(1); self.state_version = self.state_version.wrapping_add(1);
} }
true
}
persist_snapshot.map(|(storage, session_id, msg_meta)| { pub(super) fn contains_message_id(&self, message_id: &str) -> bool {
let session_meta = crate::storage::session::SessionMeta { self.message_seqs.contains_key(message_id)
id: session_id.clone(),
channel: self.id.channel.clone(),
chat_id: self.id.chat_id.clone(),
dialog_id: self.id.dialog_id.clone(),
title: self.title.clone(),
created_at: self.created_at,
last_active_at: self.last_active_at,
message_count: self.message_count,
routing_info: if self.routing_info.is_empty() {
None
} else {
Some(self.routing_info.clone())
},
archived_at: self.archived_at,
deleted_at: None,
last_consolidated_at: self.last_consolidated_at,
last_compressed_message_at: self.last_compressed_message_at,
};
(storage, session_id, msg_meta, session_meta)
})
} }
/// Roll back messages that were appended in memory but whose atomic /// Roll back messages that were appended in memory but whose atomic
@ -1905,7 +1929,7 @@ pub struct SessionManager {
provider_config: LLMProviderConfig, provider_config: LLMProviderConfig,
tools: Arc<ToolRegistry>, tools: Arc<ToolRegistry>,
skills_loader: Arc<SkillsLoader>, skills_loader: Arc<SkillsLoader>,
storage: Arc<Storage>, pub(super) storage: Arc<Storage>,
pub(super) bus: Arc<MessageBus>, pub(super) bus: Arc<MessageBus>,
memory_manager: Arc<crate::memory::MemoryManager>, memory_manager: Arc<crate::memory::MemoryManager>,
work_manager: Arc<crate::work::WorkManager>, work_manager: Arc<crate::work::WorkManager>,
@ -2276,46 +2300,6 @@ impl SessionManager {
self.work_manager.clone() self.work_manager.clone()
} }
/// 为定时任务创建一个无 session 绑定的 AgentLoop
pub fn create_cron_agent(&self) -> Result<AgentLoop, AgentError> {
let tools = self.tools.without(&["reload_config"]);
let provider = create_provider(self.provider_config.clone())
.map_err(|e| AgentError::Other(format!("failed to create cron provider: {}", e)))?;
Ok(AgentLoop::with_provider_and_tools(
Arc::from(provider),
tools,
self.provider_config.max_tool_iterations,
self.provider_config.model_id.clone(),
self.provider_config.workspace_dir.clone(),
self.provider_config.input_types.clone(),
)
.with_context_window(self.provider_config.token_limit))
}
fn create_managed_scheduled_agent(&self) -> Result<(AgentLoop, Arc<ToolRegistry>), AgentError> {
let tools = self.tools.without(&[
"send_message",
"cron_add",
"cron_update",
"cron_remove",
"cron_enable",
"cron_disable",
"reload_config",
]);
let provider = create_provider(self.provider_config.clone())
.map_err(|e| AgentError::Other(format!("failed to create scheduled provider: {e}")))?;
let agent = AgentLoop::with_provider_and_tools(
Arc::from(provider),
tools.clone(),
self.provider_config.max_tool_iterations,
self.provider_config.model_id.clone(),
self.provider_config.workspace_dir.clone(),
self.provider_config.input_types.clone(),
)
.with_context_window(self.provider_config.token_limit);
Ok((agent, tools))
}
/// 获取所有可用的斜杠命令 /// 获取所有可用的斜杠命令
pub fn get_slash_commands(&self) -> &[SlashCommand] { pub fn get_slash_commands(&self) -> &[SlashCommand] {
SLASH_COMMANDS SLASH_COMMANDS
@ -5098,103 +5082,6 @@ impl SessionManager {
} }
impl SessionManager { impl SessionManager {
///
/// Runs in a stateless manner: no session creation, no history persistence.
/// The cron system prompt instructs the LLM to deliver results via the
/// `send_message` tool, which handles both delivery and history writing
/// on the target session.
pub async fn handle_cron_message(
&self,
channel: &str,
chat_id: &str,
prompt: &str,
job_id: &str,
job_name: &str,
) -> Result<HandleResult, AgentError> {
let skills_prompt = self.skills_loader.build_skills_prompt();
let base_prompt = build_system_prompt(
&self.provider_config.workspace_dir,
&self.provider_config.model_id,
&self.tools,
);
let cron_context = format!(
"## 定时任务执行\n\n\
你正在执行定时任务「{job_name}」({job_id})。\n\
目标渠道: {channel}:{chat_id}\n\n\
规则:\n\
- 这不是聊天对话，没有用户会直接看到你的输出\n\
- 你必须使用 send_message 工具将最终结果发送到目标渠道\n\
- send_message 格式: target_chat_id=\"{channel}:{chat_id}\", content=\"消息内容\"\n\
- 可以调用其他工具收集信息、处理任务，但最终消息必须通过 send_message 发送\n\
- 只输出最终消息内容，不要输出中间思考过程或分析！"
);
let full_system_prompt =
format!("{}\n\n{}\n\n{}", base_prompt, skills_prompt, cron_context);
let history = vec![
ChatMessage::system(full_system_prompt),
ChatMessage::user(prompt),
];
let agent = self.create_cron_agent()?;
let source_session = format!("cron:{}", job_name);
let result = CURRENT_SOURCE_SESSION
.scope(Some(source_session.clone()), async {
agent
.process_with_context(
history,
ToolExecutionContext::for_session(source_session),
)
.await
})
.await
.inspect_err(|e| {
tracing::error!(error = %e, job_id = %job_id, "Cron agent processing error");
})?;
Ok(HandleResult::AgentResponse(result.final_response.content))
}
/// Execute a scheduler-managed task. The agent returns a result but cannot
/// deliver it itself; Scheduler applies the configured delivery policy.
pub async fn handle_managed_scheduled_message(
&self,
prompt: &str,
job_id: &str,
job_name: &str,
monitor: bool,
) -> Result<String, AgentError> {
let (agent, tools) = self.create_managed_scheduled_agent()?;
let base_prompt = build_system_prompt(
&self.provider_config.workspace_dir,
&self.provider_config.model_id,
&tools,
);
let skills_prompt = self.skills_loader.build_skills_prompt();
let result_contract = if monitor {
"这是无人值守巡检。完成必要检查后：一切正常且无需用户关注时，只返回 NO_REPLY[INFO]: <简短原因>；发现问题时返回简洁、可操作的告警；无法完成时返回 NO_REPLY[FAIL]: <原因>；因安全或权限拒绝时返回 NO_REPLY[REFUSE]: <原因>。不要调用 send_message，不要把不确定当作正常。"
} else {
"这是 Scheduler 托管投递的定时任务。完成任务后只返回应交付给用户的最终内容，不要调用 send_message。"
};
let system = format!(
"{base_prompt}\n\n{skills_prompt}\n\n## 定时任务执行\n任务「{job_name}」({job_id})。\n{result_contract}"
);
let history = vec![ChatMessage::system(system), ChatMessage::user(prompt)];
let source_session = format!("cron:{job_id}");
let result = CURRENT_SOURCE_SESSION
.scope(Some(source_session.clone()), async {
agent
.process_with_context(
history,
ToolExecutionContext::for_session(source_session),
)
.await
})
.await?;
Ok(result.final_response.content)
}
pub async fn clear_session_history( pub async fn clear_session_history(
&self, &self,
unified_id: &UnifiedSessionId, unified_id: &UnifiedSessionId,

2
src/storage/agent_run.rs
View File

@ -1175,7 +1175,7 @@ mod tests {
.fetch_one(storage.pool()) .fetch_one(storage.pool())
.await .await
.unwrap(); .unwrap();
assert_eq!(version, 10); assert_eq!(version, crate::storage::SCHEMA_VERSION);
for table in [ for table in [
"agent_runs", "agent_runs",
"agent_session_state", "agent_session_state",

529
src/storage/mod.rs
View File

@ -10,7 +10,10 @@ pub mod usage;
pub use context_checkpoint::{ContextCheckpoint, ContextCheckpointState, NewContextCheckpoint}; pub use context_checkpoint::{ContextCheckpoint, ContextCheckpointState, NewContextCheckpoint};
pub use error::StorageError; pub use error::StorageError;
pub use scheduler::{DeliveryPolicy, JobKind, JobRun, ScheduledJob}; pub use scheduler::{
ClaimedScheduledRun, DeliveryPolicy, JobRun, ScheduledDeliveryStatus, ScheduledJob,
ScheduledJobUpdate, ScheduledOutcomeKind, ScheduledRunCompletion, ScheduledRunStatus,
};
pub use usage::{SessionUsageTotals, TurnUsageRecord}; pub use usage::{SessionUsageTotals, TurnUsageRecord};
use sqlx::sqlite::{ use sqlx::sqlite::{
@ -20,7 +23,7 @@ use sqlx::{Pool, Row, Sqlite};
use std::path::Path; use std::path::Path;
use tokio::time::{Duration, sleep}; use tokio::time::{Duration, sleep};
const SCHEMA_VERSION: i64 = 10; const SCHEMA_VERSION: i64 = 11;
const INSERT_MESSAGE_SQL: &str = r#" const INSERT_MESSAGE_SQL: &str = r#"
INSERT INTO messages ( INSERT INTO messages (
id, session_id, seq, role, content, reasoning_content, provider_state, id, session_id, seq, role, content, reasoning_content, provider_state,
@ -29,6 +32,15 @@ const INSERT_MESSAGE_SQL: &str = r#"
) )
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
"#; "#;
const INSERT_MESSAGE_IF_ABSENT_SQL: &str = r#"
INSERT INTO messages (
id, session_id, seq, role, content, reasoning_content, provider_state,
turn_id, iteration, completion_status, client_visibility, turn_origin,
media_refs, tool_call_id, tool_name, tool_calls, source, created_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO NOTHING
"#;
pub(crate) fn insert_message_query<'a>( pub(crate) fn insert_message_query<'a>(
session_id: &'a str, session_id: &'a str,
@ -55,6 +67,31 @@ pub(crate) fn insert_message_query<'a>(
.bind(msg.created_at) .bind(msg.created_at)
} }
pub(crate) fn insert_message_if_absent_query<'a>(
session_id: &'a str,
msg: &'a crate::storage::message::MessageMeta,
) -> sqlx::query::Query<'a, Sqlite, sqlx::sqlite::SqliteArguments> {
sqlx::query(INSERT_MESSAGE_IF_ABSENT_SQL)
.bind(&msg.id)
.bind(session_id)
.bind(msg.seq)
.bind(&msg.role)
.bind(&msg.content)
.bind(&msg.reasoning_content)
.bind(&msg.provider_state)
.bind(&msg.turn_id)
.bind(msg.iteration)
.bind(msg.completion_status.as_str())
.bind(msg.client_visibility.as_str())
.bind(msg.turn_origin.as_str())
.bind(&msg.media_refs)
.bind(&msg.tool_call_id)
.bind(&msg.tool_name)
.bind(&msg.tool_calls)
.bind(&msg.source)
.bind(msg.created_at)
}
fn message_meta_from_row(row: SqliteRow) -> crate::storage::message::MessageMeta { fn message_meta_from_row(row: SqliteRow) -> crate::storage::message::MessageMeta {
let completion_status: String = row.get("completion_status"); let completion_status: String = row.get("completion_status");
crate::storage::message::MessageMeta { crate::storage::message::MessageMeta {
@ -407,7 +444,6 @@ impl Storage {
.execute(&self.pool) .execute(&self.pool)
.await?; .await?;
Self::init_scheduler_schema(&self.pool).await?;
self.migrate_schema().await?; self.migrate_schema().await?;
Ok(()) Ok(())
@ -428,7 +464,11 @@ impl Storage {
return Ok(()); return Ok(());
} }
let mut tx = self.pool.begin().await?; // Acquire the migration write lock before inspecting or modifying any
// legacy shape. This keeps the v11 rebuild on one connection and makes
// a concurrently running older Gateway fail startup cleanly instead of
// partially racing the schema migration.
let mut tx = self.pool.begin_with("BEGIN IMMEDIATE").await?;
// The legacy drops below are a pre-v8 rebuild concern: the batch // The legacy drops below are a pre-v8 rebuild concern: the batch
// "group" concept was removed in v8 and the old `background_tasks` // "group" concept was removed in v8 and the old `background_tasks`
// table in v7. Gate them on `current < 8` so a v8 -> v9 upgrade only // table in v7. Gate them on `current < 8` so a v8 -> v9 upgrade only
@ -458,6 +498,33 @@ impl Storage {
.execute(&mut *tx) .execute(&mut *tx)
.await?; .await?;
} }
let legacy_scheduler_exists: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'scheduled_jobs'",
)
.fetch_one(&mut *tx)
.await?;
let legacy_scheduler_exists = legacy_scheduler_exists == 1;
if legacy_scheduler_exists {
sqlx::query(
r#"
CREATE TABLE IF NOT EXISTS job_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
job_id TEXT NOT NULL REFERENCES scheduled_jobs(id) ON DELETE CASCADE,
started_at INTEGER NOT NULL,
finished_at INTEGER NOT NULL,
status TEXT NOT NULL,
output TEXT,
error TEXT,
duration_ms INTEGER NOT NULL,
result_kind TEXT,
delivery_status TEXT,
delivery_error TEXT
)
"#,
)
.execute(&mut *tx)
.await?;
}
for (table, column, definition) in [ for (table, column, definition) in [
("messages", "source", "source TEXT"), ("messages", "source", "source TEXT"),
("messages", "reasoning_content", "reasoning_content TEXT"), ("messages", "reasoning_content", "reasoning_content TEXT"),
@ -528,9 +595,10 @@ impl Storage {
let columns = sqlx::query(sqlx::AssertSqlSafe(pragma)) let columns = sqlx::query(sqlx::AssertSqlSafe(pragma))
.fetch_all(&mut *tx) .fetch_all(&mut *tx)
.await?; .await?;
if !columns if !columns.is_empty()
.iter() && !columns
.any(|row| row.get::<String, _>("name") == column) .iter()
.any(|row| row.get::<String, _>("name") == column)
{ {
let alter = format!("ALTER TABLE {table} ADD COLUMN {definition}"); let alter = format!("ALTER TABLE {table} ADD COLUMN {definition}");
// All identifiers and definitions come from the fixed migration list above. // All identifiers and definitions come from the fixed migration list above.
@ -562,11 +630,6 @@ impl Storage {
) )
.execute(&mut *tx) .execute(&mut *tx)
.await?; .await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_jobs_claimable ON scheduled_jobs(enabled, next_run_at, lease_until)",
)
.execute(&mut *tx)
.await?;
sqlx::query( sqlx::query(
r#" r#"
CREATE TABLE IF NOT EXISTS session_turn_usage ( CREATE TABLE IF NOT EXISTS session_turn_usage (
@ -629,6 +692,7 @@ impl Storage {
for statement in agent_run::AGENT_SCHEMA_STATEMENTS { for statement in agent_run::AGENT_SCHEMA_STATEMENTS {
sqlx::query(*statement).execute(&mut *tx).await?; sqlx::query(*statement).execute(&mut *tx).await?;
} }
scheduler::migrate_scheduler_v11(&mut tx, legacy_scheduler_exists).await?;
sqlx::query(sqlx::AssertSqlSafe(format!( sqlx::query(sqlx::AssertSqlSafe(format!(
"PRAGMA user_version = {SCHEMA_VERSION}" "PRAGMA user_version = {SCHEMA_VERSION}"
))) )))
@ -638,70 +702,6 @@ impl Storage {
Ok(()) Ok(())
} }
/// Initialize the scheduler tables (idempotent).
pub(crate) async fn init_scheduler_schema(pool: &Pool<Sqlite>) -> Result<(), StorageError> {
sqlx::query(
r#"
CREATE TABLE IF NOT EXISTS scheduled_jobs (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
schedule TEXT NOT NULL,
prompt TEXT NOT NULL,
channel TEXT NOT NULL,
chat_id TEXT NOT NULL,
model TEXT,
job_kind TEXT NOT NULL DEFAULT 'task',
delivery_policy TEXT NOT NULL DEFAULT 'direct',
enabled INTEGER NOT NULL DEFAULT 1,
delete_after_run INTEGER NOT NULL DEFAULT 0,
next_run_at INTEGER NOT NULL,
last_run_at INTEGER,
last_status TEXT,
last_error TEXT,
locked_at INTEGER,
lock_owner TEXT,
lease_until INTEGER,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)
"#,
)
.execute(pool)
.await?;
sqlx::query(
r#"
CREATE TABLE IF NOT EXISTS job_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
job_id TEXT NOT NULL REFERENCES scheduled_jobs(id) ON DELETE CASCADE,
started_at INTEGER NOT NULL,
finished_at INTEGER NOT NULL,
status TEXT NOT NULL,
output TEXT,
error TEXT,
duration_ms INTEGER NOT NULL,
result_kind TEXT,
delivery_status TEXT,
delivery_error TEXT
)
"#,
)
.execute(pool)
.await?;
sqlx::query(
"CREATE INDEX IF NOT EXISTS idx_jobs_next_run ON scheduled_jobs(enabled, next_run_at)",
)
.execute(pool)
.await?;
sqlx::query("CREATE INDEX IF NOT EXISTS idx_runs_job_id ON job_runs(job_id)")
.execute(pool)
.await?;
Ok(())
}
pub async fn append_llm_call( pub async fn append_llm_call(
&self, &self,
provider: &str, provider: &str,
@ -962,6 +962,56 @@ impl Storage {
Ok(msg.seq) Ok(msg.seq)
} }
pub async fn persist_message_if_absent(
&self,
session_id: &str,
msg: &crate::storage::message::MessageMeta,
meta: &crate::storage::session::SessionMeta,
) -> Result<bool, StorageError> {
let mut tx = self.pool.begin().await?;
let inserted = insert_message_if_absent_query(session_id, msg)
.execute(&mut *tx)
.await?
.rows_affected()
== 1;
if inserted {
sqlx::query(
r#"
INSERT INTO sessions (id, channel, chat_id, dialog_id, title, created_at,
last_active_at, message_count, routing_info, archived_at, deleted_at,
last_consolidated_at, last_compressed_message_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
title = excluded.title,
last_active_at = excluded.last_active_at,
message_count = excluded.message_count,
routing_info = excluded.routing_info,
archived_at = excluded.archived_at,
deleted_at = excluded.deleted_at,
last_consolidated_at = excluded.last_consolidated_at,
last_compressed_message_at = excluded.last_compressed_message_at
"#,
)
.bind(&meta.id)
.bind(&meta.channel)
.bind(&meta.chat_id)
.bind(&meta.dialog_id)
.bind(&meta.title)
.bind(meta.created_at)
.bind(meta.last_active_at)
.bind(meta.message_count)
.bind(&meta.routing_info)
.bind(meta.archived_at)
.bind(meta.deleted_at)
.bind(meta.last_consolidated_at)
.bind(meta.last_compressed_message_at)
.execute(&mut *tx)
.await?;
}
tx.commit().await?;
Ok(inserted)
}
/// Atomically persist all messages produced by one logical turn together /// Atomically persist all messages produced by one logical turn together
/// with the resulting session metadata. A turn is either fully visible /// with the resulting session metadata. A turn is either fully visible
/// after restart or not visible at all. /// after restart or not visible at all.
@ -1584,6 +1634,59 @@ mod tests {
(storage, dir) (storage, dir)
} }
async fn scheduler_schema_sql(storage: &Storage) -> Vec<(String, String, Option<String>)> {
sqlx::query_as::<_, (String, String, Option<String>)>(
r#"
SELECT type, name, sql FROM sqlite_master
WHERE name IN (
'scheduled_jobs', 'job_runs', 'idx_jobs_claimable',
'idx_job_runs_job_finished', 'idx_job_runs_recovery',
'idx_job_runs_delivery'
)
ORDER BY type, name
"#,
)
.fetch_all(storage.pool())
.await
.unwrap()
}
async fn create_v10_scheduler_schema(pool: &Pool<Sqlite>) {
sqlx::query(
r#"
CREATE TABLE scheduled_jobs (
id TEXT PRIMARY KEY, name TEXT NOT NULL, schedule TEXT NOT NULL,
prompt TEXT NOT NULL, channel TEXT NOT NULL, chat_id TEXT NOT NULL,
model TEXT, job_kind TEXT NOT NULL DEFAULT 'task',
delivery_policy TEXT NOT NULL DEFAULT 'direct',
enabled INTEGER NOT NULL DEFAULT 1,
delete_after_run INTEGER NOT NULL DEFAULT 0, next_run_at INTEGER NOT NULL,
last_run_at INTEGER, last_status TEXT, last_error TEXT,
locked_at INTEGER, lock_owner TEXT, lease_until INTEGER,
created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL
)
"#,
)
.execute(pool)
.await
.unwrap();
sqlx::query(
r#"
CREATE TABLE job_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
job_id TEXT NOT NULL REFERENCES scheduled_jobs(id) ON DELETE CASCADE,
started_at INTEGER NOT NULL, finished_at INTEGER NOT NULL,
status TEXT NOT NULL, output TEXT, error TEXT,
duration_ms INTEGER NOT NULL, result_kind TEXT,
delivery_status TEXT, delivery_error TEXT
)
"#,
)
.execute(pool)
.await
.unwrap();
}
#[tokio::test] #[tokio::test]
async fn sqlite_runtime_guards_are_enabled() { async fn sqlite_runtime_guards_are_enabled() {
let (storage, _dir) = create_test_storage().await; let (storage, _dir) = create_test_storage().await;
@ -1764,7 +1867,7 @@ mod tests {
} }
#[tokio::test] #[tokio::test]
async fn legacy_schema_is_migrated_without_rebuild() { async fn legacy_schema_is_migrated_to_canonical_v11() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("legacy.db"); let db_path = dir.path().join("legacy.db");
let pool = SqlitePoolOptions::new() let pool = SqlitePoolOptions::new()
@ -1860,7 +1963,14 @@ mod tests {
), ),
( (
"scheduled_jobs", "scheduled_jobs",
vec!["locked_at", "lock_owner", "lease_until"], vec![
"agent_id",
"delivery_policy",
"last_outcome",
"locked_at",
"lock_owner",
"lease_until",
],
), ),
] { ] {
let columns = sqlx::query(sqlx::AssertSqlSafe(format!("PRAGMA table_info({table})"))) let columns = sqlx::query(sqlx::AssertSqlSafe(format!("PRAGMA table_info({table})")))
@ -1876,6 +1986,25 @@ mod tests {
); );
} }
} }
let scheduler_columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
.fetch_all(storage.pool())
.await
.unwrap()
.into_iter()
.map(|row| row.get::<String, _>("name"))
.collect::<Vec<_>>();
for removed in [
"model",
"job_kind",
"delete_after_run",
"last_status",
"last_error",
] {
assert!(
!scheduler_columns.iter().any(|column| column == removed),
"legacy scheduler column survived v11: {removed}"
);
}
let schema_version: i64 = sqlx::query_scalar("PRAGMA user_version") let schema_version: i64 = sqlx::query_scalar("PRAGMA user_version")
.fetch_one(storage.pool()) .fetch_one(storage.pool())
.await .await
@ -1914,6 +2043,250 @@ mod tests {
assert_eq!(origin, "user"); assert_eq!(origin, "user");
} }
#[tokio::test]
async fn v10_scheduler_history_and_active_lease_migrate_to_v11() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("v10-scheduler.db");
let pool = SqlitePoolOptions::new()
.connect_with(
SqliteConnectOptions::new()
.filename(&db_path)
.create_if_missing(true),
)
.await
.unwrap();
create_v10_scheduler_schema(&pool).await;
sqlx::query(
r#"
INSERT INTO scheduled_jobs
(id, name, schedule, prompt, channel, chat_id, model, job_kind,
delivery_policy, enabled, next_run_at, last_run_at, last_status,
last_error, created_at, updated_at)
VALUES ('legacy-direct', 'Legacy direct', '{"type":"every","every_ms":60000}',
'check it', 'cli_chat', 'chat', 'obsolete-model', 'monitor',
'direct', 1, 2000, 1000, 'delivery_error', 'legacy diagnostic', 1, 2)
"#,
)
.execute(&pool)
.await
.unwrap();
sqlx::query(
r#"
INSERT INTO job_runs
(job_id, started_at, finished_at, status, output, error, duration_ms,
result_kind, delivery_status, delivery_error)
VALUES ('legacy-direct', 1000, 1100, 'delivery_error', 'legacy result', NULL,
100, NULL, 'failed', 'channel rejected target')
"#,
)
.execute(&pool)
.await
.unwrap();
sqlx::query(
r#"
INSERT INTO scheduled_jobs
(id, name, schedule, prompt, channel, chat_id, job_kind,
delivery_policy, enabled, next_run_at, last_run_at, last_status,
locked_at, lock_owner, lease_until, created_at, updated_at)
VALUES ('legacy-locked', 'Legacy locked', '{"type":"every","every_ms":60000}',
'check lock', 'cli_chat', 'chat', 'task', 'never', 1, 2000,
1500, NULL, 1500, 'old-owner', 900000, 1, 2)
"#,
)
.execute(&pool)
.await
.unwrap();
sqlx::query("PRAGMA user_version = 10")
.execute(&pool)
.await
.unwrap();
drop(pool);
let storage = Storage::new(&db_path).await.unwrap();
let migrated_columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
.fetch_all(storage.pool())
.await
.unwrap()
.into_iter()
.map(|row| row.get::<String, _>("name"))
.collect::<Vec<_>>();
assert!(
migrated_columns
.iter()
.any(|column| column == "last_outcome"),
"unexpected post-migration scheduler schema: {migrated_columns:?}"
);
let direct = storage.get_scheduled_job("legacy-direct").await.unwrap();
assert_eq!(
direct.delivery_policy,
crate::storage::DeliveryPolicy::Always
);
assert_eq!(
direct.last_outcome,
Some(crate::storage::ScheduledOutcomeKind::Ok)
);
let historical = storage
.list_scheduled_job_runs("legacy-direct", 10)
.await
.unwrap();
assert_eq!(historical.len(), 1);
assert_eq!(
historical[0].status,
crate::storage::ScheduledRunStatus::Completed
);
assert_eq!(
historical[0].outcome,
Some(crate::storage::ScheduledOutcomeKind::Ok)
);
assert_eq!(
historical[0].delivery_status,
crate::storage::ScheduledDeliveryStatus::Failed
);
assert_eq!(
historical[0].diagnostic.as_deref(),
Some("legacy diagnostic")
);
let recovered = storage
.list_scheduled_job_runs("legacy-locked", 10)
.await
.unwrap();
assert_eq!(recovered.len(), 1);
assert_eq!(
recovered[0].status,
crate::storage::ScheduledRunStatus::Unknown
);
assert_eq!(
recovered[0].delivery_status,
crate::storage::ScheduledDeliveryStatus::NotRequested
);
let locked = storage.get_scheduled_job("legacy-locked").await.unwrap();
assert!(locked.lock_owner.is_none());
let columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
.fetch_all(storage.pool())
.await
.unwrap()
.into_iter()
.map(|row| row.get::<String, _>("name"))
.collect::<Vec<_>>();
for removed in [
"model",
"job_kind",
"delete_after_run",
"last_status",
"last_error",
] {
assert!(!columns.iter().any(|column| column == removed));
}
let violations = sqlx::query("PRAGMA foreign_key_check")
.fetch_all(storage.pool())
.await
.unwrap();
assert!(violations.is_empty());
let fresh_dir = tempfile::tempdir().unwrap();
let fresh = Storage::new(&fresh_dir.path().join("fresh.db"))
.await
.unwrap();
assert_eq!(
scheduler_schema_sql(&storage).await,
scheduler_schema_sql(&fresh).await
);
}
#[tokio::test]
async fn invalid_v10_scheduler_data_rolls_back_v11_migration() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("invalid-v10-scheduler.db");
let pool = SqlitePoolOptions::new()
.connect_with(
SqliteConnectOptions::new()
.filename(&db_path)
.create_if_missing(true),
)
.await
.unwrap();
create_v10_scheduler_schema(&pool).await;
sqlx::query(
r#"
INSERT INTO scheduled_jobs
(id, name, schedule, prompt, channel, chat_id, delivery_policy,
enabled, next_run_at, created_at, updated_at)
VALUES ('broken', 'Broken', '{not-json}', 'check', 'cli_chat', 'chat',
'always', 1, 1000, 1, 1)
"#,
)
.execute(&pool)
.await
.unwrap();
sqlx::query("PRAGMA user_version = 10")
.execute(&pool)
.await
.unwrap();
drop(pool);
let error = match Storage::new(&db_path).await {
Ok(_) => panic!("invalid v10 scheduler data unexpectedly migrated"),
Err(error) => error,
};
assert!(matches!(error, StorageError::Migration(_)));
let pool = SqlitePoolOptions::new()
.connect_with(SqliteConnectOptions::new().filename(&db_path))
.await
.unwrap();
let version: i64 = sqlx::query_scalar("PRAGMA user_version")
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(version, 10);
let columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
.fetch_all(&pool)
.await
.unwrap()
.into_iter()
.map(|row| row.get::<String, _>("name"))
.collect::<Vec<_>>();
assert!(columns.iter().any(|column| column == "job_kind"));
assert!(!columns.iter().any(|column| column == "agent_id"));
for table in ["scheduled_jobs_v10_legacy", "job_runs_v10_legacy"] {
let exists: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?",
)
.bind(table)
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(exists, 0);
}
}
#[tokio::test]
async fn newer_schema_version_is_rejected() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("future.db");
let pool = SqlitePoolOptions::new()
.connect_with(
SqliteConnectOptions::new()
.filename(&db_path)
.create_if_missing(true),
)
.await
.unwrap();
sqlx::query("PRAGMA user_version = 12")
.execute(&pool)
.await
.unwrap();
drop(pool);
let error = match Storage::new(&db_path).await {
Ok(_) => panic!("newer schema version unexpectedly opened"),
Err(error) => error,
};
assert!(matches!(error, StorageError::Migration(_)));
}
#[tokio::test] #[tokio::test]
async fn v3_migration_preserves_existing_reasoning_and_defaults_completion() { async fn v3_migration_preserves_existing_reasoning_and_defaults_completion() {
let dir = tempfile::tempdir().unwrap(); let dir = tempfile::tempdir().unwrap();

2630
src/storage/scheduler.rs
View File

File diff suppressed because it is too large Load Diff

191
src/tools/complete_scheduled_run.rs Normal file
View File

@ -0,0 +1,191 @@
use async_trait::async_trait;
use serde_json::{Value, json};
use crate::storage::ScheduledOutcomeKind;
use crate::tools::{ScheduledOutcome, Tool, ToolExecutionContext, ToolOutput, ToolResult};
pub struct CompleteScheduledRunTool;
impl CompleteScheduledRunTool {
pub fn new() -> Self {
Self
}
}
impl Default for CompleteScheduledRunTool {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl Tool for CompleteScheduledRunTool {
fn name(&self) -> &str {
"complete_scheduled_run"
}
fn description(&self) -> &str {
"Submit the single structured final outcome (ok, alert, failed, or refused) of an unattended scheduled run and end the run immediately."
}
fn parameters_schema(&self) -> Value {
json!({
"type": "object",
"properties": {
"outcome": {
"type": "string",
"enum": ["ok", "alert", "failed", "refused"],
"description": "ok: completed with nothing needing attention; alert: completed with actionable findings; failed: did not complete reliably; refused: denied for permission or safety reasons"
},
"message": {
"type": "string",
"minLength": 1,
"maxLength": 16384,
"description": "The user-facing result or notification body"
}
},
"required": ["outcome", "message"],
"additionalProperties": false
})
}
fn runtime_injected(&self) -> bool {
true
}
fn exclusive(&self) -> bool {
true
}
async fn execute(&self, args: Value) -> anyhow::Result<ToolResult> {
self.execute_with_context(&ToolExecutionContext::default(), args)
.await
.map(|output| output.result)
}
async fn execute_with_context(
&self,
context: &ToolExecutionContext,
args: Value,
) -> anyhow::Result<ToolOutput> {
let Some(sink) = context.scheduled_completion.as_ref() else {
return Ok(failure(
"complete_scheduled_run is only available to the top-level scheduled Agent",
)
.into());
};
if !context.execution_origin.is_scheduled() {
return Ok(failure("scheduled completion context is invalid").into());
}
let Some(object) = args.as_object() else {
return Ok(failure("arguments must be an object").into());
};
if object
.keys()
.any(|key| key != "outcome" && key != "message")
{
return Ok(failure("unknown complete_scheduled_run argument").into());
}
let kind = match args.get("outcome").and_then(Value::as_str) {
Some("ok") => ScheduledOutcomeKind::Ok,
Some("alert") => ScheduledOutcomeKind::Alert,
Some("failed") => ScheduledOutcomeKind::Failed,
Some("refused") => ScheduledOutcomeKind::Refused,
Some(other) => return Ok(failure(format!("invalid scheduled outcome: {other}")).into()),
None => return Ok(failure("outcome is required").into()),
};
let message = args
.get("message")
.and_then(Value::as_str)
.unwrap_or_default()
.trim();
if message.is_empty() {
return Ok(failure("message must not be empty").into());
}
if message.chars().count() > 16_384 {
return Ok(failure("message exceeds 16384 characters").into());
}
let message = message.to_string();
if let Err(error) = sink.submit(ScheduledOutcome {
kind,
message: message.clone(),
}) {
return Ok(failure(error).into());
}
Ok(ToolResult {
success: true,
output: format!("scheduled run completed with outcome={}", kind.as_str()),
error: None,
}
.into())
}
}
fn failure(error: impl Into<String>) -> ToolResult {
ToolResult {
success: false,
output: String::new(),
error: Some(error.into()),
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tools::{ExecutionOrigin, ScheduledCompletionSink};
use std::sync::Arc;
#[tokio::test]
async fn submits_exactly_once_in_scheduled_context() {
let sink = Arc::new(ScheduledCompletionSink::default());
let context = ToolExecutionContext::for_session("scheduled-run:1")
.with_execution_origin(ExecutionOrigin::Scheduled { job_run_id: 1 })
.with_scheduled_completion(sink.clone());
let tool = CompleteScheduledRunTool::new();
let first = tool
.execute_with_context(&context, json!({"outcome":"ok","message":"healthy"}))
.await
.unwrap();
assert!(first.result.success);
assert_eq!(sink.outcome().unwrap().message, "healthy");
let second = tool
.execute_with_context(&context, json!({"outcome":"alert","message":"again"}))
.await
.unwrap();
assert!(!second.result.success);
}
#[tokio::test]
async fn rejects_empty_extra_and_oversized_arguments() {
let sink = Arc::new(ScheduledCompletionSink::default());
let context = ToolExecutionContext::for_session("scheduled-run:1")
.with_execution_origin(ExecutionOrigin::Scheduled { job_run_id: 1 })
.with_scheduled_completion(sink);
let tool = CompleteScheduledRunTool::new();
for args in [
json!({"outcome":"ok","message":" "}),
json!({"outcome":"ok","message":"fine","notify":false}),
json!({"outcome":"unknown","message":"fine"}),
] {
assert!(
!tool
.execute_with_context(&context, args)
.await
.unwrap()
.result
.success
);
}
assert!(
!tool
.execute_with_context(
&context,
json!({"outcome":"ok","message":"x".repeat(16_385)}),
)
.await
.unwrap()
.result
.success
);
}
}

1135
src/tools/cron.rs
View File

File diff suppressed because it is too large Load Diff

10
src/tools/delegate.rs
View File

@ -155,7 +155,7 @@ impl DelegateTool {
args: &Value, args: &Value,
context: &ToolExecutionContext, context: &ToolExecutionContext,
) -> anyhow::Result<ToolResult> { ) -> anyhow::Result<ToolResult> {
let mode = match args let requested_mode = match args
.get("mode") .get("mode")
.and_then(Value::as_str) .and_then(Value::as_str)
.unwrap_or("foreground") .unwrap_or("foreground")
@ -168,6 +168,13 @@ impl DelegateTool {
))); )));
} }
}; };
let background_downgraded =
requested_mode == ExecutionMode::Background && context.execution_origin.is_scheduled();
let mode = if background_downgraded {
ExecutionMode::Foreground
} else {
requested_mode
};
let task_values: Vec<&Value> = match args.get("tasks").and_then(Value::as_array) { let task_values: Vec<&Value> = match args.get("tasks").and_then(Value::as_array) {
Some(tasks) if !tasks.is_empty() => tasks.iter().collect(), Some(tasks) if !tasks.is_empty() => tasks.iter().collect(),
Some(_) => return Ok(failure("tasks must not be empty")), Some(_) => return Ok(failure("tasks must not be empty")),
@ -248,6 +255,7 @@ impl DelegateTool {
success: all_completed, success: all_completed,
output: serde_json::to_string(&json!({ output: serde_json::to_string(&json!({
"status": if all_completed { "completed" } else { "partial" }, "status": if all_completed { "completed" } else { "partial" },
"background_downgraded": background_downgraded,
"results": payload "results": payload
}))?, }))?,
error: None, error: None,

7
src/tools/mod.rs
View File

@ -3,6 +3,7 @@ pub mod bash;
pub mod browser; pub mod browser;
pub mod calculator; pub mod calculator;
pub mod chat_manager; pub mod chat_manager;
pub mod complete_scheduled_run;
pub mod content_search; pub mod content_search;
pub mod cron; pub mod cron;
pub mod delegate; pub mod delegate;
@ -32,6 +33,7 @@ pub use bash::BashTool;
pub use browser::{BrowserProfilesTool, BrowserTool}; pub use browser::{BrowserProfilesTool, BrowserTool};
pub use calculator::CalculatorTool; pub use calculator::CalculatorTool;
pub use chat_manager::ChatManagerTool; pub use chat_manager::ChatManagerTool;
pub use complete_scheduled_run::CompleteScheduledRunTool;
pub use content_search::ContentSearchTool; pub use content_search::ContentSearchTool;
pub use delegate::DelegateTool; pub use delegate::DelegateTool;
pub use emit_signal::EmitSignalTool; pub use emit_signal::EmitSignalTool;
@ -50,8 +52,9 @@ pub use reload_config::ReloadConfigTool;
pub use send_message::SendMessageTool; pub use send_message::SendMessageTool;
pub use todo::TodoTool; pub use todo::TodoTool;
pub use traits::{ pub use traits::{
OutboundDelivery, OutboundMessenger, ProcessedToolOutput, Tool, ToolArtifact, ExecutionOrigin, OutboundDelivery, OutboundMessenger, ProcessedToolOutput,
ToolArtifactAudience, ToolExecutionContext, ToolOutput, ToolOutputProcessor, ToolResult, ScheduledCompletionSink, ScheduledOutcome, Tool, ToolArtifact, ToolArtifactAudience,
ToolExecutionContext, ToolOutput, ToolOutputProcessor, ToolResult,
}; };
pub use web_fetch::WebFetchTool; pub use web_fetch::WebFetchTool;

73
src/tools/traits.rs
View File

@ -1,6 +1,60 @@
use crate::bus::{MediaItem, MediaRef, MessageSource}; use crate::bus::{MediaItem, MediaRef, MessageSource};
use async_trait::async_trait; use async_trait::async_trait;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum ExecutionOrigin {
#[default]
Interactive,
Scheduled {
job_run_id: i64,
},
}
impl ExecutionOrigin {
pub fn is_scheduled(self) -> bool {
matches!(self, Self::Scheduled { .. })
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ScheduledOutcome {
pub kind: crate::storage::ScheduledOutcomeKind,
pub message: String,
}
#[derive(Debug, Default)]
pub struct ScheduledCompletionSink {
outcome: std::sync::Mutex<Option<ScheduledOutcome>>,
}
impl ScheduledCompletionSink {
pub fn submit(&self, outcome: ScheduledOutcome) -> Result<(), &'static str> {
let mut guard = self
.outcome
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner());
if guard.is_some() {
return Err("scheduled result was already submitted");
}
*guard = Some(outcome);
Ok(())
}
pub fn outcome(&self) -> Option<ScheduledOutcome> {
self.outcome
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.clone()
}
pub fn is_completed(&self) -> bool {
self.outcome
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.is_some()
}
}
/// Session identity supplied by the runtime for tools that own external state. /// Session identity supplied by the runtime for tools that own external state.
/// Ordinary stateless tools can ignore it through the default trait method. /// Ordinary stateless tools can ignore it through the default trait method.
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@ -10,6 +64,8 @@ pub struct ToolExecutionContext {
pub agent: Option<std::sync::Arc<crate::agent::AgentExecutionContext>>, pub agent: Option<std::sync::Arc<crate::agent::AgentExecutionContext>>,
pub cancellation: tokio_util::sync::CancellationToken, pub cancellation: tokio_util::sync::CancellationToken,
pub execution_gate: Option<std::sync::Arc<crate::agent::gate::ExecutionGate>>, pub execution_gate: Option<std::sync::Arc<crate::agent::gate::ExecutionGate>>,
pub execution_origin: ExecutionOrigin,
pub scheduled_completion: Option<std::sync::Arc<ScheduledCompletionSink>>,
} }
impl Default for ToolExecutionContext { impl Default for ToolExecutionContext {
@ -20,6 +76,8 @@ impl Default for ToolExecutionContext {
agent: None, agent: None,
cancellation: tokio_util::sync::CancellationToken::new(), cancellation: tokio_util::sync::CancellationToken::new(),
execution_gate: None, execution_gate: None,
execution_origin: ExecutionOrigin::Interactive,
scheduled_completion: None,
} }
} }
} }
@ -32,6 +90,8 @@ impl ToolExecutionContext {
agent: None, agent: None,
cancellation: tokio_util::sync::CancellationToken::new(), cancellation: tokio_util::sync::CancellationToken::new(),
execution_gate: None, execution_gate: None,
execution_origin: ExecutionOrigin::Interactive,
scheduled_completion: None,
} }
} }
@ -60,6 +120,19 @@ impl ToolExecutionContext {
self.execution_gate = Some(gate); self.execution_gate = Some(gate);
self self
} }
pub fn with_execution_origin(mut self, execution_origin: ExecutionOrigin) -> Self {
self.execution_origin = execution_origin;
self
}
pub fn with_scheduled_completion(
mut self,
sink: std::sync::Arc<ScheduledCompletionSink>,
) -> Self {
self.scheduled_completion = Some(sink);
self
}
} }
#[derive(Debug, Clone)] #[derive(Debug, Clone)]

4
webui/package-lock.json generated
View File

@ -1,12 +1,12 @@
{ {
"name": "picobot-webui", "name": "picobot-webui",
"version": "1.21.0", "version": "1.22.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "picobot-webui", "name": "picobot-webui",
"version": "1.21.0", "version": "1.22.0",
"dependencies": { "dependencies": {
"bits-ui": "^2.0.0", "bits-ui": "^2.0.0",
"dompurify": "^3.4.12", "dompurify": "^3.4.12",

2
webui/package.json
View File

@ -1,7 +1,7 @@
{ {
"name": "picobot-webui", "name": "picobot-webui",
"private": true, "private": true,
"version": "1.21.0", "version": "1.22.0",
"type": "module", "type": "module",
"engines": { "engines": {
"node": ">=20" "node": ">=20"

6
webui/src/lib/StatusBadge.svelte
View File

@ -2,11 +2,11 @@
let { status = "unknown" } = $props(); let { status = "unknown" } = $props();
const normalized = $derived(String(status).toLowerCase()); const normalized = $derived(String(status).toLowerCase());
const tone = $derived( const tone = $derived(
["completed", "success", "ok", "enabled"].includes(normalized) ["completed", "success", "ok", "enabled", "delivered"].includes(normalized)
? "ok" ? "ok"
: ["failed", "error", "cancelled", "disabled"].includes(normalized) : ["failed", "error", "cancelled", "disabled", "refused", "timed_out"].includes(normalized)
? "fail" ? "fail"
: ["running", "pending"].includes(normalized) ? "run" : "" : ["running", "pending", "delivering", "alert", "unknown", "interrupted"].includes(normalized) ? "run" : ""
); );
</script> </script>

44
webui/src/pages/TasksPage.svelte
View File

@ -42,11 +42,23 @@
function dotColor(status) { function dotColor(status) {
if (status === "completed" || status === "success" || status === "ok") return "var(--signal)"; if (status === "completed" || status === "success" || status === "ok") return "var(--signal)";
if (status === "timeout") return "var(--accent)"; if (status === "timed_out" || status === "unknown" || status === "interrupted") return "var(--accent)";
if (status === "running") return "var(--info)"; if (status === "running") return "var(--info)";
return "var(--danger)"; return "var(--danger)";
} }
function scheduleLabel(schedule) {
if (!schedule) return "—";
if (schedule.type === "at") return `单次 · ${formatTime(schedule.at)}`;
if (schedule.type === "every") return `每 ${Math.max(1, Math.round(schedule.every_ms / 60000))} 分钟`;
if (schedule.type === "cron") return `${schedule.expr}${schedule.tz ? ` · ${schedule.tz}` : ""}`;
return schedule.type || "—";
}
function deliveryLabel(policy) {
return { always: "始终通知", on_alert: "异常通知", never: "从不通知" }[policy] || policy;
}
onMount(() => { onMount(() => {
load(); load();
const timer = setInterval(() => { tick += 1; }, 30000); const timer = setInterval(() => { tick += 1; }, 30000);
@ -58,7 +70,7 @@
<div class="toolbar"> <div class="toolbar">
<div> <div>
<h2 style="margin:0">定时任务</h2> <h2 style="margin:0">定时任务</h2>
<p style="margin:2px 0 0;color:var(--muted);font-size:12px">管理定时任务与巡检；后台子代理运行请到「子代理」页面查看。</p> <p style="margin:2px 0 0;color:var(--muted);font-size:12px">查看统一的定时执行、结构化结果与通知状态；Agent 运行审计请到「子代理」页面。</p>
</div> </div>
<button class="secondary" onclick={load}><Icon name="refresh" size={16} />刷新</button> <button class="secondary" onclick={load}><Icon name="refresh" size={16} />刷新</button>
</div> </div>
@ -73,8 +85,9 @@
<h3>{job.name}</h3> <h3>{job.name}</h3>
<p>{job.prompt}</p> <p>{job.prompt}</p>
<div class="meta"> <div class="meta">
<span class="mono cron">{job.cron}</span> <span class="mono cron">{scheduleLabel(job.schedule)}</span>
<span>{job.job_kind === "monitor" ? "巡检" : "任务"} · {job.delivery_policy}</span> <span>Agent：{job.agent_id || "Root"}</span>
<span>投递：{deliveryLabel(job.delivery_policy)}</span>
<span>{job.channel} · {job.chat_id}</span> <span>{job.channel} · {job.chat_id}</span>
<span>下次 {countdown(job.next_run_at)}</span> <span>下次 {countdown(job.next_run_at)}</span>
<span>上次 {formatTime(job.last_run_at)}</span> <span>上次 {formatTime(job.last_run_at)}</span>
@ -87,9 +100,23 @@
</div> </div>
{/if} {/if}
</div> </div>
<StatusBadge status={job.enabled ? (job.last_status || "enabled") : "disabled"} /> <StatusBadge status={job.enabled ? (job.last_outcome || "enabled") : "disabled"} />
</div> </div>
{#if runs[job.id]?.length}<div class="details">{#each runs[job.id] as run}<div class="card-row run-row"><span class="meta">{formatTime(run.finished_at)} · {run.duration_ms}ms{run.result_kind ? ` · ${run.result_kind}` : ""}{run.delivery_status ? ` · ${run.delivery_status}` : ""}</span><StatusBadge status={run.status} /></div>{/each}</div>{/if} {#if runs[job.id]?.length}
<div class="details">
{#each runs[job.id] as run}
<details class="run-detail">
<summary class="run-row">
<span class="meta">{formatTime(run.finished_at || run.started_at || run.scheduled_for)} · {run.duration_ms ?? "—"}ms · outcome={run.outcome || "—"} · delivery={run.delivery_status} ({run.delivery_attempts})</span>
<StatusBadge status={run.status} />
</summary>
{#if run.message}<p><b>结果</b><br />{run.message}</p>{/if}
{#if run.diagnostic}<p class="diagnostic"><b>诊断</b><br />{run.diagnostic}</p>{/if}
{#if run.delivery_error}<p class="diagnostic"><b>投递错误</b><br />{run.delivery_error}</p>{/if}
</details>
{/each}
</div>
{/if}
</article> </article>
{:else}<div class="empty-card">暂无定时任务</div>{/each} {:else}<div class="empty-card">暂无定时任务</div>{/each}
{/if} {/if}
@ -100,4 +127,9 @@
.cron { font-size: 12px; color: var(--text-soft); background: var(--code-bg); padding: 1px 6px; border-radius: 4px; border: 1px solid var(--line); font-variant-numeric: tabular-nums; } .cron { font-size: 12px; color: var(--text-soft); background: var(--code-bg); padding: 1px 6px; border-radius: 4px; border: 1px solid var(--line); font-variant-numeric: tabular-nums; }
.status-dots { display: flex; gap: 4px; margin-top: 6px; align-items: center; } .status-dots { display: flex; gap: 4px; margin-top: 6px; align-items: center; }
.run-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; } .run-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
.run-detail { padding: 7px 0; border-top: 1px solid var(--line); }
.run-detail:first-child { border-top: 0; }
.run-detail summary { cursor: pointer; justify-content: space-between; }
.run-detail p { margin: 8px 0 0; white-space: pre-wrap; overflow-wrap: anywhere; font-size: 12px; color: var(--text-soft); }
.run-detail .diagnostic { color: var(--danger); }
</style> </style>