Compare commits
No commits in common. "334e98d8941596f641403b2f3a319892fa0bb2d7" and "9ca78864a25616d090732b7e1ea2df131ad2e00b" have entirely different histories.
334e98d894
...
9ca78864a2
@ -92,7 +92,7 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
|
||||
- **TurnController** is the only owner of active Turn state; snapshots are complete latest-wins values, not token queues, and `Completed` is published only after atomic persistence succeeds
|
||||
- **DeliveryCoordinator** projects active Turn snapshots without mutating history; it owns `TurnSink` lifecycle but no platform message IDs, which remain private to each sink
|
||||
- **WorkManager** owns the single active plan per session, item state transitions, plan versions, and plan-change events; plans are optional and absent from ordinary chat context
|
||||
- **Scheduler** owns one unified Scheduled Run path: claim-time JobRun snapshots, isolated Root/named Agent execution, exactly-once `complete_scheduled_run`, structured outcome, and policy-driven outbox delivery. Scheduled origin propagates to descendants, forces background delegation to foreground, and disables direct messaging, signals, Inbox completion slots, and cron/config management tools; `on_alert` suppresses only structured `ok`
|
||||
- **Scheduler** supports legacy direct-delivery jobs and managed `task`/`monitor` jobs; managed agents cannot call `send_message`, and `on_alert` suppresses only healthy informational results
|
||||
- **AgentLoop** is stateless across turns; it receives prepared history, drains same-Turn steering only at safe model boundaries, calls LLM providers, executes tools, and returns one result
|
||||
- **Context overflow recovery** is type-driven: before tool progress Session may commit one checkpoint and retry once; after any tool batch AgentLoop may retry the current Provider step once from its in-memory transcript, preserving current tool calls/results, and Session must never restart that Turn from durable history
|
||||
- **Context compaction** keeps `messages` append-only and uses one active checkpoint per Session (`summary + first_retained_seq`) for deterministic Provider projection; `/compact`, Turn-boundary auto compaction, and overflow share the same compactor/CAS commit path, Session restoration never derives context from Timeline or calls a Provider, the Model `token_limit` (default 128K) is the hard window ceiling and an optional Agent `token_limit` can only narrow it via `min(agent, model)`, summary input is bounded from that effective window rather than a fixed cap, and the only automatic threshold is `context_tokens > context_window - effective_reserve`
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "picobot"
|
||||
version = "1.22.0"
|
||||
version = "1.21.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
|
||||
11
README.md
11
README.md
@ -17,7 +17,7 @@ PicoBot 是一个用 Rust 编写的个人 AI 助手运行时。它在本地启
|
||||
- 将同一套 Agent 能力接入飞书/Lark，并可选用单张卡片实时更新回复。
|
||||
- 让 Agent 使用本地文件、Shell、搜索、HTTP、浏览器、MCP 工具完成任务。
|
||||
- 把长期偏好、事实和历史摘要存成可检索记忆。
|
||||
- 用 Cron 运行隔离的 Root 或命名 Agent，以结构化结果决定始终通知、异常通知或静默记录。
|
||||
- 用 Cron 定时执行任务，并把结果发回目标渠道。
|
||||
- 通过 Skills 为 Agent 注入项目知识和专用操作指南。
|
||||
|
||||
## 快速开始
|
||||
@ -152,7 +152,7 @@ picobot health --json
|
||||
|
||||
缺少核心或当前配置要求的依赖时退出码为 `1`；`rg` / `fd` 等有回退实现的加速项只会标记为 `DEGRADED`。运行中的 Gateway 也提供 `/health` 斜杠命令，Agent 可调用同名 `health` 工具，WebUI 的“配置 → 健康检查”可显示相同的结构化结果并手动复查；这些入口共享同一套只读检查逻辑。
|
||||
|
||||
Debian/Ubuntu 将同一个 fd 程序安装为 `fdfind`，两者都视为首选文件搜索后端；只有退回传统 `find` 时才提示性能警告。启用浏览器工具后，Health 除了检查 agent-browser 版本和浏览器路径，还会在隔离的临时 socket namespace 中执行完整离线 doctor，分别报告浏览器安装、真实 headless 启动和运行环境，因此可发现“文件存在但 Chrome 无法启动”或缺少 Linux 共享库等问题。Gateway 内按需检查还会报告定时任务的无效 Agent/渠道引用、投递积压、最近失败/超时/unknown、静默 unknown 和执行周期覆盖；Health 不会触发任务或连接 Provider。
|
||||
Debian/Ubuntu 将同一个 fd 程序安装为 `fdfind`，两者都视为首选文件搜索后端；只有退回传统 `find` 时才提示性能警告。启用浏览器工具后，Health 除了检查 agent-browser 版本和浏览器路径，还会在隔离的临时 socket namespace 中执行完整离线 doctor，分别报告浏览器安装、真实 headless 启动和运行环境，因此可发现“文件存在但 Chrome 无法启动”或缺少 Linux 共享库等问题。
|
||||
|
||||
### 5.3 使用 WebUI
|
||||
|
||||
@ -273,7 +273,7 @@ WebUI/TUI 上传文件默认保存到 `~/.picobot/media/cli_chat`，单文件上
|
||||
| `delivery` | 活动 Turn 的展示过滤、latest-wins 节流、终态投递与 TurnSink 生命周期 |
|
||||
| `tools` | Agent 可调用工具集合 |
|
||||
| `storage` | SQLite schema、CRUD、消息和任务持久化 |
|
||||
| `scheduler` | 原子领取 occurrence，运行隔离的 Scheduled Agent，并通过持久化 outbox 按策略投递结构化结果 |
|
||||
| `scheduler` | 领取定时任务，执行普通/巡检 Agent，并按投递策略记录或发送结果 |
|
||||
| `work` | 管理 session 级单 active plan、并行子项状态和 WebSocket 变更事件 |
|
||||
| `skills` | 加载 Skill，并把 Skill 指南注入系统提示 |
|
||||
| `mcp` | 连接 MCP Server，将远端工具包装成普通 Tool |
|
||||
@ -329,7 +329,7 @@ PicoBot 有两类记忆：
|
||||
| Knowledge | 偏好、事实、项目规则、长期可复用信息 | 长期保留，手动删除 |
|
||||
| Timeline | 长对话压缩后的历史摘要 | 默认保留 90 天 |
|
||||
|
||||
每轮处理用户消息时，MemoryManager 会按用户输入召回 Knowledge，并作为运行时上下文附加到本轮用户消息。当前召回上限固定为 5；`memory.recall_limit` 已支持解析但尚未接入 worker。长会话使用一个活动 checkpoint：累计摘要加 `first_retained_seq` 之后的原始消息尾部构成模型上下文，原始消息、工具调用结果、ID 和 seq 均不会被压缩改写。旧工具结果会保留在原始历史中，但 checkpoint 边界推进后不再永久占用 Provider 上下文。成功的语义摘要还会 best-effort 保存为 Timeline，供 `timeline_recall` 检索；Timeline 不参与会话恢复正确性。Scheduler 默认创建一个每日维护任务，按 `memory.timeline_retention_days` 清理过期 Timeline；结果通过 `complete_scheduled_run` 结构化提交，Knowledge 不会被自动删除。
|
||||
每轮处理用户消息时，MemoryManager 会按用户输入召回 Knowledge，并作为运行时上下文附加到本轮用户消息。当前召回上限固定为 5；`memory.recall_limit` 已支持解析但尚未接入 worker。长会话使用一个活动 checkpoint：累计摘要加 `first_retained_seq` 之后的原始消息尾部构成模型上下文，原始消息、工具调用结果、ID 和 seq 均不会被压缩改写。旧工具结果会保留在原始历史中，但 checkpoint 边界推进后不再永久占用 Provider 上下文。成功的语义摘要还会 best-effort 保存为 Timeline，供 `timeline_recall` 检索；Timeline 不参与会话恢复正确性。Scheduler 默认创建一个每日维护巡检，按 `memory.timeline_retention_days` 清理过期 Timeline；Knowledge 不会被自动删除。
|
||||
|
||||
模型的 `models.<name>.token_limit` 给出上下文窗口上限，未配置时默认为 128,000；Agent 的 `agents.<name>.token_limit` 是可选的收紧上限，两者都有配置时有效窗口取二者最小值，因此 Agent 不能扩大模型窗口。自动压缩使用保留量阈值 `context_tokens > context_window - effective_reserve`，默认 reserve 为 16,384 tokens，并尽量原样保留最近 20,000 tokens。小窗口会自动把 reserve 限制为窗口的一半、把近期保留量限制为有效阈值的一半。摘要请求不使用固定 32K 输入上限，而是按有效窗口扣除摘要输出、提示词和安全余量；超大历史只在摘要请求副本中按“已有 checkpoint + 最新消息优先”生成有界 head/tail 转录，SQLite 原文不变。手动 `/compact` 跳过自动阈值；换成小模型后若发送前预检已发现硬超限，或首次请求返回真实 context overflow，语义摘要不可用时才使用明确标记的确定性降级裁剪，正式请求最多重试一次。若 overflow 发生在工具已经执行之后，AgentLoop 只在当前内存转录上裁掉旧完整 Turn 并重试当前模型步骤一次，不会从数据库历史重跑工具。
|
||||
|
||||
@ -353,8 +353,7 @@ PicoBot 有两类记忆：
|
||||
| `todo` | 为复杂、多轮任务创建并更新当前 session 的持久化计划 |
|
||||
| `send_message` | 向指定渠道或当前会话发送消息，可附带文件/截图；WebUI/TUI 当前 Turn 的附件并入最终回复 |
|
||||
| `chat_manager` | 查看渠道、会话和历史消息 |
|
||||
| `cron_add/list/remove/enable/disable/update` | 管理定时任务；`agent_id` 选择 Root/命名 Agent，`delivery_policy` 支持 `always/on_alert/never` |
|
||||
| `cron_runs` | 查询定时任务的结构化执行结果、诊断和投递状态，包括静默任务 |
|
||||
| `cron_add/list/remove/enable/disable/update` | 管理定时任务 |
|
||||
| `routine_maintenance` | 安全清理超过保留期的 Timeline，不删除 Knowledge |
|
||||
| `health` | 检查核心、配置相关和可选运行依赖 |
|
||||
| `browser` | 可选 agent-browser 浏览器自动化；默认按 dialog 临时使用，长期任务可用 `persistent_id` 复用个人 Profile |
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
本文档描述 PicoBot 当前实现的运行时边界、数据流、并发模型和演进约束。它面向维护者和后续参与改进的 Agent，是代码架构的主入口；行为细节仍以代码和测试为最终依据。
|
||||
|
||||
流式模型输出、reasoning 展示、活动 Turn 快照和 Channel 实时投递的详细设计与取舍见 [STREAMING_TURN_DESIGN.md](STREAMING_TURN_DESIGN.md)。用户输入路由、Session 执行拆分、终态投递确认和历史增量校准的重构方案见 [MESSAGE_FLOW_REFACTOR_DESIGN.md](MESSAGE_FLOW_REFACTOR_DESIGN.md)。已实施的 checkpoint 上下文压缩、pi 风格 reserve 阈值、统一编排和 overflow 失败语义见 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md)。配置运行代、重载边界和失败语义见 [CONFIG_HOT_RELOAD_DESIGN.md](CONFIG_HOT_RELOAD_DESIGN.md)。具名子 Agent、委托图、后台收件箱、结果传递机制与 `queue`/`steer` 信号的设计见 [SUB_AGENT_DESIGN.md](SUB_AGENT_DESIGN.md)。已实施的统一 Scheduled Run、结构化终结协议、中央投递和 v11 数据库迁移见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。
|
||||
流式模型输出、reasoning 展示、活动 Turn 快照和 Channel 实时投递的详细设计与取舍见 [STREAMING_TURN_DESIGN.md](STREAMING_TURN_DESIGN.md)。用户输入路由、Session 执行拆分、终态投递确认和历史增量校准的重构方案见 [MESSAGE_FLOW_REFACTOR_DESIGN.md](MESSAGE_FLOW_REFACTOR_DESIGN.md)。已实施的 checkpoint 上下文压缩、pi 风格 reserve 阈值、统一编排和 overflow 失败语义见 [CONTEXT_COMPACTION_DESIGN.md](CONTEXT_COMPACTION_DESIGN.md)。配置运行代、重载边界和失败语义见 [CONFIG_HOT_RELOAD_DESIGN.md](CONFIG_HOT_RELOAD_DESIGN.md)。具名子 Agent、委托图、后台收件箱、结果传递机制与 `queue`/`steer` 信号的设计见 [SUB_AGENT_DESIGN.md](SUB_AGENT_DESIGN.md)。
|
||||
|
||||
## 1. 设计目标
|
||||
|
||||
@ -78,7 +78,7 @@ flowchart LR
|
||||
| `health` | 聚合只读依赖检查，供 CLI、Tool 与 slash command 复用 | 安装、修复或连接 Provider |
|
||||
| `storage` | SQLite schema、迁移、原子 CRUD | 运行时调度策略 |
|
||||
| `memory` | Knowledge/Timeline 的存取与召回 | 直接驱动消息发送 |
|
||||
| `scheduler` | 原子领取 occurrence、运行隔离的 Root/命名 Agent、提交结构化结果并 drain 持久化投递 outbox | 解析模型自然语言、绕过 Bus 直接调用 Channel |
|
||||
| `scheduler` | 领取到期任务、执行普通/巡检 Agent、应用投递策略、原子记录结果 | 复用聊天会话历史、直接感知 Channel |
|
||||
| `work` | session 级单 active plan、并行子项状态机、版本和变更事件 | 执行模型调用、持有 Channel/WebSocket |
|
||||
| `task_supervisor` | 后台任务注册、取消、限时回收 | 业务级重试和结果语义 |
|
||||
|
||||
@ -171,18 +171,6 @@ sequenceDiagram
|
||||
|
||||
不要把“已进入 Bus”误认为“外部渠道已收到”。需要确认语义时必须使用 `deliver_outbound`。
|
||||
|
||||
### Scheduled Run
|
||||
|
||||
Scheduler 不复用聊天历史，也不根据模型正文猜测是否通知。一次到期状态在同一 SQLite 事务中取得 Job 租约、插入 `job_runs(status=claimed)`、快照 Agent/投递目标/策略，并提前推进 recurring `next_run_at`；`At` 在 claim 时立即禁用。事件循环以独立有界 JoinSet 执行 Run 和 drain delivery，长任务不阻塞其他领取或通知。
|
||||
|
||||
每次 Run 通过 `AgentCoordinator` 建立顶层 `agent_runs` 审计记录并执行隔离的 Root 或命名 Agent。Scheduled origin 贯穿所有后代，但只有顶层获得 exactly-once `complete_scheduled_run` sink；后代不继承 sink。Scheduled Agent 不获得 `send_message`、cron/config 管理或 `emit_signal`，不创建 Inbox completion slot，任何 background 委托都收敛为 foreground。普通最终文本不代表成功；没有提交 `ok/alert/failed/refused` 之一即 fail-closed。
|
||||
|
||||
顶层 AgentRun 终态、JobRun 的 lifecycle/outcome/message/diagnostic、Job 最近摘要、初始 delivery status 和租约释放在一个事务中提交。`always` 投递所有 outcome，`on_alert` 只抑制结构化 `ok`，`never` 始终只记录。`job_runs` 同时作为轻量 outbox：`pending → delivering → delivered/failed`，瞬态错误最多进行三次持久化尝试；OutboundDispatcher 返回清洗后的类型化回执，Scheduler 不解析错误字符串。
|
||||
|
||||
首次投递把目标 dialog 固定到 `target_session_id`，并先用稳定消息 ID `scheduled:<job_run_id>` 幂等写入本地历史，再调用 `MessageBus::deliver_outbound`。发送成功但 ack 提交前崩溃允许带稳定 metadata 的重复通知，不能为避免重复而丢失告警。启动恢复把遗留 claimed/running JobRun 原子收敛为 `unknown+unknown`、关联非终态 AgentRun 收敛为 interrupted，并按 claim-time 策略决定是否进入 outbox；已推进的 occurrence 不自动重跑。完整状态矩阵、v11 schema 和迁移规则见 [SCHEDULED_RUN_DESIGN.md](SCHEDULED_RUN_DESIGN.md)。
|
||||
|
||||
按需 Health 检查只读查询任务引用、投递积压、最近失败/超时/unknown、`never+unknown`、Every 周期被执行时长覆盖以及不可计算的 next run；它不执行任务、不连接 Provider，也不修改数据。
|
||||
|
||||
### Control 消息
|
||||
|
||||
WebSocket dialog 操作通过 `ControlMessage` 携带一次性回复通道。Gateway control router 以最多 64 个并发受监督任务调用 `SessionManager`，再将 `SessionEvent` 回传给发起者；慢 control 不阻塞其他聊天的 inbound 路由。Bus 只承载消息，不解释操作。
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -54,13 +54,7 @@
|
||||
"gateway": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 19876,
|
||||
"require_pairing": true,
|
||||
"scheduler": {
|
||||
"enabled": true,
|
||||
"poll_interval_secs": 60,
|
||||
"max_concurrent": 1,
|
||||
"execution_timeout_secs": 900
|
||||
}
|
||||
"require_pairing": true
|
||||
},
|
||||
"client": {
|
||||
"gateway_url": "ws://127.0.0.1:19876/ws"
|
||||
|
||||
@ -10,7 +10,7 @@ Channel → MessageBus.inbound → Gateway processor → SessionManager → per-
|
||||
AgentLoop → TurnEvent → TurnController → latest TurnSnapshot → DeliveryCoordinator → TurnSink → Channel
|
||||
|
||||
WebSocket/Channel → MessageBus.control → Gateway processor → SessionManager (dialog 操作)
|
||||
Scheduler → occurrence/JobRun claim → AgentCoordinator → isolated Scheduled Agent → complete_scheduled_run → JobRun outbox → SessionManager/MessageBus
|
||||
Scheduler → SessionManager.handle_cron_message → AgentLoop → send_message
|
||||
```
|
||||
|
||||
## 模块职责
|
||||
|
||||
@ -102,8 +102,8 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
|
||||
|------|------|------|------|
|
||||
| `enabled` | bool | true | 是否启动调度器并注册 cron 工具 |
|
||||
| `poll_interval_secs` | int | 60 | 检查到期任务的轮询间隔 |
|
||||
| `max_concurrent` | int | 1 | 同时执行的 Scheduled Run 上限，运行时限制在 1–256；投递使用独立有界并发 |
|
||||
| `execution_timeout_secs` | int | 900 | 单个定时任务 Agent 执行的硬超时；Job 执行租约额外覆盖关停宽限，投递由持久化 outbox 独立恢复 |
|
||||
| `max_concurrent` | int | 1 | 每批到期任务的最大并发数，运行时限制在 1–256 |
|
||||
| `execution_timeout_secs` | int | 900 | 单个定时任务 Agent 执行的硬超时；租约会覆盖执行和托管投递等待 |
|
||||
|
||||
## memory 字段
|
||||
|
||||
@ -116,7 +116,7 @@ Gateway WebUI 的“配置”页可以编辑实际加载的配置文件。读取
|
||||
| `timeline_retention_days` | int | 90 | 默认日常维护巡检删除超过该期限的 Timeline；Knowledge 不受影响 |
|
||||
| `max_failures_before_degrade` | int | 3 | 预留的归并失败阈值；当前无失败降级循环 |
|
||||
|
||||
注意：当前 worker 的 Knowledge 召回数量仍固定为 5；idle consolidation 和失败降级循环尚未接入。Timeline 清理由默认启用的 `picobot-routine-maintenance` Scheduled Run 执行；该任务使用 `never` 策略，结构化结果只进入运行审计和 Health。
|
||||
注意：当前 worker 的 Knowledge 召回数量仍固定为 5；idle consolidation 和失败降级循环尚未接入。Timeline 清理由默认启用的 `picobot-routine-maintenance` 定时巡检执行。
|
||||
|
||||
## channels.feishu 字段
|
||||
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
数据库为 SQLite，默认位于配置目录（`~/.picobot`）`data/` 下的 `picobot.db`，与 workspace 相互独立。
|
||||
|
||||
连接启用 WAL、`synchronous=NORMAL`、foreign keys、5 秒 busy timeout，连接池最多 8 个连接。当前 `PRAGMA user_version=11`；启动时会在单个事务内迁移旧库，遇到比程序更新的 schema version 会拒绝启动。
|
||||
连接启用 WAL、`synchronous=NORMAL`、foreign keys、5 秒 busy timeout，连接池最多 8 个连接。当前 `PRAGMA user_version=10`；启动时会在事务内补齐旧库字段和索引，遇到比程序更新的 schema version 会拒绝启动。
|
||||
|
||||
## sessions 表
|
||||
|
||||
@ -176,21 +176,22 @@ background 完成/信号投递的唯一事实源：`pending → leased → admit
|
||||
| `name` | TEXT | 任务名称 |
|
||||
| `schedule` | TEXT | 调度规则 JSON（at/every/cron） |
|
||||
| `prompt` | TEXT | 任务提示词 |
|
||||
| `agent_id` | TEXT | 可选命名 Agent；NULL 表示 Root |
|
||||
| `channel` | TEXT | 目标渠道 |
|
||||
| `channel` | TEXT | 执行渠道 |
|
||||
| `chat_id` | TEXT | 目标对话 |
|
||||
| `delivery_policy` | TEXT | `always` / `on_alert` / `never` |
|
||||
| `model` | TEXT | 可选模型标记；当前会存储/展示，但 Scheduler 执行仍使用默认 Agent 模型 |
|
||||
| `enabled` | INTEGER | 是否启用 (1/0) |
|
||||
| `delete_after_run` | INTEGER | 执行后自动删除 (1/0) |
|
||||
| `next_run_at` | INTEGER | 下次执行时间 |
|
||||
| `last_run_at` | INTEGER | 上次执行时间 |
|
||||
| `last_outcome` | TEXT | 最近结构化结果：ok/alert/failed/refused/unknown |
|
||||
| `last_status` | TEXT | 上次执行状态 |
|
||||
| `last_error` | TEXT | 上次错误信息 |
|
||||
| `locked_at` | INTEGER | 本次领取时间 |
|
||||
| `lock_owner` | TEXT | 本次 occurrence 的唯一 owner token |
|
||||
| `lease_until` | INTEGER | 租约到期时间 |
|
||||
| `lock_owner` | TEXT | 领取任务的 Scheduler owner UUID |
|
||||
| `lease_until` | INTEGER | 租约到期时间；进程崩溃后允许其他实例重新领取 |
|
||||
| `created_at` | INTEGER | 创建时间（Unix 毫秒） |
|
||||
| `updated_at` | INTEGER | 更新时间（Unix 毫秒） |
|
||||
|
||||
Scheduler 在领取事务中插入 JobRun、快照执行/投递字段并推进下次时间。`At` 在领取时立即禁用；执行失败或崩溃不重放同一个 occurrence。
|
||||
Scheduler 使用原子 `UPDATE ... RETURNING` 领取到期任务。任务结果、下次运行时间和租约释放在同一事务中提交，并校验 owner，防止过期 worker 覆盖已恢复的任务。
|
||||
|
||||
## job_runs 表
|
||||
|
||||
@ -198,26 +199,12 @@ Scheduler 在领取事务中插入 JobRun、快照执行/投递字段并推进
|
||||
|------|------|------|
|
||||
| `id` | INTEGER PK | 自增 ID |
|
||||
| `job_id` | TEXT FK | 关联任务，外键关联 scheduled_jobs(id) |
|
||||
| `scheduled_for` | INTEGER | 本 occurrence 原计划时间 |
|
||||
| `agent_run_id` | TEXT FK | 顶层 AgentRun 审计记录 |
|
||||
| `agent_id` | TEXT | claim-time Agent 快照 |
|
||||
| `delivery_policy` | TEXT | claim-time 投递策略快照 |
|
||||
| `target_channel` / `target_chat_id` | TEXT | claim-time 目标快照 |
|
||||
| `target_session_id` | TEXT | 首次投递时固定的目标 dialog |
|
||||
| `started_at` | INTEGER | 开始时间 |
|
||||
| `finished_at` | INTEGER | 结束时间 |
|
||||
| `status` | TEXT | claimed/running/completed/failed/timed_out/cancelled/interrupted/unknown |
|
||||
| `outcome` | TEXT | ok/alert/failed/refused/unknown；与 status 有联合约束 |
|
||||
| `message` | TEXT | 面向用户的结构化结果 |
|
||||
| `diagnostic` | TEXT | 有界内部诊断 |
|
||||
| `status` | TEXT | 执行状态 |
|
||||
| `output` | TEXT | 执行输出 |
|
||||
| `error` | TEXT | 错误信息 |
|
||||
| `duration_ms` | INTEGER | 耗时（毫秒） |
|
||||
| `delivery_status` | TEXT | awaiting_result/not_requested/suppressed/pending/delivering/delivered/failed |
|
||||
| `delivery_attempts` | INTEGER | 持久化投递尝试次数，最多 3 次 |
|
||||
| `delivery_next_attempt_at` | INTEGER | 瞬态失败后的退避时间 |
|
||||
| `delivery_lease_owner` / `delivery_lease_until` | TEXT / INTEGER | outbox 领取租约 |
|
||||
| `delivery_error` | TEXT | 清洗后的投递失败摘要 |
|
||||
|
||||
JobRun 是执行结果和投递状态的唯一权威。顶层 AgentRun 与 JobRun 终态、Job 最近摘要和租约释放原子提交；启动恢复将遗留运行归为 `unknown`，不会自动重跑。
|
||||
|
||||
## llm_calls 表
|
||||
|
||||
|
||||
@ -50,15 +50,14 @@
|
||||
|
||||
## Cron 定时任务工具
|
||||
|
||||
Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅在 `gateway.scheduler.enabled=true` 时注册。
|
||||
Cron 不是一个带 `action` 的统一工具，而是六个独立工具；仅在 `gateway.scheduler.enabled=true` 时注册。
|
||||
|
||||
| 工具 | 主要参数 | 说明 |
|
||||
|------|----------|------|
|
||||
| `cron_add` | `schedule`, `prompt`, `channel`, `chat_id`; 可选 `name`, `agent_id`, `delivery_policy` | 创建任务 |
|
||||
| `cron_add` | `schedule`, `prompt`, `channel`, `chat_id`; 可选 `name`, `model` | 创建任务 |
|
||||
| `cron_list` | 可选 `status=all|enabled|disabled` | 列出任务 |
|
||||
| `cron_runs` | `job_id`; 可选 `run_id`, `limit` | 查询结构化运行和投递记录，包括静默结果 |
|
||||
| `cron_update` | `job_id`; 可选 `name`, `prompt`, `schedule`, `channel`, `chat_id`, `agent_id`, `delivery_policy` | 更新指定字段；`agent_id:null` 切回 Root |
|
||||
| `cron_remove` | `job_id` | 无活动 Run 或 pending delivery 时永久删除任务 |
|
||||
| `cron_update` | `job_id`; 可选 `prompt`, `schedule`, `channel`, `chat_id`, `model` | 更新指定字段 |
|
||||
| `cron_remove` | `job_id` | 永久删除任务和关联 job runs |
|
||||
| `cron_enable` | `job_id` | 启用并重新计算下次运行时间 |
|
||||
| `cron_disable` | `job_id` | 禁用但保留任务 |
|
||||
|
||||
@ -70,9 +69,7 @@ Cron 不是一个带 `action` 的统一工具，而是七个独立工具；仅
|
||||
{"type":"cron","expr":"0 0 9 * * *","tz":"Asia/Shanghai"}
|
||||
```
|
||||
|
||||
时间戳和间隔单位为毫秒；Cron 表达式为 6 段（秒、分、时、日、月、周）。过去时间的 At 不能创建、更新或直接重新启用。定时 Agent 不复用聊天历史，`prompt` 必须包含完整上下文；`agent_id` 省略时使用 Root，否则使用当前 AgentCatalog 中的命名 Agent。
|
||||
|
||||
每次运行必须恰好一次调用 `complete_scheduled_run(outcome,message)`，outcome 只能是 `ok`、`alert`、`failed`、`refused`。普通最终文本不会被解释为结果，缺少结构化终结会 fail-closed。投递完全由 Scheduler 决定：`always` 投递所有结果，`on_alert` 只抑制 `ok`，`never` 只保留记录。Scheduled Agent 不能自行发送最终通知；子 Agent 委托会同步完成，也不会产生后台 Inbox/Signal。
|
||||
时间戳和间隔单位为毫秒；Cron 表达式为 6 段（秒、分、时、日、月、周）。定时 Agent 不复用聊天历史，`prompt` 必须包含完整上下文。`kind` 可为 `task` 或 `monitor`；`delivery_policy` 可为 `always`、`on_alert` 或 `never`。托管任务由 Scheduler 投递，巡检返回 `NO_REPLY[INFO]` 时静默，`NO_REPLY[FAIL]`/`NO_REPLY[REFUSE]` 仍视为需关注结果。升级前创建的任务保留 Agent 直接调用 `send_message` 的兼容行为。`model` 当前会持久化和展示，但执行仍使用默认 Agent Provider/Model，不能依赖它实现模型覆盖。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -78,12 +78,6 @@
|
||||
"max_files_per_message": 8,
|
||||
"max_message_bytes": 67108864,
|
||||
"pending_ttl_seconds": 3600
|
||||
},
|
||||
"scheduler": {
|
||||
"enabled": true,
|
||||
"poll_interval_secs": 60,
|
||||
"max_concurrent": 1,
|
||||
"execution_timeout_secs": 900
|
||||
}
|
||||
},
|
||||
"client": {
|
||||
|
||||
@ -1240,31 +1240,6 @@ impl AgentLoop {
|
||||
}
|
||||
completed_tool_batches = completed_tool_batches.saturating_add(1);
|
||||
|
||||
if let Some(outcome) = tool_context
|
||||
.scheduled_completion
|
||||
.as_ref()
|
||||
.and_then(|sink| sink.outcome())
|
||||
{
|
||||
Self::close_steering(turn.as_ref());
|
||||
let mut final_message = ChatMessage::assistant(outcome.message);
|
||||
attach_reply_media(&mut final_message, &reply_media_refs);
|
||||
Self::annotate_message(&mut final_message, turn.as_ref(), iteration, true);
|
||||
emitted_messages.push(final_message.clone());
|
||||
self.forward_to_transcript_sink(&final_message);
|
||||
crate::observability::metrics::global_metrics().record_turn(
|
||||
Some(&accumulated_usage),
|
||||
turn_start.elapsed().as_millis() as u64,
|
||||
);
|
||||
return Ok(AgentProcessResult {
|
||||
final_response: final_message,
|
||||
emitted_messages,
|
||||
total_tokens: Some(accumulated_tokens),
|
||||
usage: Some(accumulated_usage),
|
||||
last_request_usage,
|
||||
last_request_digest,
|
||||
});
|
||||
}
|
||||
|
||||
// A complete tool batch is the first safe steering boundary. Do
|
||||
// not drain at the final available iteration: those inputs must
|
||||
// remain in the closed mailbox for Session to queue after this
|
||||
@ -1542,17 +1517,6 @@ impl AgentLoop {
|
||||
let mut outcomes = Vec::with_capacity(tool_calls.len());
|
||||
|
||||
for tool_call in tool_calls {
|
||||
if context
|
||||
.scheduled_completion
|
||||
.as_ref()
|
||||
.is_some_and(|sink| sink.is_completed())
|
||||
{
|
||||
outcomes.push(ToolExecutionOutcome::failure(
|
||||
"Cancelled: scheduled run was already completed".to_string(),
|
||||
Some("scheduled run was already completed".to_string()),
|
||||
));
|
||||
continue;
|
||||
}
|
||||
if context.cancellation.is_cancelled() {
|
||||
return Err(AgentError::Cancelled);
|
||||
}
|
||||
@ -1702,58 +1666,6 @@ mod tests {
|
||||
requests: std::sync::atomic::AtomicUsize,
|
||||
}
|
||||
|
||||
struct ScheduledCompletionProvider {
|
||||
requests: std::sync::atomic::AtomicUsize,
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl LLMProvider for ScheduledCompletionProvider {
|
||||
async fn stream(
|
||||
&self,
|
||||
_request: ChatCompletionRequest,
|
||||
) -> Result<ProviderStream, crate::providers::DynProviderError> {
|
||||
self.requests
|
||||
.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
|
||||
Ok(crate::providers::provider_stream_for_test(
|
||||
ChatCompletionResponse {
|
||||
id: "scheduled-complete".to_string(),
|
||||
model: "scheduled-complete".to_string(),
|
||||
content: String::new(),
|
||||
reasoning_content: None,
|
||||
provider_state: None,
|
||||
tool_calls: vec![
|
||||
ToolCall {
|
||||
id: "complete".to_string(),
|
||||
name: "complete_scheduled_run".to_string(),
|
||||
arguments: serde_json::json!({
|
||||
"outcome": "ok",
|
||||
"message": "healthy"
|
||||
}),
|
||||
},
|
||||
ToolCall {
|
||||
id: "late-side-effect".to_string(),
|
||||
name: "side_effect".to_string(),
|
||||
arguments: serde_json::json!({}),
|
||||
},
|
||||
],
|
||||
usage: Usage::default(),
|
||||
},
|
||||
))
|
||||
}
|
||||
|
||||
fn ptype(&self) -> &str {
|
||||
"test"
|
||||
}
|
||||
|
||||
fn name(&self) -> &str {
|
||||
"scheduled-complete"
|
||||
}
|
||||
|
||||
fn model_id(&self) -> &str {
|
||||
"scheduled-complete"
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl LLMProvider for AlwaysOverflowProvider {
|
||||
async fn stream(
|
||||
@ -1813,53 +1725,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn scheduled_completion_ends_the_loop_and_cancels_later_batch_calls() {
|
||||
let provider = Arc::new(ScheduledCompletionProvider {
|
||||
requests: std::sync::atomic::AtomicUsize::new(0),
|
||||
});
|
||||
let executions = Arc::new(std::sync::atomic::AtomicUsize::new(0));
|
||||
let tools = Arc::new(ToolRegistry::new());
|
||||
tools.register(crate::tools::CompleteScheduledRunTool::new());
|
||||
tools.register(CountingSideEffectTool {
|
||||
executions: executions.clone(),
|
||||
});
|
||||
let agent = AgentLoop::with_provider_and_tools(
|
||||
provider.clone(),
|
||||
tools,
|
||||
3,
|
||||
"scheduled-complete".to_string(),
|
||||
PathBuf::from("."),
|
||||
Vec::new(),
|
||||
);
|
||||
let sink = Arc::new(crate::tools::ScheduledCompletionSink::default());
|
||||
let context = ToolExecutionContext::for_session("scheduled-run:1")
|
||||
.with_execution_origin(crate::tools::ExecutionOrigin::Scheduled { job_run_id: 1 })
|
||||
.with_scheduled_completion(sink.clone());
|
||||
|
||||
let result = agent
|
||||
.process_with_context(vec![ChatMessage::user("check")], context)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(result.final_response.content, "healthy");
|
||||
assert_eq!(
|
||||
sink.outcome().unwrap().kind,
|
||||
crate::storage::ScheduledOutcomeKind::Ok
|
||||
);
|
||||
assert_eq!(executions.load(std::sync::atomic::Ordering::SeqCst), 0);
|
||||
assert_eq!(
|
||||
provider.requests.load(std::sync::atomic::Ordering::SeqCst),
|
||||
1
|
||||
);
|
||||
assert!(result.emitted_messages.iter().any(|message| {
|
||||
message.role == "tool"
|
||||
&& message
|
||||
.content
|
||||
.contains("scheduled run was already completed")
|
||||
}));
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl LLMProvider for OverflowAfterToolProvider {
|
||||
async fn stream(
|
||||
|
||||
@ -32,16 +32,6 @@ pub struct BackgroundAdmission {
|
||||
pub run_ids: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ScheduledAgentExecution {
|
||||
pub agent_run_id: String,
|
||||
pub status: crate::storage::ScheduledRunStatus,
|
||||
pub outcome: Option<crate::tools::ScheduledOutcome>,
|
||||
pub error: Option<String>,
|
||||
pub agent_terminal: AgentTerminalOutcome,
|
||||
pub runtime_generation: i64,
|
||||
}
|
||||
|
||||
pub struct AgentCoordinator {
|
||||
storage: Arc<Storage>,
|
||||
manager: Arc<SubAgentManager>,
|
||||
@ -98,263 +88,6 @@ impl AgentCoordinator {
|
||||
})
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn execute_scheduled(
|
||||
self: &Arc<Self>,
|
||||
job_run_id: i64,
|
||||
lease_owner: &str,
|
||||
job_id: &str,
|
||||
job_name: &str,
|
||||
agent_id: Option<&str>,
|
||||
prompt: &str,
|
||||
timeout_secs: u64,
|
||||
) -> Result<ScheduledAgentExecution, CoordinatorError> {
|
||||
let run_id = Uuid::new_v4().to_string();
|
||||
let root_session_id = format!("scheduled-run:{job_run_id}");
|
||||
let sink = Arc::new(crate::tools::ScheduledCompletionSink::default());
|
||||
let caller = ToolExecutionContext::for_session(root_session_id.clone())
|
||||
.with_turn_id(format!("scheduled:{job_run_id}"))
|
||||
.with_execution_origin(crate::tools::ExecutionOrigin::Scheduled { job_run_id });
|
||||
let contract = format!(
|
||||
"## Unattended Scheduled Run\n\nYou are executing scheduled task “{job_name}” ({job_id}). The user will not see ordinary final text. After completing all necessary work, you must call complete_scheduled_run exactly once. Use ok only when the task completed and found nothing requiring attention; use alert for actionable findings; use failed when the task did not complete reliably; use refused for a permission or safety refusal. Legacy textual suppression and direct-messaging instructions are obsolete."
|
||||
);
|
||||
let config = SubAgentConfig {
|
||||
target: agent_id.map(str::to_string),
|
||||
prompt: prompt.to_string(),
|
||||
context: Some(contract.clone()),
|
||||
mode: ExecutionMode::Foreground,
|
||||
allowed_tools: None,
|
||||
max_iterations: None,
|
||||
timeout_secs: Some(timeout_secs),
|
||||
plan_item_id: None,
|
||||
session_id: Some(root_session_id.clone()),
|
||||
};
|
||||
let mut resolution = if agent_id.is_some() {
|
||||
let mut resolution = self.manager.resolve_agent(&config, &caller, &run_id)?;
|
||||
resolution
|
||||
.tools
|
||||
.register(crate::tools::CompleteScheduledRunTool::new());
|
||||
resolution.tool_context.session_id = Some(root_session_id.clone());
|
||||
resolution.tool_context.scheduled_completion = Some(sink.clone());
|
||||
resolution.timeout_secs = resolution.timeout_secs.min(timeout_secs);
|
||||
resolution.signal_contract = None;
|
||||
resolution
|
||||
} else {
|
||||
self.manager
|
||||
.resolve_scheduled_root(&caller, &run_id, timeout_secs, sink.clone())?
|
||||
};
|
||||
resolution.tool_context.execution_origin =
|
||||
crate::tools::ExecutionOrigin::Scheduled { job_run_id };
|
||||
|
||||
let now = chrono::Utc::now().timestamp_millis();
|
||||
let deadline_at = now.saturating_add((resolution.timeout_secs * 1000) as i64);
|
||||
let new_run = NewAgentRun {
|
||||
id: run_id.clone(),
|
||||
root_session_id,
|
||||
root_turn_id: None,
|
||||
parent_run_id: None,
|
||||
caller_agent_id: "SCHEDULER".to_string(),
|
||||
caller_scope_id: format!("scheduled:{job_id}"),
|
||||
idempotency_key: Some(format!("scheduled:{job_run_id}")),
|
||||
agent_id: agent_id.unwrap_or("ROOT").to_string(),
|
||||
definition_hash: resolution.definition_hash.clone().unwrap_or_default(),
|
||||
provider_profile: resolution.llm_profile.clone().unwrap_or_default(),
|
||||
provider_name: resolution.provider_config.name.clone(),
|
||||
model_id: resolution.provider_config.model_id.clone(),
|
||||
mode: AgentRunMode::Foreground,
|
||||
depth: 1,
|
||||
plan_item_id: None,
|
||||
execution_id: run_id.clone(),
|
||||
task: prompt.to_string(),
|
||||
context_json: None,
|
||||
budget_json: serde_json::json!({
|
||||
"remaining_runs": self.manager.catalog().max_runs_per_tree().saturating_sub(1),
|
||||
"remaining_depth": self.manager.catalog().max_tree_depth().saturating_sub(1),
|
||||
})
|
||||
.to_string(),
|
||||
signal_contract_json: None,
|
||||
signal_delivery: None,
|
||||
deadline_at,
|
||||
runtime_generation: self.runtime_generation,
|
||||
completion_slot_reserved: false,
|
||||
};
|
||||
match self
|
||||
.storage
|
||||
.accept_agent_runs(AcceptAgentRequest {
|
||||
runs: vec![new_run],
|
||||
now,
|
||||
})
|
||||
.await?
|
||||
{
|
||||
AcceptedAgentRuns::Accepted { .. } => {}
|
||||
AcceptedAgentRuns::Existing { .. } => {
|
||||
return Err(CoordinatorError::Rejected(format!(
|
||||
"scheduled occurrence {job_run_id} already has an Agent run"
|
||||
)));
|
||||
}
|
||||
}
|
||||
let mut mark_attempt = 0_u64;
|
||||
let marked = match loop {
|
||||
match self
|
||||
.storage
|
||||
.mark_scheduled_run_running(job_run_id, lease_owner, Some(&run_id), now)
|
||||
.await
|
||||
{
|
||||
Err(error) if error.is_transient() && mark_attempt < 2 => {
|
||||
mark_attempt += 1;
|
||||
tokio::time::sleep(std::time::Duration::from_millis(50 * mark_attempt)).await;
|
||||
}
|
||||
result => break result,
|
||||
}
|
||||
} {
|
||||
Ok(marked) => marked,
|
||||
Err(error) => {
|
||||
let _ = self
|
||||
.storage
|
||||
.cancel_agent_run_with_completion(
|
||||
&run_id,
|
||||
"scheduled occurrence could not enter running state",
|
||||
true,
|
||||
now,
|
||||
)
|
||||
.await;
|
||||
return Err(error.into());
|
||||
}
|
||||
};
|
||||
if !marked {
|
||||
let _ = self
|
||||
.storage
|
||||
.cancel_agent_run_with_completion(
|
||||
&run_id,
|
||||
"scheduled occurrence was no longer active",
|
||||
true,
|
||||
now,
|
||||
)
|
||||
.await;
|
||||
return Err(CoordinatorError::Rejected(format!(
|
||||
"scheduled occurrence {job_run_id} lost its lease"
|
||||
)));
|
||||
}
|
||||
|
||||
let (execution, agent_terminal) = match self
|
||||
.execute_scheduled_agent_run(&run_id, &config, resolution)
|
||||
.await
|
||||
{
|
||||
Ok(execution) => execution,
|
||||
Err(error) => (
|
||||
Err(error),
|
||||
AgentTerminalOutcome::Failed {
|
||||
error: "scheduled Agent could not enter its execution lifecycle".to_string(),
|
||||
prompt_tokens: None,
|
||||
completion_tokens: None,
|
||||
cost: None,
|
||||
signal_ids: Vec::new(),
|
||||
},
|
||||
),
|
||||
};
|
||||
let status = match &execution {
|
||||
Ok(result) => match &result.status {
|
||||
TaskStatus::Completed => crate::storage::ScheduledRunStatus::Completed,
|
||||
TaskStatus::Failed(_) => crate::storage::ScheduledRunStatus::Failed,
|
||||
TaskStatus::TimedOut => crate::storage::ScheduledRunStatus::TimedOut,
|
||||
TaskStatus::Cancelled => crate::storage::ScheduledRunStatus::Interrupted,
|
||||
},
|
||||
Err(_) => crate::storage::ScheduledRunStatus::Failed,
|
||||
};
|
||||
let error = match &execution {
|
||||
Ok(result) => match &result.status {
|
||||
TaskStatus::Completed => None,
|
||||
TaskStatus::Failed(_) => Some(
|
||||
"scheduled Agent execution failed; inspect Gateway logs for details"
|
||||
.to_string(),
|
||||
),
|
||||
TaskStatus::TimedOut => Some("scheduled Agent timed out".to_string()),
|
||||
TaskStatus::Cancelled => Some("scheduled Agent was cancelled".to_string()),
|
||||
},
|
||||
Err(_) => Some(
|
||||
"scheduled Agent execution failed; inspect Gateway logs for details".to_string(),
|
||||
),
|
||||
};
|
||||
Ok(ScheduledAgentExecution {
|
||||
agent_run_id: run_id,
|
||||
status,
|
||||
outcome: sink.outcome(),
|
||||
error,
|
||||
agent_terminal,
|
||||
runtime_generation: self.runtime_generation,
|
||||
})
|
||||
}
|
||||
|
||||
async fn execute_scheduled_agent_run(
|
||||
self: &Arc<Self>,
|
||||
run_id: &str,
|
||||
config: &SubAgentConfig,
|
||||
resolution: ResolvedAgentRun,
|
||||
) -> Result<
|
||||
(
|
||||
Result<SubAgentResult, CoordinatorError>,
|
||||
AgentTerminalOutcome,
|
||||
),
|
||||
CoordinatorError,
|
||||
> {
|
||||
let execution_id = run_id.to_string();
|
||||
let token = resolution.tool_context.cancellation.clone();
|
||||
self.active_tokens.insert(run_id.to_string(), token);
|
||||
let started = self
|
||||
.storage
|
||||
.mark_agent_run_running(run_id, &execution_id, chrono::Utc::now().timestamp_millis())
|
||||
.await?;
|
||||
if !started {
|
||||
self.active_tokens.remove(run_id);
|
||||
return Err(CoordinatorError::Rejected(format!(
|
||||
"scheduled Agent run {run_id} was closed before execution started"
|
||||
)));
|
||||
}
|
||||
|
||||
let result = self
|
||||
.manager
|
||||
.execute_resolved(config, resolution, run_id)
|
||||
.await
|
||||
.map_err(CoordinatorError::SubAgent);
|
||||
self.active_tokens.remove(run_id);
|
||||
let terminal = match &result {
|
||||
Ok(result) => match &result.status {
|
||||
TaskStatus::Completed => AgentTerminalOutcome::Completed {
|
||||
result: result.full_content.clone(),
|
||||
prompt_tokens: None,
|
||||
completion_tokens: None,
|
||||
cost: None,
|
||||
tool_calls: result.tool_calls_count as i64,
|
||||
iterations: result.iterations as i64,
|
||||
signal_ids: Vec::new(),
|
||||
},
|
||||
TaskStatus::Failed(error) => AgentTerminalOutcome::Failed {
|
||||
error: error.clone(),
|
||||
prompt_tokens: None,
|
||||
completion_tokens: None,
|
||||
cost: None,
|
||||
signal_ids: Vec::new(),
|
||||
},
|
||||
TaskStatus::TimedOut => AgentTerminalOutcome::TimedOut {
|
||||
deadline_at: chrono::Utc::now().timestamp_millis(),
|
||||
signal_ids: Vec::new(),
|
||||
},
|
||||
TaskStatus::Cancelled => AgentTerminalOutcome::Interrupted {
|
||||
reason: "scheduled Agent interrupted by shutdown".to_string(),
|
||||
signal_ids: Vec::new(),
|
||||
},
|
||||
},
|
||||
Err(error) => AgentTerminalOutcome::Failed {
|
||||
error: error.to_string(),
|
||||
prompt_tokens: None,
|
||||
completion_tokens: None,
|
||||
cost: None,
|
||||
signal_ids: Vec::new(),
|
||||
},
|
||||
};
|
||||
Ok((result, terminal))
|
||||
}
|
||||
|
||||
/// Admit a named background run for the root caller and spawn its runner.
|
||||
/// Completion is guaranteed by the reserved inbox slot; the returned ID
|
||||
/// is only valid when every durable step succeeded.
|
||||
@ -367,11 +100,6 @@ impl AgentCoordinator {
|
||||
caller: &ToolExecutionContext,
|
||||
configs: Vec<SubAgentConfig>,
|
||||
) -> Result<BackgroundAdmission, CoordinatorError> {
|
||||
if caller.execution_origin.is_scheduled() {
|
||||
return Err(CoordinatorError::Rejected(
|
||||
"scheduled Agents cannot create background runs".to_string(),
|
||||
));
|
||||
}
|
||||
if caller.agent.is_some() {
|
||||
return Err(CoordinatorError::Rejected(
|
||||
"nested background runs are not available yet; only the root Agent may delegate background work".to_string(),
|
||||
|
||||
@ -21,7 +21,7 @@ pub use context_compaction::{
|
||||
ContextRequestKey, ContextUsageTracker, PreviousCheckpoint, SequencedMessage,
|
||||
context_request_digest, estimate_tokens,
|
||||
};
|
||||
pub use coordinator::{AgentCoordinator, CoordinatorError, ScheduledAgentExecution};
|
||||
pub use coordinator::{AgentCoordinator, CoordinatorError};
|
||||
pub use definition::{AgentDefinition, AgentLimits};
|
||||
pub use gate::ExecutionGate;
|
||||
pub use inbox::{AgentInboxNotifier, AgentInboxWakeTarget};
|
||||
|
||||
@ -10,22 +10,6 @@ use crate::providers::{LLMProvider, create_provider};
|
||||
use crate::skills::SkillsLoader;
|
||||
use crate::tools::{ToolExecutionContext, ToolRegistry};
|
||||
|
||||
const SCHEDULED_DISABLED_TOOLS: &[&str] = &[
|
||||
"send_message",
|
||||
"cron_add",
|
||||
"cron_update",
|
||||
"cron_remove",
|
||||
"cron_enable",
|
||||
"cron_disable",
|
||||
"cron_list",
|
||||
"cron_runs",
|
||||
"reload_config",
|
||||
"agent_task",
|
||||
"chat_manager",
|
||||
"todo",
|
||||
"emit_signal",
|
||||
];
|
||||
|
||||
const DEFAULT_MAX_ITERATIONS: usize = 99;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
@ -207,13 +191,7 @@ impl SubAgentManager {
|
||||
})?;
|
||||
let cancellation = caller.cancellation.child_token();
|
||||
let execution = if let Some(parent) = caller.agent.as_ref() {
|
||||
let allowed =
|
||||
if caller.execution_origin.is_scheduled() && parent.current_agent_id == "ROOT" {
|
||||
self.catalog.root_can_delegate(target)
|
||||
} else {
|
||||
self.catalog.can_delegate(&parent.current_agent_id, target)
|
||||
};
|
||||
if !allowed {
|
||||
if !self.catalog.can_delegate(&parent.current_agent_id, target) {
|
||||
return Err(SubAgentError::Other(format!(
|
||||
"Agent '{}' is not allowed to delegate to '{target}'",
|
||||
parent.current_agent_id
|
||||
@ -257,14 +235,10 @@ impl SubAgentManager {
|
||||
.budget
|
||||
.remaining_depth
|
||||
.min(definition.limits.max_depth);
|
||||
child.signal_contract = if caller.execution_origin.is_scheduled() {
|
||||
None
|
||||
} else {
|
||||
definition
|
||||
.signal_contract
|
||||
.as_ref()
|
||||
.map(|contract| Arc::new(contract.clone()))
|
||||
};
|
||||
child.signal_contract = definition
|
||||
.signal_contract
|
||||
.as_ref()
|
||||
.map(|contract| Arc::new(contract.clone()));
|
||||
Arc::new(child)
|
||||
} else {
|
||||
if !self.catalog.root_can_delegate(target) {
|
||||
@ -278,11 +252,7 @@ impl SubAgentManager {
|
||||
run_id: task_id.to_string(),
|
||||
execution_id: task_id.to_string(),
|
||||
parent_run_id: None,
|
||||
caller_agent_id: if caller.execution_origin.is_scheduled() {
|
||||
"SCHEDULER".to_string()
|
||||
} else {
|
||||
"ROOT".to_string()
|
||||
},
|
||||
caller_agent_id: "ROOT".to_string(),
|
||||
current_agent_id: target.to_string(),
|
||||
ancestry: vec![target.to_string()],
|
||||
depth: 1,
|
||||
@ -297,14 +267,10 @@ impl SubAgentManager {
|
||||
.min(definition.limits.max_depth),
|
||||
},
|
||||
tree_runs: Arc::new(std::sync::atomic::AtomicUsize::new(1)),
|
||||
signal_contract: if caller.execution_origin.is_scheduled() {
|
||||
None
|
||||
} else {
|
||||
definition
|
||||
.signal_contract
|
||||
.as_ref()
|
||||
.map(|contract| Arc::new(contract.clone()))
|
||||
},
|
||||
signal_contract: definition
|
||||
.signal_contract
|
||||
.as_ref()
|
||||
.map(|contract| Arc::new(contract.clone())),
|
||||
emitted_signals: Arc::new(std::sync::Mutex::new(Vec::new())),
|
||||
})
|
||||
};
|
||||
@ -313,9 +279,6 @@ impl SubAgentManager {
|
||||
if let Some(allowed) = config.allowed_tools.as_ref() {
|
||||
effective_names.retain(|name| allowed.iter().any(|allowed| allowed == name));
|
||||
}
|
||||
if caller.execution_origin.is_scheduled() {
|
||||
effective_names.retain(|name| !SCHEDULED_DISABLED_TOOLS.contains(&name.as_str()));
|
||||
}
|
||||
let has_get_skill = effective_names.iter().any(|name| name == "get_skill");
|
||||
let mut names = effective_names;
|
||||
names.retain(|name| name != "get_skill");
|
||||
@ -347,7 +310,7 @@ impl SubAgentManager {
|
||||
// The signal tool is contract-bound: it exists only when the
|
||||
// definition declares a signal block and the durable Coordinator is
|
||||
// live. If either is missing the run cannot emit signals.
|
||||
if definition.signal_contract.is_some() && !caller.execution_origin.is_scheduled() {
|
||||
if definition.signal_contract.is_some() {
|
||||
match self.coordinator() {
|
||||
Some(coordinator) => {
|
||||
let contract = definition.signal_contract.clone().unwrap();
|
||||
@ -380,91 +343,17 @@ impl SubAgentManager {
|
||||
agent_id: Some(target.to_string()),
|
||||
definition_hash: Some(definition.definition_hash.clone()),
|
||||
llm_profile: definition.llm_profile.clone(),
|
||||
signal_contract: (!caller.execution_origin.is_scheduled())
|
||||
.then(|| definition.signal_contract.clone())
|
||||
.flatten(),
|
||||
tool_context: ToolExecutionContext::for_session(
|
||||
if caller.execution_origin.is_scheduled() {
|
||||
root_session_id
|
||||
} else {
|
||||
format!("agent-run:{task_id}")
|
||||
},
|
||||
)
|
||||
.with_turn_id(
|
||||
caller
|
||||
.turn_id
|
||||
.clone()
|
||||
.unwrap_or_else(|| task_id.to_string()),
|
||||
)
|
||||
.with_agent(execution)
|
||||
.with_cancellation(cancellation)
|
||||
.with_execution_gate(self.execution_gate.clone())
|
||||
.with_execution_origin(caller.execution_origin),
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn resolve_scheduled_root(
|
||||
&self,
|
||||
caller: &ToolExecutionContext,
|
||||
task_id: &str,
|
||||
timeout_secs: u64,
|
||||
sink: Arc<crate::tools::ScheduledCompletionSink>,
|
||||
) -> Result<ResolvedAgentRun, SubAgentError> {
|
||||
if !caller.execution_origin.is_scheduled() {
|
||||
return Err(SubAgentError::Other(
|
||||
"scheduled root resolution requires Scheduled execution origin".to_string(),
|
||||
));
|
||||
}
|
||||
let root_session_id = caller.session_id.clone().ok_or_else(|| {
|
||||
SubAgentError::Other("scheduled root requires an execution scope".to_string())
|
||||
})?;
|
||||
let run_id = task_id.to_string();
|
||||
let cancellation = caller.cancellation.child_token();
|
||||
let execution = Arc::new(crate::agent::AgentExecutionContext {
|
||||
root_session_id: root_session_id.clone(),
|
||||
root_turn_id: None,
|
||||
run_id: run_id.clone(),
|
||||
execution_id: run_id,
|
||||
parent_run_id: None,
|
||||
caller_agent_id: "SCHEDULER".to_string(),
|
||||
current_agent_id: "ROOT".to_string(),
|
||||
ancestry: vec!["ROOT".to_string()],
|
||||
depth: 1,
|
||||
plan_item_id: None,
|
||||
cancellation: cancellation.clone(),
|
||||
budget: crate::agent::AgentBudget {
|
||||
remaining_runs: self.catalog.max_runs_per_tree().saturating_sub(1),
|
||||
remaining_depth: self.catalog.max_tree_depth().saturating_sub(1),
|
||||
},
|
||||
tree_runs: Arc::new(std::sync::atomic::AtomicUsize::new(1)),
|
||||
signal_contract: None,
|
||||
emitted_signals: Arc::new(std::sync::Mutex::new(Vec::new())),
|
||||
});
|
||||
let tools = self.full_tools.without(SCHEDULED_DISABLED_TOOLS);
|
||||
tools.register(crate::tools::CompleteScheduledRunTool::new());
|
||||
let skills_prompt = self
|
||||
.skills_loader
|
||||
.as_ref()
|
||||
.map(|loader| loader.build_skills_prompt())
|
||||
.filter(|prompt| !prompt.is_empty());
|
||||
Ok(ResolvedAgentRun {
|
||||
provider_config: Arc::new(self.provider_config.clone()),
|
||||
tools,
|
||||
timeout_secs,
|
||||
max_iterations: self.provider_config.max_tool_iterations,
|
||||
max_result_chars: 16_384,
|
||||
role_prompt: None,
|
||||
skills_prompt,
|
||||
tool_context: ToolExecutionContext::for_session(root_session_id)
|
||||
signal_contract: definition.signal_contract.clone(),
|
||||
tool_context: ToolExecutionContext::for_session(format!("agent-run:{task_id}"))
|
||||
.with_turn_id(
|
||||
caller
|
||||
.turn_id
|
||||
.clone()
|
||||
.unwrap_or_else(|| task_id.to_string()),
|
||||
)
|
||||
.with_agent(execution)
|
||||
.with_cancellation(cancellation)
|
||||
.with_execution_gate(self.execution_gate.clone())
|
||||
.with_execution_origin(caller.execution_origin)
|
||||
.with_scheduled_completion(sink),
|
||||
agent_id: None,
|
||||
definition_hash: None,
|
||||
llm_profile: None,
|
||||
signal_contract: None,
|
||||
.with_execution_gate(self.execution_gate.clone()),
|
||||
})
|
||||
}
|
||||
|
||||
@ -775,25 +664,6 @@ mod tests {
|
||||
assert!(!crate::tools::Tool::runtime_injected(&reload));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scheduled_runs_remove_direct_and_interactive_control_tools() {
|
||||
for name in [
|
||||
"send_message",
|
||||
"cron_add",
|
||||
"cron_runs",
|
||||
"reload_config",
|
||||
"agent_task",
|
||||
"chat_manager",
|
||||
"todo",
|
||||
"emit_signal",
|
||||
] {
|
||||
assert!(SCHEDULED_DISABLED_TOOLS.contains(&name));
|
||||
}
|
||||
assert!(!SCHEDULED_DISABLED_TOOLS.contains(&"bash"));
|
||||
assert!(!SCHEDULED_DISABLED_TOOLS.contains(&"delegate"));
|
||||
assert!(!SCHEDULED_DISABLED_TOOLS.contains(&"complete_scheduled_run"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolve_agent_rejects_missing_target() {
|
||||
let manager = manager();
|
||||
|
||||
@ -5,7 +5,7 @@ use std::time::Duration;
|
||||
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
use crate::bus::{DeliveryReceipt, MessageBus, OutboundMessage};
|
||||
use crate::bus::{MessageBus, OutboundMessage};
|
||||
use crate::channels::ChannelManager;
|
||||
use crate::channels::base::{Channel, ChannelError};
|
||||
use crate::delivery::ConversationWriteLocks;
|
||||
@ -64,14 +64,12 @@ impl OutboundDispatcher {
|
||||
if sender.as_ref().is_none_or(mpsc::Sender::is_closed) {
|
||||
let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else {
|
||||
tracing::warn!(channel = %msg.channel, "No channel found for message");
|
||||
msg.complete_delivery(DeliveryReceipt::PermanentFailure {
|
||||
summary: format!("channel not found: {}", msg.channel),
|
||||
});
|
||||
msg.complete_delivery(Err(format!("channel not found: {}", msg.channel)));
|
||||
continue;
|
||||
};
|
||||
let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY);
|
||||
if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone()) {
|
||||
msg.complete_delivery(DeliveryReceipt::DispatcherClosed);
|
||||
msg.complete_delivery(Err("dispatcher is shutting down".to_string()));
|
||||
continue;
|
||||
}
|
||||
lanes.insert(lane_key.clone(), new_sender.clone());
|
||||
@ -91,9 +89,7 @@ impl OutboundDispatcher {
|
||||
capacity = LANE_CAPACITY,
|
||||
"Outbound lane full; rejecting message instead of blocking other destinations"
|
||||
);
|
||||
msg.complete_delivery(DeliveryReceipt::TransientFailure {
|
||||
summary: "outbound lane is full".to_string(),
|
||||
});
|
||||
msg.complete_delivery(Err("outbound lane is full".to_string()));
|
||||
}
|
||||
Err(mpsc::error::TrySendError::Closed(msg)) => {
|
||||
// The lane may have expired between the closed check and
|
||||
@ -101,15 +97,13 @@ impl OutboundDispatcher {
|
||||
lanes.remove(&lane_key);
|
||||
let Some(channel) = self.channel_manager.get_channel(&msg.channel).await else {
|
||||
tracing::warn!(channel = %msg.channel, "No channel found for message");
|
||||
msg.complete_delivery(DeliveryReceipt::PermanentFailure {
|
||||
summary: format!("channel not found: {}", msg.channel),
|
||||
});
|
||||
msg.complete_delivery(Err(format!("channel not found: {}", msg.channel)));
|
||||
continue;
|
||||
};
|
||||
let (new_sender, receiver) = mpsc::channel(LANE_CAPACITY);
|
||||
if !self.spawn_lane(channel, receiver, msg.channel.clone(), msg.chat_id.clone())
|
||||
{
|
||||
msg.complete_delivery(DeliveryReceipt::DispatcherClosed);
|
||||
msg.complete_delivery(Err("dispatcher is shutting down".to_string()));
|
||||
continue;
|
||||
}
|
||||
match new_sender.try_send(msg) {
|
||||
@ -117,9 +111,9 @@ impl OutboundDispatcher {
|
||||
lanes.insert(lane_key, new_sender);
|
||||
}
|
||||
Err(error) => {
|
||||
error
|
||||
.into_inner()
|
||||
.complete_delivery(DeliveryReceipt::DispatcherClosed);
|
||||
error.into_inner().complete_delivery(Err(
|
||||
"outbound lane could not be restarted during shutdown".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -149,15 +143,15 @@ impl OutboundDispatcher {
|
||||
Ok(None) | Err(_) => break,
|
||||
};
|
||||
let result = Self::send_with_retry(&*channel, &msg, &target_lock).await;
|
||||
if result != DeliveryReceipt::Delivered {
|
||||
if let Err(error) = &result {
|
||||
tracing::error!(
|
||||
channel = %channel_name,
|
||||
chat_id = %chat_id,
|
||||
result = ?result,
|
||||
error = %error,
|
||||
"Failed to send message after retries"
|
||||
);
|
||||
}
|
||||
msg.complete_delivery(result);
|
||||
msg.complete_delivery(result.map_err(|error| error.to_string()));
|
||||
}
|
||||
},
|
||||
)
|
||||
@ -167,28 +161,26 @@ impl OutboundDispatcher {
|
||||
channel: &dyn Channel,
|
||||
msg: &OutboundMessage,
|
||||
target_lock: &tokio::sync::Mutex<()>,
|
||||
) -> DeliveryReceipt {
|
||||
) -> Result<(), ChannelError> {
|
||||
let _guard = target_lock.lock().await;
|
||||
const DELAYS: &[u64] = &[1, 2, 4];
|
||||
|
||||
for (attempt, &delay) in DELAYS.iter().enumerate() {
|
||||
let result = tokio::time::timeout(SEND_TIMEOUT, channel.send(msg.clone())).await;
|
||||
match result {
|
||||
Ok(Ok(())) => return DeliveryReceipt::Delivered,
|
||||
Ok(Ok(())) => return Ok(()),
|
||||
Ok(Err(error)) if attempt < DELAYS.len() - 1 && error.is_transient() => {
|
||||
tracing::warn!(
|
||||
attempt = attempt + 1,
|
||||
delay,
|
||||
error_class = channel_error_class(&error),
|
||||
"Send failed, retrying"
|
||||
);
|
||||
tracing::warn!(attempt = attempt + 1, delay, error = %error, "Send failed, retrying");
|
||||
}
|
||||
Ok(Err(error)) => return receipt_from_channel_error(error),
|
||||
Ok(Err(error)) => return Err(error),
|
||||
Err(_) if attempt < DELAYS.len() - 1 => {
|
||||
tracing::warn!(attempt = attempt + 1, delay, "Send timed out, retrying");
|
||||
}
|
||||
Err(_) => {
|
||||
return DeliveryReceipt::TimedOut;
|
||||
return Err(ChannelError::Other(format!(
|
||||
"send timed out after {} seconds",
|
||||
SEND_TIMEOUT.as_secs()
|
||||
)));
|
||||
}
|
||||
}
|
||||
tokio::time::sleep(Duration::from_secs(delay)).await;
|
||||
@ -197,36 +189,6 @@ impl OutboundDispatcher {
|
||||
}
|
||||
}
|
||||
|
||||
fn channel_error_class(error: &ChannelError) -> &'static str {
|
||||
match error {
|
||||
ChannelError::ConnectionError(_) => "connection",
|
||||
ChannelError::SendError(_) => "send",
|
||||
ChannelError::BusError(_) => "bus",
|
||||
ChannelError::ConfigError(_) => "config",
|
||||
ChannelError::Other(_) => "other",
|
||||
}
|
||||
}
|
||||
|
||||
fn receipt_from_channel_error(error: ChannelError) -> DeliveryReceipt {
|
||||
match error {
|
||||
ChannelError::ConnectionError(_) => DeliveryReceipt::TransientFailure {
|
||||
summary: "channel connection failed after retries".to_string(),
|
||||
},
|
||||
ChannelError::SendError(_) => DeliveryReceipt::TransientFailure {
|
||||
summary: "channel send failed after retries".to_string(),
|
||||
},
|
||||
ChannelError::BusError(_) => DeliveryReceipt::TransientFailure {
|
||||
summary: "channel bus was unavailable".to_string(),
|
||||
},
|
||||
ChannelError::ConfigError(_) => DeliveryReceipt::PermanentFailure {
|
||||
summary: "channel configuration rejected delivery".to_string(),
|
||||
},
|
||||
ChannelError::Other(_) => DeliveryReceipt::PermanentFailure {
|
||||
summary: "channel rejected delivery".to_string(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Decrements the active-lane counter exactly once when a lane task ends,
|
||||
/// whether it exits normally, is cancelled, or is aborted.
|
||||
struct LaneGuard {
|
||||
@ -386,7 +348,7 @@ mod tests {
|
||||
message.channel = "missing".to_string();
|
||||
let error = bus.deliver_outbound(message).await.unwrap_err();
|
||||
|
||||
assert!(matches!(error, crate::bus::BusError::DeliveryPermanent(_)));
|
||||
assert!(matches!(error, crate::bus::BusError::DeliveryFailed(_)));
|
||||
task.abort();
|
||||
supervisor.shutdown(Duration::from_secs(1)).await;
|
||||
}
|
||||
@ -473,40 +435,18 @@ mod tests {
|
||||
};
|
||||
|
||||
let target_lock = tokio::sync::Mutex::new(());
|
||||
let receipt = OutboundDispatcher::send_with_retry(
|
||||
let error = OutboundDispatcher::send_with_retry(
|
||||
&channel,
|
||||
&outbound("invalid", "message"),
|
||||
&target_lock,
|
||||
)
|
||||
.await;
|
||||
.await
|
||||
.unwrap_err();
|
||||
|
||||
assert!(matches!(receipt, DeliveryReceipt::PermanentFailure { .. }));
|
||||
assert!(matches!(error, ChannelError::Other(_)));
|
||||
assert_eq!(channel.attempts.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn channel_errors_map_to_typed_sanitized_receipts() {
|
||||
for error in [
|
||||
ChannelError::ConnectionError("https://secret.example/?token=x".to_string()),
|
||||
ChannelError::SendError("private response body".to_string()),
|
||||
ChannelError::BusError("private queue detail".to_string()),
|
||||
] {
|
||||
let receipt = receipt_from_channel_error(error);
|
||||
assert!(matches!(receipt, DeliveryReceipt::TransientFailure { .. }));
|
||||
assert!(!format!("{receipt:?}").contains("private"));
|
||||
assert!(!format!("{receipt:?}").contains("secret"));
|
||||
}
|
||||
for error in [
|
||||
ChannelError::ConfigError("api_key=x".to_string()),
|
||||
ChannelError::Other("private platform payload".to_string()),
|
||||
] {
|
||||
let receipt = receipt_from_channel_error(error);
|
||||
assert!(matches!(receipt, DeliveryReceipt::PermanentFailure { .. }));
|
||||
assert!(!format!("{receipt:?}").contains("private"));
|
||||
assert!(!format!("{receipt:?}").contains("api_key"));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn shared_write_lock_orders_dispatcher_with_live_turn_writes() {
|
||||
let channel = RecordingChannel {
|
||||
@ -528,7 +468,7 @@ mod tests {
|
||||
assert!(channel.sent.lock().await.is_empty());
|
||||
|
||||
drop(live_write);
|
||||
assert_eq!(send.await, DeliveryReceipt::Delivered);
|
||||
send.await.unwrap();
|
||||
assert_eq!(channel.sent.lock().await.as_slice(), &["after-live-update"]);
|
||||
}
|
||||
}
|
||||
|
||||
@ -509,26 +509,17 @@ pub struct OutboundMessage {
|
||||
pub reply_to: Option<String>,
|
||||
pub media: Vec<MediaItem>,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub(crate) delivery: Option<tokio::sync::watch::Sender<Option<DeliveryReceipt>>>,
|
||||
pub(crate) delivery: Option<tokio::sync::watch::Sender<Option<Result<(), String>>>>,
|
||||
}
|
||||
|
||||
impl OutboundMessage {
|
||||
pub(crate) fn complete_delivery(&self, result: DeliveryReceipt) {
|
||||
pub(crate) fn complete_delivery(&self, result: Result<(), String>) {
|
||||
if let Some(delivery) = &self.delivery {
|
||||
delivery.send_replace(Some(result));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum DeliveryReceipt {
|
||||
Delivered,
|
||||
TransientFailure { summary: String },
|
||||
PermanentFailure { summary: String },
|
||||
TimedOut,
|
||||
DispatcherClosed,
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// ControlMessage - Message for control channel (session management)
|
||||
// Uses SessionCommand from session module
|
||||
|
||||
@ -4,8 +4,8 @@ pub mod message;
|
||||
pub use dispatcher::OutboundDispatcher;
|
||||
pub use message::{
|
||||
ChannelContext, ChatMessage, ClientVisibility, CommittedMessage, CommittedTurnDelta,
|
||||
CompletionStatus, ContentBlock, ControlMessage, DeliveryReceipt, InboundMessage, MediaItem,
|
||||
MediaRef, MessageSource, OutboundMessage, ProviderReasoningState, SourceKind, TurnOrigin,
|
||||
CompletionStatus, ContentBlock, ControlMessage, InboundMessage, MediaItem, MediaRef,
|
||||
MessageSource, OutboundMessage, ProviderReasoningState, SourceKind, TurnOrigin,
|
||||
};
|
||||
|
||||
use std::sync::Arc;
|
||||
@ -80,17 +80,7 @@ impl MessageBus {
|
||||
loop {
|
||||
delivery_rx.changed().await.map_err(|_| BusError::Closed)?;
|
||||
if let Some(result) = delivery_rx.borrow().clone() {
|
||||
return match result {
|
||||
DeliveryReceipt::Delivered => Ok(()),
|
||||
DeliveryReceipt::TransientFailure { summary } => {
|
||||
Err(BusError::DeliveryTransient(summary))
|
||||
}
|
||||
DeliveryReceipt::PermanentFailure { summary } => {
|
||||
Err(BusError::DeliveryPermanent(summary))
|
||||
}
|
||||
DeliveryReceipt::TimedOut => Err(BusError::DeliveryTimedOut),
|
||||
DeliveryReceipt::DispatcherClosed => Err(BusError::Closed),
|
||||
};
|
||||
return result.map_err(BusError::DeliveryFailed);
|
||||
}
|
||||
}
|
||||
})
|
||||
@ -148,8 +138,7 @@ pub struct QueueDepths {
|
||||
#[derive(Debug)]
|
||||
pub enum BusError {
|
||||
Closed,
|
||||
DeliveryTransient(String),
|
||||
DeliveryPermanent(String),
|
||||
DeliveryFailed(String),
|
||||
DeliveryTimedOut,
|
||||
}
|
||||
|
||||
@ -157,12 +146,7 @@ impl std::fmt::Display for BusError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
BusError::Closed => write!(f, "Bus channel closed"),
|
||||
BusError::DeliveryTransient(error) => {
|
||||
write!(f, "Transient outbound delivery failure: {error}")
|
||||
}
|
||||
BusError::DeliveryPermanent(error) => {
|
||||
write!(f, "Permanent outbound delivery failure: {error}")
|
||||
}
|
||||
BusError::DeliveryFailed(error) => write!(f, "Outbound delivery failed: {error}"),
|
||||
BusError::DeliveryTimedOut => write!(f, "Outbound delivery confirmation timed out"),
|
||||
}
|
||||
}
|
||||
|
||||
@ -812,12 +812,8 @@ fn scheduler_snapshot(jobs: &[crate::storage::ScheduledJob]) -> Value {
|
||||
.iter()
|
||||
.filter(|job| {
|
||||
matches!(
|
||||
job.last_outcome,
|
||||
Some(
|
||||
crate::storage::ScheduledOutcomeKind::Failed
|
||||
| crate::storage::ScheduledOutcomeKind::Refused
|
||||
| crate::storage::ScheduledOutcomeKind::Unknown
|
||||
)
|
||||
job.last_status.as_deref(),
|
||||
Some("error" | "timeout" | "delivery_error")
|
||||
)
|
||||
})
|
||||
.count();
|
||||
@ -1469,26 +1465,6 @@ pub async fn get_job_runs(
|
||||
.list_scheduled_job_runs(&id, limit)
|
||||
.await
|
||||
.map_err(ApiError::internal)?;
|
||||
let runs = runs
|
||||
.into_iter()
|
||||
.map(|run| {
|
||||
json!({
|
||||
"id": run.id,
|
||||
"job_id": run.job_id,
|
||||
"scheduled_for": run.scheduled_for,
|
||||
"started_at": run.started_at,
|
||||
"finished_at": run.finished_at,
|
||||
"status": run.status,
|
||||
"outcome": run.outcome,
|
||||
"message": run.message,
|
||||
"diagnostic": run.diagnostic,
|
||||
"duration_ms": run.duration_ms,
|
||||
"delivery_status": run.delivery_status,
|
||||
"delivery_attempts": run.delivery_attempts,
|
||||
"delivery_error": run.delivery_error,
|
||||
})
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
Ok(Json(json!({ "runs": runs })))
|
||||
}
|
||||
|
||||
@ -1588,57 +1564,37 @@ mod tests {
|
||||
id: &str,
|
||||
enabled: bool,
|
||||
next_run_at: i64,
|
||||
last_outcome: Option<crate::storage::ScheduledOutcomeKind>,
|
||||
last_status: Option<&str>,
|
||||
) -> crate::storage::ScheduledJob {
|
||||
crate::storage::ScheduledJob {
|
||||
id: id.to_string(),
|
||||
name: id.to_string(),
|
||||
schedule: crate::scheduler::Schedule::Every { every_ms: 60_000 },
|
||||
prompt: String::new(),
|
||||
agent_id: None,
|
||||
channel: "cli_chat".to_string(),
|
||||
chat_id: "test".to_string(),
|
||||
model: None,
|
||||
job_kind: crate::storage::JobKind::Task,
|
||||
delivery_policy: crate::storage::DeliveryPolicy::Never,
|
||||
enabled,
|
||||
delete_after_run: false,
|
||||
next_run_at,
|
||||
last_run_at: None,
|
||||
last_outcome,
|
||||
last_status: last_status.map(str::to_string),
|
||||
last_error: None,
|
||||
created_at: 0,
|
||||
updated_at: 0,
|
||||
locked_at: None,
|
||||
lock_owner: None,
|
||||
lease_until: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scheduler_snapshot_classifies_failures_and_next_enabled_run() {
|
||||
let jobs = vec![
|
||||
scheduled_job(
|
||||
"healthy",
|
||||
true,
|
||||
300,
|
||||
Some(crate::storage::ScheduledOutcomeKind::Ok),
|
||||
),
|
||||
scheduled_job(
|
||||
"error",
|
||||
true,
|
||||
200,
|
||||
Some(crate::storage::ScheduledOutcomeKind::Failed),
|
||||
),
|
||||
scheduled_job(
|
||||
"refused",
|
||||
false,
|
||||
100,
|
||||
Some(crate::storage::ScheduledOutcomeKind::Refused),
|
||||
),
|
||||
scheduled_job(
|
||||
"unknown",
|
||||
true,
|
||||
400,
|
||||
Some(crate::storage::ScheduledOutcomeKind::Unknown),
|
||||
),
|
||||
scheduled_job("other", false, 50, None),
|
||||
scheduled_job("healthy", true, 300, Some("ok")),
|
||||
scheduled_job("error", true, 200, Some("error")),
|
||||
scheduled_job("timeout", false, 100, Some("timeout")),
|
||||
scheduled_job("delivery", true, 400, Some("delivery_error")),
|
||||
scheduled_job("other", false, 50, Some("cancelled")),
|
||||
];
|
||||
|
||||
assert_eq!(
|
||||
|
||||
@ -178,7 +178,6 @@ impl GatewayState {
|
||||
.init(&config, workspace_path.clone())
|
||||
.await
|
||||
.map_err(|e| format!("Failed to init channels: {}", e))?;
|
||||
let available_channels = channel_manager.list_channel_names().await;
|
||||
let turn_delivery = TurnDeliveryService::new(
|
||||
delivery_coordinator.clone(),
|
||||
channel_manager.clone(),
|
||||
@ -190,10 +189,7 @@ impl GatewayState {
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let health = Arc::new(
|
||||
crate::health::HealthService::new(config.clone())
|
||||
.with_scheduler_runtime(storage.clone(), available_channels.clone()),
|
||||
);
|
||||
let health = Arc::new(crate::health::HealthService::new(config.clone()));
|
||||
let provider_profiles: std::collections::HashMap<String, _> = config
|
||||
.agents
|
||||
.keys()
|
||||
@ -249,9 +245,9 @@ impl GatewayState {
|
||||
let session_manager = Arc::new(session_manager);
|
||||
session_manager.bind_inbox_wake();
|
||||
let agent_catalog = session_manager.agent_catalog();
|
||||
health.bind_agent_catalog(agent_catalog.clone());
|
||||
|
||||
// Register send_message tool with available channel names
|
||||
let available_channels = channel_manager.list_channel_names().await;
|
||||
let valid_channels = available_channels.clone();
|
||||
session_manager.register_outbound_tool(available_channels);
|
||||
|
||||
@ -281,15 +277,11 @@ impl GatewayState {
|
||||
.tools()
|
||||
.register(crate::tools::cron::CronAddTool::new(
|
||||
storage.clone(),
|
||||
valid_channels.clone(),
|
||||
agent_catalog.clone(),
|
||||
valid_channels,
|
||||
));
|
||||
session_manager
|
||||
.tools()
|
||||
.register(crate::tools::cron::CronListTool::new(storage.clone()));
|
||||
session_manager
|
||||
.tools()
|
||||
.register(crate::tools::cron::CronRunsTool::new(storage.clone()));
|
||||
session_manager
|
||||
.tools()
|
||||
.register(crate::tools::cron::CronRemoveTool::new(storage.clone()));
|
||||
@ -301,11 +293,7 @@ impl GatewayState {
|
||||
.register(crate::tools::cron::CronDisableTool::new(storage.clone()));
|
||||
session_manager
|
||||
.tools()
|
||||
.register(crate::tools::cron::CronUpdateTool::new(
|
||||
storage.clone(),
|
||||
valid_channels,
|
||||
agent_catalog.clone(),
|
||||
));
|
||||
.register(crate::tools::cron::CronUpdateTool::new(storage.clone()));
|
||||
tracing::info!("Cron tools registered");
|
||||
}
|
||||
|
||||
@ -344,25 +332,7 @@ impl GatewayState {
|
||||
}
|
||||
|
||||
/// Start the message processing loops
|
||||
pub async fn start_message_processing(&self) -> Result<(), String> {
|
||||
match self
|
||||
.storage
|
||||
.recover_scheduled_runs(chrono::Utc::now().timestamp_millis())
|
||||
.await
|
||||
{
|
||||
Ok(recovered) if recovered > 0 => {
|
||||
tracing::warn!(
|
||||
recovered,
|
||||
"Scheduled runs recovered as unknown on activation"
|
||||
);
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(error) => {
|
||||
return Err(format!(
|
||||
"Scheduled run recovery failed on activation: {error}"
|
||||
));
|
||||
}
|
||||
}
|
||||
pub async fn start_message_processing(&self) {
|
||||
// Recover durable Agent state for this runtime generation: interrupt
|
||||
// runs of older generations, expire stale inbox leases and reconcile
|
||||
// capacity rows. Runs never recover while the generation is still a candidate.
|
||||
@ -381,9 +351,7 @@ impl GatewayState {
|
||||
}
|
||||
}
|
||||
Err(error) => {
|
||||
return Err(format!(
|
||||
"Agent state recovery failed on activation: {error}"
|
||||
));
|
||||
tracing::error!(error = %error, "Agent state recovery failed on activation");
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -493,7 +461,6 @@ impl GatewayState {
|
||||
});
|
||||
tracing::info!("Scheduler background task spawned");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
@ -558,7 +525,7 @@ pub async fn run(
|
||||
reload_controller.set_failed(current_generation, error.to_string());
|
||||
return Err(error.into());
|
||||
}
|
||||
state.start_message_processing().await?;
|
||||
state.start_message_processing().await;
|
||||
reload_controller.set_phase(current_generation, reload::ReloadPhase::Active);
|
||||
let app = build_router(state.clone());
|
||||
let generation_listener = TcpListener::from_std(listener.try_clone()?)?;
|
||||
|
||||
@ -459,7 +459,7 @@ mod tests {
|
||||
Some("command")
|
||||
);
|
||||
assert!(!publish_task.is_finished());
|
||||
output.complete_delivery(crate::bus::DeliveryReceipt::Delivered);
|
||||
output.complete_delivery(Ok(()));
|
||||
publish_task.await.unwrap();
|
||||
}
|
||||
|
||||
|
||||
221
src/health.rs
221
src/health.rs
@ -1,7 +1,6 @@
|
||||
use std::collections::HashSet;
|
||||
use std::path::Path;
|
||||
use std::process::{Output, Stdio};
|
||||
use std::sync::{Arc, RwLock};
|
||||
use std::time::Duration;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
@ -118,41 +117,11 @@ impl HealthReport {
|
||||
#[derive(Clone)]
|
||||
pub struct HealthService {
|
||||
config: Config,
|
||||
scheduler_runtime: Arc<RwLock<Option<SchedulerHealthRuntime>>>,
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct SchedulerHealthRuntime {
|
||||
storage: Arc<crate::storage::Storage>,
|
||||
channels: HashSet<String>,
|
||||
catalog: Option<Arc<crate::agent::AgentCatalog>>,
|
||||
}
|
||||
|
||||
impl HealthService {
|
||||
pub fn new(config: Config) -> Self {
|
||||
Self {
|
||||
config,
|
||||
scheduler_runtime: Arc::new(RwLock::new(None)),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_scheduler_runtime(
|
||||
self,
|
||||
storage: Arc<crate::storage::Storage>,
|
||||
channels: Vec<String>,
|
||||
) -> Self {
|
||||
*self.scheduler_runtime.write().unwrap() = Some(SchedulerHealthRuntime {
|
||||
storage,
|
||||
channels: channels.into_iter().collect(),
|
||||
catalog: None,
|
||||
});
|
||||
self
|
||||
}
|
||||
|
||||
pub fn bind_agent_catalog(&self, catalog: Arc<crate::agent::AgentCatalog>) {
|
||||
if let Some(runtime) = self.scheduler_runtime.write().unwrap().as_mut() {
|
||||
runtime.catalog = Some(catalog);
|
||||
}
|
||||
Self { config }
|
||||
}
|
||||
|
||||
pub async fn check(&self) -> HealthReport {
|
||||
@ -170,165 +139,9 @@ impl HealthService {
|
||||
];
|
||||
checks.extend(self.check_mcp_commands());
|
||||
checks.extend(self.check_browser().await);
|
||||
checks.extend(self.check_scheduler().await);
|
||||
HealthReport::from_checks(checks)
|
||||
}
|
||||
|
||||
async fn check_scheduler(&self) -> Vec<HealthCheck> {
|
||||
let scheduler_config = self.config.gateway.scheduler.clone().unwrap_or_default();
|
||||
if !scheduler_config.enabled {
|
||||
return vec![HealthCheck {
|
||||
name: "scheduled runs".to_string(),
|
||||
category: "configured".to_string(),
|
||||
required: false,
|
||||
status: HealthStatus::Pass,
|
||||
detail: "scheduler disabled; persisted jobs are not executed".to_string(),
|
||||
remediation: None,
|
||||
}];
|
||||
}
|
||||
let runtime = self.scheduler_runtime.read().unwrap().clone();
|
||||
let Some(runtime) = runtime else {
|
||||
return vec![HealthCheck {
|
||||
name: "scheduled runs".to_string(),
|
||||
category: "runtime".to_string(),
|
||||
required: false,
|
||||
status: HealthStatus::Pass,
|
||||
detail: "runtime scheduler state is checked by the running Gateway".to_string(),
|
||||
remediation: None,
|
||||
}];
|
||||
};
|
||||
let jobs = match runtime.storage.list_scheduled_jobs().await {
|
||||
Ok(jobs) => jobs,
|
||||
Err(error) => {
|
||||
return vec![HealthCheck {
|
||||
name: "scheduled run storage".to_string(),
|
||||
category: "runtime".to_string(),
|
||||
required: true,
|
||||
status: HealthStatus::Fail,
|
||||
detail: format!("cannot read scheduled jobs: {error}"),
|
||||
remediation: Some(
|
||||
"Check the configured SQLite database and Gateway logs.".to_string(),
|
||||
),
|
||||
}];
|
||||
}
|
||||
};
|
||||
let now = chrono::Utc::now().timestamp_millis();
|
||||
let stale_after = scheduler_config
|
||||
.poll_interval_secs
|
||||
.max(60)
|
||||
.saturating_mul(10_000)
|
||||
.min(i64::MAX as u64) as i64;
|
||||
let mut bad_references = Vec::new();
|
||||
let mut stale_deliveries = Vec::new();
|
||||
let mut unhealthy_latest = Vec::new();
|
||||
let mut silent_unknown = Vec::new();
|
||||
let mut covered_intervals = Vec::new();
|
||||
let mut invalid_next = Vec::new();
|
||||
|
||||
match runtime
|
||||
.storage
|
||||
.list_stale_scheduled_deliveries(now.saturating_sub(stale_after), 100)
|
||||
.await
|
||||
{
|
||||
Ok(runs) => {
|
||||
stale_deliveries.extend(
|
||||
runs.into_iter()
|
||||
.map(|run| format!("{}#{}", run.job_id, run.id)),
|
||||
);
|
||||
}
|
||||
Err(error) => bad_references.push(format!("delivery backlog unavailable ({error})")),
|
||||
}
|
||||
|
||||
for job in &jobs {
|
||||
if !runtime.channels.contains(&job.channel) {
|
||||
bad_references.push(format!("{}: channel {}", job.id, job.channel));
|
||||
}
|
||||
if let Some(agent_id) = job.agent_id.as_deref()
|
||||
&& runtime
|
||||
.catalog
|
||||
.as_ref()
|
||||
.is_none_or(|catalog| catalog.get(agent_id).is_none())
|
||||
{
|
||||
bad_references.push(format!("{}: agent {}", job.id, agent_id));
|
||||
}
|
||||
if job.enabled && crate::scheduler::next_run_for_schedule(&job.schedule, now).is_none()
|
||||
{
|
||||
invalid_next.push(job.id.clone());
|
||||
}
|
||||
|
||||
let runs = match runtime.storage.list_scheduled_job_runs(&job.id, 20).await {
|
||||
Ok(runs) => runs,
|
||||
Err(error) => {
|
||||
bad_references.push(format!("{}: run history unavailable ({error})", job.id));
|
||||
continue;
|
||||
}
|
||||
};
|
||||
if let Some(latest) = runs.first() {
|
||||
if matches!(
|
||||
latest.status,
|
||||
crate::storage::ScheduledRunStatus::Unknown
|
||||
| crate::storage::ScheduledRunStatus::Failed
|
||||
| crate::storage::ScheduledRunStatus::TimedOut
|
||||
) {
|
||||
unhealthy_latest.push(format!("{}#{}", job.id, latest.id));
|
||||
}
|
||||
if job.delivery_policy == crate::storage::DeliveryPolicy::Never
|
||||
&& latest.outcome == Some(crate::storage::ScheduledOutcomeKind::Unknown)
|
||||
{
|
||||
silent_unknown.push(format!("{}#{}", job.id, latest.id));
|
||||
}
|
||||
if let crate::scheduler::Schedule::Every { every_ms } = job.schedule
|
||||
&& i64::try_from(every_ms).ok().is_some_and(|interval| {
|
||||
latest
|
||||
.duration_ms
|
||||
.is_some_and(|duration| duration >= interval)
|
||||
})
|
||||
{
|
||||
covered_intervals.push(format!("{}#{}", job.id, latest.id));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
vec![
|
||||
scheduler_health_check(
|
||||
"scheduled references",
|
||||
bad_references,
|
||||
"all Agent and channel references are available",
|
||||
"Update or disable jobs that reference missing Agents or channels.",
|
||||
),
|
||||
scheduler_health_check(
|
||||
"scheduled delivery backlog",
|
||||
stale_deliveries,
|
||||
"no stale pending or delivering notifications",
|
||||
"Inspect cron_runs and the target channel configuration.",
|
||||
),
|
||||
scheduler_health_check(
|
||||
"scheduled latest outcomes",
|
||||
unhealthy_latest,
|
||||
"no latest run is failed, timed out, or unknown",
|
||||
"Inspect cron_runs for the diagnostic and assess external side effects before retrying.",
|
||||
),
|
||||
scheduler_health_check(
|
||||
"silent unknown scheduled runs",
|
||||
silent_unknown,
|
||||
"no never-delivery job has an unknown latest outcome",
|
||||
"Inspect the run manually; delivery_policy=never prevents automatic notification.",
|
||||
),
|
||||
scheduler_health_check(
|
||||
"scheduled execution intervals",
|
||||
covered_intervals,
|
||||
"recent execution durations fit their Every intervals",
|
||||
"Increase the interval or split long-running jobs.",
|
||||
),
|
||||
scheduler_health_check(
|
||||
"scheduled next runs",
|
||||
invalid_next,
|
||||
"all enabled schedules can compute a next run",
|
||||
"Correct the schedule expression or disable the job.",
|
||||
),
|
||||
]
|
||||
}
|
||||
|
||||
fn check_mcp_commands(&self) -> Vec<HealthCheck> {
|
||||
let mut seen = HashSet::new();
|
||||
let mut checks = Vec::new();
|
||||
@ -552,38 +365,6 @@ impl HealthService {
|
||||
}
|
||||
}
|
||||
|
||||
fn scheduler_health_check(
|
||||
name: &str,
|
||||
findings: Vec<String>,
|
||||
healthy_detail: &str,
|
||||
remediation: &str,
|
||||
) -> HealthCheck {
|
||||
if findings.is_empty() {
|
||||
HealthCheck {
|
||||
name: name.to_string(),
|
||||
category: "runtime".to_string(),
|
||||
required: false,
|
||||
status: HealthStatus::Pass,
|
||||
detail: healthy_detail.to_string(),
|
||||
remediation: None,
|
||||
}
|
||||
} else {
|
||||
let total = findings.len();
|
||||
let mut sample = findings.into_iter().take(5).collect::<Vec<_>>().join(", ");
|
||||
if total > 5 {
|
||||
sample.push_str(&format!(", and {} more", total - 5));
|
||||
}
|
||||
HealthCheck {
|
||||
name: name.to_string(),
|
||||
category: "runtime".to_string(),
|
||||
required: false,
|
||||
status: HealthStatus::Warning,
|
||||
detail: format!("{total} finding(s): {sample}"),
|
||||
remediation: Some(remediation.to_string()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn check_configuration_recovery(config: &Config) -> HealthCheck {
|
||||
if config.diagnostics.is_empty() {
|
||||
return HealthCheck {
|
||||
|
||||
@ -3,41 +3,81 @@ pub mod types;
|
||||
use std::sync::Arc;
|
||||
use std::time::Instant;
|
||||
|
||||
use tokio::task::JoinSet;
|
||||
use futures_util::stream::{self, StreamExt};
|
||||
use tokio::time;
|
||||
|
||||
use crate::config::SchedulerConfig;
|
||||
use crate::session::{ScheduledDeliveryError, SessionManager};
|
||||
use crate::storage::{
|
||||
ClaimedScheduledRun, JobRun, ScheduledOutcomeKind, ScheduledRunCompletion, ScheduledRunStatus,
|
||||
Storage,
|
||||
};
|
||||
use crate::session::SessionManager;
|
||||
use crate::session::session::HandleResult;
|
||||
use crate::storage::ScheduledJob;
|
||||
use crate::storage::Storage;
|
||||
use crate::storage::{DeliveryPolicy, JobKind, JobRun};
|
||||
|
||||
pub use types::Schedule;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
enum ScheduledDisposition {
|
||||
Content(String),
|
||||
Quiet(String),
|
||||
ReportedFailure(String),
|
||||
Refused(String),
|
||||
}
|
||||
|
||||
fn parse_scheduled_disposition(output: &str) -> ScheduledDisposition {
|
||||
let trimmed = output.trim();
|
||||
if trimmed.eq_ignore_ascii_case("NO_REPLY") {
|
||||
return ScheduledDisposition::Quiet(String::new());
|
||||
}
|
||||
let upper = trimmed.to_ascii_uppercase();
|
||||
for (prefix, kind) in [
|
||||
("NO_REPLY[INFO]", "info"),
|
||||
("NO_REPLY[FAIL]", "fail"),
|
||||
("NO_REPLY[REFUSE]", "refuse"),
|
||||
] {
|
||||
if upper.starts_with(prefix) {
|
||||
let suffix = &trimmed[prefix.len()..];
|
||||
if !suffix.is_empty() && !suffix.trim_start().starts_with(':') {
|
||||
continue;
|
||||
}
|
||||
let reason = suffix.trim().trim_start_matches(':').trim().to_string();
|
||||
return match kind {
|
||||
"info" => ScheduledDisposition::Quiet(reason),
|
||||
"fail" => ScheduledDisposition::ReportedFailure(reason),
|
||||
_ => ScheduledDisposition::Refused(reason),
|
||||
};
|
||||
}
|
||||
}
|
||||
if trimmed.is_empty() {
|
||||
ScheduledDisposition::ReportedFailure("scheduled agent returned empty output".into())
|
||||
} else {
|
||||
ScheduledDisposition::Content(trimmed.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
/// Compute the next execution time (Unix ms) for a schedule, given `from` (Unix ms).
|
||||
/// Returns `None` if no next time can be determined (e.g. an invalid cron expression).
|
||||
/// Returns `None` if no next time can be determined (e.g., invalid cron expression).
|
||||
pub fn next_run_for_schedule(schedule: &Schedule, from: i64) -> Option<i64> {
|
||||
use chrono::{TimeZone, Utc};
|
||||
use std::str::FromStr;
|
||||
|
||||
match schedule {
|
||||
Schedule::At { at } => Some(*at),
|
||||
Schedule::Every { every_ms } => Some(from.saturating_add(i64::try_from(*every_ms).ok()?)),
|
||||
Schedule::Every { every_ms } => Some(from + *every_ms as i64),
|
||||
Schedule::Cron { expr, tz } => {
|
||||
let cron_schedule = cron::Schedule::from_str(expr.as_str()).ok()?;
|
||||
let from_secs = from / 1000;
|
||||
let from_nanos = ((from % 1000) * 1_000_000) as u32;
|
||||
let from_dt = Utc.timestamp_opt(from_secs, from_nanos).single()?;
|
||||
|
||||
let next_utc = if let Some(tz_str) = tz {
|
||||
let tz: chrono_tz::Tz = tz_str.parse().ok()?;
|
||||
cron_schedule
|
||||
.after(&from_dt.with_timezone(&tz))
|
||||
.next()?
|
||||
.with_timezone(&Utc)
|
||||
let from_local = from_dt.with_timezone(&tz);
|
||||
let next_local = cron_schedule.after(&from_local).next()?;
|
||||
next_local.with_timezone(&Utc)
|
||||
} else {
|
||||
cron_schedule.after(&from_dt).next()?
|
||||
};
|
||||
|
||||
Some(next_utc.timestamp_millis())
|
||||
}
|
||||
}
|
||||
@ -50,6 +90,8 @@ fn now_ms() -> i64 {
|
||||
.as_millis() as i64
|
||||
}
|
||||
|
||||
/// The scheduler runs as a background tokio task, periodically checking for due jobs
|
||||
/// and executing them via `SessionManager::handle_cron_message`.
|
||||
pub struct Scheduler {
|
||||
storage: Arc<Storage>,
|
||||
session_manager: Arc<SessionManager>,
|
||||
@ -87,16 +129,15 @@ impl Scheduler {
|
||||
}
|
||||
}
|
||||
|
||||
/// Non-blocking event loop. Execution and delivery use separate bounded
|
||||
/// JoinSets so one long Agent run cannot delay other claims or outbox work.
|
||||
/// Claim due jobs with a durable lease, then execute the claimed batch with
|
||||
/// bounded concurrency.
|
||||
pub async fn run(self: Arc<Self>) {
|
||||
let poll_duration = time::Duration::from_secs(self.config.poll_interval_secs.max(1));
|
||||
let mut interval = time::interval(poll_duration);
|
||||
interval.set_missed_tick_behavior(time::MissedTickBehavior::Skip);
|
||||
// Keep accidental configuration values from claiming an unbounded
|
||||
// batch and overwhelming the runtime or SQLite parameter conversion.
|
||||
let max_concurrent = self.config.max_concurrent.clamp(1, 256);
|
||||
let max_delivery = max_concurrent.clamp(1, 16);
|
||||
let mut runs = JoinSet::new();
|
||||
let mut deliveries = JoinSet::new();
|
||||
|
||||
tracing::info!(
|
||||
poll_interval_secs = self.config.poll_interval_secs,
|
||||
@ -106,306 +147,272 @@ impl Scheduler {
|
||||
);
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
_ = interval.tick() => {}
|
||||
Some(result) = runs.join_next(), if !runs.is_empty() => {
|
||||
if let Err(error) = result {
|
||||
tracing::error!(error = %error, "scheduled run task panicked");
|
||||
}
|
||||
}
|
||||
Some(result) = deliveries.join_next(), if !deliveries.is_empty() => {
|
||||
if let Err(error) = result {
|
||||
tracing::error!(error = %error, "scheduled delivery task panicked");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
while let Some(result) = runs.try_join_next() {
|
||||
if let Err(error) = result {
|
||||
tracing::error!(error = %error, "scheduled run task panicked");
|
||||
}
|
||||
}
|
||||
while let Some(result) = deliveries.try_join_next() {
|
||||
if let Err(error) = result {
|
||||
tracing::error!(error = %error, "scheduled delivery task panicked");
|
||||
}
|
||||
}
|
||||
interval.tick().await;
|
||||
if !self.admission.is_accepting() {
|
||||
continue;
|
||||
}
|
||||
|
||||
let now = now_ms();
|
||||
let delivery_slots = max_delivery.saturating_sub(deliveries.len());
|
||||
if delivery_slots > 0 {
|
||||
let lease_until = now.saturating_add(180_000);
|
||||
let delivery_owner = format!("{}:delivery:{}", self.owner, uuid::Uuid::new_v4());
|
||||
match self
|
||||
.storage
|
||||
.claim_scheduled_deliveries(now, lease_until, &delivery_owner, delivery_slots)
|
||||
.await
|
||||
{
|
||||
Ok(claimed) => {
|
||||
for run in claimed {
|
||||
let scheduler = self.clone();
|
||||
deliveries.spawn(async move {
|
||||
scheduler.deliver_claimed_run(run).await;
|
||||
});
|
||||
}
|
||||
}
|
||||
Err(error) => {
|
||||
tracing::error!(error = %error, "scheduler: failed to claim deliveries");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let run_slots = max_concurrent.saturating_sub(runs.len());
|
||||
if run_slots == 0 {
|
||||
continue;
|
||||
}
|
||||
let lease_ms = self
|
||||
.config
|
||||
.execution_timeout_secs
|
||||
.saturating_add(150)
|
||||
.saturating_mul(1000)
|
||||
.min(i64::MAX as u64) as i64;
|
||||
let run_owner = format!("{}:run:{}", self.owner, uuid::Uuid::new_v4());
|
||||
match self
|
||||
let lease_until = now.saturating_add(lease_ms);
|
||||
let jobs = match self
|
||||
.storage
|
||||
.claim_due_scheduled_runs(now, now.saturating_add(lease_ms), &run_owner, run_slots)
|
||||
.claim_due_scheduled_jobs(now, lease_until, &self.owner, max_concurrent)
|
||||
.await
|
||||
{
|
||||
Ok(claimed) => {
|
||||
for run in claimed {
|
||||
let scheduler = self.clone();
|
||||
runs.spawn(async move {
|
||||
scheduler.execute_claimed_run(run).await;
|
||||
});
|
||||
}
|
||||
}
|
||||
Ok(jobs) => jobs,
|
||||
Err(error) => {
|
||||
tracing::error!(error = %error, "scheduler: failed to claim due runs");
|
||||
tracing::error!(error = %error, "scheduler: failed to claim due jobs");
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
if jobs.is_empty() {
|
||||
continue;
|
||||
}
|
||||
tracing::info!(count = jobs.len(), "scheduler: claimed due jobs");
|
||||
|
||||
stream::iter(jobs)
|
||||
.for_each_concurrent(max_concurrent, |job| {
|
||||
let scheduler = self.clone();
|
||||
async move { scheduler.execute_claimed_job(job).await }
|
||||
})
|
||||
.await;
|
||||
}
|
||||
}
|
||||
|
||||
async fn execute_claimed_run(self: Arc<Self>, claimed: ClaimedScheduledRun) {
|
||||
async fn execute_claimed_job(self: Arc<Self>, job: ScheduledJob) {
|
||||
let Some(_activity) = self.admission.try_enter() else {
|
||||
if let Err(error) = self
|
||||
.storage
|
||||
.release_scheduled_job_lease(&job.id, &self.owner)
|
||||
.await
|
||||
{
|
||||
tracing::error!(job_id = %job.id, error = %error, "scheduler: failed to release job claimed during reload drain");
|
||||
}
|
||||
return;
|
||||
};
|
||||
let start = Instant::now();
|
||||
let job = &claimed.job;
|
||||
let (completion, agent_execution) = if let Some(_activity) = self.admission.try_enter() {
|
||||
match self.session_manager.agent_coordinator() {
|
||||
Some(coordinator) => match coordinator
|
||||
.execute_scheduled(
|
||||
claimed.run_id,
|
||||
&claimed.owner,
|
||||
let started_at = now_ms();
|
||||
tracing::info!(job_id = %job.id, job_name = %job.name, "scheduler: executing claimed job");
|
||||
|
||||
let managed = job.delivery_policy != DeliveryPolicy::Direct;
|
||||
let execution = async {
|
||||
if managed {
|
||||
self.session_manager
|
||||
.handle_managed_scheduled_message(
|
||||
&job.prompt,
|
||||
&job.id,
|
||||
&job.name,
|
||||
job.agent_id.as_deref(),
|
||||
&job.prompt,
|
||||
self.config.execution_timeout_secs.max(1),
|
||||
job.job_kind == JobKind::Monitor,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(execution) => {
|
||||
if execution.status == ScheduledRunStatus::Completed
|
||||
&& let Some(outcome) = execution.outcome.clone()
|
||||
{
|
||||
(
|
||||
ScheduledRunCompletion {
|
||||
status: ScheduledRunStatus::Completed,
|
||||
outcome: outcome.kind,
|
||||
message: outcome.message,
|
||||
diagnostic: execution.error.clone(),
|
||||
duration_ms: start.elapsed().as_millis() as i64,
|
||||
},
|
||||
Some(execution),
|
||||
)
|
||||
.map(HandleResult::AgentResponse)
|
||||
} else {
|
||||
self.session_manager
|
||||
.handle_cron_message(
|
||||
&job.channel,
|
||||
&job.chat_id,
|
||||
&job.prompt,
|
||||
&job.id,
|
||||
&job.name,
|
||||
)
|
||||
.await
|
||||
}
|
||||
};
|
||||
let result = time::timeout(
|
||||
time::Duration::from_secs(self.config.execution_timeout_secs.max(1)),
|
||||
execution,
|
||||
)
|
||||
.await;
|
||||
let finished_at = now_ms();
|
||||
let duration_ms = start.elapsed().as_millis() as i64;
|
||||
|
||||
let (mut status, output, error, result_kind, mut delivery_status, mut delivery_error) =
|
||||
match result {
|
||||
Ok(Ok(
|
||||
HandleResult::AgentResponse(output) | HandleResult::CommandOutput(output),
|
||||
)) => {
|
||||
let output = if output.len() > 8000 {
|
||||
format!(
|
||||
"{}...[truncated]",
|
||||
&output[..output.ceil_char_boundary(8000)]
|
||||
)
|
||||
} else {
|
||||
output
|
||||
};
|
||||
if !managed {
|
||||
(
|
||||
"ok".into(),
|
||||
Some(output),
|
||||
None,
|
||||
None,
|
||||
Some("direct".into()),
|
||||
None,
|
||||
)
|
||||
} else {
|
||||
let disposition = parse_scheduled_disposition(&output);
|
||||
let (kind, content, alert) = match &disposition {
|
||||
ScheduledDisposition::Content(value) => {
|
||||
("content", Some(value.as_str()), true)
|
||||
}
|
||||
ScheduledDisposition::Quiet(_) => ("quiet", None, false),
|
||||
ScheduledDisposition::ReportedFailure(value) => {
|
||||
("reported_failure", Some(value.as_str()), true)
|
||||
}
|
||||
ScheduledDisposition::Refused(value) => {
|
||||
("refused", Some(value.as_str()), true)
|
||||
}
|
||||
};
|
||||
let should_deliver = match job.delivery_policy {
|
||||
DeliveryPolicy::Always => true,
|
||||
DeliveryPolicy::OnAlert => alert,
|
||||
DeliveryPolicy::Never => false,
|
||||
DeliveryPolicy::Direct => false,
|
||||
};
|
||||
if should_deliver {
|
||||
let message = content.unwrap_or("巡检完成，未发现需要关注的问题。");
|
||||
match self
|
||||
.session_manager
|
||||
.deliver_scheduled_message(
|
||||
&job.channel,
|
||||
&job.chat_id,
|
||||
&job.id,
|
||||
&job.name,
|
||||
message,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(()) => (
|
||||
"ok".into(),
|
||||
Some(output),
|
||||
None,
|
||||
Some(kind.into()),
|
||||
Some("delivered".into()),
|
||||
None,
|
||||
),
|
||||
Err(delivery_error) => (
|
||||
"delivery_error".into(),
|
||||
Some(output),
|
||||
None,
|
||||
Some(kind.into()),
|
||||
Some("failed".into()),
|
||||
Some(delivery_error),
|
||||
),
|
||||
}
|
||||
} else {
|
||||
let diagnostic = execution.error.clone().unwrap_or_else(|| {
|
||||
"scheduled Agent returned ordinary text without calling complete_scheduled_run"
|
||||
.to_string()
|
||||
});
|
||||
let status = match execution.status {
|
||||
ScheduledRunStatus::TimedOut => ScheduledRunStatus::TimedOut,
|
||||
ScheduledRunStatus::Interrupted | ScheduledRunStatus::Cancelled => {
|
||||
ScheduledRunStatus::Interrupted
|
||||
}
|
||||
_ => ScheduledRunStatus::Failed,
|
||||
let delivery = if job.delivery_policy == DeliveryPolicy::Never {
|
||||
"skipped"
|
||||
} else {
|
||||
"suppressed"
|
||||
};
|
||||
(
|
||||
ScheduledRunCompletion {
|
||||
status,
|
||||
outcome: ScheduledOutcomeKind::Failed,
|
||||
message: if status == ScheduledRunStatus::TimedOut {
|
||||
format!("定时任务「{}」执行超时。", job.name)
|
||||
} else if status == ScheduledRunStatus::Interrupted {
|
||||
format!("定时任务「{}」在系统关停时被中断。", job.name)
|
||||
} else {
|
||||
format!(
|
||||
"定时任务「{}」未可靠完成：Agent 未提交结构化运行结果。",
|
||||
job.name
|
||||
)
|
||||
},
|
||||
diagnostic: Some(diagnostic),
|
||||
duration_ms: start.elapsed().as_millis() as i64,
|
||||
},
|
||||
Some(execution),
|
||||
"ok".into(),
|
||||
Some(output),
|
||||
None,
|
||||
Some(kind.into()),
|
||||
Some(delivery.into()),
|
||||
None,
|
||||
)
|
||||
}
|
||||
}
|
||||
Err(error) => (
|
||||
ScheduledRunCompletion {
|
||||
status: ScheduledRunStatus::Failed,
|
||||
outcome: ScheduledOutcomeKind::Failed,
|
||||
message: job.agent_id.as_deref().map_or_else(
|
||||
|| format!("定时任务「{}」未能启动 Root Agent 执行。", job.name),
|
||||
|agent_id| {
|
||||
format!(
|
||||
"定时任务「{}」无法使用 Agent「{}」执行。请恢复该 Agent 定义，或更新任务的 agent_id。",
|
||||
job.name, agent_id
|
||||
)
|
||||
},
|
||||
),
|
||||
diagnostic: Some(error.to_string()),
|
||||
duration_ms: start.elapsed().as_millis() as i64,
|
||||
},
|
||||
None,
|
||||
),
|
||||
},
|
||||
None => (
|
||||
ScheduledRunCompletion {
|
||||
status: ScheduledRunStatus::Failed,
|
||||
outcome: ScheduledOutcomeKind::Failed,
|
||||
message: format!("定时任务「{}」未能启动执行。", job.name),
|
||||
diagnostic: Some("AgentCoordinator is unavailable".to_string()),
|
||||
duration_ms: start.elapsed().as_millis() as i64,
|
||||
},
|
||||
}
|
||||
Ok(Ok(HandleResult::AgentProcessing)) => (
|
||||
"error".to_string(),
|
||||
None,
|
||||
Some("cron execution returned asynchronous processing".to_string()),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
),
|
||||
Ok(Err(error)) => (
|
||||
"error".to_string(),
|
||||
None,
|
||||
Some(error.to_string()),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
),
|
||||
Err(_) => (
|
||||
"timeout".to_string(),
|
||||
None,
|
||||
Some(format!(
|
||||
"execution exceeded {} seconds",
|
||||
self.config.execution_timeout_secs.max(1)
|
||||
)),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
),
|
||||
}
|
||||
} else {
|
||||
(
|
||||
ScheduledRunCompletion {
|
||||
status: ScheduledRunStatus::Interrupted,
|
||||
outcome: ScheduledOutcomeKind::Failed,
|
||||
message: format!("定时任务「{}」因 Gateway 重载而中断。", job.name),
|
||||
diagnostic: Some("runtime admission closed".to_string()),
|
||||
duration_ms: start.elapsed().as_millis() as i64,
|
||||
},
|
||||
None,
|
||||
)
|
||||
};
|
||||
let finished_at = now_ms();
|
||||
let mut commit_attempt = 0_u64;
|
||||
let commit = loop {
|
||||
let result = match agent_execution.as_ref() {
|
||||
Some(execution) => {
|
||||
self.storage
|
||||
.finish_scheduled_run_with_agent(
|
||||
claimed.run_id,
|
||||
&claimed.owner,
|
||||
&completion,
|
||||
&execution.agent_run_id,
|
||||
&execution.agent_run_id,
|
||||
execution.runtime_generation,
|
||||
&execution.agent_terminal,
|
||||
finished_at,
|
||||
)
|
||||
.await
|
||||
}
|
||||
None => {
|
||||
self.storage
|
||||
.finish_scheduled_run(
|
||||
claimed.run_id,
|
||||
&claimed.owner,
|
||||
&completion,
|
||||
finished_at,
|
||||
)
|
||||
.await
|
||||
}
|
||||
};
|
||||
match result {
|
||||
Err(error) if error.is_transient() && commit_attempt < 2 => {
|
||||
commit_attempt += 1;
|
||||
tracing::warn!(
|
||||
job_id = %job.id,
|
||||
run_id = claimed.run_id,
|
||||
attempt = commit_attempt + 1,
|
||||
error = %error,
|
||||
"scheduler: retrying transient run completion commit"
|
||||
);
|
||||
time::sleep(time::Duration::from_millis(50 * commit_attempt)).await;
|
||||
}
|
||||
result => break result,
|
||||
}
|
||||
};
|
||||
match commit {
|
||||
Ok(true) => tracing::info!(
|
||||
job_id = %job.id,
|
||||
run_id = claimed.run_id,
|
||||
status = completion.status.as_str(),
|
||||
outcome = completion.outcome.as_str(),
|
||||
duration_ms = completion.duration_ms,
|
||||
"scheduler: run completed"
|
||||
),
|
||||
Ok(false) => tracing::warn!(
|
||||
job_id = %job.id,
|
||||
run_id = claimed.run_id,
|
||||
"scheduler: late run result discarded"
|
||||
),
|
||||
Err(error) => tracing::error!(
|
||||
job_id = %job.id,
|
||||
run_id = claimed.run_id,
|
||||
error = %error,
|
||||
"scheduler: failed to commit run completion"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
async fn deliver_claimed_run(self: Arc<Self>, run: JobRun) {
|
||||
let Some(delivery_owner) = run.delivery_lease_owner.clone() else {
|
||||
tracing::error!(
|
||||
run_id = run.id,
|
||||
"scheduler: claimed delivery has no lease owner"
|
||||
);
|
||||
return;
|
||||
};
|
||||
let result = self
|
||||
.session_manager
|
||||
.deliver_scheduled_run(&run, &delivery_owner)
|
||||
.await;
|
||||
let (delivered, permanent, error) = match result {
|
||||
Ok(()) => (true, false, None),
|
||||
Err(ScheduledDeliveryError::Transient(error)) => {
|
||||
(false, false, Some(sanitize_error(&error)))
|
||||
if managed
|
||||
&& delivery_status.is_none()
|
||||
&& job.delivery_policy != DeliveryPolicy::Never
|
||||
&& let Some(message) = error.as_deref()
|
||||
{
|
||||
let notice = format!("定时任务「{}」执行失败：{}", job.name, message);
|
||||
match self
|
||||
.session_manager
|
||||
.deliver_scheduled_message(&job.channel, &job.chat_id, &job.id, &job.name, ¬ice)
|
||||
.await
|
||||
{
|
||||
Ok(()) => delivery_status = Some("delivered".into()),
|
||||
Err(error) => {
|
||||
status = "delivery_error".into();
|
||||
delivery_status = Some("failed".into());
|
||||
delivery_error = Some(error);
|
||||
}
|
||||
}
|
||||
Err(ScheduledDeliveryError::Permanent(error)) => {
|
||||
(false, true, Some(sanitize_error(&error)))
|
||||
}
|
||||
|
||||
let (next_run_at, disable, delete) = match &job.schedule {
|
||||
Schedule::At { .. } => (None, !job.delete_after_run, job.delete_after_run),
|
||||
Schedule::Every { .. } | Schedule::Cron { .. } => {
|
||||
match next_run_for_schedule(&job.schedule, finished_at) {
|
||||
Some(next) => (Some(next), false, false),
|
||||
None => (None, true, false),
|
||||
}
|
||||
}
|
||||
};
|
||||
if let Err(commit_error) = self
|
||||
let run = JobRun {
|
||||
id: 0,
|
||||
job_id: job.id.clone(),
|
||||
started_at,
|
||||
finished_at,
|
||||
status,
|
||||
output,
|
||||
error,
|
||||
duration_ms,
|
||||
result_kind,
|
||||
delivery_status,
|
||||
delivery_error,
|
||||
};
|
||||
|
||||
if let Err(error) = self
|
||||
.storage
|
||||
.complete_scheduled_delivery(
|
||||
run.id,
|
||||
&delivery_owner,
|
||||
delivered,
|
||||
permanent,
|
||||
error.as_deref(),
|
||||
now_ms(),
|
||||
)
|
||||
.complete_scheduled_job(&run, &self.owner, next_run_at, disable, delete)
|
||||
.await
|
||||
{
|
||||
tracing::error!(
|
||||
run_id = run.id,
|
||||
error = %commit_error,
|
||||
"scheduler: failed to commit delivery receipt"
|
||||
);
|
||||
tracing::error!(job_id = %job.id, error = %error, "scheduler: failed to commit job completion");
|
||||
let _ = self
|
||||
.storage
|
||||
.release_scheduled_job_lease(&job.id, &self.owner)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn sanitize_error(error: &str) -> String {
|
||||
error.chars().take(1_024).collect()
|
||||
tracing::info!(
|
||||
job_id = %job.id,
|
||||
status = %run.status,
|
||||
duration_ms,
|
||||
"scheduler: job completed"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@ -413,33 +420,102 @@ mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn next_run_for_every_uses_claim_time() {
|
||||
assert_eq!(
|
||||
next_run_for_schedule(&Schedule::Every { every_ms: 5_000 }, 1_000),
|
||||
Some(6_000)
|
||||
);
|
||||
fn test_next_run_at_schedule() {
|
||||
let now = 1000000;
|
||||
let next = next_run_for_schedule(&Schedule::At { at: 2000000 }, now);
|
||||
assert_eq!(next, Some(2000000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn next_run_for_at_keeps_absolute_timestamp() {
|
||||
assert_eq!(
|
||||
next_run_for_schedule(&Schedule::At { at: 2_000 }, 1_000),
|
||||
Some(2_000)
|
||||
);
|
||||
fn test_next_run_every_schedule() {
|
||||
let now = 1000000;
|
||||
let next = next_run_for_schedule(&Schedule::Every { every_ms: 5000 }, now);
|
||||
assert_eq!(next, Some(1005000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cron_timezone_uses_from_argument() {
|
||||
fn test_next_run_cron_every_minute() {
|
||||
let expr = "0 * * * * *".to_string();
|
||||
let schedule = Schedule::Cron { expr, tz: None };
|
||||
let now = 1000000;
|
||||
let next = next_run_for_schedule(&schedule, now);
|
||||
assert!(next.is_some());
|
||||
assert!(next.unwrap() > now);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_next_run_cron_every_day_at_9am() {
|
||||
let expr = "0 0 9 * * *".to_string();
|
||||
let schedule = Schedule::Cron { expr, tz: None };
|
||||
let now = 1000000;
|
||||
let next = next_run_for_schedule(&schedule, now);
|
||||
assert!(next.is_some());
|
||||
let next_ms = next.unwrap();
|
||||
assert!(next_ms > now);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_next_run_cron_uses_from_argument() {
|
||||
let expr = "0 * * * * *".to_string();
|
||||
let schedule = Schedule::Cron { expr, tz: None };
|
||||
let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T12:34:20Z")
|
||||
.unwrap()
|
||||
.timestamp_millis();
|
||||
|
||||
let next = next_run_for_schedule(&schedule, from).unwrap();
|
||||
let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T12:35:00Z")
|
||||
.unwrap()
|
||||
.timestamp_millis();
|
||||
assert_eq!(next, expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scheduled_disposition_is_fail_safe() {
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition("NO_REPLY"),
|
||||
ScheduledDisposition::Quiet(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition("NO_REPLY[INFO]: healthy"),
|
||||
ScheduledDisposition::Quiet(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition("NO_REPLY[FAIL]: timeout"),
|
||||
ScheduledDisposition::ReportedFailure(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition("NO_REPLY[REFUSE]: denied"),
|
||||
ScheduledDisposition::Refused(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition("text mentioning NO_REPLY"),
|
||||
ScheduledDisposition::Content(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition("NO_REPLY[INFO] but this is content"),
|
||||
ScheduledDisposition::Content(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
parse_scheduled_disposition(""),
|
||||
ScheduledDisposition::ReportedFailure(_)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_next_run_cron_timezone_uses_from_argument() {
|
||||
let expr = "0 0 9 * * *".to_string();
|
||||
let schedule = Schedule::Cron {
|
||||
expr: "0 0 9 * * *".to_string(),
|
||||
expr,
|
||||
tz: Some("Asia/Shanghai".to_string()),
|
||||
};
|
||||
let from = chrono::DateTime::parse_from_rfc3339("2026-06-16T00:30:00Z")
|
||||
.unwrap()
|
||||
.timestamp_millis();
|
||||
|
||||
let next = next_run_for_schedule(&schedule, from).unwrap();
|
||||
let expected = chrono::DateTime::parse_from_rfc3339("2026-06-16T01:00:00Z")
|
||||
.unwrap()
|
||||
.timestamp_millis();
|
||||
assert_eq!(next_run_for_schedule(&schedule, from), Some(expected));
|
||||
assert_eq!(next, expected);
|
||||
}
|
||||
}
|
||||
|
||||
@ -4,9 +4,7 @@ use crate::bus::{ChatMessage, MediaItem, MessageSource, OutboundMessage, SourceK
|
||||
use crate::session::UnifiedSessionId;
|
||||
use crate::tools::{OutboundDelivery, OutboundMessenger};
|
||||
|
||||
use super::persistence::{
|
||||
append_active_turn_message, append_persisted_message_if_absent, append_persisted_messages,
|
||||
};
|
||||
use super::persistence::{append_active_turn_message, append_persisted_messages};
|
||||
use super::session::{
|
||||
CURRENT_SOURCE_SESSION, CURRENT_TURN_DELIVERIES, CURRENT_TURN_ID, PendingTurnDelivery,
|
||||
SessionManager,
|
||||
@ -120,108 +118,37 @@ impl OutboundMessenger for SessionManager {
|
||||
}
|
||||
|
||||
impl SessionManager {
|
||||
pub async fn deliver_scheduled_run(
|
||||
pub async fn deliver_scheduled_message(
|
||||
&self,
|
||||
run: &crate::storage::JobRun,
|
||||
delivery_owner: &str,
|
||||
) -> Result<(), ScheduledDeliveryError> {
|
||||
let content = run
|
||||
.message
|
||||
.as_deref()
|
||||
.unwrap_or("定时任务已结束，但没有生成可投递的结果。请在任务运行记录中查看诊断信息。");
|
||||
let target_sid = if let Some(session_id) = run.target_session_id.as_deref() {
|
||||
UnifiedSessionId::parse(session_id).ok_or_else(|| {
|
||||
ScheduledDeliveryError::Permanent("stored target session is invalid".to_string())
|
||||
})?
|
||||
} else {
|
||||
let resolved = self
|
||||
.resolve_dialog_id(&run.target_channel, &run.target_chat_id)
|
||||
.await
|
||||
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
|
||||
let fixed = self
|
||||
.storage
|
||||
.set_scheduled_delivery_target_session(
|
||||
run.id,
|
||||
delivery_owner,
|
||||
&resolved.to_string(),
|
||||
chrono::Utc::now().timestamp_millis(),
|
||||
)
|
||||
.await
|
||||
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?
|
||||
.ok_or_else(|| {
|
||||
ScheduledDeliveryError::Transient(
|
||||
"scheduled delivery lost its claim before fixing target session"
|
||||
.to_string(),
|
||||
)
|
||||
})?;
|
||||
UnifiedSessionId::parse(&fixed).ok_or_else(|| {
|
||||
ScheduledDeliveryError::Permanent("fixed target session is invalid".to_string())
|
||||
})?
|
||||
};
|
||||
if target_sid.channel != run.target_channel || target_sid.chat_id != run.target_chat_id {
|
||||
return Err(ScheduledDeliveryError::Permanent(
|
||||
"fixed target session does not belong to the scheduled destination".to_string(),
|
||||
));
|
||||
}
|
||||
let session = self
|
||||
.get_or_activate_session(&target_sid)
|
||||
.await
|
||||
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
|
||||
let source = MessageSource {
|
||||
kind: SourceKind::ExternalTrigger,
|
||||
from_channel: Some("scheduler".to_string()),
|
||||
from_session: Some(format!("scheduled-run:{}", run.id)),
|
||||
from_user_id: None,
|
||||
system_name: Some("scheduled task".to_string()),
|
||||
task_id: Some(run.job_id.clone()),
|
||||
from_run_id: run.agent_run_id.clone(),
|
||||
from_agent_id: run.agent_id.clone(),
|
||||
};
|
||||
let mut message = outbound_history_message(content, source, &[]);
|
||||
message.id = format!("scheduled:{}", run.id);
|
||||
let message_id = message.id.clone();
|
||||
append_persisted_message_if_absent(&session, message)
|
||||
.await
|
||||
.map_err(|error| ScheduledDeliveryError::Transient(error.to_string()))?;
|
||||
|
||||
let metadata = HashMap::from([
|
||||
("_session_id".to_string(), target_sid.to_string()),
|
||||
("_message_id".to_string(), message_id),
|
||||
("scheduled_delivery_id".to_string(), run.id.to_string()),
|
||||
]);
|
||||
self.bus
|
||||
.deliver_outbound(OutboundMessage {
|
||||
channel: run.target_channel.clone(),
|
||||
chat_id: run.target_chat_id.clone(),
|
||||
content: content.to_string(),
|
||||
reply_to: None,
|
||||
media: Vec::new(),
|
||||
metadata,
|
||||
delivery: None,
|
||||
})
|
||||
.await
|
||||
.map_err(|error| match error {
|
||||
crate::bus::BusError::Closed
|
||||
| crate::bus::BusError::DeliveryTimedOut
|
||||
| crate::bus::BusError::DeliveryTransient(_) => {
|
||||
ScheduledDeliveryError::Transient(error.to_string())
|
||||
}
|
||||
crate::bus::BusError::DeliveryPermanent(summary) => {
|
||||
ScheduledDeliveryError::Permanent(summary)
|
||||
}
|
||||
})?;
|
||||
Ok(())
|
||||
channel: &str,
|
||||
chat_id: &str,
|
||||
job_id: &str,
|
||||
job_name: &str,
|
||||
content: &str,
|
||||
) -> Result<(), String> {
|
||||
<Self as OutboundMessenger>::send_message(
|
||||
self,
|
||||
channel,
|
||||
chat_id,
|
||||
None,
|
||||
content,
|
||||
MessageSource {
|
||||
kind: SourceKind::ExternalTrigger,
|
||||
from_channel: Some("scheduler".to_string()),
|
||||
from_session: Some(format!("cron:{job_id}")),
|
||||
from_user_id: None,
|
||||
system_name: Some(job_name.to_string()),
|
||||
task_id: Some(job_id.to_string()),
|
||||
from_run_id: None,
|
||||
from_agent_id: None,
|
||||
},
|
||||
Vec::new(),
|
||||
)
|
||||
.await
|
||||
.map(|_| ())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum ScheduledDeliveryError {
|
||||
#[error("transient scheduled delivery failure: {0}")]
|
||||
Transient(String),
|
||||
#[error("permanent scheduled delivery failure: {0}")]
|
||||
Permanent(String),
|
||||
}
|
||||
|
||||
fn outbound_history_message(
|
||||
content: impl Into<String>,
|
||||
source: MessageSource,
|
||||
|
||||
@ -14,7 +14,6 @@ pub mod turn;
|
||||
pub use commands::SessionCommand;
|
||||
pub use error::SessionError;
|
||||
pub use events::{DialogInfo, SessionEvent};
|
||||
pub use messenger::ScheduledDeliveryError;
|
||||
pub use session::{
|
||||
AgentCatalogPreparation, SLASH_COMMANDS, Session, SessionManager, SessionManagerServices,
|
||||
SlashCommand,
|
||||
|
||||
@ -101,41 +101,6 @@ pub(super) async fn append_persisted_messages_with_meta(
|
||||
append_persisted_messages_inner(session, messages, VersionPolicy::Advance, None, None).await
|
||||
}
|
||||
|
||||
pub(super) async fn append_persisted_message_if_absent(
|
||||
session: &Arc<Mutex<Session>>,
|
||||
message: ChatMessage,
|
||||
) -> Result<bool, StorageError> {
|
||||
let persistence_lock = { session.lock().await.persistence_lock.clone() };
|
||||
let _persistence_guard = persistence_lock.lock().await;
|
||||
let message_id = message.id.clone();
|
||||
let snapshot = {
|
||||
let guard = session.lock().await;
|
||||
if guard.contains_message_id(&message_id) {
|
||||
return Ok(false);
|
||||
}
|
||||
guard.prepare_message_persist_snapshot(&message)
|
||||
};
|
||||
let Some((storage, session_id, persisted, meta)) = snapshot else {
|
||||
return Ok(false);
|
||||
};
|
||||
let inserted = storage
|
||||
.persist_message_if_absent(&session_id, &persisted, &meta)
|
||||
.await?;
|
||||
if !inserted {
|
||||
return Ok(false);
|
||||
}
|
||||
let mut guard = session.lock().await;
|
||||
if guard.contains_message_id(&message_id) {
|
||||
return Ok(true);
|
||||
}
|
||||
if !guard.apply_prepared_message_in_memory(message, persisted.seq, persisted.created_at, true) {
|
||||
return Err(StorageError::Conflict(format!(
|
||||
"session changed while committing idempotent message {message_id}"
|
||||
)));
|
||||
}
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
pub(super) async fn append_persisted_turn_messages(
|
||||
session: &Arc<Mutex<Session>>,
|
||||
messages: Vec<ChatMessage>,
|
||||
@ -282,7 +247,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn side_effect_messages_apply_the_expected_session_version_once() {
|
||||
async fn active_turn_side_effect_does_not_invalidate_its_session_version() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let storage = Arc::new(
|
||||
crate::storage::Storage::new(&dir.path().join("memory.db"))
|
||||
@ -290,7 +255,7 @@ mod tests {
|
||||
.unwrap(),
|
||||
);
|
||||
let memory_manager = Arc::new(MemoryManager::new(
|
||||
storage.clone(),
|
||||
storage,
|
||||
"test".to_string(),
|
||||
"test".to_string(),
|
||||
));
|
||||
@ -311,32 +276,12 @@ mod tests {
|
||||
price_input_per_million: None,
|
||||
price_output_per_million: None,
|
||||
};
|
||||
let unified_id = crate::session::UnifiedSessionId::new("cli_chat", "chat", "dialog");
|
||||
let now = chrono::Utc::now().timestamp_millis();
|
||||
storage
|
||||
.upsert_session(&crate::storage::session::SessionMeta {
|
||||
id: unified_id.to_string(),
|
||||
channel: "cli_chat".to_string(),
|
||||
chat_id: "chat".to_string(),
|
||||
dialog_id: "dialog".to_string(),
|
||||
title: "test".to_string(),
|
||||
created_at: now,
|
||||
last_active_at: now,
|
||||
message_count: 0,
|
||||
routing_info: None,
|
||||
archived_at: None,
|
||||
deleted_at: None,
|
||||
last_consolidated_at: None,
|
||||
last_compressed_message_at: None,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
let session = Arc::new(Mutex::new(
|
||||
Session::new(
|
||||
unified_id,
|
||||
crate::session::UnifiedSessionId::new("cli_chat", "chat", "dialog"),
|
||||
config,
|
||||
Arc::new(ToolRegistry::new()),
|
||||
Some(storage.clone()),
|
||||
None,
|
||||
String::new(),
|
||||
"test".to_string(),
|
||||
super::super::session::SessionContextServices {
|
||||
@ -375,37 +320,5 @@ mod tests {
|
||||
let guard = session.lock().await;
|
||||
assert_eq!(guard.state_version_for_test(), base_version + 1);
|
||||
assert_eq!(guard.get_history().len(), 2);
|
||||
drop(guard);
|
||||
|
||||
let idempotent_base_version = session.lock().await.state_version_for_test();
|
||||
let mut scheduled = ChatMessage::assistant("scheduled result");
|
||||
scheduled.id = "scheduled:42".to_string();
|
||||
assert!(
|
||||
append_persisted_message_if_absent(&session, scheduled.clone())
|
||||
.await
|
||||
.unwrap()
|
||||
);
|
||||
assert!(
|
||||
!append_persisted_message_if_absent(&session, scheduled)
|
||||
.await
|
||||
.unwrap()
|
||||
);
|
||||
let guard = session.lock().await;
|
||||
assert_eq!(guard.state_version_for_test(), idempotent_base_version + 1);
|
||||
assert_eq!(
|
||||
guard
|
||||
.get_history()
|
||||
.iter()
|
||||
.filter(|message| message.id == "scheduled:42")
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
drop(guard);
|
||||
let persisted: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM messages WHERE id = ?")
|
||||
.bind("scheduled:42")
|
||||
.fetch_one(storage.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(persisted, 1);
|
||||
}
|
||||
}
|
||||
|
||||
@ -1083,101 +1083,59 @@ impl Session {
|
||||
persist: bool,
|
||||
advance_state_version: bool,
|
||||
) -> Option<MessagePersistSnapshot> {
|
||||
let persist_snapshot = persist
|
||||
.then(|| self.prepare_message_persist_snapshot(&message))
|
||||
.flatten();
|
||||
let (seq, now) = persist_snapshot
|
||||
.as_ref()
|
||||
.map(|(_, _, message, _)| (message.seq, message.created_at))
|
||||
.unwrap_or_else(|| (self.seq_counter, chrono::Utc::now().timestamp_millis()));
|
||||
let applied =
|
||||
self.apply_prepared_message_in_memory(message, seq, now, advance_state_version);
|
||||
debug_assert!(applied, "fresh in-memory message snapshot must apply");
|
||||
persist_snapshot
|
||||
}
|
||||
|
||||
pub(super) fn prepare_message_persist_snapshot(
|
||||
&self,
|
||||
message: &ChatMessage,
|
||||
) -> Option<MessagePersistSnapshot> {
|
||||
let storage = self.storage.clone()?;
|
||||
let is_user = message.role == "user";
|
||||
let counts_as_user_input =
|
||||
is_user && message.client_visibility == crate::bus::ClientVisibility::Visible;
|
||||
let now = chrono::Utc::now().timestamp_millis();
|
||||
let seq = self.seq_counter;
|
||||
let msg_meta = crate::storage::message::MessageMeta {
|
||||
id: message.id.clone(),
|
||||
session_id: self.id.to_string(),
|
||||
seq,
|
||||
role: message.role.clone(),
|
||||
content: message.content.clone(),
|
||||
reasoning_content: message.reasoning_content.clone(),
|
||||
provider_state: message
|
||||
.provider_state
|
||||
.as_ref()
|
||||
.and_then(|state| serde_json::to_string(state).ok()),
|
||||
turn_id: message.turn_id.clone(),
|
||||
iteration: message.iteration.map(i64::from),
|
||||
completion_status: message.completion_status,
|
||||
client_visibility: message.client_visibility,
|
||||
turn_origin: message.turn_origin,
|
||||
media_refs: if message.media_refs.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(serde_json::to_string(&message.media_refs).unwrap_or_default())
|
||||
},
|
||||
tool_call_id: message.tool_call_id.clone(),
|
||||
tool_name: message.tool_name.clone(),
|
||||
tool_calls: message
|
||||
.tool_calls
|
||||
.as_ref()
|
||||
.and_then(|tc| serde_json::to_string(tc).ok()),
|
||||
source: message
|
||||
.source
|
||||
.as_ref()
|
||||
.map(|s| serde_json::to_string(s).unwrap_or_default()),
|
||||
created_at: now,
|
||||
};
|
||||
let session_id = self.id.to_string();
|
||||
let session_meta = crate::storage::session::SessionMeta {
|
||||
id: session_id.clone(),
|
||||
channel: self.id.channel.clone(),
|
||||
chat_id: self.id.chat_id.clone(),
|
||||
dialog_id: self.id.dialog_id.clone(),
|
||||
title: self.title.clone(),
|
||||
created_at: self.created_at,
|
||||
last_active_at: now,
|
||||
message_count: self.message_count + i64::from(counts_as_user_input),
|
||||
routing_info: if self.routing_info.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(self.routing_info.clone())
|
||||
},
|
||||
archived_at: self.archived_at,
|
||||
deleted_at: None,
|
||||
last_consolidated_at: self.last_consolidated_at,
|
||||
last_compressed_message_at: self.last_compressed_message_at,
|
||||
};
|
||||
Some((storage, session_id, msg_meta, session_meta))
|
||||
}
|
||||
|
||||
pub(super) fn apply_prepared_message_in_memory(
|
||||
&mut self,
|
||||
message: ChatMessage,
|
||||
seq: i64,
|
||||
now: i64,
|
||||
advance_state_version: bool,
|
||||
) -> bool {
|
||||
if self.seq_counter != seq || self.contains_message_id(&message.id) {
|
||||
return false;
|
||||
}
|
||||
let is_user = message.role == "user";
|
||||
let counts_as_user_input =
|
||||
is_user && message.client_visibility == crate::bus::ClientVisibility::Visible;
|
||||
// Assign seq
|
||||
let seq = self.seq_counter;
|
||||
self.seq_counter += 1;
|
||||
|
||||
let persist_snapshot = if persist {
|
||||
self.storage.clone().map(|storage| {
|
||||
let msg_meta = crate::storage::message::MessageMeta {
|
||||
id: message.id.clone(),
|
||||
session_id: self.id.to_string(),
|
||||
seq,
|
||||
role: message.role.clone(),
|
||||
content: message.content.clone(),
|
||||
reasoning_content: message.reasoning_content.clone(),
|
||||
provider_state: message
|
||||
.provider_state
|
||||
.as_ref()
|
||||
.and_then(|state| serde_json::to_string(state).ok()),
|
||||
turn_id: message.turn_id.clone(),
|
||||
iteration: message.iteration.map(i64::from),
|
||||
completion_status: message.completion_status,
|
||||
client_visibility: message.client_visibility,
|
||||
turn_origin: message.turn_origin,
|
||||
media_refs: if message.media_refs.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(serde_json::to_string(&message.media_refs).unwrap_or_default())
|
||||
},
|
||||
tool_call_id: message.tool_call_id.clone(),
|
||||
tool_name: message.tool_name.clone(),
|
||||
tool_calls: message
|
||||
.tool_calls
|
||||
.as_ref()
|
||||
.and_then(|tc| serde_json::to_string(tc).ok()),
|
||||
source: message
|
||||
.source
|
||||
.as_ref()
|
||||
.map(|s| serde_json::to_string(s).unwrap_or_default()),
|
||||
created_at: now,
|
||||
};
|
||||
(storage, self.id.to_string(), msg_meta)
|
||||
})
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
// Update in-memory state
|
||||
self.message_seqs.insert(message.id.clone(), seq);
|
||||
self.messages.push(message);
|
||||
self.seq_counter += 1;
|
||||
self.total_message_count += 1;
|
||||
if counts_as_user_input {
|
||||
self.message_count += 1;
|
||||
@ -1186,11 +1144,29 @@ impl Session {
|
||||
if advance_state_version {
|
||||
self.state_version = self.state_version.wrapping_add(1);
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
pub(super) fn contains_message_id(&self, message_id: &str) -> bool {
|
||||
self.message_seqs.contains_key(message_id)
|
||||
persist_snapshot.map(|(storage, session_id, msg_meta)| {
|
||||
let session_meta = crate::storage::session::SessionMeta {
|
||||
id: session_id.clone(),
|
||||
channel: self.id.channel.clone(),
|
||||
chat_id: self.id.chat_id.clone(),
|
||||
dialog_id: self.id.dialog_id.clone(),
|
||||
title: self.title.clone(),
|
||||
created_at: self.created_at,
|
||||
last_active_at: self.last_active_at,
|
||||
message_count: self.message_count,
|
||||
routing_info: if self.routing_info.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(self.routing_info.clone())
|
||||
},
|
||||
archived_at: self.archived_at,
|
||||
deleted_at: None,
|
||||
last_consolidated_at: self.last_consolidated_at,
|
||||
last_compressed_message_at: self.last_compressed_message_at,
|
||||
};
|
||||
(storage, session_id, msg_meta, session_meta)
|
||||
})
|
||||
}
|
||||
|
||||
/// Roll back messages that were appended in memory but whose atomic
|
||||
@ -1929,7 +1905,7 @@ pub struct SessionManager {
|
||||
provider_config: LLMProviderConfig,
|
||||
tools: Arc<ToolRegistry>,
|
||||
skills_loader: Arc<SkillsLoader>,
|
||||
pub(super) storage: Arc<Storage>,
|
||||
storage: Arc<Storage>,
|
||||
pub(super) bus: Arc<MessageBus>,
|
||||
memory_manager: Arc<crate::memory::MemoryManager>,
|
||||
work_manager: Arc<crate::work::WorkManager>,
|
||||
@ -2300,6 +2276,46 @@ impl SessionManager {
|
||||
self.work_manager.clone()
|
||||
}
|
||||
|
||||
/// 为定时任务创建一个无 session 绑定的 AgentLoop
|
||||
pub fn create_cron_agent(&self) -> Result<AgentLoop, AgentError> {
|
||||
let tools = self.tools.without(&["reload_config"]);
|
||||
let provider = create_provider(self.provider_config.clone())
|
||||
.map_err(|e| AgentError::Other(format!("failed to create cron provider: {}", e)))?;
|
||||
Ok(AgentLoop::with_provider_and_tools(
|
||||
Arc::from(provider),
|
||||
tools,
|
||||
self.provider_config.max_tool_iterations,
|
||||
self.provider_config.model_id.clone(),
|
||||
self.provider_config.workspace_dir.clone(),
|
||||
self.provider_config.input_types.clone(),
|
||||
)
|
||||
.with_context_window(self.provider_config.token_limit))
|
||||
}
|
||||
|
||||
fn create_managed_scheduled_agent(&self) -> Result<(AgentLoop, Arc<ToolRegistry>), AgentError> {
|
||||
let tools = self.tools.without(&[
|
||||
"send_message",
|
||||
"cron_add",
|
||||
"cron_update",
|
||||
"cron_remove",
|
||||
"cron_enable",
|
||||
"cron_disable",
|
||||
"reload_config",
|
||||
]);
|
||||
let provider = create_provider(self.provider_config.clone())
|
||||
.map_err(|e| AgentError::Other(format!("failed to create scheduled provider: {e}")))?;
|
||||
let agent = AgentLoop::with_provider_and_tools(
|
||||
Arc::from(provider),
|
||||
tools.clone(),
|
||||
self.provider_config.max_tool_iterations,
|
||||
self.provider_config.model_id.clone(),
|
||||
self.provider_config.workspace_dir.clone(),
|
||||
self.provider_config.input_types.clone(),
|
||||
)
|
||||
.with_context_window(self.provider_config.token_limit);
|
||||
Ok((agent, tools))
|
||||
}
|
||||
|
||||
/// 获取所有可用的斜杠命令
|
||||
pub fn get_slash_commands(&self) -> &[SlashCommand] {
|
||||
SLASH_COMMANDS
|
||||
@ -5082,6 +5098,103 @@ impl SessionManager {
|
||||
}
|
||||
|
||||
impl SessionManager {
|
||||
///
|
||||
/// Runs in a stateless manner: no session creation, no history persistence.
|
||||
/// The cron system prompt instructs the LLM to deliver results via the
|
||||
/// `send_message` tool, which handles both delivery and history writing
|
||||
/// on the target session.
|
||||
pub async fn handle_cron_message(
|
||||
&self,
|
||||
channel: &str,
|
||||
chat_id: &str,
|
||||
prompt: &str,
|
||||
job_id: &str,
|
||||
job_name: &str,
|
||||
) -> Result<HandleResult, AgentError> {
|
||||
let skills_prompt = self.skills_loader.build_skills_prompt();
|
||||
|
||||
let base_prompt = build_system_prompt(
|
||||
&self.provider_config.workspace_dir,
|
||||
&self.provider_config.model_id,
|
||||
&self.tools,
|
||||
);
|
||||
let cron_context = format!(
|
||||
"## 定时任务执行\n\n\
|
||||
你正在执行定时任务「{job_name}」({job_id})。\n\
|
||||
目标渠道: {channel}:{chat_id}\n\n\
|
||||
规则:\n\
|
||||
- 这不是聊天对话，没有用户会直接看到你的输出\n\
|
||||
- 你必须使用 send_message 工具将最终结果发送到目标渠道\n\
|
||||
- send_message 格式: target_chat_id=\"{channel}:{chat_id}\", content=\"消息内容\"\n\
|
||||
- 可以调用其他工具收集信息、处理任务，但最终消息必须通过 send_message 发送\n\
|
||||
- 只输出最终消息内容，不要输出中间思考过程或分析！"
|
||||
);
|
||||
let full_system_prompt =
|
||||
format!("{}\n\n{}\n\n{}", base_prompt, skills_prompt, cron_context);
|
||||
|
||||
let history = vec![
|
||||
ChatMessage::system(full_system_prompt),
|
||||
ChatMessage::user(prompt),
|
||||
];
|
||||
|
||||
let agent = self.create_cron_agent()?;
|
||||
let source_session = format!("cron:{}", job_name);
|
||||
let result = CURRENT_SOURCE_SESSION
|
||||
.scope(Some(source_session.clone()), async {
|
||||
agent
|
||||
.process_with_context(
|
||||
history,
|
||||
ToolExecutionContext::for_session(source_session),
|
||||
)
|
||||
.await
|
||||
})
|
||||
.await
|
||||
.inspect_err(|e| {
|
||||
tracing::error!(error = %e, job_id = %job_id, "Cron agent processing error");
|
||||
})?;
|
||||
|
||||
Ok(HandleResult::AgentResponse(result.final_response.content))
|
||||
}
|
||||
|
||||
/// Execute a scheduler-managed task. The agent returns a result but cannot
|
||||
/// deliver it itself; Scheduler applies the configured delivery policy.
|
||||
pub async fn handle_managed_scheduled_message(
|
||||
&self,
|
||||
prompt: &str,
|
||||
job_id: &str,
|
||||
job_name: &str,
|
||||
monitor: bool,
|
||||
) -> Result<String, AgentError> {
|
||||
let (agent, tools) = self.create_managed_scheduled_agent()?;
|
||||
let base_prompt = build_system_prompt(
|
||||
&self.provider_config.workspace_dir,
|
||||
&self.provider_config.model_id,
|
||||
&tools,
|
||||
);
|
||||
let skills_prompt = self.skills_loader.build_skills_prompt();
|
||||
let result_contract = if monitor {
|
||||
"这是无人值守巡检。完成必要检查后：一切正常且无需用户关注时，只返回 NO_REPLY[INFO]: <简短原因>；发现问题时返回简洁、可操作的告警；无法完成时返回 NO_REPLY[FAIL]: <原因>；因安全或权限拒绝时返回 NO_REPLY[REFUSE]: <原因>。不要调用 send_message，不要把不确定当作正常。"
|
||||
} else {
|
||||
"这是 Scheduler 托管投递的定时任务。完成任务后只返回应交付给用户的最终内容，不要调用 send_message。"
|
||||
};
|
||||
let system = format!(
|
||||
"{base_prompt}\n\n{skills_prompt}\n\n## 定时任务执行\n任务「{job_name}」({job_id})。\n{result_contract}"
|
||||
);
|
||||
let history = vec![ChatMessage::system(system), ChatMessage::user(prompt)];
|
||||
let source_session = format!("cron:{job_id}");
|
||||
let result = CURRENT_SOURCE_SESSION
|
||||
.scope(Some(source_session.clone()), async {
|
||||
agent
|
||||
.process_with_context(
|
||||
history,
|
||||
ToolExecutionContext::for_session(source_session),
|
||||
)
|
||||
.await
|
||||
})
|
||||
.await?;
|
||||
Ok(result.final_response.content)
|
||||
}
|
||||
|
||||
pub async fn clear_session_history(
|
||||
&self,
|
||||
unified_id: &UnifiedSessionId,
|
||||
|
||||
@ -1175,7 +1175,7 @@ mod tests {
|
||||
.fetch_one(storage.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(version, crate::storage::SCHEMA_VERSION);
|
||||
assert_eq!(version, 10);
|
||||
for table in [
|
||||
"agent_runs",
|
||||
"agent_session_state",
|
||||
|
||||
@ -10,10 +10,7 @@ pub mod usage;
|
||||
|
||||
pub use context_checkpoint::{ContextCheckpoint, ContextCheckpointState, NewContextCheckpoint};
|
||||
pub use error::StorageError;
|
||||
pub use scheduler::{
|
||||
ClaimedScheduledRun, DeliveryPolicy, JobRun, ScheduledDeliveryStatus, ScheduledJob,
|
||||
ScheduledJobUpdate, ScheduledOutcomeKind, ScheduledRunCompletion, ScheduledRunStatus,
|
||||
};
|
||||
pub use scheduler::{DeliveryPolicy, JobKind, JobRun, ScheduledJob};
|
||||
pub use usage::{SessionUsageTotals, TurnUsageRecord};
|
||||
|
||||
use sqlx::sqlite::{
|
||||
@ -23,7 +20,7 @@ use sqlx::{Pool, Row, Sqlite};
|
||||
use std::path::Path;
|
||||
use tokio::time::{Duration, sleep};
|
||||
|
||||
const SCHEMA_VERSION: i64 = 11;
|
||||
const SCHEMA_VERSION: i64 = 10;
|
||||
const INSERT_MESSAGE_SQL: &str = r#"
|
||||
INSERT INTO messages (
|
||||
id, session_id, seq, role, content, reasoning_content, provider_state,
|
||||
@ -32,15 +29,6 @@ const INSERT_MESSAGE_SQL: &str = r#"
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
"#;
|
||||
const INSERT_MESSAGE_IF_ABSENT_SQL: &str = r#"
|
||||
INSERT INTO messages (
|
||||
id, session_id, seq, role, content, reasoning_content, provider_state,
|
||||
turn_id, iteration, completion_status, client_visibility, turn_origin,
|
||||
media_refs, tool_call_id, tool_name, tool_calls, source, created_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(id) DO NOTHING
|
||||
"#;
|
||||
|
||||
pub(crate) fn insert_message_query<'a>(
|
||||
session_id: &'a str,
|
||||
@ -67,31 +55,6 @@ pub(crate) fn insert_message_query<'a>(
|
||||
.bind(msg.created_at)
|
||||
}
|
||||
|
||||
pub(crate) fn insert_message_if_absent_query<'a>(
|
||||
session_id: &'a str,
|
||||
msg: &'a crate::storage::message::MessageMeta,
|
||||
) -> sqlx::query::Query<'a, Sqlite, sqlx::sqlite::SqliteArguments> {
|
||||
sqlx::query(INSERT_MESSAGE_IF_ABSENT_SQL)
|
||||
.bind(&msg.id)
|
||||
.bind(session_id)
|
||||
.bind(msg.seq)
|
||||
.bind(&msg.role)
|
||||
.bind(&msg.content)
|
||||
.bind(&msg.reasoning_content)
|
||||
.bind(&msg.provider_state)
|
||||
.bind(&msg.turn_id)
|
||||
.bind(msg.iteration)
|
||||
.bind(msg.completion_status.as_str())
|
||||
.bind(msg.client_visibility.as_str())
|
||||
.bind(msg.turn_origin.as_str())
|
||||
.bind(&msg.media_refs)
|
||||
.bind(&msg.tool_call_id)
|
||||
.bind(&msg.tool_name)
|
||||
.bind(&msg.tool_calls)
|
||||
.bind(&msg.source)
|
||||
.bind(msg.created_at)
|
||||
}
|
||||
|
||||
fn message_meta_from_row(row: SqliteRow) -> crate::storage::message::MessageMeta {
|
||||
let completion_status: String = row.get("completion_status");
|
||||
crate::storage::message::MessageMeta {
|
||||
@ -444,6 +407,7 @@ impl Storage {
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
|
||||
Self::init_scheduler_schema(&self.pool).await?;
|
||||
self.migrate_schema().await?;
|
||||
|
||||
Ok(())
|
||||
@ -464,11 +428,7 @@ impl Storage {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
// Acquire the migration write lock before inspecting or modifying any
|
||||
// legacy shape. This keeps the v11 rebuild on one connection and makes
|
||||
// a concurrently running older Gateway fail startup cleanly instead of
|
||||
// partially racing the schema migration.
|
||||
let mut tx = self.pool.begin_with("BEGIN IMMEDIATE").await?;
|
||||
let mut tx = self.pool.begin().await?;
|
||||
// The legacy drops below are a pre-v8 rebuild concern: the batch
|
||||
// "group" concept was removed in v8 and the old `background_tasks`
|
||||
// table in v7. Gate them on `current < 8` so a v8 -> v9 upgrade only
|
||||
@ -498,33 +458,6 @@ impl Storage {
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
let legacy_scheduler_exists: i64 = sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'scheduled_jobs'",
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
let legacy_scheduler_exists = legacy_scheduler_exists == 1;
|
||||
if legacy_scheduler_exists {
|
||||
sqlx::query(
|
||||
r#"
|
||||
CREATE TABLE IF NOT EXISTS job_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
job_id TEXT NOT NULL REFERENCES scheduled_jobs(id) ON DELETE CASCADE,
|
||||
started_at INTEGER NOT NULL,
|
||||
finished_at INTEGER NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
output TEXT,
|
||||
error TEXT,
|
||||
duration_ms INTEGER NOT NULL,
|
||||
result_kind TEXT,
|
||||
delivery_status TEXT,
|
||||
delivery_error TEXT
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
for (table, column, definition) in [
|
||||
("messages", "source", "source TEXT"),
|
||||
("messages", "reasoning_content", "reasoning_content TEXT"),
|
||||
@ -595,10 +528,9 @@ impl Storage {
|
||||
let columns = sqlx::query(sqlx::AssertSqlSafe(pragma))
|
||||
.fetch_all(&mut *tx)
|
||||
.await?;
|
||||
if !columns.is_empty()
|
||||
&& !columns
|
||||
.iter()
|
||||
.any(|row| row.get::<String, _>("name") == column)
|
||||
if !columns
|
||||
.iter()
|
||||
.any(|row| row.get::<String, _>("name") == column)
|
||||
{
|
||||
let alter = format!("ALTER TABLE {table} ADD COLUMN {definition}");
|
||||
// All identifiers and definitions come from the fixed migration list above.
|
||||
@ -630,6 +562,11 @@ impl Storage {
|
||||
)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_jobs_claimable ON scheduled_jobs(enabled, next_run_at, lease_until)",
|
||||
)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
sqlx::query(
|
||||
r#"
|
||||
CREATE TABLE IF NOT EXISTS session_turn_usage (
|
||||
@ -692,7 +629,6 @@ impl Storage {
|
||||
for statement in agent_run::AGENT_SCHEMA_STATEMENTS {
|
||||
sqlx::query(*statement).execute(&mut *tx).await?;
|
||||
}
|
||||
scheduler::migrate_scheduler_v11(&mut tx, legacy_scheduler_exists).await?;
|
||||
sqlx::query(sqlx::AssertSqlSafe(format!(
|
||||
"PRAGMA user_version = {SCHEMA_VERSION}"
|
||||
)))
|
||||
@ -702,6 +638,70 @@ impl Storage {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Initialize the scheduler tables (idempotent).
|
||||
pub(crate) async fn init_scheduler_schema(pool: &Pool<Sqlite>) -> Result<(), StorageError> {
|
||||
sqlx::query(
|
||||
r#"
|
||||
CREATE TABLE IF NOT EXISTS scheduled_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
schedule TEXT NOT NULL,
|
||||
prompt TEXT NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
chat_id TEXT NOT NULL,
|
||||
model TEXT,
|
||||
job_kind TEXT NOT NULL DEFAULT 'task',
|
||||
delivery_policy TEXT NOT NULL DEFAULT 'direct',
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
delete_after_run INTEGER NOT NULL DEFAULT 0,
|
||||
next_run_at INTEGER NOT NULL,
|
||||
last_run_at INTEGER,
|
||||
last_status TEXT,
|
||||
last_error TEXT,
|
||||
locked_at INTEGER,
|
||||
lock_owner TEXT,
|
||||
lease_until INTEGER,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
r#"
|
||||
CREATE TABLE IF NOT EXISTS job_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
job_id TEXT NOT NULL REFERENCES scheduled_jobs(id) ON DELETE CASCADE,
|
||||
started_at INTEGER NOT NULL,
|
||||
finished_at INTEGER NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
output TEXT,
|
||||
error TEXT,
|
||||
duration_ms INTEGER NOT NULL,
|
||||
result_kind TEXT,
|
||||
delivery_status TEXT,
|
||||
delivery_error TEXT
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_jobs_next_run ON scheduled_jobs(enabled, next_run_at)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
sqlx::query("CREATE INDEX IF NOT EXISTS idx_runs_job_id ON job_runs(job_id)")
|
||||
.execute(pool)
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub async fn append_llm_call(
|
||||
&self,
|
||||
provider: &str,
|
||||
@ -962,56 +962,6 @@ impl Storage {
|
||||
Ok(msg.seq)
|
||||
}
|
||||
|
||||
pub async fn persist_message_if_absent(
|
||||
&self,
|
||||
session_id: &str,
|
||||
msg: &crate::storage::message::MessageMeta,
|
||||
meta: &crate::storage::session::SessionMeta,
|
||||
) -> Result<bool, StorageError> {
|
||||
let mut tx = self.pool.begin().await?;
|
||||
let inserted = insert_message_if_absent_query(session_id, msg)
|
||||
.execute(&mut *tx)
|
||||
.await?
|
||||
.rows_affected()
|
||||
== 1;
|
||||
if inserted {
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO sessions (id, channel, chat_id, dialog_id, title, created_at,
|
||||
last_active_at, message_count, routing_info, archived_at, deleted_at,
|
||||
last_consolidated_at, last_compressed_message_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
title = excluded.title,
|
||||
last_active_at = excluded.last_active_at,
|
||||
message_count = excluded.message_count,
|
||||
routing_info = excluded.routing_info,
|
||||
archived_at = excluded.archived_at,
|
||||
deleted_at = excluded.deleted_at,
|
||||
last_consolidated_at = excluded.last_consolidated_at,
|
||||
last_compressed_message_at = excluded.last_compressed_message_at
|
||||
"#,
|
||||
)
|
||||
.bind(&meta.id)
|
||||
.bind(&meta.channel)
|
||||
.bind(&meta.chat_id)
|
||||
.bind(&meta.dialog_id)
|
||||
.bind(&meta.title)
|
||||
.bind(meta.created_at)
|
||||
.bind(meta.last_active_at)
|
||||
.bind(meta.message_count)
|
||||
.bind(&meta.routing_info)
|
||||
.bind(meta.archived_at)
|
||||
.bind(meta.deleted_at)
|
||||
.bind(meta.last_consolidated_at)
|
||||
.bind(meta.last_compressed_message_at)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
tx.commit().await?;
|
||||
Ok(inserted)
|
||||
}
|
||||
|
||||
/// Atomically persist all messages produced by one logical turn together
|
||||
/// with the resulting session metadata. A turn is either fully visible
|
||||
/// after restart or not visible at all.
|
||||
@ -1634,59 +1584,6 @@ mod tests {
|
||||
(storage, dir)
|
||||
}
|
||||
|
||||
async fn scheduler_schema_sql(storage: &Storage) -> Vec<(String, String, Option<String>)> {
|
||||
sqlx::query_as::<_, (String, String, Option<String>)>(
|
||||
r#"
|
||||
SELECT type, name, sql FROM sqlite_master
|
||||
WHERE name IN (
|
||||
'scheduled_jobs', 'job_runs', 'idx_jobs_claimable',
|
||||
'idx_job_runs_job_finished', 'idx_job_runs_recovery',
|
||||
'idx_job_runs_delivery'
|
||||
)
|
||||
ORDER BY type, name
|
||||
"#,
|
||||
)
|
||||
.fetch_all(storage.pool())
|
||||
.await
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
async fn create_v10_scheduler_schema(pool: &Pool<Sqlite>) {
|
||||
sqlx::query(
|
||||
r#"
|
||||
CREATE TABLE scheduled_jobs (
|
||||
id TEXT PRIMARY KEY, name TEXT NOT NULL, schedule TEXT NOT NULL,
|
||||
prompt TEXT NOT NULL, channel TEXT NOT NULL, chat_id TEXT NOT NULL,
|
||||
model TEXT, job_kind TEXT NOT NULL DEFAULT 'task',
|
||||
delivery_policy TEXT NOT NULL DEFAULT 'direct',
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
delete_after_run INTEGER NOT NULL DEFAULT 0, next_run_at INTEGER NOT NULL,
|
||||
last_run_at INTEGER, last_status TEXT, last_error TEXT,
|
||||
locked_at INTEGER, lock_owner TEXT, lease_until INTEGER,
|
||||
created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.execute(pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
r#"
|
||||
CREATE TABLE job_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
job_id TEXT NOT NULL REFERENCES scheduled_jobs(id) ON DELETE CASCADE,
|
||||
started_at INTEGER NOT NULL, finished_at INTEGER NOT NULL,
|
||||
status TEXT NOT NULL, output TEXT, error TEXT,
|
||||
duration_ms INTEGER NOT NULL, result_kind TEXT,
|
||||
delivery_status TEXT, delivery_error TEXT
|
||||
)
|
||||
"#,
|
||||
)
|
||||
.execute(pool)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn sqlite_runtime_guards_are_enabled() {
|
||||
let (storage, _dir) = create_test_storage().await;
|
||||
@ -1867,7 +1764,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn legacy_schema_is_migrated_to_canonical_v11() {
|
||||
async fn legacy_schema_is_migrated_without_rebuild() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let db_path = dir.path().join("legacy.db");
|
||||
let pool = SqlitePoolOptions::new()
|
||||
@ -1963,14 +1860,7 @@ mod tests {
|
||||
),
|
||||
(
|
||||
"scheduled_jobs",
|
||||
vec![
|
||||
"agent_id",
|
||||
"delivery_policy",
|
||||
"last_outcome",
|
||||
"locked_at",
|
||||
"lock_owner",
|
||||
"lease_until",
|
||||
],
|
||||
vec!["locked_at", "lock_owner", "lease_until"],
|
||||
),
|
||||
] {
|
||||
let columns = sqlx::query(sqlx::AssertSqlSafe(format!("PRAGMA table_info({table})")))
|
||||
@ -1986,25 +1876,6 @@ mod tests {
|
||||
);
|
||||
}
|
||||
}
|
||||
let scheduler_columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
|
||||
.fetch_all(storage.pool())
|
||||
.await
|
||||
.unwrap()
|
||||
.into_iter()
|
||||
.map(|row| row.get::<String, _>("name"))
|
||||
.collect::<Vec<_>>();
|
||||
for removed in [
|
||||
"model",
|
||||
"job_kind",
|
||||
"delete_after_run",
|
||||
"last_status",
|
||||
"last_error",
|
||||
] {
|
||||
assert!(
|
||||
!scheduler_columns.iter().any(|column| column == removed),
|
||||
"legacy scheduler column survived v11: {removed}"
|
||||
);
|
||||
}
|
||||
let schema_version: i64 = sqlx::query_scalar("PRAGMA user_version")
|
||||
.fetch_one(storage.pool())
|
||||
.await
|
||||
@ -2043,250 +1914,6 @@ mod tests {
|
||||
assert_eq!(origin, "user");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn v10_scheduler_history_and_active_lease_migrate_to_v11() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let db_path = dir.path().join("v10-scheduler.db");
|
||||
let pool = SqlitePoolOptions::new()
|
||||
.connect_with(
|
||||
SqliteConnectOptions::new()
|
||||
.filename(&db_path)
|
||||
.create_if_missing(true),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
create_v10_scheduler_schema(&pool).await;
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO scheduled_jobs
|
||||
(id, name, schedule, prompt, channel, chat_id, model, job_kind,
|
||||
delivery_policy, enabled, next_run_at, last_run_at, last_status,
|
||||
last_error, created_at, updated_at)
|
||||
VALUES ('legacy-direct', 'Legacy direct', '{"type":"every","every_ms":60000}',
|
||||
'check it', 'cli_chat', 'chat', 'obsolete-model', 'monitor',
|
||||
'direct', 1, 2000, 1000, 'delivery_error', 'legacy diagnostic', 1, 2)
|
||||
"#,
|
||||
)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO job_runs
|
||||
(job_id, started_at, finished_at, status, output, error, duration_ms,
|
||||
result_kind, delivery_status, delivery_error)
|
||||
VALUES ('legacy-direct', 1000, 1100, 'delivery_error', 'legacy result', NULL,
|
||||
100, NULL, 'failed', 'channel rejected target')
|
||||
"#,
|
||||
)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO scheduled_jobs
|
||||
(id, name, schedule, prompt, channel, chat_id, job_kind,
|
||||
delivery_policy, enabled, next_run_at, last_run_at, last_status,
|
||||
locked_at, lock_owner, lease_until, created_at, updated_at)
|
||||
VALUES ('legacy-locked', 'Legacy locked', '{"type":"every","every_ms":60000}',
|
||||
'check lock', 'cli_chat', 'chat', 'task', 'never', 1, 2000,
|
||||
1500, NULL, 1500, 'old-owner', 900000, 1, 2)
|
||||
"#,
|
||||
)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("PRAGMA user_version = 10")
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
drop(pool);
|
||||
|
||||
let storage = Storage::new(&db_path).await.unwrap();
|
||||
let migrated_columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
|
||||
.fetch_all(storage.pool())
|
||||
.await
|
||||
.unwrap()
|
||||
.into_iter()
|
||||
.map(|row| row.get::<String, _>("name"))
|
||||
.collect::<Vec<_>>();
|
||||
assert!(
|
||||
migrated_columns
|
||||
.iter()
|
||||
.any(|column| column == "last_outcome"),
|
||||
"unexpected post-migration scheduler schema: {migrated_columns:?}"
|
||||
);
|
||||
let direct = storage.get_scheduled_job("legacy-direct").await.unwrap();
|
||||
assert_eq!(
|
||||
direct.delivery_policy,
|
||||
crate::storage::DeliveryPolicy::Always
|
||||
);
|
||||
assert_eq!(
|
||||
direct.last_outcome,
|
||||
Some(crate::storage::ScheduledOutcomeKind::Ok)
|
||||
);
|
||||
let historical = storage
|
||||
.list_scheduled_job_runs("legacy-direct", 10)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(historical.len(), 1);
|
||||
assert_eq!(
|
||||
historical[0].status,
|
||||
crate::storage::ScheduledRunStatus::Completed
|
||||
);
|
||||
assert_eq!(
|
||||
historical[0].outcome,
|
||||
Some(crate::storage::ScheduledOutcomeKind::Ok)
|
||||
);
|
||||
assert_eq!(
|
||||
historical[0].delivery_status,
|
||||
crate::storage::ScheduledDeliveryStatus::Failed
|
||||
);
|
||||
assert_eq!(
|
||||
historical[0].diagnostic.as_deref(),
|
||||
Some("legacy diagnostic")
|
||||
);
|
||||
|
||||
let recovered = storage
|
||||
.list_scheduled_job_runs("legacy-locked", 10)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(recovered.len(), 1);
|
||||
assert_eq!(
|
||||
recovered[0].status,
|
||||
crate::storage::ScheduledRunStatus::Unknown
|
||||
);
|
||||
assert_eq!(
|
||||
recovered[0].delivery_status,
|
||||
crate::storage::ScheduledDeliveryStatus::NotRequested
|
||||
);
|
||||
let locked = storage.get_scheduled_job("legacy-locked").await.unwrap();
|
||||
assert!(locked.lock_owner.is_none());
|
||||
|
||||
let columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
|
||||
.fetch_all(storage.pool())
|
||||
.await
|
||||
.unwrap()
|
||||
.into_iter()
|
||||
.map(|row| row.get::<String, _>("name"))
|
||||
.collect::<Vec<_>>();
|
||||
for removed in [
|
||||
"model",
|
||||
"job_kind",
|
||||
"delete_after_run",
|
||||
"last_status",
|
||||
"last_error",
|
||||
] {
|
||||
assert!(!columns.iter().any(|column| column == removed));
|
||||
}
|
||||
let violations = sqlx::query("PRAGMA foreign_key_check")
|
||||
.fetch_all(storage.pool())
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(violations.is_empty());
|
||||
|
||||
let fresh_dir = tempfile::tempdir().unwrap();
|
||||
let fresh = Storage::new(&fresh_dir.path().join("fresh.db"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
scheduler_schema_sql(&storage).await,
|
||||
scheduler_schema_sql(&fresh).await
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn invalid_v10_scheduler_data_rolls_back_v11_migration() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let db_path = dir.path().join("invalid-v10-scheduler.db");
|
||||
let pool = SqlitePoolOptions::new()
|
||||
.connect_with(
|
||||
SqliteConnectOptions::new()
|
||||
.filename(&db_path)
|
||||
.create_if_missing(true),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
create_v10_scheduler_schema(&pool).await;
|
||||
sqlx::query(
|
||||
r#"
|
||||
INSERT INTO scheduled_jobs
|
||||
(id, name, schedule, prompt, channel, chat_id, delivery_policy,
|
||||
enabled, next_run_at, created_at, updated_at)
|
||||
VALUES ('broken', 'Broken', '{not-json}', 'check', 'cli_chat', 'chat',
|
||||
'always', 1, 1000, 1, 1)
|
||||
"#,
|
||||
)
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("PRAGMA user_version = 10")
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
drop(pool);
|
||||
|
||||
let error = match Storage::new(&db_path).await {
|
||||
Ok(_) => panic!("invalid v10 scheduler data unexpectedly migrated"),
|
||||
Err(error) => error,
|
||||
};
|
||||
assert!(matches!(error, StorageError::Migration(_)));
|
||||
|
||||
let pool = SqlitePoolOptions::new()
|
||||
.connect_with(SqliteConnectOptions::new().filename(&db_path))
|
||||
.await
|
||||
.unwrap();
|
||||
let version: i64 = sqlx::query_scalar("PRAGMA user_version")
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(version, 10);
|
||||
let columns = sqlx::query("PRAGMA table_info(scheduled_jobs)")
|
||||
.fetch_all(&pool)
|
||||
.await
|
||||
.unwrap()
|
||||
.into_iter()
|
||||
.map(|row| row.get::<String, _>("name"))
|
||||
.collect::<Vec<_>>();
|
||||
assert!(columns.iter().any(|column| column == "job_kind"));
|
||||
assert!(!columns.iter().any(|column| column == "agent_id"));
|
||||
for table in ["scheduled_jobs_v10_legacy", "job_runs_v10_legacy"] {
|
||||
let exists: i64 = sqlx::query_scalar(
|
||||
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?",
|
||||
)
|
||||
.bind(table)
|
||||
.fetch_one(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(exists, 0);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn newer_schema_version_is_rejected() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let db_path = dir.path().join("future.db");
|
||||
let pool = SqlitePoolOptions::new()
|
||||
.connect_with(
|
||||
SqliteConnectOptions::new()
|
||||
.filename(&db_path)
|
||||
.create_if_missing(true),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("PRAGMA user_version = 12")
|
||||
.execute(&pool)
|
||||
.await
|
||||
.unwrap();
|
||||
drop(pool);
|
||||
|
||||
let error = match Storage::new(&db_path).await {
|
||||
Ok(_) => panic!("newer schema version unexpectedly opened"),
|
||||
Err(error) => error,
|
||||
};
|
||||
assert!(matches!(error, StorageError::Migration(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn v3_migration_preserves_existing_reasoning_and_defaults_completion() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,191 +0,0 @@
|
||||
use async_trait::async_trait;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::storage::ScheduledOutcomeKind;
|
||||
use crate::tools::{ScheduledOutcome, Tool, ToolExecutionContext, ToolOutput, ToolResult};
|
||||
|
||||
pub struct CompleteScheduledRunTool;
|
||||
|
||||
impl CompleteScheduledRunTool {
|
||||
pub fn new() -> Self {
|
||||
Self
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for CompleteScheduledRunTool {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Tool for CompleteScheduledRunTool {
|
||||
fn name(&self) -> &str {
|
||||
"complete_scheduled_run"
|
||||
}
|
||||
|
||||
fn description(&self) -> &str {
|
||||
"Submit the single structured final outcome (ok, alert, failed, or refused) of an unattended scheduled run and end the run immediately."
|
||||
}
|
||||
|
||||
fn parameters_schema(&self) -> Value {
|
||||
json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"outcome": {
|
||||
"type": "string",
|
||||
"enum": ["ok", "alert", "failed", "refused"],
|
||||
"description": "ok: completed with nothing needing attention; alert: completed with actionable findings; failed: did not complete reliably; refused: denied for permission or safety reasons"
|
||||
},
|
||||
"message": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"maxLength": 16384,
|
||||
"description": "The user-facing result or notification body"
|
||||
}
|
||||
},
|
||||
"required": ["outcome", "message"],
|
||||
"additionalProperties": false
|
||||
})
|
||||
}
|
||||
|
||||
fn runtime_injected(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
fn exclusive(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
async fn execute(&self, args: Value) -> anyhow::Result<ToolResult> {
|
||||
self.execute_with_context(&ToolExecutionContext::default(), args)
|
||||
.await
|
||||
.map(|output| output.result)
|
||||
}
|
||||
|
||||
async fn execute_with_context(
|
||||
&self,
|
||||
context: &ToolExecutionContext,
|
||||
args: Value,
|
||||
) -> anyhow::Result<ToolOutput> {
|
||||
let Some(sink) = context.scheduled_completion.as_ref() else {
|
||||
return Ok(failure(
|
||||
"complete_scheduled_run is only available to the top-level scheduled Agent",
|
||||
)
|
||||
.into());
|
||||
};
|
||||
if !context.execution_origin.is_scheduled() {
|
||||
return Ok(failure("scheduled completion context is invalid").into());
|
||||
}
|
||||
let Some(object) = args.as_object() else {
|
||||
return Ok(failure("arguments must be an object").into());
|
||||
};
|
||||
if object
|
||||
.keys()
|
||||
.any(|key| key != "outcome" && key != "message")
|
||||
{
|
||||
return Ok(failure("unknown complete_scheduled_run argument").into());
|
||||
}
|
||||
let kind = match args.get("outcome").and_then(Value::as_str) {
|
||||
Some("ok") => ScheduledOutcomeKind::Ok,
|
||||
Some("alert") => ScheduledOutcomeKind::Alert,
|
||||
Some("failed") => ScheduledOutcomeKind::Failed,
|
||||
Some("refused") => ScheduledOutcomeKind::Refused,
|
||||
Some(other) => return Ok(failure(format!("invalid scheduled outcome: {other}")).into()),
|
||||
None => return Ok(failure("outcome is required").into()),
|
||||
};
|
||||
let message = args
|
||||
.get("message")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default()
|
||||
.trim();
|
||||
if message.is_empty() {
|
||||
return Ok(failure("message must not be empty").into());
|
||||
}
|
||||
if message.chars().count() > 16_384 {
|
||||
return Ok(failure("message exceeds 16384 characters").into());
|
||||
}
|
||||
let message = message.to_string();
|
||||
if let Err(error) = sink.submit(ScheduledOutcome {
|
||||
kind,
|
||||
message: message.clone(),
|
||||
}) {
|
||||
return Ok(failure(error).into());
|
||||
}
|
||||
Ok(ToolResult {
|
||||
success: true,
|
||||
output: format!("scheduled run completed with outcome={}", kind.as_str()),
|
||||
error: None,
|
||||
}
|
||||
.into())
|
||||
}
|
||||
}
|
||||
|
||||
fn failure(error: impl Into<String>) -> ToolResult {
|
||||
ToolResult {
|
||||
success: false,
|
||||
output: String::new(),
|
||||
error: Some(error.into()),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::tools::{ExecutionOrigin, ScheduledCompletionSink};
|
||||
use std::sync::Arc;
|
||||
|
||||
#[tokio::test]
|
||||
async fn submits_exactly_once_in_scheduled_context() {
|
||||
let sink = Arc::new(ScheduledCompletionSink::default());
|
||||
let context = ToolExecutionContext::for_session("scheduled-run:1")
|
||||
.with_execution_origin(ExecutionOrigin::Scheduled { job_run_id: 1 })
|
||||
.with_scheduled_completion(sink.clone());
|
||||
let tool = CompleteScheduledRunTool::new();
|
||||
let first = tool
|
||||
.execute_with_context(&context, json!({"outcome":"ok","message":"healthy"}))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(first.result.success);
|
||||
assert_eq!(sink.outcome().unwrap().message, "healthy");
|
||||
let second = tool
|
||||
.execute_with_context(&context, json!({"outcome":"alert","message":"again"}))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!second.result.success);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn rejects_empty_extra_and_oversized_arguments() {
|
||||
let sink = Arc::new(ScheduledCompletionSink::default());
|
||||
let context = ToolExecutionContext::for_session("scheduled-run:1")
|
||||
.with_execution_origin(ExecutionOrigin::Scheduled { job_run_id: 1 })
|
||||
.with_scheduled_completion(sink);
|
||||
let tool = CompleteScheduledRunTool::new();
|
||||
for args in [
|
||||
json!({"outcome":"ok","message":" "}),
|
||||
json!({"outcome":"ok","message":"fine","notify":false}),
|
||||
json!({"outcome":"unknown","message":"fine"}),
|
||||
] {
|
||||
assert!(
|
||||
!tool
|
||||
.execute_with_context(&context, args)
|
||||
.await
|
||||
.unwrap()
|
||||
.result
|
||||
.success
|
||||
);
|
||||
}
|
||||
assert!(
|
||||
!tool
|
||||
.execute_with_context(
|
||||
&context,
|
||||
json!({"outcome":"ok","message":"x".repeat(16_385)}),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.result
|
||||
.success
|
||||
);
|
||||
}
|
||||
}
|
||||
1125
src/tools/cron.rs
1125
src/tools/cron.rs
File diff suppressed because it is too large
Load Diff
@ -155,7 +155,7 @@ impl DelegateTool {
|
||||
args: &Value,
|
||||
context: &ToolExecutionContext,
|
||||
) -> anyhow::Result<ToolResult> {
|
||||
let requested_mode = match args
|
||||
let mode = match args
|
||||
.get("mode")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("foreground")
|
||||
@ -168,13 +168,6 @@ impl DelegateTool {
|
||||
)));
|
||||
}
|
||||
};
|
||||
let background_downgraded =
|
||||
requested_mode == ExecutionMode::Background && context.execution_origin.is_scheduled();
|
||||
let mode = if background_downgraded {
|
||||
ExecutionMode::Foreground
|
||||
} else {
|
||||
requested_mode
|
||||
};
|
||||
let task_values: Vec<&Value> = match args.get("tasks").and_then(Value::as_array) {
|
||||
Some(tasks) if !tasks.is_empty() => tasks.iter().collect(),
|
||||
Some(_) => return Ok(failure("tasks must not be empty")),
|
||||
@ -255,7 +248,6 @@ impl DelegateTool {
|
||||
success: all_completed,
|
||||
output: serde_json::to_string(&json!({
|
||||
"status": if all_completed { "completed" } else { "partial" },
|
||||
"background_downgraded": background_downgraded,
|
||||
"results": payload
|
||||
}))?,
|
||||
error: None,
|
||||
|
||||
@ -3,7 +3,6 @@ pub mod bash;
|
||||
pub mod browser;
|
||||
pub mod calculator;
|
||||
pub mod chat_manager;
|
||||
pub mod complete_scheduled_run;
|
||||
pub mod content_search;
|
||||
pub mod cron;
|
||||
pub mod delegate;
|
||||
@ -33,7 +32,6 @@ pub use bash::BashTool;
|
||||
pub use browser::{BrowserProfilesTool, BrowserTool};
|
||||
pub use calculator::CalculatorTool;
|
||||
pub use chat_manager::ChatManagerTool;
|
||||
pub use complete_scheduled_run::CompleteScheduledRunTool;
|
||||
pub use content_search::ContentSearchTool;
|
||||
pub use delegate::DelegateTool;
|
||||
pub use emit_signal::EmitSignalTool;
|
||||
@ -52,9 +50,8 @@ pub use reload_config::ReloadConfigTool;
|
||||
pub use send_message::SendMessageTool;
|
||||
pub use todo::TodoTool;
|
||||
pub use traits::{
|
||||
ExecutionOrigin, OutboundDelivery, OutboundMessenger, ProcessedToolOutput,
|
||||
ScheduledCompletionSink, ScheduledOutcome, Tool, ToolArtifact, ToolArtifactAudience,
|
||||
ToolExecutionContext, ToolOutput, ToolOutputProcessor, ToolResult,
|
||||
OutboundDelivery, OutboundMessenger, ProcessedToolOutput, Tool, ToolArtifact,
|
||||
ToolArtifactAudience, ToolExecutionContext, ToolOutput, ToolOutputProcessor, ToolResult,
|
||||
};
|
||||
pub use web_fetch::WebFetchTool;
|
||||
|
||||
|
||||
@ -1,60 +1,6 @@
|
||||
use crate::bus::{MediaItem, MediaRef, MessageSource};
|
||||
use async_trait::async_trait;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum ExecutionOrigin {
|
||||
#[default]
|
||||
Interactive,
|
||||
Scheduled {
|
||||
job_run_id: i64,
|
||||
},
|
||||
}
|
||||
|
||||
impl ExecutionOrigin {
|
||||
pub fn is_scheduled(self) -> bool {
|
||||
matches!(self, Self::Scheduled { .. })
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ScheduledOutcome {
|
||||
pub kind: crate::storage::ScheduledOutcomeKind,
|
||||
pub message: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub struct ScheduledCompletionSink {
|
||||
outcome: std::sync::Mutex<Option<ScheduledOutcome>>,
|
||||
}
|
||||
|
||||
impl ScheduledCompletionSink {
|
||||
pub fn submit(&self, outcome: ScheduledOutcome) -> Result<(), &'static str> {
|
||||
let mut guard = self
|
||||
.outcome
|
||||
.lock()
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner());
|
||||
if guard.is_some() {
|
||||
return Err("scheduled result was already submitted");
|
||||
}
|
||||
*guard = Some(outcome);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn outcome(&self) -> Option<ScheduledOutcome> {
|
||||
self.outcome
|
||||
.lock()
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner())
|
||||
.clone()
|
||||
}
|
||||
|
||||
pub fn is_completed(&self) -> bool {
|
||||
self.outcome
|
||||
.lock()
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner())
|
||||
.is_some()
|
||||
}
|
||||
}
|
||||
|
||||
/// Session identity supplied by the runtime for tools that own external state.
|
||||
/// Ordinary stateless tools can ignore it through the default trait method.
|
||||
#[derive(Debug, Clone)]
|
||||
@ -64,8 +10,6 @@ pub struct ToolExecutionContext {
|
||||
pub agent: Option<std::sync::Arc<crate::agent::AgentExecutionContext>>,
|
||||
pub cancellation: tokio_util::sync::CancellationToken,
|
||||
pub execution_gate: Option<std::sync::Arc<crate::agent::gate::ExecutionGate>>,
|
||||
pub execution_origin: ExecutionOrigin,
|
||||
pub scheduled_completion: Option<std::sync::Arc<ScheduledCompletionSink>>,
|
||||
}
|
||||
|
||||
impl Default for ToolExecutionContext {
|
||||
@ -76,8 +20,6 @@ impl Default for ToolExecutionContext {
|
||||
agent: None,
|
||||
cancellation: tokio_util::sync::CancellationToken::new(),
|
||||
execution_gate: None,
|
||||
execution_origin: ExecutionOrigin::Interactive,
|
||||
scheduled_completion: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -90,8 +32,6 @@ impl ToolExecutionContext {
|
||||
agent: None,
|
||||
cancellation: tokio_util::sync::CancellationToken::new(),
|
||||
execution_gate: None,
|
||||
execution_origin: ExecutionOrigin::Interactive,
|
||||
scheduled_completion: None,
|
||||
}
|
||||
}
|
||||
|
||||
@ -120,19 +60,6 @@ impl ToolExecutionContext {
|
||||
self.execution_gate = Some(gate);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_execution_origin(mut self, execution_origin: ExecutionOrigin) -> Self {
|
||||
self.execution_origin = execution_origin;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_scheduled_completion(
|
||||
mut self,
|
||||
sink: std::sync::Arc<ScheduledCompletionSink>,
|
||||
) -> Self {
|
||||
self.scheduled_completion = Some(sink);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
|
||||
4
webui/package-lock.json
generated
4
webui/package-lock.json
generated
@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "picobot-webui",
|
||||
"version": "1.22.0",
|
||||
"version": "1.21.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "picobot-webui",
|
||||
"version": "1.22.0",
|
||||
"version": "1.21.0",
|
||||
"dependencies": {
|
||||
"bits-ui": "^2.0.0",
|
||||
"dompurify": "^3.4.12",
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "picobot-webui",
|
||||
"private": true,
|
||||
"version": "1.22.0",
|
||||
"version": "1.21.0",
|
||||
"type": "module",
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
|
||||
@ -2,11 +2,11 @@
|
||||
let { status = "unknown" } = $props();
|
||||
const normalized = $derived(String(status).toLowerCase());
|
||||
const tone = $derived(
|
||||
["completed", "success", "ok", "enabled", "delivered"].includes(normalized)
|
||||
["completed", "success", "ok", "enabled"].includes(normalized)
|
||||
? "ok"
|
||||
: ["failed", "error", "cancelled", "disabled", "refused", "timed_out"].includes(normalized)
|
||||
: ["failed", "error", "cancelled", "disabled"].includes(normalized)
|
||||
? "fail"
|
||||
: ["running", "pending", "delivering", "alert", "unknown", "interrupted"].includes(normalized) ? "run" : ""
|
||||
: ["running", "pending"].includes(normalized) ? "run" : ""
|
||||
);
|
||||
</script>
|
||||
|
||||
|
||||
@ -42,23 +42,11 @@
|
||||
|
||||
function dotColor(status) {
|
||||
if (status === "completed" || status === "success" || status === "ok") return "var(--signal)";
|
||||
if (status === "timed_out" || status === "unknown" || status === "interrupted") return "var(--accent)";
|
||||
if (status === "timeout") return "var(--accent)";
|
||||
if (status === "running") return "var(--info)";
|
||||
return "var(--danger)";
|
||||
}
|
||||
|
||||
function scheduleLabel(schedule) {
|
||||
if (!schedule) return "—";
|
||||
if (schedule.type === "at") return `单次 · ${formatTime(schedule.at)}`;
|
||||
if (schedule.type === "every") return `每 ${Math.max(1, Math.round(schedule.every_ms / 60000))} 分钟`;
|
||||
if (schedule.type === "cron") return `${schedule.expr}${schedule.tz ? ` · ${schedule.tz}` : ""}`;
|
||||
return schedule.type || "—";
|
||||
}
|
||||
|
||||
function deliveryLabel(policy) {
|
||||
return { always: "始终通知", on_alert: "异常通知", never: "从不通知" }[policy] || policy;
|
||||
}
|
||||
|
||||
onMount(() => {
|
||||
load();
|
||||
const timer = setInterval(() => { tick += 1; }, 30000);
|
||||
@ -70,7 +58,7 @@
|
||||
<div class="toolbar">
|
||||
<div>
|
||||
<h2 style="margin:0">定时任务</h2>
|
||||
<p style="margin:2px 0 0;color:var(--muted);font-size:12px">查看统一的定时执行、结构化结果与通知状态；Agent 运行审计请到「子代理」页面。</p>
|
||||
<p style="margin:2px 0 0;color:var(--muted);font-size:12px">管理定时任务与巡检；后台子代理运行请到「子代理」页面查看。</p>
|
||||
</div>
|
||||
<button class="secondary" onclick={load}><Icon name="refresh" size={16} />刷新</button>
|
||||
</div>
|
||||
@ -85,9 +73,8 @@
|
||||
<h3>{job.name}</h3>
|
||||
<p>{job.prompt}</p>
|
||||
<div class="meta">
|
||||
<span class="mono cron">{scheduleLabel(job.schedule)}</span>
|
||||
<span>Agent：{job.agent_id || "Root"}</span>
|
||||
<span>投递：{deliveryLabel(job.delivery_policy)}</span>
|
||||
<span class="mono cron">{job.cron}</span>
|
||||
<span>{job.job_kind === "monitor" ? "巡检" : "任务"} · {job.delivery_policy}</span>
|
||||
<span>{job.channel} · {job.chat_id}</span>
|
||||
<span>下次 {countdown(job.next_run_at)}</span>
|
||||
<span>上次 {formatTime(job.last_run_at)}</span>
|
||||
@ -100,23 +87,9 @@
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
<StatusBadge status={job.enabled ? (job.last_outcome || "enabled") : "disabled"} />
|
||||
<StatusBadge status={job.enabled ? (job.last_status || "enabled") : "disabled"} />
|
||||
</div>
|
||||
{#if runs[job.id]?.length}
|
||||
<div class="details">
|
||||
{#each runs[job.id] as run}
|
||||
<details class="run-detail">
|
||||
<summary class="run-row">
|
||||
<span class="meta">{formatTime(run.finished_at || run.started_at || run.scheduled_for)} · {run.duration_ms ?? "—"}ms · outcome={run.outcome || "—"} · delivery={run.delivery_status} ({run.delivery_attempts})</span>
|
||||
<StatusBadge status={run.status} />
|
||||
</summary>
|
||||
{#if run.message}<p><b>结果</b><br />{run.message}</p>{/if}
|
||||
{#if run.diagnostic}<p class="diagnostic"><b>诊断</b><br />{run.diagnostic}</p>{/if}
|
||||
{#if run.delivery_error}<p class="diagnostic"><b>投递错误</b><br />{run.delivery_error}</p>{/if}
|
||||
</details>
|
||||
{/each}
|
||||
</div>
|
||||
{/if}
|
||||
{#if runs[job.id]?.length}<div class="details">{#each runs[job.id] as run}<div class="card-row run-row"><span class="meta">{formatTime(run.finished_at)} · {run.duration_ms}ms{run.result_kind ? ` · ${run.result_kind}` : ""}{run.delivery_status ? ` · ${run.delivery_status}` : ""}</span><StatusBadge status={run.status} /></div>{/each}</div>{/if}
|
||||
</article>
|
||||
{:else}<div class="empty-card">暂无定时任务</div>{/each}
|
||||
{/if}
|
||||
@ -127,9 +100,4 @@
|
||||
.cron { font-size: 12px; color: var(--text-soft); background: var(--code-bg); padding: 1px 6px; border-radius: 4px; border: 1px solid var(--line); font-variant-numeric: tabular-nums; }
|
||||
.status-dots { display: flex; gap: 4px; margin-top: 6px; align-items: center; }
|
||||
.run-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
|
||||
.run-detail { padding: 7px 0; border-top: 1px solid var(--line); }
|
||||
.run-detail:first-child { border-top: 0; }
|
||||
.run-detail summary { cursor: pointer; justify-content: space-between; }
|
||||
.run-detail p { margin: 8px 0 0; white-space: pre-wrap; overflow-wrap: anywhere; font-size: 12px; color: var(--text-soft); }
|
||||
.run-detail .diagnostic { color: var(--danger); }
|
||||
</style>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user