实现tui和webui的附件收发,优化文件在消息中的处理

This commit is contained in:
xiaoxixi 2026-07-16 17:49:51 +08:00
parent e37581c909
commit f4172fea38
28 changed files with 1926 additions and 60 deletions

View File

@ -83,6 +83,7 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
- **WebUI management APIs** only expose allowlisted config/profile/log/storage operations; keep response limits, secret redaction, atomic config writes, and profile path allowlists intact - **WebUI management APIs** only expose allowlisted config/profile/log/storage operations; keep response limits, secret redaction, atomic config writes, and profile path allowlists intact
- **WebUI slash completion** must consume the existing `get_slash_commands` WebSocket response; do not duplicate the backend command list in frontend source - **WebUI slash completion** must consume the existing `get_slash_commands` WebSocket response; do not duplicate the backend command list in frontend source
- **WebUI chat rendering** sanitizes Markdown before inserting HTML; session history must preserve structured tool-call metadata so calls and results remain independently collapsible - **WebUI chat rendering** sanitizes Markdown before inserting HTML; session history must preserve structured tool-call metadata so calls and results remain independently collapsible
- **WebUI/TUI file transfer** streams bytes over authenticated HTTP and sends only short-lived upload IDs/attachment metadata over WebSocket; messages persist local media paths without guaranteeing later availability, and client responses must never expose those paths
- **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token; never put bearer tokens in URLs or logs - **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token; never put bearer tokens in URLs or logs
- **Providers** are pure HTTP clients; no bus/session/channel awareness - **Providers** are pure HTTP clients; no bus/session/channel awareness
- **Tools** are executed by `AgentLoop`; they receive raw arguments and return string results - **Tools** are executed by `AgentLoop`; they receive raw arguments and return string results

View File

@ -11,10 +11,10 @@ serde_json = "1.0"
async-trait = "0.1" async-trait = "0.1"
thiserror = "2.0.18" thiserror = "2.0.18"
tokio = { version = "1.52", features = ["full"] } tokio = { version = "1.52", features = ["full"] }
tokio-util = { version = "0.7", features = ["rt"] } tokio-util = { version = "0.7", features = ["rt", "io"] }
dashmap = "6.1" dashmap = "6.1"
uuid = { version = "1.23", features = ["v4"] } uuid = { version = "1.23", features = ["v4"] }
axum = { version = "0.8", features = ["ws"] } axum = { version = "0.8", features = ["ws", "multipart"] }
tokio-tungstenite = { version = "0.29.0", features = ["rustls-tls-webpki-roots", "rustls"] } tokio-tungstenite = { version = "0.29.0", features = ["rustls-tls-webpki-roots", "rustls"] }
futures-util = "0.3" futures-util = "0.3"
clap = { version = "4", features = ["derive"] } clap = { version = "4", features = ["derive"] }

View File

@ -112,6 +112,7 @@ picobot pair
WebUI 随二进制嵌入,不需要 Node.js、npm 或单独部署静态文件,提供: WebUI 随二进制嵌入,不需要 Node.js、npm 或单独部署静态文件,提供:
- 在线聊天、会话创建/切换、历史回放、Markdown 消息、可折叠工具调用卡片,以及基于 Gateway 实时命令清单的 `/` 斜杠命令补全。 - 在线聊天、会话创建/切换、历史回放、Markdown 消息、可折叠工具调用卡片,以及基于 Gateway 实时命令清单的 `/` 斜杠命令补全。
- 文件选择、拖放和剪贴板图片上传;消息中的附件可预览或下载。附件按服务端路径引用,原文件移动或删除后历史附件可能不可用。
- 可持久化的浅色/深色主题,首次访问时跟随系统偏好。 - 可持久化的浅色/深色主题,首次访问时跟随系统偏好。
- Cron 定时任务、最近运行记录和后台子任务状态。 - Cron 定时任务、最近运行记录和后台子任务状态。
- 当前聊天 session 的可展开 Todo 侧栏;计划变化时自动展开,其他 session 的变化显示未读提示。 - 当前聊天 session 的可展开 Todo 侧栏;计划变化时自动展开,其他 session 的变化显示未读提示。
@ -156,7 +157,7 @@ picobot service uninstall
unit 位于 `~/.config/systemd/user/picobot.service`,以执行 `service install` 时的当前目录作为初始工作目录。服务异常退出时由 systemd 自动重启;`stop``restart` 会通过 SIGTERM 触发 Gateway 的有界优雅关停。 unit 位于 `~/.config/systemd/user/picobot.service`,以执行 `service install` 时的当前目录作为初始工作目录。服务异常退出时由 systemd 自动重启;`stop``restart` 会通过 SIGTERM 触发 Gateway 的有界优雅关停。
TUI 会把一个随机客户端标识保存到 `~/.picobot/tui_client_id`,因此关闭并重新打开客户端后会恢复同一组 dialog 和最近使用的会话。界面支持历史回放、会话列表与归档筛选、命令补全、Unicode/中文编辑、括号粘贴和多行输入 TUI 会把一个随机客户端标识保存到 `~/.picobot/tui_client_id`,因此关闭并重新打开客户端后会恢复同一组 dialog 和最近使用的会话。界面支持历史回放、会话列表与归档筛选、命令补全、Unicode/中文编辑、括号粘贴、多行输入和文件传输
常用快捷键: 常用快捷键:
@ -168,6 +169,9 @@ TUI 会把一个随机客户端标识保存到 `~/.picobot/tui_client_id`,因
| `Ctrl+R` / `Ctrl+A` / `Ctrl+D` | 重命名 / 归档 / 删除所选会话 | | `Ctrl+R` / `Ctrl+A` / `Ctrl+D` | 重命名 / 归档 / 删除所选会话 |
| `Ctrl+L` / `Ctrl+O` | 清空历史 / 显示归档会话 | | `Ctrl+L` / `Ctrl+O` | 清空历史 / 显示归档会话 |
| `Enter` / `Shift+Enter` | 发送 / 换行 | | `Enter` / `Shift+Enter` | 发送 / 换行 |
| `Ctrl+F` / `F2` | 添加附件 / 下载历史中最近的附件 |
WebUI/TUI 上传文件默认保存到 `~/.picobot/media/cli_chat`,单文件上限 25 MiB可通过 `gateway.file_transfer` 调整目录、上限和待发送上传的有效期。消息只保存文件路径,不保证文件被移动或删除后仍能下载。
| `PageUp` / `PageDown` | 滚动对话历史 | | `PageUp` / `PageDown` | 滚动对话历史 |
| 连按两次 `Ctrl+C` | 退出客户端 | | 连按两次 `Ctrl+C` | 退出客户端 |
@ -402,6 +406,7 @@ docs/ 面向维护者和 Agent 的架构与开发文档
## 进一步阅读 ## 进一步阅读
- [维护者架构文档](docs/ARCHITECTURE.md) - [维护者架构文档](docs/ARCHITECTURE.md)
- [WebUI 与 TUI 文件收发设计](docs/FILE_TRANSFER_DESIGN.md)
- [内置 Skill架构机制](resources/skills/about-picobot/references/architecture.md) - [内置 Skill架构机制](resources/skills/about-picobot/references/architecture.md)
- [配置说明](resources/skills/about-picobot/references/config.md) - [配置说明](resources/skills/about-picobot/references/config.md)
- [命令说明](resources/skills/about-picobot/references/commands.md) - [命令说明](resources/skills/about-picobot/references/commands.md)

View File

@ -199,6 +199,8 @@ WebSocket 每个客户端的 writer task 是连接局部任务,由连接 handl
Gateway 在 `/` 提供随二进制编译的 HTML/CSS/JavaScript不依赖外部 CDN 或前端运行服务。前端源码位于 `webui/`,由 Svelte 5 + Vite 构建Bits UI 提供无样式的可访问组件原语。`build.rs` 监听前端源码、锁文件和构建配置,增量地将生产资源生成到 Cargo `OUT_DIR`Rust 再从该目录编译嵌入;前端产物不进入仓库,最终用户使用发布二进制时不需要 Node.js。浏览器聊天继续使用 `/ws``cli_chat` 渠道,因此复用现有 dialog scope、每会话串行 worker、历史持久化和出站 lane会话历史帧保留工具调用 ID、名称、参数和工具结果角色WebUI 在本轮完成后刷新历史并将其渲染为默认折叠的工具卡片;聊天页的 Todo 侧栏默认隐藏,按 session 保存快照和未读状态,并通过结构化 `session_plan`/`plan_updated` 帧刷新,计划变化不会写入聊天历史;斜杠命令补全通过 `get_slash_commands` 获取 Gateway 的实时命令与别名清单不在前端重复定义WebUI 不直接调用 Provider 或 SessionManager。 Gateway 在 `/` 提供随二进制编译的 HTML/CSS/JavaScript不依赖外部 CDN 或前端运行服务。前端源码位于 `webui/`,由 Svelte 5 + Vite 构建Bits UI 提供无样式的可访问组件原语。`build.rs` 监听前端源码、锁文件和构建配置,增量地将生产资源生成到 Cargo `OUT_DIR`Rust 再从该目录编译嵌入;前端产物不进入仓库,最终用户使用发布二进制时不需要 Node.js。浏览器聊天继续使用 `/ws``cli_chat` 渠道,因此复用现有 dialog scope、每会话串行 worker、历史持久化和出站 lane会话历史帧保留工具调用 ID、名称、参数和工具结果角色WebUI 在本轮完成后刷新历史并将其渲染为默认折叠的工具卡片;聊天页的 Todo 侧栏默认隐藏,按 session 保存快照和未读状态,并通过结构化 `session_plan`/`plan_updated` 帧刷新,计划变化不会写入聊天历史;斜杠命令补全通过 `get_slash_commands` 获取 Gateway 的实时命令与别名清单不在前端重复定义WebUI 不直接调用 Provider 或 SessionManager。
WebUI/TUI 文件字节通过受鉴权的 HTTP 接口流式传输WebSocket 只携带短期 `upload_id` 和结构化附件描述。`UploadRegistry` 在内存中按 `cli_chat` chat scope 校验并消费待发送上传;消息继续以 `media_refs` 保存 Gateway 本地路径,不建立永久附件资产。下载接口必须通过 client、session、message 和附件序号反查路径,不能接受客户端路径。历史附件路径失效属于正常状态,不得影响历史文本读取。待发送但未进入消息的上传由 `TaskSupervisor` 所有的限时清理任务回收。Agent 上下文会为所有媒体注入内部路径清单,客户端响应不得暴露该路径。
`AuthManager` 默认保护所有管理 API 和 `/ws`。静态资源、`/health``/api/auth/status``/api/auth/pair` 保持公开,使未配对浏览器只能加载配对界面。`picobot pair` 使用权限为 `0600` 的本机管理密钥调用仅接受真实回环连接的 `/api/auth/code`;反向代理即使从回环连接也无法在没有该密钥时签发代码。配对码为 8 位、5 分钟有效、单次消费,并按来源实施失败锁定。浏览器收到 HttpOnly、SameSite=Strict CookieCLI 使用 Bearer token服务端仅持久化 SHA-256 哈希。`--revoke-all` 的持久化成功后才清空内存令牌,活动 WebSocket 每 5 秒复核身份并回收已撤销连接。 `AuthManager` 默认保护所有管理 API 和 `/ws`。静态资源、`/health``/api/auth/status``/api/auth/pair` 保持公开,使未配对浏览器只能加载配对界面。`picobot pair` 使用权限为 `0600` 的本机管理密钥调用仅接受真实回环连接的 `/api/auth/code`;反向代理即使从回环连接也无法在没有该密钥时签发代码。配对码为 8 位、5 分钟有效、单次消费,并按来源实施失败锁定。浏览器收到 HttpOnly、SameSite=Strict CookieCLI 使用 Bearer token服务端仅持久化 SHA-256 哈希。`--revoke-all` 的持久化成功后才清空内存令牌,活动 WebSocket 每 5 秒复核身份并回收已撤销连接。
同源 `/api/*` 管理接口只提供显式白名单能力: 同源 `/api/*` 管理接口只提供显式白名单能力:
@ -207,6 +209,7 @@ Gateway 在 `/` 提供随二进制编译的 HTML/CSS/JavaScript不依赖外
- `USER.md``AGENTS.md` 只允许固定文件名,不接受任意路径。 - `USER.md``AGENTS.md` 只允许固定文件名,不接受任意路径。
- 日志、记忆、任务和运行记录均限制单次返回数量;日志目录固定为 `~/.picobot/logs` - 日志、记忆、任务和运行记录均限制单次返回数量;日志目录固定为 `~/.picobot/logs`
- 任务与记忆读取复用 Storage API不允许 WebUI 直接持有 SQLite 连接或拼接任意查询。 - 任务与记忆读取复用 Storage API不允许 WebUI 直接持有 SQLite 连接或拼接任意查询。
- 文件上传/下载复用设备鉴权并校验 chat/session/message scope客户端不能提交或获得服务端路径inline 预览仅允许安全 MIME 白名单。
- 前端依赖只存在于源码构建阶段;生产页面不加载 CDN。`build.rs``package-lock.json` 的依赖 stamp 判断是否需要 `npm ci`,并依靠 Cargo `rerun-if-changed` 避免后端代码变化触发前端重建。前端开发仍须运行 `npm run check`,并以 `cargo build` 验证最终嵌入路径。 - 前端依赖只存在于源码构建阶段;生产页面不加载 CDN。`build.rs``package-lock.json` 的依赖 stamp 判断是否需要 `npm ci`,并依靠 Cargo `rerun-if-changed` 避免后端代码变化触发前端重建。前端开发仍须运行 `npm run check`,并以 `cargo build` 验证最终嵌入路径。
配对鉴权只证明设备持有凭据,不提供机密性。非回环部署仍必须由反向代理或其他外层提供 TLS显式设置 `gateway.require_pairing=false` 会恢复无鉴权模式,仅适合隔离环境。 配对鉴权只证明设备持有凭据,不提供机密性。非回环部署仍必须由反向代理或其他外层提供 TLS显式设置 `gateway.require_pairing=false` 会恢复无鉴权模式,仅适合隔离环境。

View File

@ -0,0 +1,374 @@
# WebUI 与 TUI 文件收发设计
## 1. 目标与非目标
当前统一消息链路已经有 `MediaItem``MediaRef``messages.media_refs`Agent 能处理图片,飞书及 `send_message(files=...)` 也能收发媒体。缺口集中在 `cli_chat`WebSocket 入站把 `media` 固定为空,实时响应和历史协议也没有附件字段,因此 WebUI/TUI 无法使用已有能力。
本设计补齐:
- WebUI 文件选择、拖放、粘贴图片、上传进度、附件展示和下载;
- TUI 通过本地路径上传、展示附件和下载文件;
- 文本加附件及纯附件消息进入现有 session worker
- Agent 通过现有 `send_message(files=...)` 发出的文件可被 WebUI/TUI 获取;
- 文件大小、并发、scope、路径和 MIME 安全边界。
附件采用“路径引用”语义,而不是持久化文件资产:
- 消息仍在 `messages.media_refs` 中记录服务端本地路径和媒体类型;
- 不新增附件表、消息附件关联表、内容寻址、引用计数或永久 blob 存储;
- 不保证历史附件可获取。路径变化、文件被覆盖或删除后,预览/下载可以返回不可用;
- 历史消息必须仍能正常显示,文件不可用不能导致整段历史读取失败;
- 客户端永远看不到或提交服务端路径,路径只在 Gateway 内部使用。
首版不包含目录上传、断点续传、对象存储、跨 Gateway 部署、缩略图服务和自动文档解析。
## 2. 总体方案
采用“HTTP 传字节WebSocket 传短期上传引用和消息事件”:
```mermaid
sequenceDiagram
participant C as WebUI / TUI
participant H as Attachment HTTP API
participant U as UploadRegistry
participant W as cli_chat WebSocket
participant S as Session worker
C->>H: POST multipart file
H->>H: 流式保存到本地媒体目录
H->>U: 注册 upload_id -> path + chat scope
H-->>C: UploadDescriptor(upload_id, name, size, type)
C->>W: user_input(content, upload_ids)
W->>U: 校验并消费 upload IDs
W->>S: InboundMessage(media paths)
S->>S: 按现有 media_refs 持久化路径
S-->>C: assistant_response / session_history
C->>H: GET message attachment by message ID + index
H->>H: 查询 media_refs按当前路径流式读取
H-->>C: bytes 或 404/410
```
不把文件塞入 WebSocket binary frame 或 base64 JSON现有 WebSocket writer、Bus 和 session 队列面向小型消息大帧会造成内存峰值与队头阻塞HTTP 更适合流式 I/O、上传进度、状态码和后续 Range 支持。
## 3. 模型与协议
### 3.1 短期上传描述
上传成功后返回只在短期内有效的 `UploadDescriptor`
```rust
pub struct UploadDescriptor {
pub upload_id: String, // UUID v4仅用于随后提交 user_input
pub name: String,
pub media_type: String, // image | audio | video | file
pub mime_type: String,
pub size: u64,
pub expires_at: i64,
}
```
`upload_id` 不是持久化附件 ID。Gateway 重启、过期或被成功消费后均可失效。
### 3.2 消息附件描述
WebSocket 对客户端返回的描述不含路径:
```rust
pub struct MessageAttachment {
pub index: u32, // 在该消息 media_refs 中的位置
pub name: String, // 从当前 path basename 安全派生
pub media_type: String,
pub mime_type: String, // 按扩展名推断,仅用于展示/响应默认值
}
```
下载定位使用 `session_id + message_id + index`。不在描述中放服务端路径,也不使用可映射回路径的编码 ID。
是否增加 `available` 字段:首版不增加。历史查询不为最多 2000 条消息逐个执行文件 `stat`;客户端点击预览/下载时,由 HTTP 状态码反映文件是否仍存在。WebUI/TUI 收到 `404/410` 后把该附件标记为“文件已不存在”。
### 3.3 WebSocket 扩展
保持旧 JSON 可解析:
```rust
WsInbound::UserInput {
content: String,
upload_ids: Vec<String>, // serde(default),最多 8 个
client_message_id: Option<String>,
// 现有字段保留
}
HistoryMessage {
// 现有字段保留
attachments: Vec<MessageAttachment>, // serde(default)
}
WsOutbound::AssistantResponse {
// 现有字段保留
attachments: Vec<MessageAttachment>, // serde(default)
}
```
上传引用无效、跨 scope、重复/过期、文件数量或总量超限时Gateway 使用现有 `error` 帧返回明确原因。成功提交沿用普通消息的处理/响应语义,不新增附件专用确认帧。
`SessionEstablished` 增加默认空的 `capabilities`;支持本方案的 Gateway 返回 `file_transfer_v1`。新 TUI 连接旧 Gateway 时隐藏文件功能并提示升级,旧客户端忽略新增字段。
## 4. UploadRegistry
新增轻量 `UploadRegistry`,由 `GatewayState` 持有,只管理尚未提交到消息的上传:
```rust
struct PendingUpload {
upload_id: String,
owner_channel: String,
owner_chat_id: String,
path: PathBuf,
name: String,
media_type: String,
mime_type: String,
size: u64,
expires_at: Instant,
state: Pending | Consuming,
}
```
Registry 存于内存,不写 SQLite。默认文件目录建议为 `~/.picobot/media/cli_chat`,配置归入 `gateway.file_transfer`
```json
{
"enabled": true,
"upload_dir": "~/.picobot/media/cli_chat",
"max_file_bytes": 26214400,
"max_files_per_message": 8,
"max_message_bytes": 67108864,
"pending_ttl_seconds": 3600
}
```
文件布局不使用用户文件名定位:
```text
cli_chat/<client-scope-hash>/<uuid>.upload
cli_chat/.staging/<uuid>.part
```
原始文件名只保存在 registry 中;消息最终记录的路径可以带安全扩展名,但不能包含未经清理的目录组件。
状态规则:
1. 上传完成后为 `Pending`
2. `user_input` 批量校验所有 upload ID 后,从 registry 原子取出。
3. 成功发布到 inbound Bus 后不删除文件;后续消息持有路径引用。
4. 发布到 inbound Bus 失败时恢复 registry 记录,允许客户端重试。
5. 到期且仍在 registry 中的孤儿上传由 GC 删除。
6. Gateway 重启后 registry 丢失;残留文件由启动时按文件年龄限量清理。它们没有进入消息,因此可以删除。
一旦路径已经写入消息该文件不再由“pending upload GC”追踪。项目不承诺它的保存期限用户、工具、外部清理任务或后续保留策略都可以移动或删除它。
GC 任务必须由 `TaskSupervisor` 持有,观察取消,并限制每轮扫描数和总执行时间。
## 5. HTTP API
路由加入现有受保护 Router复用 Cookie/Bearer 鉴权。
### 5.1 上传
```http
POST /api/chat/{client_id}/uploads
Content-Type: multipart/form-data
file=<binary>
```
成功返回 `201 Created``UploadDescriptor`。上传 handler
1. 校验与 WebSocket 相同规则的 `client_id`
2. 流式读取 multipart不聚合整个文件
3. 边写临时文件边计算实际大小,超限立即停止并清理;
4. 清理文件名,限制 UTF-8 长度,去除路径、控制字符和保留名称;
5. MIME 以有限 magic-byte 检测为主、扩展名为辅,未知为 `application/octet-stream`
6. `sync_all` 后原子 rename再注册 upload ID注册失败删除文件
7. 每个身份/chat scope 使用 semaphore 限制并发上传。
主要错误为 `400` 非法请求、`401` 未认证、`413` 超限、`429` 并发或暂存量超限、`507` 空间不足。
### 5.2 下载与预览
```http
GET /api/chat/{client_id}/sessions/{session_id}/messages/{message_id}/attachments/{index}
GET /api/chat/{client_id}/sessions/{session_id}/messages/{message_id}/attachments/{index}?disposition=inline
```
handler 必须:
1. 验证设备身份和 `client_id`
2. 使用 Session/Storage API 验证该 session 属于 `cli_chat:{client_id}`
3. 查询指定 message并从其 `media_refs[index]` 取得内部路径;
4. 重新检查路径当前指向普通文件;不存在、已移动、是目录或不可读时返回 `404``410`
5. 流式读取当前文件内容,不在打开前把整个文件载入内存。
响应设置 `Content-Length`、推断的 `Content-Type`、安全编码的 `Content-Disposition``X-Content-Type-Options: nosniff``Cache-Control: private, no-store``inline` 仅允许白名单图片/音视频 MIME其余强制下载。
下载路由不能接受 path 查询参数。错误响应和日志都不能包含服务端路径。路径在消息存在不等于文件存在,这属于正常的可预期状态。
`client_id` 是非秘密 chat scope可以出现在路径Bearer token 仍只能进入 Authorization header不能放入 URL。WebUI `<img>` 使用同源 HttpOnly CookieTUI 使用 Bearer header。
## 6. 入站处理
`cli_chat` 收到带上传的 `user_input` 后:
1. 允许正文为空,但正文和 `upload_ids` 不能同时为空;
2. 校验数量、去重并批量验证所有上传属于 `client.chat_id`
3. 检查上传未过期、文件当前仍是普通文件且总大小未超限;
4. 原子取得 upload IDs转换为带本地路径的 `MediaItem`
5. 走现有 Bus → SessionManager → session worker
6. 成功发布到 inbound Bus 后消费 registry 记录;发布失败则恢复记录并返回现有 error 帧;
7. worker 按现有流程把路径写入 `messages.media_refs`
为此,`SessionManager::handle_message` 应返回结构化的“已入队/拒绝”不能再把队列满包装为普通命令输出。Gateway 主循环仍只等待快速入队,不等待模型。
Slash command 不接受上传。正文识别为 slash command 且带 `upload_ids` 时返回 `UPLOADS_NOT_ALLOWED_FOR_COMMAND`,避免文件被静默忽略。
## 7. 出站和历史
### 7.1 出站文件
Agent 继续使用 `send_message(files=...)`。文件路径由服务端内部产生,不需要复制到附件资产目录:
1. `SendMessageTool` 将路径转换为现有 `MediaItem`
2. 发送前检查每个路径当前是可读普通文件且未超过发送限制;
3. `OutboundMessenger` 按现有逻辑把路径写入 assistant 消息的 `media_refs`
4. `OutboundMessage.media` 传给 `CliChatChannel`
5. `cli_chat` 只返回文件名、类型和消息内 index不返回路径
6. 用户点击下载时再次读取当前路径,因此发送后路径变化或文件删除会导致下载失败。
多个文件建议在发送前全部校验;校验通过后仍可能发生 TOCTOU 删除,下载端必须把这种情况作为正常不可用处理。
即时 `AssistantResponse` 需要携带已持久化消息的真实 `message_id`,不能继续使用与历史无关的临时短 ID否则客户端无法构造安全下载地址。若当前投递链路暂时拿不到 message ID客户端收到响应后立即刷新历史以历史记录为附件展示权威来源。
### 7.2 历史读取
`HistoryMessage.attachments` 从已保存的 `media_refs` 映射而来。映射只做字符串解析和 basename/MIME 推断,不访问文件系统,因此:
- 历史查询性能不依赖附件文件数量和存储速度;
- 文件缺失不会导致历史帧失败;
- 展示附件不代表下载一定成功;
- 旧 `media_refs` 数据无需 schema 迁移即可生效。
客户端在下载返回不可用后只更新本地 UI 状态,不修改历史消息。
### 7.3 Agent 上下文中的路径
构造发给模型的用户消息时,无论附件类型是否被模型原生支持,都先增加一个结构化附件清单。清单包含安全文件名、媒体类型和 Gateway 内部路径,例如:
```text
[附件清单path 是 Gateway 内部存储路径,可供文件工具读取]
[
{
"name": "report.pdf",
"media_type": "file",
"path": "/gateway/media/report.pdf"
}
]
```
随后再追加图片等原生多模态 content block。这样支持视觉输入的模型既能看到图片内容也知道其文件路径普通文件同样可以由 LLM 使用 `file_read`、Bash 等工具读取。附件路径只进入服务端到 Provider 的模型上下文,不进入 WebSocket/HTTP 客户端响应。
历史路径已经失效时,清单仍反映消息所记录的原路径;工具读取失败应作为普通、可解释的“文件已移动或删除”结果返回,不能导致 Agent loop panic。
## 8. WebUI 交互
Composer 增加回形针按钮、隐藏多选 file input并支持拖放和剪贴板图片。待发送区域展示文件名、大小、进度、失败、重试和移除。
- 选中文件后立即通过 HTTP 上传,并发不超过 3
- 全部上传成功后才可发送;文本为空但有上传时允许发送;
- 上传或消息提交返回 error 时给出明确提示,过期项要求重新上传;
- 切换 session 时按 session 保存内存草稿,刷新页面后的孤儿上传由 TTL 清理;
- 历史消息在正文下显示附件卡;图片可尝试 inline 预览;
- 下载/预览为 404/410 时显示“原文件已移动或删除”;
- 不把文件字节、服务端路径或 upload ID 长期写入 localStorage
- object URL 在移除或销毁时 revoke。
Markdown 继续走现有 sanitizer附件使用结构化字段渲染文件名不能拼接成 HTML。
## 9. TUI 交互
TUI 保存 Gateway HTTP base URL、Bearer token 和 `client_id`
- `Ctrl+F` 打开“添加附件”路径输入框;相对路径按 TUI 启动目录解释;
- TUI 读取本机文件并通过 HTTP 上传,不能把客户端路径直接传给 Gateway
- Composer 上方显示待发送文件和上传状态,支持移除;
- 消息显示 `[附件 1] report.pdf`;按 `F2` 将历史中最近的附件下载到当前目录;
- 下载先写目标目录临时文件,再原子 rename默认不覆盖已有文件
- 服务端返回不可用时显示“原文件已移动或删除”;
- HTTP 上传/下载通过受管理任务和内部 channel 回报进度,不阻塞键盘或 WebSocket退出时取消并有界 join。
不新增伪服务端 `/attach``/download` 命令,避免与 `get_slash_commands` 的权威列表冲突。
## 10. 安全与资源边界
- **路径穿越**:上传目标路径完全由 Gateway 生成;清理后的名字只用于展示。
- **任意文件读取**:客户端只提交 upload ID下载只通过已归属该 session/message 的 `media_refs` 定位,不接受路径参数。
- **跨 scope**上传消费、session 查询和下载都验证 `cli_chat:{client_id}`
- **符号链接与特殊文件**上传落盘为普通文件出站和下载每次打开前拒绝目录、设备、FIFO 等非普通文件。对于工具路径是否跟随 symlink应保留现有工具权限语义并在打开后检查 metadata。
- **资源耗尽**限制单文件、单消息总量、文件数、并发上传、multipart body、pending TTL 和每轮 GC 数量。
- **内容伪装**:不信任客户端 MIMEinline 采用白名单并设置 `nosniff`
- **恶意文件**Gateway 不自动解压或解析任意文档;图片进入模型前另设格式、像素和编码后大小限制。
- **提示注入**:附件是不可信用户内容;普通文件只提供路径提示,由 Agent 显式调用工具读取。
- **路径泄漏**协议、HTTP 错误和常规日志不得包含完整服务端路径。
- **传输安全**:非回环部署仍要求外层 HTTPS/WSS设备配对本身不提供机密性。
## 11. 生命周期与并发
UploadRegistry 的 pending GC 和启动残留清理由 `TaskSupervisor` 持有,观察 cancellation 并有硬超时。已经进入消息的路径不属于该 GC。
HTTP 连接断开或 Gateway shutdown 时停止读写并删除未提交 `.part` 文件。任何 session mutex 都不得跨上传、下载、文件 `stat/open` 或数据库 I/O 持有。慢文件 I/O 完成后提交会话结果时继续验证 `worker_generation`/`state_version`
## 12. 分阶段实施
### 阶段 A协议与服务端
1. 新增 file-transfer 配置、UploadRegistry、HTTP 上传/下载路由和 pending GC。
2. 扩展 WebSocket capability、upload IDs 和附件描述,并复用现有 error 帧。
3. `cli_chat` 入站恢复 `media`,出站和历史不再丢弃 `media_refs`
4. 给 Storage/SessionManager 增加按 scope 查询单条消息的安全 API。
本阶段不修改数据库 schema。
### 阶段 BWebUI
实现选择/拖放/粘贴、上传进度、附件卡、预览和不可用提示;运行 `npm run check``npm run build``cargo build`
### 阶段 CTUI
实现路径 modal、HTTP 传输任务、状态渲染和安全下载;补齐断线、退出取消和旧 Gateway capability 降级。
### 阶段 D收口
完善 `send_message(files=...)` 校验、可观测性和保留策略说明。实现后同步更新 `README.md``docs/ARCHITECTURE.md``AGENTS.md` 和配置示例。
## 13. 测试与验收
重点测试:
- 上传分片计数、超限中止、文件名清理、MIME 映射和 orphan 清理;
- upload ID 的 scope、过期、重复消费、批量原子取得与入队失败恢复
- 文本+附件、纯附件、slash command 带附件拒绝和 session 队列满;
- 历史 `media_refs``MessageAttachment` 映射绝不泄露路径;
- 文件存在时可下载,路径删除/移动/变成目录时返回不可用且历史仍正常;
- 跨 client/session/message/index 访问被拒绝且不泄露文件是否存在;
- Cookie 与 Bearer 认证、UTF-8 文件名、inline 白名单和下载中断;
- Agent 图片输入和 `send_message(files)` 的在线/历史展示;
- Gateway shutdown 时传输与 GC 有界退出,无裸后台任务;
- 旧文本客户端继续工作,新 TUI 对旧 Gateway 安全降级。
验收标准:
1. WebUI/TUI 可发送限制内的本地文件,不使用 WebSocket base64。
2. 纯附件消息可靠入队,失败有明确、可重试反馈。
3. 在线响应和历史均能显示附件引用;文件仍存在时可下载。
4. 文件路径变化或删除后允许下载失败,但历史消息、文本和其他附件不受影响。
5. 客户端看不到服务端路径,也不能利用接口读取未归属当前 chat scope 的路径。
6. 同一 session 顺序、跨 session 并发和现有锁/取消不变量保持不变。
Rust 实现完成后运行目标测试、`cargo test --lib`、离线协议/调度集成测试、Clippy warnings denied 和 `cargo build`WebUI 额外执行独立 check/build。

View File

@ -49,7 +49,15 @@
"gateway": { "gateway": {
"host": "127.0.0.1", "host": "127.0.0.1",
"port": 19876, "port": 19876,
"require_pairing": true "require_pairing": true,
"file_transfer": {
"enabled": true,
"upload_dir": "~/.picobot/media/cli_chat",
"max_file_bytes": 26214400,
"max_files_per_message": 8,
"max_message_bytes": 67108864,
"pending_ttl_seconds": 3600
}
}, },
"client": { "client": {
"gateway_url": "ws://127.0.0.1:19876/ws" "gateway_url": "ws://127.0.0.1:19876/ws"

View File

@ -28,13 +28,53 @@ fn build_content_blocks(
) -> Vec<ContentBlock> { ) -> Vec<ContentBlock> {
let mut blocks = Vec::new(); let mut blocks = Vec::new();
if !text.is_empty() {
blocks.push(ContentBlock::text(text));
}
if !media_refs.is_empty() { if !media_refs.is_empty() {
let attachments = media_refs
.iter()
.map(|media_ref| {
let path = std::path::Path::new(&media_ref.path);
let name = path
.file_name()
.map(|name| name.to_string_lossy().into_owned())
.unwrap_or_else(|| media_ref.path.clone());
let extension = path
.extension()
.map(|extension| extension.to_string_lossy().into_owned());
let mime_type = mime_guess::from_path(path)
.first_or_octet_stream()
.to_string();
let size_bytes = std::fs::metadata(path).ok().map(|metadata| metadata.len());
let native_input = input_types.contains(&media_ref.media_type)
&& registry.supports(&media_ref.media_type);
serde_json::json!({
"name": name,
"extension": extension,
"media_type": media_ref.media_type,
"mime_type": mime_type,
"size_bytes": size_bytes,
"path": media_ref.path,
"content_delivery": if native_input {
"also included as a model-native content block"
} else {
"content is not embedded; use a file tool with path to inspect it"
},
})
})
.collect::<Vec<_>>();
let manifest = serde_json::Value::Array(attachments).to_string();
let message_text = if text.is_empty() {
format!(
"[用户发送了以下附件。path 是 Gateway 内部存储路径,可供文件工具读取;附件内容未必已嵌入模型输入。]\n{manifest}"
)
} else {
format!(
"{text}\n\n[随本条用户消息同时提交的附件。path 是 Gateway 内部存储路径可供文件工具读取content_delivery 说明附件内容是否另以模型原生内容块提供。]\n{manifest}"
)
};
blocks.push(ContentBlock::text(message_text));
for mr in media_refs { for mr in media_refs {
if input_types.contains(&mr.media_type) { if input_types.contains(&mr.media_type) && registry.supports(&mr.media_type) {
match registry.handle(&mr.media_type, &mr.path) { match registry.handle(&mr.media_type, &mr.path) {
Ok(content_blocks) => blocks.extend(content_blocks), Ok(content_blocks) => blocks.extend(content_blocks),
Err(e) => { Err(e) => {
@ -55,14 +95,12 @@ fn build_content_blocks(
path = %mr.path, path = %mr.path,
media_type = %mr.media_type, media_type = %mr.media_type,
model_input_types = ?input_types, model_input_types = ?input_types,
"Media type not supported by model, using text placeholder" "Media type not supported by model; attachment manifest remains available"
); );
blocks.push(ContentBlock::text(format!(
"[用户发来了一个文件: {}]",
mr.path
)));
} }
} }
} else if !text.is_empty() {
blocks.push(ContentBlock::text(text));
} }
if blocks.is_empty() { if blocks.is_empty() {
@ -868,12 +906,63 @@ mod tests {
&registry, &registry,
); );
assert!( assert_eq!(blocks.len(), 1);
matches!(blocks.first(), Some(ContentBlock::Text { text }) if text == "先看这段文字") assert!(matches!(blocks.first(), Some(ContentBlock::Text { text })
if text.starts_with("先看这段文字\n\n")
&& text.contains("随本条用户消息同时提交的附件")
&& text.contains("missing.png")
&& text.contains("\"media_type\":\"image\"")
&& text.contains("content is not embedded")));
}
#[test]
fn test_build_content_blocks_describes_attachment_only_message() {
let registry = MediaHandlerRegistry::new();
let blocks = build_content_blocks(
"",
&[MediaRef {
path: "/tmp/report.docx".to_string(),
media_type: "file".to_string(),
}],
&[],
&registry,
); );
assert!(
matches!(blocks.get(1), Some(ContentBlock::Text { text }) if text.contains("用户发来了一个文件")) assert_eq!(blocks.len(), 1);
assert!(matches!(blocks.first(), Some(ContentBlock::Text { text })
if text.starts_with("[用户发送了以下附件")
&& text.contains("report.docx")
&& text.contains("application/vnd.openxmlformats-officedocument.wordprocessingml.document")
&& text.contains("\"extension\":\"docx\"")
&& text.contains("\"size_bytes\":null")
&& text.contains("content is not embedded")));
}
#[test]
fn test_build_content_blocks_includes_path_for_supported_images() {
use std::io::Write;
let mut image = tempfile::Builder::new().suffix(".png").tempfile().unwrap();
image.write_all(b"image bytes").unwrap();
let path = image.path().to_string_lossy().into_owned();
let registry = MediaHandlerRegistry::with_defaults();
let blocks = build_content_blocks(
"分析图片",
&[MediaRef {
path: path.clone(),
media_type: "image".to_string(),
}],
&["image".to_string()],
&registry,
); );
assert!(matches!(blocks.first(), Some(ContentBlock::Text { text })
if text.starts_with("分析图片\n\n")
&& text.contains("随本条用户消息同时提交的附件")
&& text.contains(&path)
&& text.contains("model-native content block")));
assert!(matches!(blocks.get(1), Some(ContentBlock::ImageUrl { .. })));
} }
} }

View File

@ -99,6 +99,10 @@ impl MediaHandlerRegistry {
} }
} }
pub fn supports(&self, media_type: &str) -> bool {
self.handlers.contains_key(media_type)
}
pub fn with_defaults() -> Self { pub fn with_defaults() -> Self {
let mut reg = Self::new(); let mut reg = Self::new();
reg.register(Box::new(ImageHandler)); reg.register(Box::new(ImageHandler));

View File

@ -4,7 +4,10 @@ use std::sync::Arc;
use tokio::sync::{Mutex, mpsc}; use tokio::sync::{Mutex, mpsc};
use crate::bus::{ControlMessage, InboundMessage, MessageBus, OutboundMessage}; use crate::bus::{ControlMessage, InboundMessage, MessageBus, OutboundMessage};
use crate::protocol::{HistoryMessage, SlashCommandInfo, WsInbound, WsOutbound, parse_inbound}; use crate::gateway::uploads::UploadRegistry;
use crate::protocol::{
HistoryMessage, MessageAttachment, SlashCommandInfo, WsInbound, WsOutbound, parse_inbound,
};
use crate::session::{SessionCommand, SessionEvent, UnifiedSessionId}; use crate::session::{SessionCommand, SessionEvent, UnifiedSessionId};
use super::base::{Channel, ChannelError}; use super::base::{Channel, ChannelError};
@ -32,6 +35,7 @@ impl Client {
pub struct CliChatChannel { pub struct CliChatChannel {
bus: std::sync::Mutex<Option<Arc<MessageBus>>>, bus: std::sync::Mutex<Option<Arc<MessageBus>>>,
clients: Mutex<HashMap<String, Arc<Client>>>, clients: Mutex<HashMap<String, Arc<Client>>>,
uploads: UploadRegistry,
} }
impl Default for CliChatChannel { impl Default for CliChatChannel {
@ -42,9 +46,14 @@ impl Default for CliChatChannel {
impl CliChatChannel { impl CliChatChannel {
pub fn new() -> Self { pub fn new() -> Self {
Self::with_upload_registry(UploadRegistry::disabled())
}
pub fn with_upload_registry(uploads: UploadRegistry) -> Self {
Self { Self {
bus: std::sync::Mutex::new(None), bus: std::sync::Mutex::new(None),
clients: Mutex::new(HashMap::new()), clients: Mutex::new(HashMap::new()),
uploads,
} }
} }
@ -166,21 +175,49 @@ impl CliChatChannel {
match inbound { match inbound {
WsInbound::UserInput { WsInbound::UserInput {
content, chat_id, .. content,
upload_ids,
chat_id,
..
} => { } => {
// All messages (including slash commands) go through the normal inbound flow // All messages (including slash commands) go through the normal inbound flow
// SessionManager handles session creation/reuse internally // SessionManager handles session creation/reuse internally
if content.trim().is_empty() && upload_ids.is_empty() {
return Err(ChannelError::Other("Message is empty".to_string()));
}
if !upload_ids.is_empty()
&& crate::channels::parse_slash_command(&content).is_some()
{
return Err(ChannelError::Other(
"Attachments cannot be sent with slash commands".to_string(),
));
}
let target_chat_id = chat_id.unwrap_or_else(|| client.chat_id.clone());
if target_chat_id != client.chat_id {
return Err(ChannelError::Other(
"Chat does not belong to this client".to_string(),
));
}
let uploads = self
.uploads
.take_many(&client.chat_id, &upload_ids)
.await
.map_err(|error| ChannelError::Other(error.to_string()))?;
let media = uploads.iter().map(UploadRegistry::as_media).collect();
let msg = InboundMessage { let msg = InboundMessage {
channel: self.name().to_string(), channel: self.name().to_string(),
sender_id: "cli".to_string(), sender_id: "cli".to_string(),
chat_id: chat_id.unwrap_or_else(|| client.chat_id.clone()), chat_id: target_chat_id,
content, content,
timestamp: crate::bus::message::current_timestamp(), timestamp: crate::bus::message::current_timestamp(),
media: Vec::new(), media,
metadata: Default::default(), metadata: Default::default(),
forwarded_metadata: Default::default(), forwarded_metadata: Default::default(),
}; };
bus.publish_inbound(msg).await?; if let Err(error) = bus.publish_inbound(msg).await {
self.uploads.restore(uploads).await;
return Err(error.into());
}
} }
WsInbound::ClearHistory { WsInbound::ClearHistory {
chat_id, chat_id,
@ -391,10 +428,25 @@ impl CliChatChannel {
.into_iter() .into_iter()
.filter(|message| { .filter(|message| {
!message.content.is_empty() !message.content.is_empty()
|| message.media_refs.is_some()
|| message.tool_calls.is_some() || message.tool_calls.is_some()
|| message.role == "tool" || message.role == "tool"
}) })
.map(|message| HistoryMessage { .map(|message| {
let attachments = message
.media_refs
.as_deref()
.and_then(|refs| {
serde_json::from_str::<Vec<crate::bus::MediaRef>>(refs).ok()
})
.unwrap_or_default()
.iter()
.enumerate()
.map(|(index, media_ref)| {
MessageAttachment::from_media_ref(index, media_ref)
})
.collect();
HistoryMessage {
id: message.id, id: message.id,
seq: message.seq, seq: message.seq,
role: message.role, role: message.role,
@ -405,6 +457,8 @@ impl CliChatChannel {
tool_calls: message tool_calls: message
.tool_calls .tool_calls
.and_then(|calls| serde_json::from_str(&calls).ok()), .and_then(|calls| serde_json::from_str(&calls).ok()),
attachments,
}
}) })
.collect(); .collect();
let _ = client let _ = client
@ -778,6 +832,17 @@ impl Channel for CliChatChannel {
}; };
let message_type = msg.metadata.get("_type").map(String::as_str); let message_type = msg.metadata.get("_type").map(String::as_str);
let session_id = msg.metadata.get("_session_id").cloned(); let session_id = msg.metadata.get("_session_id").cloned();
let message_id = msg
.metadata
.get("_message_id")
.cloned()
.unwrap_or_else(crate::util::short_id);
let attachments = msg
.media
.iter()
.enumerate()
.map(|(index, media)| MessageAttachment::from_media_ref(index, &media.to_media_ref()))
.collect();
let outbound = if message_type == Some("notification") { let outbound = if message_type == Some("notification") {
WsOutbound::SystemNotification { WsOutbound::SystemNotification {
content: msg.content, content: msg.content,
@ -789,9 +854,10 @@ impl Channel for CliChatChannel {
} }
} else { } else {
WsOutbound::AssistantResponse { WsOutbound::AssistantResponse {
id: crate::util::short_id(), id: message_id,
content: msg.content, content: msg.content,
role: "assistant".to_string(), role: "assistant".to_string(),
attachments,
session_id, session_id,
} }
}; };
@ -874,4 +940,90 @@ mod tests {
.is_some_and(|client| Arc::ptr_eq(client, &replacement)) .is_some_and(|client| Arc::ptr_eq(client, &replacement))
); );
} }
#[tokio::test]
async fn upload_ids_become_inbound_media_paths() {
let uploads = UploadRegistry::new(crate::config::FileTransferConfig::default());
uploads
.register(
"upload-1".into(),
"client".into(),
std::path::PathBuf::from("/tmp/report.pdf"),
crate::protocol::UploadDescriptor {
upload_id: String::new(),
name: "report.pdf".into(),
media_type: "file".into(),
mime_type: "application/pdf".into(),
size: 42,
expires_at: 0,
},
)
.await
.unwrap();
let channel = CliChatChannel::with_upload_registry(uploads);
let bus = MessageBus::new(4);
channel.start(bus.clone()).await.unwrap();
let (sender, _receiver) = mpsc::channel(1);
let client = Arc::new(Client {
sender,
chat_id: "client".into(),
current_session_id: Mutex::new(None),
});
channel
.handle_ws_inbound(
client,
WsInbound::UserInput {
content: "处理附件".into(),
upload_ids: vec!["upload-1".into()],
channel: None,
chat_id: None,
sender_id: None,
},
)
.await
.unwrap();
let inbound = bus.consume_inbound().await.unwrap();
assert_eq!(inbound.media.len(), 1);
assert_eq!(inbound.media[0].path, "/tmp/report.pdf");
assert_eq!(inbound.media[0].media_type, "file");
}
#[tokio::test]
async fn outbound_media_is_exposed_as_safe_attachment_metadata() {
let channel = CliChatChannel::new();
let (sender, mut receiver) = mpsc::channel(1);
let client = Arc::new(Client {
sender,
chat_id: "client".into(),
current_session_id: Mutex::new(None),
});
channel.clients.lock().await.insert("client".into(), client);
channel
.send(OutboundMessage {
channel: "cli_chat".into(),
chat_id: "client".into(),
content: "报告".into(),
reply_to: None,
media: vec![crate::bus::MediaItem::new("/secret/report.pdf", "file")],
metadata: HashMap::from([("_message_id".into(), "message-1".into())]),
delivery: None,
})
.await
.unwrap();
match receiver.recv().await.unwrap() {
WsOutbound::AssistantResponse {
id, attachments, ..
} => {
assert_eq!(id, "message-1");
assert_eq!(attachments[0].name, "report.pdf");
let json = serde_json::to_string(&attachments).unwrap();
assert!(!json.contains("/secret"));
}
other => panic!("unexpected outbound: {other:?}"),
}
}
} }

View File

@ -36,7 +36,7 @@ pub async fn run(
load_auth_token() load_auth_token()
}; };
let mut request = connect_url.into_client_request()?; let mut request = connect_url.into_client_request()?;
if let Some(token) = token { if let Some(token) = &token {
request request
.headers_mut() .headers_mut()
.insert(header::AUTHORIZATION, format!("Bearer {token}").parse()?); .insert(header::AUTHORIZATION, format!("Bearer {token}").parse()?);
@ -51,6 +51,9 @@ pub async fn run(
let (ws_sender, ws_receiver) = ws_stream.split(); let (ws_sender, ws_receiver) = ws_stream.split();
let mut app = App::new(); let mut app = App::new();
app.http_base_url = gateway_http_base_url(gateway_url)?;
app.auth_token = token;
app.client_id = client_id;
app.ws_sender = Some(ws_sender); app.ws_sender = Some(ws_sender);
app.ws_receiver = Some(ws_receiver); app.ws_receiver = Some(ws_receiver);
@ -75,6 +78,23 @@ pub async fn run(
result result
} }
fn gateway_http_base_url(gateway_url: &str) -> Result<String, Box<dyn std::error::Error>> {
let mut url = reqwest::Url::parse(gateway_url)?;
let scheme = match url.scheme() {
"ws" => "http",
"wss" => "https",
"http" => "http",
"https" => "https",
other => return Err(format!("unsupported gateway URL scheme: {other}").into()),
};
url.set_scheme(scheme)
.map_err(|_| "failed to set gateway URL scheme")?;
url.set_path("");
url.set_query(None);
url.set_fragment(None);
Ok(url.to_string().trim_end_matches('/').to_string())
}
async fn exchange_pairing_code( async fn exchange_pairing_code(
gateway_url: &str, gateway_url: &str,
code: &str, code: &str,
@ -234,7 +254,9 @@ async fn run_app(
async fn handle_ws_message(app: &mut App, outbound: WsOutbound) { async fn handle_ws_message(app: &mut App, outbound: WsOutbound) {
match outbound { match outbound {
WsOutbound::AssistantResponse { WsOutbound::AssistantResponse {
id,
content, content,
attachments,
session_id, session_id,
.. ..
} => { } => {
@ -244,7 +266,10 @@ async fn handle_ws_message(app: &mut App, outbound: WsOutbound) {
.as_ref() .as_ref()
.is_none_or(|session_id| app.current_session_id.as_ref() == Some(session_id)) .is_none_or(|session_id| app.current_session_id.as_ref() == Some(session_id))
{ {
app.add_message(MessageRole::Assistant, content); app.add_message_with_attachments(id, MessageRole::Assistant, content, attachments);
if let Some(current) = app.current_session_id.clone() {
request_history(app, current).await;
}
} else { } else {
app.status_message = Some("另一个会话已完成响应".to_string()); app.status_message = Some("另一个会话已完成响应".to_string());
} }
@ -255,8 +280,13 @@ async fn handle_ws_message(app: &mut App, outbound: WsOutbound) {
app.status_message = Some(message.clone()); app.status_message = Some(message.clone());
app.add_message(MessageRole::System, format!("Error: {}", message)); app.add_message(MessageRole::System, format!("Error: {}", message));
} }
WsOutbound::SessionEstablished { session_id } => { WsOutbound::SessionEstablished {
session_id,
capabilities,
} => {
app.connected = true; app.connected = true;
app.file_transfer_supported =
capabilities.iter().any(|value| value == "file_transfer_v1");
app.set_current_session(Some(session_id.clone())); app.set_current_session(Some(session_id.clone()));
request_history(app, session_id).await; request_history(app, session_id).await;
request_session_list(app).await; request_session_list(app).await;

View File

@ -1,4 +1,6 @@
use crate::protocol::{HistoryMessage, SessionSummary, SlashCommandInfo}; use crate::protocol::{
HistoryMessage, MessageAttachment, SessionSummary, SlashCommandInfo, UploadDescriptor,
};
use std::collections::VecDeque; use std::collections::VecDeque;
use tokio_tungstenite::tungstenite::Message; use tokio_tungstenite::tungstenite::Message;
@ -14,8 +16,10 @@ pub enum MessageRole {
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct ChatMessage { pub struct ChatMessage {
pub id: String,
pub role: MessageRole, pub role: MessageRole,
pub content: String, pub content: String,
pub attachments: Vec<MessageAttachment>,
} }
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
@ -34,6 +38,7 @@ pub enum ConfirmAction {
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub enum Modal { pub enum Modal {
Rename { input: String, cursor: usize }, Rename { input: String, cursor: usize },
AttachPath { input: String, cursor: usize },
Confirm(ConfirmAction), Confirm(ConfirmAction),
} }
@ -73,6 +78,11 @@ pub struct App {
pub commands: Vec<SlashCommandInfo>, pub commands: Vec<SlashCommandInfo>,
pub show_command_menu: bool, pub show_command_menu: bool,
pub selected_command_idx: usize, pub selected_command_idx: usize,
pub http_base_url: String,
pub auth_token: Option<String>,
pub client_id: String,
pub file_transfer_supported: bool,
pub pending_uploads: Vec<UploadDescriptor>,
} }
impl App { impl App {
@ -99,11 +109,31 @@ impl App {
commands: Vec::new(), commands: Vec::new(),
show_command_menu: false, show_command_menu: false,
selected_command_idx: 0, selected_command_idx: 0,
http_base_url: String::new(),
auth_token: None,
client_id: String::new(),
file_transfer_supported: false,
pending_uploads: Vec::new(),
} }
} }
pub fn add_message(&mut self, role: MessageRole, content: String) { pub fn add_message(&mut self, role: MessageRole, content: String) {
self.messages.push_back(ChatMessage { role, content }); self.add_message_with_attachments(crate::util::short_id(), role, content, Vec::new());
}
pub fn add_message_with_attachments(
&mut self,
id: String,
role: MessageRole,
content: String,
attachments: Vec<MessageAttachment>,
) {
self.messages.push_back(ChatMessage {
id,
role,
content,
attachments,
});
while self.messages.len() > MAX_MESSAGES { while self.messages.len() > MAX_MESSAGES {
self.messages.pop_front(); self.messages.pop_front();
} }
@ -124,8 +154,10 @@ impl App {
_ => return None, _ => return None,
}; };
Some(ChatMessage { Some(ChatMessage {
id: message.id,
role, role,
content: message.content, content: message.content,
attachments: message.attachments,
}) })
}) })
.collect(); .collect();
@ -153,6 +185,7 @@ impl App {
if self.current_session_id != session_id { if self.current_session_id != session_id {
self.current_session_id = session_id; self.current_session_id = session_id;
self.messages.clear(); self.messages.clear();
self.pending_uploads.clear();
self.chat_scroll_from_bottom = 0; self.chat_scroll_from_bottom = 0;
} }
if let Some(current) = &self.current_session_id if let Some(current) = &self.current_session_id

View File

@ -35,6 +35,12 @@ pub fn render(frame: &mut Frame, area: Rect, app: &App) {
); );
} }
} }
for attachment in &message.attachments {
lines.push(Line::from(Span::styled(
format!(" [附件 {}] {}", attachment.index + 1, attachment.name),
Style::default().fg(Color::Cyan),
)));
}
lines.push(Line::from("")); lines.push(Line::from(""));
} }
if app.pending_responses > 0 { if app.pending_responses > 0 {

View File

@ -18,6 +18,8 @@ pub fn render(frame: &mut Frame, area: Rect) {
Line::from(" Ctrl+R 重命名 Ctrl+O 显示/隐藏归档"), Line::from(" Ctrl+R 重命名 Ctrl+O 显示/隐藏归档"),
Line::from(" Ctrl+A 归档 Ctrl+D 删除"), Line::from(" Ctrl+A 归档 Ctrl+D 删除"),
Line::from(" Ctrl+L 清空历史 Ctrl+C 两次 退出"), Line::from(" Ctrl+L 清空历史 Ctrl+C 两次 退出"),
Line::from(" Ctrl+F 添加附件 F2 下载最近附件"),
Line::from(" Ctrl+X 清除待发送附件"),
Line::from(""), Line::from(""),
Line::from(Span::styled( Line::from(Span::styled(
"输入", "输入",

View File

@ -15,7 +15,11 @@ pub fn render(frame: &mut Frame, area: Rect, app: &App) {
Style::default() Style::default()
}; };
let title = if app.connected { let title = if app.connected {
" 输入 · Enter 发送 / Shift+Enter 换行 " if app.pending_uploads.is_empty() {
" 输入 · Enter 发送 / Ctrl+F 附件 "
} else {
" 输入 · Enter 发送(已有附件)"
}
} else { } else {
" 输入 · Gateway 已断开 " " 输入 · Gateway 已断开 "
}; };

View File

@ -1,8 +1,10 @@
use crate::client::tui::app::{App, ConfirmAction, Focus, MessageRole, Modal}; use crate::client::tui::app::{App, ConfirmAction, Focus, MessageRole, Modal};
use crate::protocol::{WsInbound, serialize_inbound}; use crate::protocol::{MessageAttachment, UploadDescriptor, WsInbound, serialize_inbound};
use crossterm::event::{KeyCode, KeyEvent, KeyModifiers}; use crossterm::event::{KeyCode, KeyEvent, KeyModifiers};
use futures_util::SinkExt; use futures_util::SinkExt;
use tokio::io::AsyncWriteExt;
use tokio_tungstenite::tungstenite::Message; use tokio_tungstenite::tungstenite::Message;
use tokio_util::io::ReaderStream;
pub async fn handle_key_event(app: &mut App, key: KeyEvent) { pub async fn handle_key_event(app: &mut App, key: KeyEvent) {
if app.show_help { if app.show_help {
@ -64,10 +66,24 @@ pub async fn handle_key_event(app: &mut App, key: KeyEvent) {
app.show_archived = !app.show_archived; app.show_archived = !app.show_archived;
request_session_list(app).await; request_session_list(app).await;
} }
KeyCode::Char('f') => {
if app.file_transfer_supported {
app.modal = Some(Modal::AttachPath {
input: String::new(),
cursor: 0,
});
} else {
app.status_message = Some("当前 Gateway 不支持文件传输".to_string());
}
}
KeyCode::Char('u') if app.focus == Focus::Input => { KeyCode::Char('u') if app.focus == Focus::Input => {
app.input.clear(); app.input.clear();
app.input_cursor_pos = 0; app.input_cursor_pos = 0;
} }
KeyCode::Char('x') if app.focus == Focus::Input => {
app.pending_uploads.clear();
app.status_message = Some("已移除待发送附件".to_string());
}
KeyCode::Up => app.scroll_chat_up(3), KeyCode::Up => app.scroll_chat_up(3),
KeyCode::Down => app.scroll_chat_down(3), KeyCode::Down => app.scroll_chat_down(3),
_ => {} _ => {}
@ -85,6 +101,7 @@ pub async fn handle_key_event(app: &mut App, key: KeyEvent) {
KeyCode::Esc => app.focus = Focus::Input, KeyCode::Esc => app.focus = Focus::Input,
KeyCode::PageUp => app.scroll_chat_up(10), KeyCode::PageUp => app.scroll_chat_up(10),
KeyCode::PageDown => app.scroll_chat_down(10), KeyCode::PageDown => app.scroll_chat_down(10),
KeyCode::F(2) => download_latest_attachment(app).await,
KeyCode::Home if app.focus == Focus::Sessions => app.selected_session = 0, KeyCode::Home if app.focus == Focus::Sessions => app.selected_session = 0,
KeyCode::End if app.focus == Focus::Sessions => { KeyCode::End if app.focus == Focus::Sessions => {
app.selected_session = app.sessions.len().saturating_sub(1); app.selected_session = app.sessions.len().saturating_sub(1);
@ -95,7 +112,9 @@ pub async fn handle_key_event(app: &mut App, key: KeyEvent) {
} }
pub fn handle_paste(app: &mut App, text: &str) { pub fn handle_paste(app: &mut App, text: &str) {
if let Some(Modal::Rename { input, cursor }) = &mut app.modal { if let Some(Modal::Rename { input, cursor } | Modal::AttachPath { input, cursor }) =
&mut app.modal
{
let remaining = 256_usize.saturating_sub(input.len()); let remaining = 256_usize.saturating_sub(input.len());
let mut end = text.len().min(remaining); let mut end = text.len().min(remaining);
while !text.is_char_boundary(end) { while !text.is_char_boundary(end) {
@ -157,14 +176,36 @@ async fn handle_input_key(app: &mut App, key: KeyEvent) {
KeyCode::Enter => { KeyCode::Enter => {
let input = app.take_input(); let input = app.take_input();
close_command_menu(app); close_command_menu(app);
if !input.trim().is_empty() { if !input.trim().is_empty() || !app.pending_uploads.is_empty() {
app.add_message(MessageRole::User, input.clone()); let upload_ids = app
.pending_uploads
.iter()
.map(|upload| upload.upload_id.clone())
.collect::<Vec<_>>();
let attachments = app
.pending_uploads
.iter()
.enumerate()
.map(|(index, upload)| MessageAttachment {
index: u32::try_from(index).unwrap_or(u32::MAX),
name: upload.name.clone(),
media_type: upload.media_type.clone(),
mime_type: upload.mime_type.clone(),
})
.collect();
app.add_message_with_attachments(
crate::util::short_id(),
MessageRole::User,
input.clone(),
attachments,
);
app.pending_responses = app.pending_responses.saturating_add(1); app.pending_responses = app.pending_responses.saturating_add(1);
app.status_message = Some("PicoBot 正在处理…".to_string()); app.status_message = Some("PicoBot 正在处理…".to_string());
let sent = send( let sent = send(
app, app,
WsInbound::UserInput { WsInbound::UserInput {
content: input, content: input,
upload_ids,
channel: None, channel: None,
// Session routing is owned by the server. A full session // Session routing is owned by the server. A full session
// id is not a chat id and must never be sent here. // id is not a chat id and must never be sent here.
@ -175,6 +216,8 @@ async fn handle_input_key(app: &mut App, key: KeyEvent) {
.await; .await;
if !sent { if !sent {
app.pending_responses = app.pending_responses.saturating_sub(1); app.pending_responses = app.pending_responses.saturating_sub(1);
} else {
app.pending_uploads.clear();
} }
} }
} }
@ -259,10 +302,202 @@ async fn handle_modal_key(app: &mut App, key: KeyEvent) {
} }
_ => app.modal = Some(Modal::Rename { input, cursor }), _ => app.modal = Some(Modal::Rename { input, cursor }),
}, },
Some(Modal::AttachPath {
mut input,
mut cursor,
}) => match key.code {
KeyCode::Esc => {}
KeyCode::Enter => {
let path = input.trim().to_string();
if !path.is_empty() {
upload_tui_file(app, &path).await;
}
}
KeyCode::Char(character) => {
input.insert(cursor, character);
cursor += character.len_utf8();
app.modal = Some(Modal::AttachPath { input, cursor });
}
KeyCode::Backspace => {
if let Some((index, _)) = input[..cursor].char_indices().next_back() {
input.drain(index..cursor);
cursor = index;
}
app.modal = Some(Modal::AttachPath { input, cursor });
}
KeyCode::Delete => {
if let Some(character) = input[cursor..].chars().next() {
input.drain(cursor..cursor + character.len_utf8());
}
app.modal = Some(Modal::AttachPath { input, cursor });
}
KeyCode::Left => {
if let Some((index, _)) = input[..cursor].char_indices().next_back() {
cursor = index;
}
app.modal = Some(Modal::AttachPath { input, cursor });
}
KeyCode::Right => {
if let Some(character) = input[cursor..].chars().next() {
cursor += character.len_utf8();
}
app.modal = Some(Modal::AttachPath { input, cursor });
}
_ => app.modal = Some(Modal::AttachPath { input, cursor }),
},
None => {} None => {}
} }
} }
async fn upload_tui_file(app: &mut App, raw_path: &str) {
let path = std::path::PathBuf::from(raw_path);
let metadata = match tokio::fs::metadata(&path).await {
Ok(metadata) if metadata.is_file() => metadata,
_ => {
app.status_message = Some("附件路径不是可读普通文件".to_string());
return;
}
};
let file_name = path
.file_name()
.and_then(|value| value.to_str())
.unwrap_or("attachment")
.to_string();
let file = match tokio::fs::File::open(&path).await {
Ok(file) => file,
Err(error) => {
app.status_message = Some(format!("无法打开附件:{error}"));
return;
}
};
app.status_message = Some(format!("正在上传 {file_name}"));
let body = reqwest::Body::wrap_stream(ReaderStream::new(file));
let part = reqwest::multipart::Part::stream_with_length(body, metadata.len())
.file_name(file_name.clone());
let form = reqwest::multipart::Form::new().part("file", part);
let url = format!("{}/api/chat/{}/uploads", app.http_base_url, app.client_id);
let client = reqwest::Client::new();
let mut request = client.post(url).multipart(form);
if let Some(token) = &app.auth_token {
request = request.bearer_auth(token);
}
match request.send().await {
Ok(response) if response.status().is_success() => {
match response.json::<UploadDescriptor>().await {
Ok(upload) => {
app.status_message = Some(format!("已添加附件:{}", upload.name));
app.pending_uploads.push(upload);
}
Err(error) => app.status_message = Some(format!("上传响应无效:{error}")),
}
}
Ok(response) => {
let status = response.status();
let detail = response.text().await.unwrap_or_default();
app.status_message = Some(format!("上传失败 ({status}){detail}"));
}
Err(error) => app.status_message = Some(format!("上传失败:{error}")),
}
}
async fn download_latest_attachment(app: &mut App) {
let Some(session_id) = app.current_session_id.clone() else {
return;
};
let Some((message_id, attachment)) = app.messages.iter().rev().find_map(|message| {
message
.attachments
.first()
.map(|attachment| (message.id.clone(), attachment.clone()))
}) else {
app.status_message = Some("当前历史中没有附件".to_string());
return;
};
let mut url = match reqwest::Url::parse(&app.http_base_url) {
Ok(url) => url,
Err(error) => {
app.status_message = Some(format!("下载地址无效:{error}"));
return;
}
};
if let Ok(mut segments) = url.path_segments_mut() {
segments.extend([
"api",
"chat",
&app.client_id,
"sessions",
&session_id,
"messages",
&message_id,
"attachments",
&attachment.index.to_string(),
]);
}
let client = reqwest::Client::new();
let mut request = client.get(url);
if let Some(token) = &app.auth_token {
request = request.bearer_auth(token);
}
let response = match request.send().await {
Ok(response) if response.status().is_success() => response,
Ok(response) => {
app.status_message = Some(format!("附件不可用 ({})", response.status()));
return;
}
Err(error) => {
app.status_message = Some(format!("下载失败:{error}"));
return;
}
};
let target = unique_download_path(&attachment.name);
let temporary = target.with_extension("picobot.part");
let mut output = match tokio::fs::File::create(&temporary).await {
Ok(output) => output,
Err(error) => {
app.status_message = Some(format!("无法创建下载文件:{error}"));
return;
}
};
let mut stream = response.bytes_stream();
while let Some(chunk) = futures_util::StreamExt::next(&mut stream).await {
match chunk {
Ok(chunk) => {
if let Err(error) = output.write_all(&chunk).await {
let _ = tokio::fs::remove_file(&temporary).await;
app.status_message = Some(format!("写入下载文件失败:{error}"));
return;
}
}
Err(error) => {
let _ = tokio::fs::remove_file(&temporary).await;
app.status_message = Some(format!("下载中断:{error}"));
return;
}
}
}
drop(output);
if let Err(error) = tokio::fs::rename(&temporary, &target).await {
let _ = tokio::fs::remove_file(&temporary).await;
app.status_message = Some(format!("保存附件失败:{error}"));
return;
}
app.status_message = Some(format!("附件已保存到 {}", target.display()));
}
fn unique_download_path(name: &str) -> std::path::PathBuf {
let safe = std::path::Path::new(name)
.file_name()
.and_then(|value| value.to_str())
.unwrap_or("attachment");
let mut candidate = std::path::PathBuf::from(safe);
let mut counter = 1_u32;
while candidate.exists() {
candidate = std::path::PathBuf::from(format!("{safe}.{counter}"));
counter += 1;
}
candidate
}
fn open_rename(app: &mut App) { fn open_rename(app: &mut App) {
if target_session_id(app).is_some() { if target_session_id(app).is_some() {
let input = if app.focus == Focus::Sessions { let input = if app.focus == Focus::Sessions {

View File

@ -110,6 +110,33 @@ fn render_modal(frame: &mut Frame, area: Rect, modal: &Modal) {
frame.set_cursor_position((cursor_x, cursor_y)); frame.set_cursor_position((cursor_x, cursor_y));
} }
} }
Modal::AttachPath { input, cursor } => {
frame.render_widget(
Paragraph::new(vec![
Line::from(Span::styled(
"添加附件路径",
Style::default().add_modifier(Modifier::BOLD),
)),
Line::from(""),
Line::from(input.as_str()),
Line::from(Span::styled(
"Enter 上传 · Esc 取消",
Style::default().fg(Color::DarkGray),
)),
])
.block(block)
.wrap(Wrap { trim: false }),
area,
);
let cursor_x = area.x
+ 1
+ UnicodeWidthStr::width(&input[..*cursor])
.min(area.width.saturating_sub(3) as usize) as u16;
let cursor_y = area.y.saturating_add(3);
if cursor_x < area.right() && cursor_y < area.bottom() {
frame.set_cursor_position((cursor_x, cursor_y));
}
}
Modal::Confirm(action) => { Modal::Confirm(action) => {
let prompt = match action { let prompt = match action {
ConfirmAction::Archive => "归档所选会话?", ConfirmAction::Archive => "归档所选会话?",

View File

@ -158,6 +158,8 @@ pub struct GatewayConfig {
pub max_concurrent_background_tasks: usize, pub max_concurrent_background_tasks: usize,
#[serde(default)] #[serde(default)]
pub scheduler: Option<SchedulerConfig>, pub scheduler: Option<SchedulerConfig>,
#[serde(default)]
pub file_transfer: FileTransferConfig,
} }
impl Default for GatewayConfig { impl Default for GatewayConfig {
@ -171,10 +173,67 @@ impl Default for GatewayConfig {
session_db_path: None, session_db_path: None,
max_concurrent_background_tasks: 10, max_concurrent_background_tasks: 10,
scheduler: None, scheduler: None,
file_transfer: FileTransferConfig::default(),
} }
} }
} }
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FileTransferConfig {
#[serde(default = "default_file_transfer_enabled")]
pub enabled: bool,
#[serde(default = "default_upload_dir")]
pub upload_dir: String,
#[serde(default = "default_max_file_bytes")]
pub max_file_bytes: u64,
#[serde(default = "default_max_files_per_message")]
pub max_files_per_message: usize,
#[serde(default = "default_max_message_bytes")]
pub max_message_bytes: u64,
#[serde(default = "default_pending_ttl_seconds")]
pub pending_ttl_seconds: u64,
}
impl Default for FileTransferConfig {
fn default() -> Self {
Self {
enabled: default_file_transfer_enabled(),
upload_dir: default_upload_dir(),
max_file_bytes: default_max_file_bytes(),
max_files_per_message: default_max_files_per_message(),
max_message_bytes: default_max_message_bytes(),
pending_ttl_seconds: default_pending_ttl_seconds(),
}
}
}
fn default_file_transfer_enabled() -> bool {
true
}
fn default_upload_dir() -> String {
get_user_config_dir()
.join("media/cli_chat")
.to_string_lossy()
.to_string()
}
fn default_max_file_bytes() -> u64 {
25 * 1024 * 1024
}
fn default_max_files_per_message() -> usize {
8
}
fn default_max_message_bytes() -> u64 {
64 * 1024 * 1024
}
fn default_pending_ttl_seconds() -> u64 {
60 * 60
}
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SchedulerConfig { pub struct SchedulerConfig {
/// Whether the scheduler is enabled /// Whether the scheduler is enabled
@ -639,6 +698,11 @@ mod tests {
assert_eq!(config.gateway.host, "0.0.0.0"); assert_eq!(config.gateway.host, "0.0.0.0");
assert_eq!(config.gateway.port, 19876); assert_eq!(config.gateway.port, 19876);
assert!(config.gateway.require_pairing); assert!(config.gateway.require_pairing);
assert!(config.gateway.file_transfer.enabled);
assert_eq!(
config.gateway.file_transfer.max_file_bytes,
25 * 1024 * 1024
);
} }
#[test] #[test]

View File

@ -3,7 +3,7 @@ use crate::config::Config;
use crate::memory::MemoryCategory; use crate::memory::MemoryCategory;
use axum::Json; use axum::Json;
use axum::body::Body; use axum::body::Body;
use axum::extract::{Path, Query, State}; use axum::extract::{Multipart, Path, Query, State};
use axum::http::{StatusCode, header}; use axum::http::{StatusCode, header};
use axum::response::{IntoResponse, Response}; use axum::response::{IntoResponse, Response};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@ -11,12 +11,26 @@ use serde_json::{Value, json};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::path::{Path as FsPath, PathBuf}; use std::path::{Path as FsPath, PathBuf};
use std::sync::Arc; use std::sync::Arc;
use tokio::io::{AsyncReadExt, AsyncSeekExt}; use tokio::io::{AsyncReadExt, AsyncSeekExt, AsyncWriteExt};
use tokio_util::io::ReaderStream;
const REDACTED: &str = "********"; const REDACTED: &str = "********";
const MAX_CONFIG_BYTES: usize = 1024 * 1024; const MAX_CONFIG_BYTES: usize = 1024 * 1024;
const MAX_PROFILE_BYTES: usize = 256 * 1024; const MAX_PROFILE_BYTES: usize = 256 * 1024;
struct TemporaryUpload {
path: PathBuf,
committed: bool,
}
impl Drop for TemporaryUpload {
fn drop(&mut self) {
if !self.committed {
let _ = std::fs::remove_file(&self.path);
}
}
}
#[derive(Serialize)] #[derive(Serialize)]
pub struct HealthResponse { pub struct HealthResponse {
status: String, status: String,
@ -85,6 +99,13 @@ impl ApiError {
} }
} }
fn payload_too_large(message: impl Into<String>) -> Self {
Self {
status: StatusCode::PAYLOAD_TOO_LARGE,
message: message.into(),
}
}
fn internal(error: impl std::fmt::Display) -> Self { fn internal(error: impl std::fmt::Display) -> Self {
tracing::error!(error = %error, "WebUI API request failed"); tracing::error!(error = %error, "WebUI API request failed");
Self { Self {
@ -94,6 +115,219 @@ impl ApiError {
} }
} }
pub async fn upload_file(
State(state): State<Arc<GatewayState>>,
Path(client_id): Path<String>,
mut multipart: Multipart,
) -> Result<(StatusCode, Json<crate::protocol::UploadDescriptor>), ApiError> {
if !valid_client_id(&client_id) {
return Err(ApiError::bad_request("invalid client id"));
}
if !state.uploads.enabled() {
return Err(ApiError::bad_request("file transfer is disabled"));
}
let mut field = multipart
.next_field()
.await
.map_err(|error| ApiError::bad_request(format!("invalid multipart body: {error}")))?
.ok_or_else(|| ApiError::bad_request("file field is required"))?;
if field.name() != Some("file") {
return Err(ApiError::bad_request("expected multipart field named file"));
}
let original_name = field.file_name().unwrap_or("attachment").to_string();
let safe_name = crate::gateway::uploads::UploadRegistry::safe_file_name(&original_name);
let (temporary_path, final_path, upload_id) = state
.uploads
.allocate_path(&client_id, &safe_name)
.await
.map_err(ApiError::internal)?;
let mut temporary = TemporaryUpload {
path: temporary_path.clone(),
committed: false,
};
let mut output = tokio::fs::File::create(&temporary_path)
.await
.map_err(ApiError::internal)?;
let mut size = 0_u64;
while let Some(chunk) = field
.chunk()
.await
.map_err(|error| ApiError::bad_request(format!("failed to read upload: {error}")))?
{
size = size.saturating_add(chunk.len() as u64);
if size > state.uploads.max_file_bytes() {
drop(output);
let _ = tokio::fs::remove_file(&temporary_path).await;
return Err(ApiError::payload_too_large(format!(
"file exceeds {} bytes",
state.uploads.max_file_bytes()
)));
}
if let Err(error) = output.write_all(&chunk).await {
drop(output);
let _ = tokio::fs::remove_file(&temporary_path).await;
return Err(ApiError::internal(error));
}
}
if size == 0 {
drop(output);
let _ = tokio::fs::remove_file(&temporary_path).await;
return Err(ApiError::bad_request("empty files are not supported"));
}
output.sync_all().await.map_err(ApiError::internal)?;
drop(output);
tokio::fs::rename(&temporary_path, &final_path)
.await
.map_err(ApiError::internal)?;
temporary.committed = true;
let mime_type = mime_guess::from_path(&safe_name)
.first_or_octet_stream()
.to_string();
let media_type = media_type_for_mime(&mime_type).to_string();
let descriptor = match state
.uploads
.register(
upload_id,
client_id,
final_path.clone(),
crate::protocol::UploadDescriptor {
upload_id: String::new(),
name: safe_name,
media_type,
mime_type,
size,
expires_at: 0,
},
)
.await
{
Ok(descriptor) => descriptor,
Err(error) => {
let _ = tokio::fs::remove_file(final_path).await;
return Err(ApiError::bad_request(error.to_string()));
}
};
Ok((StatusCode::CREATED, Json(descriptor)))
}
#[derive(Debug, Default, Deserialize)]
pub struct AttachmentQuery {
disposition: Option<String>,
}
pub async fn download_attachment(
State(state): State<Arc<GatewayState>>,
Path((client_id, session_id, message_id, index)): Path<(String, String, String, usize)>,
Query(query): Query<AttachmentQuery>,
) -> Result<Response, ApiError> {
if !valid_client_id(&client_id) {
return Err(ApiError::not_found("attachment not found"));
}
let unified = crate::session::UnifiedSessionId::parse(&session_id)
.filter(|id| id.channel == "cli_chat" && id.chat_id == client_id)
.ok_or_else(|| ApiError::not_found("attachment not found"))?;
let message = state
.storage
.get_message(&unified.to_string(), &message_id)
.await
.map_err(ApiError::internal)?
.ok_or_else(|| ApiError::not_found("attachment not found"))?;
let media_refs = message
.media_refs
.as_deref()
.and_then(|value| serde_json::from_str::<Vec<crate::bus::MediaRef>>(value).ok())
.unwrap_or_default();
let media_ref = media_refs
.get(index)
.ok_or_else(|| ApiError::not_found("attachment not found"))?;
let path = FsPath::new(&media_ref.path);
let metadata = tokio::fs::metadata(path)
.await
.map_err(|_| ApiError::not_found("the original file was moved or deleted"))?;
if !metadata.is_file() {
return Err(ApiError::not_found(
"the original file was moved or deleted",
));
}
let file = tokio::fs::File::open(path)
.await
.map_err(|_| ApiError::not_found("the original file was moved or deleted"))?;
let name = path
.file_name()
.and_then(|value| value.to_str())
.map(crate::gateway::uploads::UploadRegistry::safe_file_name)
.unwrap_or_else(|| "attachment".to_string());
let mime_type = mime_guess::from_path(&name)
.first_or_octet_stream()
.to_string();
let inline = query.disposition.as_deref() == Some("inline") && inline_mime_allowed(&mime_type);
let disposition = if inline { "inline" } else { "attachment" };
let encoded_name = encode_header_filename(&name);
Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, mime_type)
.header(header::CONTENT_LENGTH, metadata.len())
.header(
header::CONTENT_DISPOSITION,
format!("{disposition}; filename*=UTF-8''{encoded_name}"),
)
.header("X-Content-Type-Options", "nosniff")
.header(header::CACHE_CONTROL, "private, no-store")
.body(Body::from_stream(ReaderStream::new(file)))
.map_err(ApiError::internal)
}
fn valid_client_id(value: &str) -> bool {
!value.is_empty()
&& value.len() <= 64
&& value
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'-' || byte == b'_')
}
fn media_type_for_mime(mime: &str) -> &'static str {
if mime.starts_with("image/") {
"image"
} else if mime.starts_with("audio/") {
"audio"
} else if mime.starts_with("video/") {
"video"
} else {
"file"
}
}
fn inline_mime_allowed(mime: &str) -> bool {
matches!(
mime,
"image/png"
| "image/jpeg"
| "image/gif"
| "image/webp"
| "audio/mpeg"
| "audio/ogg"
| "audio/wav"
| "video/mp4"
| "video/webm"
)
}
fn encode_header_filename(value: &str) -> String {
value
.as_bytes()
.iter()
.map(|byte| match *byte {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'.' | b'_' | b'-' => {
(*byte as char).to_string()
}
other => format!("%{other:02X}"),
})
.collect()
}
impl IntoResponse for ApiError { impl IntoResponse for ApiError {
fn into_response(self) -> Response { fn into_response(self) -> Response {
(self.status, Json(json!({ "error": self.message }))).into_response() (self.status, Json(json!({ "error": self.message }))).into_response()
@ -482,4 +716,15 @@ mod tests {
); );
assert!(response.headers().contains_key("Content-Security-Policy")); assert!(response.headers().contains_key("Content-Security-Policy"));
} }
#[test]
fn attachment_headers_are_safely_encoded_and_inline_is_allowlisted() {
assert_eq!(
encode_header_filename("报告 1.pdf"),
"%E6%8A%A5%E5%91%8A%201.pdf"
);
assert!(inline_mime_allowed("image/png"));
assert!(!inline_mime_allowed("image/svg+xml"));
assert!(!inline_mime_allowed("text/html"));
}
} }

View File

@ -1,5 +1,6 @@
pub mod auth; pub mod auth;
pub mod http; pub mod http;
pub mod uploads;
pub mod ws; pub mod ws;
use axum::{Router, middleware, routing}; use axum::{Router, middleware, routing};
@ -28,6 +29,7 @@ pub struct GatewayState {
pub task_supervisor: TaskSupervisor, pub task_supervisor: TaskSupervisor,
pub connection_shutdown: tokio_util::sync::CancellationToken, pub connection_shutdown: tokio_util::sync::CancellationToken,
pub auth: auth::AuthManager, pub auth: auth::AuthManager,
pub uploads: uploads::UploadRegistry,
} }
impl GatewayState { impl GatewayState {
@ -41,6 +43,7 @@ impl GatewayState {
crate::config::get_user_config_dir().join("web_auth.json"), crate::config::get_user_config_dir().join("web_auth.json"),
) )
.await?; .await?;
let uploads = uploads::UploadRegistry::new(config.gateway.file_transfer.clone());
// Initialize workspace directory: expand path and ensure it exists // Initialize workspace directory: expand path and ensure it exists
let workspace_path = expand_path(&config.workspace_dir); let workspace_path = expand_path(&config.workspace_dir);
@ -118,7 +121,7 @@ impl GatewayState {
let session_manager = Arc::new(session_manager); let session_manager = Arc::new(session_manager);
// Create ChannelManager and init channels // Create ChannelManager and init channels
let cli_chat_channel = Arc::new(CliChatChannel::new()); let cli_chat_channel = Arc::new(CliChatChannel::with_upload_registry(uploads.clone()));
let channel_manager = ChannelManager::with_bus(cli_chat_channel, bus); let channel_manager = ChannelManager::with_bus(cli_chat_channel, bus);
channel_manager channel_manager
.init(&config, workspace_path.clone()) .init(&config, workspace_path.clone())
@ -201,6 +204,7 @@ impl GatewayState {
task_supervisor, task_supervisor,
connection_shutdown, connection_shutdown,
auth, auth,
uploads,
}) })
} }
@ -220,6 +224,22 @@ impl GatewayState {
let bus_for_outbound = bus.clone(); let bus_for_outbound = bus.clone();
let session_manager = self.session_manager.clone(); let session_manager = self.session_manager.clone();
if self.uploads.enabled() {
let uploads = self.uploads.clone();
self.task_supervisor
.spawn("pending-upload-cleanup", async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(300));
interval.tick().await;
loop {
interval.tick().await;
let removed = uploads.cleanup_expired().await;
if removed > 0 {
tracing::debug!(removed, "Expired pending uploads removed");
}
}
});
}
// Relay structured plan changes to WebSocket clients. This remains // Relay structured plan changes to WebSocket clients. This remains
// separate from chat messages, so task UI updates never pollute history. // separate from chat messages, so task UI updates never pollute history.
let mut plan_events = self.session_manager.work_manager().subscribe(); let mut plan_events = self.session_manager.work_manager().subscribe();
@ -480,6 +500,14 @@ pub async fn run(
.route("/api/jobs", routing::get(http::get_jobs)) .route("/api/jobs", routing::get(http::get_jobs))
.route("/api/jobs/{id}/runs", routing::get(http::get_job_runs)) .route("/api/jobs/{id}/runs", routing::get(http::get_job_runs))
.route("/api/memories", routing::get(http::get_memories)) .route("/api/memories", routing::get(http::get_memories))
.route(
"/api/chat/{client_id}/uploads",
routing::post(http::upload_file).layer(axum::extract::DefaultBodyLimit::disable()),
)
.route(
"/api/chat/{client_id}/sessions/{session_id}/messages/{message_id}/attachments/{index}",
routing::get(http::download_attachment),
)
.route("/ws", routing::get(ws::ws_handler)) .route("/ws", routing::get(ws::ws_handler))
.route_layer(middleware::from_fn_with_state( .route_layer(middleware::from_fn_with_state(
state.auth.clone(), state.auth.clone(),

310
src/gateway/uploads.rs Normal file
View File

@ -0,0 +1,310 @@
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use sha2::{Digest, Sha256};
use tokio::sync::Mutex;
use crate::bus::MediaItem;
use crate::config::{FileTransferConfig, expand_path};
use crate::protocol::UploadDescriptor;
#[derive(Debug, Clone)]
pub struct PendingUpload {
pub descriptor: UploadDescriptor,
pub owner_chat_id: String,
pub path: PathBuf,
expires_at: Instant,
}
#[derive(Debug, thiserror::Error)]
pub enum UploadError {
#[error("file transfer is disabled")]
Disabled,
#[error("too many files; maximum is {0}")]
TooManyFiles(usize),
#[error("duplicate upload id")]
Duplicate,
#[error("upload not found or expired")]
NotFound,
#[error("upload does not belong to this client")]
WrongOwner,
#[error("attachments exceed the per-message limit of {0} bytes")]
MessageTooLarge(u64),
}
#[derive(Clone)]
pub struct UploadRegistry {
config: FileTransferConfig,
upload_dir: PathBuf,
pending: Arc<Mutex<HashMap<String, PendingUpload>>>,
}
impl UploadRegistry {
pub fn new(config: FileTransferConfig) -> Self {
let upload_dir = expand_path(&config.upload_dir);
Self {
config,
upload_dir,
pending: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn disabled() -> Self {
let config = FileTransferConfig {
enabled: false,
..FileTransferConfig::default()
};
Self::new(config)
}
pub fn enabled(&self) -> bool {
self.config.enabled
}
pub fn max_file_bytes(&self) -> u64 {
self.config.max_file_bytes
}
pub fn safe_file_name(name: &str) -> String {
let base = Path::new(name)
.file_name()
.and_then(|value| value.to_str())
.unwrap_or("attachment");
let cleaned = base
.chars()
.filter(|character| !character.is_control() && *character != '/' && *character != '\\')
.collect::<String>();
let cleaned = cleaned.trim().trim_matches('.');
let mut result = if cleaned.is_empty() {
"attachment".to_string()
} else {
cleaned.to_string()
};
while result.len() > 180 {
result.pop();
}
result
}
pub async fn allocate_path(
&self,
chat_id: &str,
file_name: &str,
) -> Result<(PathBuf, PathBuf, String), std::io::Error> {
let scope_hash = hex_digest(chat_id.as_bytes());
let upload_id = uuid::Uuid::new_v4().to_string();
let directory = self.upload_dir.join(&scope_hash[..16]).join(&upload_id);
tokio::fs::create_dir_all(&directory).await?;
let safe_name = Self::safe_file_name(file_name);
let final_path = directory.join(&safe_name);
let temporary_path = directory.join(".upload.part");
Ok((temporary_path, final_path, upload_id))
}
pub async fn register(
&self,
upload_id: String,
chat_id: String,
path: PathBuf,
mut descriptor: UploadDescriptor,
) -> Result<UploadDescriptor, UploadError> {
if !self.enabled() {
return Err(UploadError::Disabled);
}
let ttl = Duration::from_secs(self.config.pending_ttl_seconds.max(1));
let expires_at = Instant::now() + ttl;
let expires_at_unix = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs()
.saturating_add(ttl.as_secs()) as i64;
descriptor.upload_id = upload_id.clone();
descriptor.expires_at = expires_at_unix;
let mut pending = self.pending.lock().await;
pending.insert(
upload_id,
PendingUpload {
descriptor: descriptor.clone(),
owner_chat_id: chat_id,
path,
expires_at,
},
);
Ok(descriptor)
}
pub async fn take_many(
&self,
chat_id: &str,
upload_ids: &[String],
) -> Result<Vec<PendingUpload>, UploadError> {
if upload_ids.is_empty() {
return Ok(Vec::new());
}
if !self.enabled() {
return Err(UploadError::Disabled);
}
if upload_ids.len() > self.config.max_files_per_message {
return Err(UploadError::TooManyFiles(self.config.max_files_per_message));
}
let unique = upload_ids.iter().collect::<HashSet<_>>();
if unique.len() != upload_ids.len() {
return Err(UploadError::Duplicate);
}
let now = Instant::now();
let mut pending = self.pending.lock().await;
let mut total = 0_u64;
for id in upload_ids {
let upload = pending.get(id).ok_or(UploadError::NotFound)?;
if upload.expires_at <= now {
return Err(UploadError::NotFound);
}
if upload.owner_chat_id != chat_id {
return Err(UploadError::WrongOwner);
}
total = total.saturating_add(upload.descriptor.size);
}
if total > self.config.max_message_bytes {
return Err(UploadError::MessageTooLarge(self.config.max_message_bytes));
}
Ok(upload_ids
.iter()
.filter_map(|id| pending.remove(id))
.collect())
}
pub async fn restore(&self, uploads: Vec<PendingUpload>) {
let mut pending = self.pending.lock().await;
for upload in uploads {
pending.insert(upload.descriptor.upload_id.clone(), upload);
}
}
pub async fn cleanup_expired(&self) -> usize {
let now = Instant::now();
let expired = {
let mut pending = self.pending.lock().await;
let ids = pending
.iter()
.filter(|(_, upload)| upload.expires_at <= now)
.map(|(id, _)| id.clone())
.collect::<Vec<_>>();
ids.into_iter()
.filter_map(|id| pending.remove(&id))
.map(|upload| upload.path)
.collect::<Vec<_>>()
};
let count = expired.len();
for path in expired {
let _ = tokio::fs::remove_file(&path).await;
if let Some(parent) = path.parent() {
let _ = tokio::fs::remove_dir(parent).await;
}
}
count
}
pub fn as_media(upload: &PendingUpload) -> MediaItem {
MediaItem {
path: upload.path.to_string_lossy().into_owned(),
media_type: upload.descriptor.media_type.clone(),
mime_type: Some(upload.descriptor.mime_type.clone()),
original_key: None,
}
}
}
fn hex_digest(value: &[u8]) -> String {
let digest = Sha256::digest(value);
digest.iter().map(|byte| format!("{byte:02x}")).collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn sanitizes_file_names() {
assert_eq!(
UploadRegistry::safe_file_name("../../report.pdf"),
"report.pdf"
);
assert_eq!(UploadRegistry::safe_file_name(".."), "attachment");
assert_eq!(UploadRegistry::safe_file_name("a\n.txt"), "a.txt");
}
#[tokio::test]
async fn upload_ids_are_scoped_and_consumed() {
let registry = UploadRegistry::new(FileTransferConfig::default());
registry
.register(
"upload-1".into(),
"client-a".into(),
PathBuf::from("/tmp/a"),
UploadDescriptor {
upload_id: String::new(),
name: "a.txt".into(),
media_type: "file".into(),
mime_type: "text/plain".into(),
size: 10,
expires_at: 0,
},
)
.await
.unwrap();
assert!(matches!(
registry.take_many("client-b", &["upload-1".into()]).await,
Err(UploadError::WrongOwner)
));
assert_eq!(
registry
.take_many("client-a", &["upload-1".into()])
.await
.unwrap()
.len(),
1
);
assert!(matches!(
registry.take_many("client-a", &["upload-1".into()]).await,
Err(UploadError::NotFound)
));
}
#[tokio::test]
async fn expired_pending_uploads_are_removed() {
let directory = tempfile::tempdir().unwrap();
let config = FileTransferConfig {
upload_dir: directory.path().to_string_lossy().into_owned(),
pending_ttl_seconds: 1,
..FileTransferConfig::default()
};
let registry = UploadRegistry::new(config);
let path = directory.path().join("expired.txt");
tokio::fs::write(&path, b"expired").await.unwrap();
registry
.register(
"expired".into(),
"client".into(),
path.clone(),
UploadDescriptor {
upload_id: String::new(),
name: "expired.txt".into(),
media_type: "file".into(),
mime_type: "text/plain".into(),
size: 7,
expires_at: 0,
},
)
.await
.unwrap();
tokio::time::sleep(Duration::from_millis(1_050)).await;
assert_eq!(registry.cleanup_expired().await, 1);
assert!(!path.exists());
}
}

View File

@ -56,6 +56,11 @@ async fn handle_socket(
let _ = sender let _ = sender
.send(WsOutbound::SessionEstablished { .send(WsOutbound::SessionEstablished {
session_id: session_id.clone(), session_id: session_id.clone(),
capabilities: if state.uploads.enabled() {
vec!["file_transfer_v1".to_string()]
} else {
Vec::new()
},
}) })
.await; .await;

View File

@ -19,6 +19,43 @@ pub struct SlashCommandInfo {
pub aliases: Vec<String>, pub aliases: Vec<String>,
} }
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct UploadDescriptor {
pub upload_id: String,
pub name: String,
pub media_type: String,
pub mime_type: String,
pub size: u64,
pub expires_at: i64,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct MessageAttachment {
pub index: u32,
pub name: String,
pub media_type: String,
pub mime_type: String,
}
impl MessageAttachment {
pub fn from_media_ref(index: usize, media_ref: &crate::bus::MediaRef) -> Self {
let name = std::path::Path::new(&media_ref.path)
.file_name()
.map(|name| name.to_string_lossy().into_owned())
.filter(|name| !name.is_empty())
.unwrap_or_else(|| "attachment".to_string());
let mime_type = mime_guess::from_path(&name)
.first_or_octet_stream()
.to_string();
Self {
index: u32::try_from(index).unwrap_or(u32::MAX),
name,
media_type: media_ref.media_type.clone(),
mime_type,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HistoryMessage { pub struct HistoryMessage {
pub id: String, pub id: String,
@ -32,6 +69,8 @@ pub struct HistoryMessage {
pub tool_name: Option<String>, pub tool_name: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<Vec<crate::providers::ToolCall>>, pub tool_calls: Option<Vec<crate::providers::ToolCall>>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub attachments: Vec<MessageAttachment>,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@ -40,6 +79,8 @@ pub enum WsInbound {
#[serde(rename = "user_input")] #[serde(rename = "user_input")]
UserInput { UserInput {
content: String, content: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
upload_ids: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
channel: Option<String>, channel: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
@ -104,13 +145,19 @@ pub enum WsOutbound {
id: String, id: String,
content: String, content: String,
role: String, role: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
attachments: Vec<MessageAttachment>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
session_id: Option<String>, session_id: Option<String>,
}, },
#[serde(rename = "error")] #[serde(rename = "error")]
Error { code: String, message: String }, Error { code: String, message: String },
#[serde(rename = "session_established")] #[serde(rename = "session_established")]
SessionEstablished { session_id: String }, SessionEstablished {
session_id: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
capabilities: Vec<String>,
},
#[serde(rename = "session_created")] #[serde(rename = "session_created")]
SessionCreated { session_id: String, title: String }, SessionCreated { session_id: String, title: String },
#[serde(rename = "session_list")] #[serde(rename = "session_list")]

View File

@ -54,6 +54,7 @@ impl OutboundMessenger for SessionManager {
}; };
let message = outbound_history_message(marked_content.clone(), source, &media); let message = outbound_history_message(marked_content.clone(), source, &media);
let message_id = message.id.clone();
append_persisted_messages(&session, vec![message]) append_persisted_messages(&session, vec![message])
.await .await
.map_err(|error| error.to_string())?; .map_err(|error| error.to_string())?;
@ -62,6 +63,10 @@ impl OutboundMessenger for SessionManager {
self.restore_origin_dialog(&origin_id, &target_sid).await; self.restore_origin_dialog(&origin_id, &target_sid).await;
} }
let metadata = HashMap::from([
("_session_id".to_string(), target_sid.to_string()),
("_message_id".to_string(), message_id),
]);
self.bus self.bus
.deliver_outbound(OutboundMessage { .deliver_outbound(OutboundMessage {
channel: channel.to_string(), channel: channel.to_string(),
@ -69,7 +74,7 @@ impl OutboundMessenger for SessionManager {
content: marked_content, content: marked_content,
reply_to: None, reply_to: None,
media, media,
metadata: HashMap::new(), metadata,
delivery: None, delivery: None,
}) })
.await .await

View File

@ -859,6 +859,40 @@ impl Storage {
.collect()) .collect())
} }
pub async fn get_message(
&self,
session_id: &str,
message_id: &str,
) -> Result<Option<crate::storage::message::MessageMeta>, StorageError> {
let row = sqlx::query(
r#"
SELECT id, session_id, seq, role, content, reasoning_content, media_refs,
tool_call_id, tool_name, tool_calls, source, created_at
FROM messages
WHERE session_id = ? AND id = ?
"#,
)
.bind(session_id)
.bind(message_id)
.fetch_optional(self.pool())
.await?;
Ok(row.map(|row| crate::storage::message::MessageMeta {
id: row.get("id"),
session_id: row.get("session_id"),
seq: row.get("seq"),
role: row.get("role"),
content: row.get("content"),
reasoning_content: row.get("reasoning_content"),
media_refs: row.get("media_refs"),
tool_call_id: row.get("tool_call_id"),
tool_name: row.get("tool_name"),
tool_calls: row.get("tool_calls"),
source: row.get("source"),
created_at: row.get("created_at"),
}))
}
pub async fn get_max_message_seq(&self, session_id: &str) -> Result<i64, StorageError> { pub async fn get_max_message_seq(&self, session_id: &str) -> Result<i64, StorageError> {
let row = sqlx::query( let row = sqlx::query(
"SELECT COALESCE(MAX(seq), 0) as max_seq FROM messages WHERE session_id = ?", "SELECT COALESCE(MAX(seq), 0) as max_seq FROM messages WHERE session_id = ?",

View File

@ -139,6 +139,7 @@ fn test_bounded_session_history_protocol() {
tool_call_id: None, tool_call_id: None,
tool_name: None, tool_name: None,
tool_calls: None, tool_calls: None,
attachments: Vec::new(),
}], }],
}; };
let decoded: WsOutbound = let decoded: WsOutbound =
@ -152,6 +153,26 @@ fn test_bounded_session_history_protocol() {
} }
} }
#[test]
fn test_user_input_accepts_upload_ids_and_old_payloads() {
let old: WsInbound =
serde_json::from_str(r#"{"type":"user_input","content":"hello"}"#).unwrap();
match old {
WsInbound::UserInput { upload_ids, .. } => assert!(upload_ids.is_empty()),
other => panic!("unexpected decoded variant: {other:?}"),
}
let message = WsInbound::UserInput {
content: "处理文件".to_string(),
upload_ids: vec!["upload-1".to_string()],
channel: None,
chat_id: None,
sender_id: None,
};
let json = serde_json::to_string(&message).unwrap();
assert!(json.contains(r#""upload_ids":["upload-1"]"#));
}
#[test] #[test]
fn test_session_history_preserves_tool_call_metadata() { fn test_session_history_preserves_tool_call_metadata() {
let outbound = WsOutbound::SessionHistory { let outbound = WsOutbound::SessionHistory {
@ -169,6 +190,7 @@ fn test_session_history_preserves_tool_call_metadata() {
name: "read_file".to_string(), name: "read_file".to_string(),
arguments: serde_json::json!({ "path": "README.md" }), arguments: serde_json::json!({ "path": "README.md" }),
}]), }]),
attachments: Vec::new(),
}], }],
}; };

BIN
tmp.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 242 KiB

View File

@ -17,6 +17,8 @@
let selectedCommand = $state(0); let selectedCommand = $state(0);
let commandMenuDismissed = $state(false); let commandMenuDismissed = $state(false);
let thinking = $state(false); let thinking = $state(false);
let pendingUploads = $state([]);
let fileInput;
let plansBySession = $state({}); let plansBySession = $state({});
let unseenPlanSessions = $state({}); let unseenPlanSessions = $state({});
let todoOpen = $state(false); let todoOpen = $state(false);
@ -126,7 +128,7 @@
case "assistant_response": case "assistant_response":
thinking = false; thinking = false;
if (!frame.session_id || frame.session_id === currentId) { if (!frame.session_id || frame.session_id === currentId) {
appendMessage(frame.role || "assistant", frame.content); appendMessage(frame.role || "assistant", frame.content, frame.attachments || [], frame.id);
if (currentId) send({ type: "get_session_history", session_id: currentId, limit: 1000 }); if (currentId) send({ type: "get_session_history", session_id: currentId, limit: 1000 });
} }
send({ type: "list_sessions", include_archived: false }); send({ type: "list_sessions", include_archived: false });
@ -144,14 +146,15 @@
if (messageBox) messageBox.scrollTop = messageBox.scrollHeight; if (messageBox) messageBox.scrollTop = messageBox.scrollHeight;
} }
function appendMessage(role, content) { function appendMessage(role, content, attachments = [], id = crypto.randomUUID()) {
messages = [...messages, { id: crypto.randomUUID(), role, content }]; messages = [...messages, { id, role, content, attachments }];
scrollToBottom(); scrollToBottom();
} }
function loadSession(id) { function loadSession(id) {
if (!id) return; if (!id) return;
currentId = id; currentId = id;
clearPendingUploads();
messages = []; messages = [];
todoOpen = Boolean(unseenPlanSessions[id]); todoOpen = Boolean(unseenPlanSessions[id]);
unseenPlanSessions[id] = false; unseenPlanSessions[id] = false;
@ -171,16 +174,108 @@
function submit() { function submit() {
const content = draft.trim(); const content = draft.trim();
if (!content || !connected) return; const ready = pendingUploads.filter((upload) => upload.status === "ready");
appendMessage("user", content); if ((!content && !ready.length) || !connected || pendingUploads.some((upload) => upload.status === "uploading")) return;
appendMessage("user", content, ready.map((upload, index) => ({
index,
name: upload.name,
media_type: upload.media_type,
mime_type: upload.mime_type,
local_url: upload.localUrl
})));
thinking = true; thinking = true;
send({ type: "user_input", content }); send({ type: "user_input", content, upload_ids: ready.map((upload) => upload.upload_id) });
draft = ""; draft = "";
pendingUploads = [];
commandMenuDismissed = false; commandMenuDismissed = false;
selectedCommand = 0; selectedCommand = 0;
if (input) input.style.height = "auto"; if (input) input.style.height = "auto";
} }
function attachmentUrl(message, attachment, inline = false) {
if (attachment.local_url) return attachment.local_url;
if (!currentId || !message.id) return "";
const base = `/api/chat/${encodeURIComponent(clientId())}/sessions/${encodeURIComponent(currentId)}/messages/${encodeURIComponent(message.id)}/attachments/${attachment.index}`;
return inline ? `${base}?disposition=inline` : base;
}
function formatBytes(value) {
if (!Number.isFinite(value)) return "";
if (value < 1024) return `${value} B`;
if (value < 1024 * 1024) return `${(value / 1024).toFixed(1)} KiB`;
return `${(value / 1024 / 1024).toFixed(1)} MiB`;
}
function canPreview(attachment) {
return ["image/png", "image/jpeg", "image/gif", "image/webp"].includes(attachment.mime_type);
}
function addFiles(files) {
for (const file of Array.from(files || [])) uploadFile(file);
}
function uploadFile(file) {
if (!connected) return notify("聊天连接尚未就绪", true);
const localId = crypto.randomUUID();
const localUrl = file.type.startsWith("image/") ? URL.createObjectURL(file) : null;
pendingUploads = [...pendingUploads, {
localId, name: file.name, size: file.size, progress: 0, status: "uploading", localUrl
}];
const body = new FormData();
body.append("file", file, file.name);
const request = new XMLHttpRequest();
request.open("POST", `/api/chat/${encodeURIComponent(clientId())}/uploads`);
request.upload.onprogress = (event) => {
if (!event.lengthComputable) return;
pendingUploads = pendingUploads.map((item) => item.localId === localId
? { ...item, progress: Math.round(event.loaded / event.total * 100) }
: item);
};
request.onload = () => {
let response = {};
try { response = JSON.parse(request.responseText || "{}"); } catch {}
if (request.status >= 200 && request.status < 300) {
pendingUploads = pendingUploads.map((item) => item.localId === localId
? { ...item, ...response, status: "ready", progress: 100 }
: item);
} else {
pendingUploads = pendingUploads.map((item) => item.localId === localId
? { ...item, status: "error", error: response.error || `上传失败 (${request.status})` }
: item);
}
};
request.onerror = () => {
pendingUploads = pendingUploads.map((item) => item.localId === localId
? { ...item, status: "error", error: "网络错误" }
: item);
};
request.send(body);
}
function removeUpload(localId) {
const upload = pendingUploads.find((item) => item.localId === localId);
if (upload?.localUrl) URL.revokeObjectURL(upload.localUrl);
pendingUploads = pendingUploads.filter((item) => item.localId !== localId);
}
function clearPendingUploads() {
for (const upload of pendingUploads) if (upload.localUrl) URL.revokeObjectURL(upload.localUrl);
pendingUploads = [];
}
function dropFiles(event) {
event.preventDefault();
addFiles(event.dataTransfer?.files);
}
function pasteFiles(event) {
const files = Array.from(event.clipboardData?.items || [])
.filter((item) => item.kind === "file")
.map((item) => item.getAsFile())
.filter(Boolean);
if (files.length) addFiles(files);
}
async function moveCommandSelection(offset) { async function moveCommandSelection(offset) {
const length = commandSuggestions.length; const length = commandSuggestions.length;
if (!length) return; if (!length) return;
@ -247,6 +342,7 @@
stopped = true; stopped = true;
clearTimeout(reconnectTimer); clearTimeout(reconnectTimer);
socket?.close(); socket?.close();
clearPendingUploads();
}; };
}); });
</script> </script>
@ -283,11 +379,24 @@
<div class="empty"><div class="empty-logo">P</div><h2>今天想做些什么?</h2><p>消息与 CLI 客户端使用同一套会话、记忆和工具能力。</p></div> <div class="empty"><div class="empty-logo">P</div><h2>今天想做些什么?</h2><p>消息与 CLI 客户端使用同一套会话、记忆和工具能力。</p></div>
{/if} {/if}
{#each messages as message (message.id)} {#each messages as message (message.id)}
{#if message.role !== "tool" && (message.content || message.tool_calls?.length)} {#if message.role !== "tool" && (message.content || message.tool_calls?.length || message.attachments?.length)}
<div class:user={message.role === "user"} class:assistant={message.role !== "user"} class:has-tools={message.tool_calls?.length} class="message"> <div class:user={message.role === "user"} class:assistant={message.role !== "user"} class:has-tools={message.tool_calls?.length} class="message">
<div class="avatar">{message.role === "user" ? "你" : "P"}</div> <div class="avatar">{message.role === "user" ? "你" : "P"}</div>
<div class="message-content"> <div class="message-content">
{#if message.content}<div class="bubble"><Markdown content={message.content} /></div>{/if} {#if message.content}<div class="bubble"><Markdown content={message.content} /></div>{/if}
{#if message.attachments?.length}
<div class="message-attachments">
{#each message.attachments as attachment (`${message.id}:${attachment.index}`)}
<article class="attachment-card">
{#if canPreview(attachment)}
<img src={attachmentUrl(message, attachment, true)} alt={attachment.name} />
{:else}<span class="attachment-icon"></span>{/if}
<div><strong>{attachment.name}</strong><small>{attachment.mime_type || attachment.media_type}</small></div>
<a href={attachmentUrl(message, attachment)} download={attachment.name} aria-label={`下载 ${attachment.name}`}>↓</a>
</article>
{/each}
</div>
{/if}
{#if message.tool_calls?.length} {#if message.tool_calls?.length}
<div class="tool-calls"> <div class="tool-calls">
{#each message.tool_calls as call (call.id)} {#each message.tool_calls as call (call.id)}
@ -301,7 +410,7 @@
{/each} {/each}
{#if thinking}<div class="message assistant typing"><div class="avatar">P</div><div class="bubble"><span class="pulse"></span>正在思考…</div></div>{/if} {#if thinking}<div class="message assistant typing"><div class="avatar">P</div><div class="bubble"><span class="pulse"></span>正在思考…</div></div>{/if}
</div> </div>
<form class="composer" onsubmit={(event) => { event.preventDefault(); submit(); }}> <form class="composer" onsubmit={(event) => { event.preventDefault(); submit(); }} ondragover={(event) => event.preventDefault()} ondrop={dropFiles}>
{#if commandSuggestions.length} {#if commandSuggestions.length}
<div class="command-menu" id="slash-command-menu" role="listbox" aria-label="斜杠命令"> <div class="command-menu" id="slash-command-menu" role="listbox" aria-label="斜杠命令">
<div class="command-menu-heading"><span>斜杠命令</span><kbd>↑↓ 选择 · Tab/Enter 补全 · Esc 关闭</kbd></div> <div class="command-menu-heading"><span>斜杠命令</span><kbd>↑↓ 选择 · Tab/Enter 补全 · Esc 关闭</kbd></div>
@ -320,11 +429,24 @@
{/each} {/each}
</div> </div>
{/if} {/if}
{#if pendingUploads.length}
<div class="pending-uploads">
{#each pendingUploads as upload (upload.localId)}
<div class:error={upload.status === "error"} class="pending-upload">
<span></span><div><strong>{upload.name}</strong><small>{upload.status === "uploading" ? `上传中 ${upload.progress}%` : upload.status === "error" ? upload.error : formatBytes(upload.size)}</small></div>
<button type="button" aria-label={`移除 ${upload.name}`} onclick={() => removeUpload(upload.localId)}>×</button>
</div>
{/each}
</div>
{/if}
<input class="file-input" bind:this={fileInput} type="file" multiple onchange={(event) => { addFiles(event.currentTarget.files); event.currentTarget.value = ""; }} />
<button class="attach" type="button" aria-label="添加附件" onclick={() => fileInput?.click()}></button>
<textarea <textarea
bind:this={input} bind:this={input}
bind:value={draft} bind:value={draft}
onkeydown={keydown} onkeydown={keydown}
oninput={inputChanged} oninput={inputChanged}
onpaste={pasteFiles}
rows="1" rows="1"
role="combobox" role="combobox"
aria-autocomplete="list" aria-autocomplete="list"
@ -333,7 +455,7 @@
aria-activedescendant={commandSuggestions.length ? `slash-command-${selectedCommand}` : undefined} aria-activedescendant={commandSuggestions.length ? `slash-command-${selectedCommand}` : undefined}
placeholder="输入消息,输入 / 查看命令" placeholder="输入消息,输入 / 查看命令"
></textarea> ></textarea>
<button class="send" type="submit" aria-label="发送" disabled={!connected || !draft.trim()}>↑</button> <button class="send" type="submit" aria-label="发送" disabled={!connected || (!draft.trim() && !pendingUploads.some((upload) => upload.status === "ready")) || pendingUploads.some((upload) => upload.status === "uploading")}>↑</button>
<small><span class:online={connected}>{connected ? "已连接" : "已断开,正在重连"}</span><span>/ 打开命令 · Shift+Enter 换行</span></small> <small><span class:online={connected}>{connected ? "已连接" : "已断开,正在重连"}</span><span>/ 打开命令 · Shift+Enter 换行</span></small>
</form> </form>
</div> </div>

View File

@ -143,6 +143,12 @@ main { min-width: 0; height: 100vh; display: flex; flex-direction: column; }
.message.user .message-content { justify-items: end; } .message.user .message-content { justify-items: end; }
.bubble { max-width: 100%; padding: 11px 14px; border: 1px solid var(--line); border-radius: 12px; background: var(--panel); font-size: 14px; line-height: 1.62; overflow-wrap: anywhere; } .bubble { max-width: 100%; padding: 11px 14px; border: 1px solid var(--line); border-radius: 12px; background: var(--panel); font-size: 14px; line-height: 1.62; overflow-wrap: anywhere; }
.message.user .bubble { background: var(--user-bubble); border-color: var(--accent-border); } .message.user .bubble { background: var(--user-bubble); border-color: var(--accent-border); }
.message-attachments { display: grid; gap: 7px; width: min(100%, 520px); }
.attachment-card { display: grid; grid-template-columns: 42px minmax(0, 1fr) 30px; gap: 9px; align-items: center; padding: 8px; border: 1px solid var(--line); border-radius: 10px; background: var(--panel); }
.attachment-card img, .attachment-icon { width: 42px; height: 42px; border-radius: 7px; object-fit: cover; background: var(--panel-2); display: grid; place-items: center; color: var(--accent); }
.attachment-card strong, .attachment-card small { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.attachment-card strong { font-size: 12px; }.attachment-card small { margin-top: 3px; color: var(--muted); font-size: 9px; }
.attachment-card a { width: 28px; height: 28px; display: grid; place-items: center; border: 1px solid var(--line); border-radius: 7px; color: var(--accent); text-decoration: none; }
.typing .bubble { color: var(--muted); } .typing .bubble { color: var(--muted); }
.pulse { display: inline-block; width: 6px; height: 6px; margin-right: 8px; border-radius: 50%; background: var(--accent); animation: pulse 1.1s infinite; } .pulse { display: inline-block; width: 6px; height: 6px; margin-right: 8px; border-radius: 50%; background: var(--accent); animation: pulse 1.1s infinite; }
@keyframes pulse { 50% { opacity: .25; transform: scale(.8); } } @keyframes pulse { 50% { opacity: .25; transform: scale(.8); } }
@ -182,8 +188,13 @@ main { min-width: 0; height: 100vh; display: flex; flex-direction: column; }
.tool-call-details > div > span { display: block; margin-bottom: 6px; color: var(--muted); font-size: 9px; font-weight: 700; letter-spacing: .08em; text-transform: uppercase; } .tool-call-details > div > span { display: block; margin-bottom: 6px; color: var(--muted); font-size: 9px; font-weight: 700; letter-spacing: .08em; text-transform: uppercase; }
.tool-call-details pre, .tool-result { max-height: 300px; margin: 0; padding: 10px 11px; overflow: auto; border: 1px solid var(--line); border-radius: 8px; color: var(--text-soft); background: var(--code-bg); font: 11px/1.55 ui-monospace, SFMono-Regular, Consolas, monospace; white-space: pre-wrap; overflow-wrap: anywhere; } .tool-call-details pre, .tool-result { max-height: 300px; margin: 0; padding: 10px 11px; overflow: auto; border: 1px solid var(--line); border-radius: 8px; color: var(--text-soft); background: var(--code-bg); font: 11px/1.55 ui-monospace, SFMono-Regular, Consolas, monospace; white-space: pre-wrap; overflow-wrap: anywhere; }
.tool-result .markdown-body { font: inherit; } .tool-result .markdown-body { font: inherit; }
.composer { position: relative; margin: 0 max(18px, calc((100% - 850px) / 2)) 18px; border: 1px solid var(--line); background: var(--panel); border-radius: 14px; padding: 10px 11px 6px; display: grid; grid-template-columns: 1fr 38px; box-shadow: var(--shadow); } .composer { position: relative; margin: 0 max(18px, calc((100% - 850px) / 2)) 18px; border: 1px solid var(--line); background: var(--panel); border-radius: 14px; padding: 10px 11px 6px; display: grid; grid-template-columns: 38px 1fr 38px; box-shadow: var(--shadow); }
.composer textarea { resize: none; max-height: 180px; background: transparent; border: 0; outline: 0; color: var(--text); padding: 7px; line-height: 1.5; } .composer textarea { resize: none; max-height: 180px; background: transparent; border: 0; outline: 0; color: var(--text); padding: 7px; line-height: 1.5; }
.file-input { display: none; }
.attach { width: 34px; height: 34px; align-self: end; border: 1px solid var(--line); border-radius: 9px; color: var(--muted); background: var(--panel-2); cursor: pointer; font-size: 18px; }
.pending-uploads { grid-column: 1 / -1; display: flex; gap: 7px; padding: 2px 4px 8px; overflow-x: auto; }
.pending-upload { min-width: 170px; max-width: 260px; display: grid; grid-template-columns: 24px minmax(0, 1fr) 22px; gap: 6px; align-items: center; padding: 7px; border: 1px solid var(--line); border-radius: 9px; background: var(--panel-2); }
.pending-upload.error { border-color: var(--danger); }.pending-upload strong, .pending-upload small { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }.pending-upload strong { font-size: 10px; }.pending-upload small { color: var(--muted); font-size: 9px; }.pending-upload button { border: 0; color: var(--muted); background: none; cursor: pointer; }
.send { width: 36px; height: 36px; border-radius: 10px; color: var(--accent-contrast); background: var(--accent); border: 0; font-size: 19px; cursor: pointer; } .send { width: 36px; height: 36px; border-radius: 10px; color: var(--accent-contrast); background: var(--accent); border: 0; font-size: 19px; cursor: pointer; }
.composer small { grid-column: 1 / -1; display: flex; justify-content: space-between; padding: 3px 7px; color: var(--muted); font-size: 10px; } .composer small { grid-column: 1 / -1; display: flex; justify-content: space-between; padding: 3px 7px; color: var(--muted); font-size: 10px; }
.composer small .online { color: var(--accent); } .composer small .online { color: var(--accent); }