Compare commits

...

6 Commits

22 changed files with 526 additions and 43 deletions

View File

@ -94,6 +94,7 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
- **WebUI slash completion** must consume the existing `get_slash_commands` WebSocket response; do not duplicate the backend command list in frontend source
- **WebUI chat rendering** sanitizes Markdown before inserting HTML; durable `turn_committed` deltas calibrate normal terminal Turns without a full history reload, and history must preserve structured tool-call metadata so calls and results remain independently collapsible
- **WebUI/TUI file transfer** streams bytes over authenticated HTTP and sends only short-lived upload IDs/attachment metadata over WebSocket; messages persist local media paths without guaranteeing later availability, and client responses must never expose those paths
- **WebUI/TUI same-turn media delivery** stages same-session `send_message(files=...)` media on the active Turn and commits it on the final assistant message, after durable tool-call history; safe raster formats should render as an inline preview with download fallback
- **WebUI authentication** protects every management API and `/ws`; only static pairing assets, public health/status, and pairing submission may bypass device auth. Pair-code issuance requires both a real loopback peer and the filesystem-held admin token. The same conjunction may authenticate only `/ws` for local one-shot `run`; it must never authorize management APIs. Never put bearer or admin tokens in URLs or logs
- **Providers** are pure HTTP clients; no bus/session/channel awareness
- **Provider reasoning state** is private replay data: persist it, replay it only to the matching provider, and never expose it to clients, channels, or logs
@ -107,6 +108,7 @@ Scheduler → SessionManager scheduled execution → AgentLoop → Scheduler del
- Slow Turn consumers may skip intermediate snapshots but must receive an explicit bounded terminal delivery; shutdown must call sink abort so platform cleanup remains possible
- Never hold a Session mutex across model, network, or database I/O unless a documented invariant requires it
- Slow work derived from session state must validate `worker_generation`/`state_version` before committing results
- A same-session `send_message` write may avoid advancing `state_version` only when its task-local Turn ID still owns that session's active Turn; all other writes remain versioned
- Related durable mutations use Storage transaction APIs; persistence failure must not leave silent memory/database divergence
- Long-lived gateway tasks must be owned by `TaskSupervisor`; connection-local tasks must be explicitly joined or aborted by their owner
- Connection, retry sleep, queue wait, and shutdown join paths must observe cancellation and have hard time bounds

View File

@ -1,6 +1,6 @@
[package]
name = "picobot"
version = "1.2.0"
version = "1.2.2"
edition = "2024"
[dependencies]

View File

@ -320,7 +320,7 @@ PicoBot 有两类记忆:
| `memory_store` / `memory_recall` / `timeline_recall` / `memory_forget` | 长期记忆操作 |
| `delegate` | 启动 inline、background 或 parallel 子 Agent |
| `todo` | 为复杂、多轮任务创建并更新当前 session 的持久化计划 |
| `send_message` | 向指定渠道发送消息 |
| `send_message` | 向指定渠道或当前会话发送消息,可附带文件/截图WebUI/TUI 当前 Turn 的附件并入最终回复 |
| `chat_manager` | 查看渠道、会话和历史消息 |
| `cron_add/list/remove/enable/disable/update` | 管理定时任务 |
| `routine_maintenance` | 安全清理超过保留期的 Timeline不删除 Knowledge |

View File

@ -3,7 +3,7 @@ services:
build:
context: .
dockerfile: Dockerfile
image: picobot:1.2.0
image: picobot:1.2.2
container_name: picobot-test
restart: unless-stopped
ports:

View File

@ -189,6 +189,8 @@ Session ID 格式为:
5. 持久化写入由 `persistence_lock` 串行化;多条相关记录应使用 Storage 的原子接口。
6. 内存先变更但持久化失败时,必须回滚精确匹配的消息后缀,不能删除无关的新状态。
当前 WebUI/TUI Turn 通过 `send_message(files=...)` 向自身 session 投递文件时,文件先进入 task-local Turn delivery 暂存区,成功结束后附加到最终 assistant 消息,与工具链一起原子提交;因此持久化和刷新后的顺序都是工具调用/结果在前、携带附件的最终回复在后,也不会生成带 `[message from ...]` 的自投递气泡。其他同 Turn 自投递仍是受控例外:只有 task-local Turn ID 仍匹配该 session 的 active Turn写入才允许不递增 `state_version`。跨 Turn、跨 session 以及无法证明所有权的写入仍必须递增版本。Provider 回放历史附件时,只有 user 输入和当前工具结果可生成模型原生媒体块assistant/system 附件只回放文本清单,避免把图片放到供应商不接受的角色。
SessionManager 负责组装会话上下文系统提示、Skills、召回的 Knowledge、压缩后的 Timeline、可选的 active plan 摘要和当前消息历史。`session::turn_input` 在 Session 锁外并行读取 Knowledge、active plan 并压缩历史,然后通过同一个 assembly 路径生成首次请求和 context-overflow 重试输入;重试不得复制系统提示或 runtime context 拼接逻辑。普通闲聊 session 没有 plan 摘要;计划状态由 `WorkManager` 从 SQLite 读取,因此不以自然语言摘要作为权威来源。`AgentLoop` 接收完整输入执行一次模型/工具循环,本身不拥有会话状态。
当前 Turn 的工具进度只从 `AgentLoop` 的结构化 `TurnEvent` 进入 `TurnController`,不能另建字符串 notification 通道重复投递。后台子 Agent 的 `TaskNotification` 表达跨 Turn 的任务完成仍由独立的受监督消费者投递。自动标题属于非关键派生工作Turn 持久化完成后由 `TaskSupervisor` 调度Session worker 不等待模型生成;同一 Session 同时最多有一个标题任务,提交时仍校验标题保持默认值,避免覆盖用户改名。

BIN
docs/avatar.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 692 KiB

BIN
docs/logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 732 KiB

View File

@ -146,6 +146,8 @@ Worker 的处理原则:
4. Session 持久化由独立 `persistence_lock` 串行化;批量消息使用原子写入,失败时精确回滚内存后缀。
5. 上下文溢出时按 Provider 返回的真实限制重新压缩并重试。
WebUI/TUI 的 Active Turn 使用 `send_message(files=...)` 向自身 session 投递附件时,附件暂存到 task-local Turn delivery成功结束后并入最终 assistant 消息,因此工具链始终排在附件回复之前且不会出现自引用来源前缀。其他自投递要求 task-local Turn ID 与 session 的 active Turn 匹配;历史中的 assistant/system 附件只作为文本清单提供给模型,原生媒体块仅用于 user 输入和当前工具结果。
### 活动 Turn
每个主 Agent 请求会创建一个内存 Turn。Provider delta 经 AgentLoop 转换为 reasoning、正文、工具开始/完成等语义事件TurnController 归约为有序 block 和单调 revision 的完整快照。TUI/WebUI 使用 `history + active_turn` 渲染,不自行拼接 token中间帧可丢下一快照会自动收敛。

View File

@ -15,6 +15,8 @@
`files` 支持绝对路径和 workspace 相对路径,媒体类型由文件扩展名/MIME 自动判断。目前 schema 不支持手工指定 `file_types`
目标可以是当前会话例如把浏览器截图或生成文件直接交付给正在聊天的用户。WebUI/TUI 中,同一 active Turn 的附件会并入本轮最终回复:先显示工具调用与结果,再显示自然回复和内联图片,不生成 `[message from ...]` 自引用前缀。后续模型回放只读取 assistant 附件的文本清单,不会把图片放入 assistant 内容块。
### 示例
```json

View File

@ -548,10 +548,19 @@ impl AgentLoop {
"tool" => MediaOrigin::Tool(m.tool_name.as_deref().unwrap_or("unknown")),
_ => MediaOrigin::Message,
};
// Provider APIs generally allow native image/audio blocks only in
// user input (and, through provider-specific adaptation, current
// tool results). Persisted assistant attachments are delivery
// artifacts: replay their manifest as text, never as native media.
let native_input_types = if matches!(m.role.as_str(), "user" | "tool") {
self.input_types.as_slice()
} else {
&[]
};
build_content_blocks(
&m.content,
&m.media_refs,
&self.input_types,
native_input_types,
&self.media_registry,
origin,
)
@ -1465,6 +1474,49 @@ mod tests {
assert!(matches!(blocks.get(1), Some(ContentBlock::ImageUrl { .. })));
}
#[test]
fn assistant_attachments_replay_as_text_without_native_image_blocks() {
use std::io::Write;
let mut image = tempfile::Builder::new().suffix(".png").tempfile().unwrap();
image.write_all(b"\x89PNG\r\n\x1a\nminimal").unwrap();
let path = image.path().to_string_lossy().into_owned();
let provider = Arc::new(ToolMediaProvider {
image_path: path.clone(),
requests: std::sync::Mutex::new(Vec::new()),
});
let agent = AgentLoop::with_provider_and_tools(
provider,
Arc::new(ToolRegistry::new()),
1,
"vision-test".to_string(),
std::env::current_dir().unwrap(),
vec!["text".to_string(), "image".to_string()],
);
let mut assistant = ChatMessage::assistant("截图已发送");
assistant.media_refs = vec![MediaRef {
path: path.clone(),
media_type: "image".to_string(),
}];
let converted = agent.chat_message_to_llm_message(&assistant, true);
assert_eq!(converted.role, "assistant");
assert_eq!(converted.content.len(), 1);
assert!(
matches!(converted.content.first(), Some(ContentBlock::Text { text })
if text.contains("截图已发送")
&& text.contains(&path)
&& text.contains("not embedded in this model request"))
);
assert!(
!converted
.content
.iter()
.any(|block| matches!(block, ContentBlock::ImageUrl { .. }))
);
}
#[test]
fn only_the_trailing_tool_batch_replays_tool_media() {
let mut messages = vec![

View File

@ -307,6 +307,9 @@ fn inline_mime_allowed(mime: &str) -> bool {
| "image/jpeg"
| "image/gif"
| "image/webp"
| "image/bmp"
| "image/avif"
| "image/x-icon"
| "audio/mpeg"
| "audio/ogg"
| "audio/wav"
@ -724,6 +727,7 @@ mod tests {
"%E6%8A%A5%E5%91%8A%201.pdf"
);
assert!(inline_mime_allowed("image/png"));
assert!(inline_mime_allowed("image/bmp"));
assert!(!inline_mime_allowed("image/svg+xml"));
assert!(!inline_mime_allowed("text/html"));
}

View File

@ -47,6 +47,18 @@ fn text_content(blocks: &[ContentBlock]) -> String {
.join("\n")
}
fn role_safe_content(role: &str, blocks: &[ContentBlock]) -> Value {
if role == "user" {
convert_content_blocks(blocks)
} else {
// OpenAI-compatible endpoints commonly reject native image parts on
// system/assistant messages. AgentLoop already supplies a textual
// attachment manifest for persisted assistant media; keep this final
// provider boundary defensive for all callers.
Value::String(text_content(blocks))
}
}
fn regular_message_json(message: &Message) -> Value {
if message.role == "tool" {
json!({
@ -63,7 +75,7 @@ fn regular_message_json(message: &Message) -> Value {
{
let mut value = json!({
"role": message.role,
"content": convert_content_blocks(&message.content),
"content": role_safe_content(&message.role, &message.content),
"tool_calls": message.tool_calls.as_ref().map(|calls| {
calls.iter().map(|call| json!({
"id": call.id,
@ -82,7 +94,7 @@ fn regular_message_json(message: &Message) -> Value {
} else {
let mut value = json!({
"role": message.role,
"content": convert_content_blocks(&message.content)
"content": role_safe_content(&message.role, &message.content)
});
if message.role == "assistant"
&& let Some(ref reasoning_content) = message.reasoning_content
@ -749,6 +761,26 @@ mod tests {
assert_eq!(converted[1]["content"], "second image");
}
#[test]
fn assistant_images_are_never_serialized_as_native_content_parts() {
let converted = convert_messages(&[Message {
role: "assistant".to_string(),
content: vec![
ContentBlock::text("screenshot delivered"),
ContentBlock::image_url("data:image/png;base64,AAAA"),
],
reasoning_content: None,
provider_state: None,
tool_call_id: None,
name: None,
tool_calls: None,
}]);
assert_eq!(converted.len(), 1);
assert_eq!(converted[0]["role"], "assistant");
assert_eq!(converted[0]["content"], "screenshot delivered");
}
#[tokio::test]
async fn sse_decoder_handles_byte_boundaries_reasoning_content_and_usage() {
let input = concat!(

View File

@ -2,10 +2,13 @@ use std::collections::HashMap;
use crate::bus::{ChatMessage, MediaItem, MessageSource, OutboundMessage, SourceKind};
use crate::session::UnifiedSessionId;
use crate::tools::OutboundMessenger;
use crate::tools::{OutboundDelivery, OutboundMessenger};
use super::persistence::append_persisted_messages;
use super::session::{CURRENT_SOURCE_SESSION, SessionManager};
use super::persistence::{append_active_turn_message, append_persisted_messages};
use super::session::{
CURRENT_SOURCE_SESSION, CURRENT_TURN_DELIVERIES, CURRENT_TURN_ID, PendingTurnDelivery,
SessionManager,
};
#[async_trait::async_trait]
impl OutboundMessenger for SessionManager {
@ -17,7 +20,7 @@ impl OutboundMessenger for SessionManager {
content: &str,
mut source: MessageSource,
media: Vec<MediaItem>,
) -> Result<(), String> {
) -> Result<OutboundDelivery, String> {
if source.from_session.is_none() {
source.from_session = CURRENT_SOURCE_SESSION
.try_with(|value| value.clone())
@ -47,6 +50,32 @@ impl OutboundMessenger for SessionManager {
let origin = source.from_session.as_deref().unwrap_or("unknown");
let origin_id = source.from_session.clone();
let same_session = source.from_session.as_deref() == Some(target_sid.to_string().as_str());
let current_turn_id = CURRENT_TURN_ID
.try_with(|value| value.clone())
.ok()
.flatten();
let current_turn_deliveries = CURRENT_TURN_DELIVERIES
.try_with(|value| value.clone())
.ok()
.flatten();
if same_session
&& channel == "cli_chat"
&& !media.is_empty()
&& let (Some(turn_id), Some(deliveries)) =
(current_turn_id.clone(), current_turn_deliveries)
{
let guard = session.lock().await;
if guard.owns_active_turn(&turn_id) {
deliveries
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.push(PendingTurnDelivery {
content: content.to_string(),
media,
});
return Ok(OutboundDelivery::AttachedToCurrentTurn);
}
}
let marked_content = if content.trim().is_empty() && !media.is_empty() && same_session {
String::new()
} else {
@ -55,9 +84,14 @@ impl OutboundMessenger for SessionManager {
let message = outbound_history_message(marked_content.clone(), source, &media);
let message_id = message.id.clone();
append_persisted_messages(&session, vec![message])
.await
.map_err(|error| error.to_string())?;
if same_session && let Some(turn_id) = current_turn_id {
// Ownership is revalidated atomically with the in-memory append;
// a concurrent /stop therefore falls back to a versioned write.
append_active_turn_message(&session, message, turn_id).await
} else {
append_persisted_messages(&session, vec![message]).await
}
.map_err(|error| error.to_string())?;
if let Some(origin_id) = origin_id {
self.restore_origin_dialog(&origin_id, &target_sid).await;
@ -78,7 +112,8 @@ impl OutboundMessenger for SessionManager {
delivery: None,
})
.await
.map_err(|error| error.to_string())
.map_err(|error| error.to_string())?;
Ok(OutboundDelivery::Delivered)
}
}
@ -108,6 +143,7 @@ impl SessionManager {
Vec::new(),
)
.await
.map(|_| ())
}
}

View File

@ -49,9 +49,42 @@ pub(super) async fn append_persisted_messages(
.map(|_| ())
}
enum VersionPolicy {
Advance,
PreserveForOwnedTurn(String),
}
/// Persist a message emitted as a side effect of the currently active Turn
/// without invalidating that same Turn's captured session version.
///
/// The task-local Turn ID is revalidated while holding the session lock. If it
/// no longer owns the target session, the write advances `state_version` like
/// any unrelated mutation so stale model work is still rejected.
pub(super) async fn append_active_turn_message(
session: &Arc<Mutex<Session>>,
message: ChatMessage,
turn_id: String,
) -> Result<(), StorageError> {
append_persisted_messages_inner(
session,
vec![message],
VersionPolicy::PreserveForOwnedTurn(turn_id),
)
.await
.map(|_| ())
}
pub(super) async fn append_persisted_messages_with_meta(
session: &Arc<Mutex<Session>>,
messages: Vec<ChatMessage>,
) -> Result<Vec<crate::storage::message::MessageMeta>, StorageError> {
append_persisted_messages_inner(session, messages, VersionPolicy::Advance).await
}
async fn append_persisted_messages_inner(
session: &Arc<Mutex<Session>>,
messages: Vec<ChatMessage>,
version_policy: VersionPolicy,
) -> Result<Vec<crate::storage::message::MessageMeta>, StorageError> {
if messages.is_empty() {
return Ok(Vec::new());
@ -60,12 +93,19 @@ pub(super) async fn append_persisted_messages_with_meta(
let persistence_lock = { session.lock().await.persistence_lock.clone() };
let _persistence_guard = persistence_lock.lock().await;
let message_ids: Vec<_> = messages.iter().map(|message| message.id.clone()).collect();
let snapshots: Vec<Option<MessagePersistSnapshot>> = {
let (snapshots, advance_state_version): (Vec<Option<MessagePersistSnapshot>>, bool) = {
let mut guard = session.lock().await;
messages
let advance_state_version = match &version_policy {
VersionPolicy::Advance => true,
VersionPolicy::PreserveForOwnedTurn(turn_id) => !guard.owns_active_turn(turn_id),
};
let snapshots = messages
.into_iter()
.map(|message| guard.add_message_in_memory(message, true))
.collect()
.map(|message| {
guard.add_message_in_memory_with_version(message, true, advance_state_version)
})
.collect();
(snapshots, advance_state_version)
};
let committed = snapshots
.iter()
@ -74,7 +114,10 @@ pub(super) async fn append_persisted_messages_with_meta(
.collect();
if let Err(error) = persist_added_messages(snapshots).await {
session.lock().await.rollback_message_suffix(&message_ids);
session
.lock()
.await
.rollback_message_suffix_with_version(&message_ids, advance_state_version);
return Err(error);
}
Ok(committed)
@ -109,7 +152,12 @@ where
#[cfg(test)]
mod tests {
use super::*;
use crate::config::LLMProviderConfig;
use crate::memory::MemoryManager;
use crate::session::{TurnController, TurnStatus};
use crate::tools::ToolRegistry;
use std::collections::HashMap;
use std::path::PathBuf;
#[tokio::test]
async fn completed_is_published_only_after_persistence_succeeds() {
@ -149,4 +197,75 @@ mod tests {
Some("failed to persist turn: database down")
);
}
#[tokio::test]
async fn active_turn_side_effect_does_not_invalidate_its_session_version() {
let dir = tempfile::tempdir().unwrap();
let storage = Arc::new(
crate::storage::Storage::new(&dir.path().join("memory.db"))
.await
.unwrap(),
);
let memory_manager = Arc::new(MemoryManager::new(
storage,
"test".to_string(),
"test".to_string(),
));
let config = LLMProviderConfig {
provider_type: "openai".to_string(),
name: "test".to_string(),
base_url: "http://127.0.0.1".to_string(),
api_key: "test".to_string(),
extra_headers: HashMap::new(),
model_id: "test".to_string(),
temperature: None,
max_tokens: None,
model_extra: HashMap::new(),
max_tool_iterations: 1,
token_limit: 8_192,
workspace_dir: PathBuf::from("."),
input_types: vec!["text".to_string(), "image".to_string()],
};
let session = Arc::new(Mutex::new(
Session::new(
crate::session::UnifiedSessionId::new("cli_chat", "chat", "dialog"),
config,
Arc::new(ToolRegistry::new()),
None,
String::new(),
"test".to_string(),
memory_manager,
)
.await
.unwrap(),
));
let base_version = session.lock().await.state_version_for_test();
session.lock().await.set_active_turn_for_test("turn-1");
append_active_turn_message(
&session,
ChatMessage::assistant("sent screenshot"),
"turn-1".to_string(),
)
.await
.unwrap();
{
let guard = session.lock().await;
assert_eq!(guard.state_version_for_test(), base_version);
assert_eq!(guard.get_history().len(), 1);
}
append_active_turn_message(
&session,
ChatMessage::assistant("late screenshot"),
"stale-turn".to_string(),
)
.await
.unwrap();
let guard = session.lock().await;
assert_eq!(guard.state_version_for_test(), base_version + 1);
assert_eq!(guard.get_history().len(), 2);
}
}

View File

@ -73,6 +73,117 @@ fn committed_turn_delta(
tokio::task_local! {
pub(super) static CURRENT_SOURCE_SESSION: Option<String>;
pub(super) static CURRENT_TURN_ID: Option<String>;
pub(super) static CURRENT_TURN_DELIVERIES: Option<Arc<std::sync::Mutex<Vec<PendingTurnDelivery>>>>;
}
pub(super) struct PendingTurnDelivery {
pub content: String,
pub media: Vec<MediaItem>,
}
fn take_pending_turn_deliveries(
deliveries: &Arc<std::sync::Mutex<Vec<PendingTurnDelivery>>>,
) -> Vec<PendingTurnDelivery> {
std::mem::take(
&mut *deliveries
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner()),
)
}
fn take_current_turn_deliveries() -> Vec<PendingTurnDelivery> {
CURRENT_TURN_DELIVERIES
.try_with(|deliveries| {
deliveries
.as_ref()
.map(take_pending_turn_deliveries)
.unwrap_or_default()
})
.unwrap_or_default()
}
fn collapse_pending_turn_deliveries(pending: Vec<PendingTurnDelivery>) -> (String, Vec<MediaRef>) {
let fallback_content = pending
.iter()
.map(|delivery| delivery.content.trim())
.filter(|content| !content.is_empty())
.collect::<Vec<_>>()
.join("\n\n");
let mut media_refs = Vec::new();
for media_ref in pending
.into_iter()
.flat_map(|delivery| delivery.media.into_iter().map(|media| media.to_media_ref()))
{
if !media_refs.iter().any(|existing: &MediaRef| {
existing.path == media_ref.path && existing.media_type == media_ref.media_type
}) {
media_refs.push(media_ref);
}
}
(fallback_content, media_refs)
}
fn attach_pending_to_message(
message: &mut ChatMessage,
fallback_content: &str,
media_refs: &[MediaRef],
) {
if message.content.trim().is_empty() && !fallback_content.is_empty() {
message.content = fallback_content.to_string();
}
for media_ref in media_refs {
if !message.media_refs.iter().any(|existing| {
existing.path == media_ref.path && existing.media_type == media_ref.media_type
}) {
message.media_refs.push(media_ref.clone());
}
}
}
fn attach_pending_turn_deliveries(
result: &mut crate::agent::AgentProcessResult,
pending: Vec<PendingTurnDelivery>,
) {
let (fallback_content, media_refs) = collapse_pending_turn_deliveries(pending);
if media_refs.is_empty() && fallback_content.is_empty() {
return;
}
attach_pending_to_message(&mut result.final_response, &fallback_content, &media_refs);
if let Some(final_message) = result
.emitted_messages
.iter_mut()
.rev()
.find(|message| message.id == result.final_response.id)
{
final_message
.content
.clone_from(&result.final_response.content);
final_message
.media_refs
.clone_from(&result.final_response.media_refs);
}
}
fn partial_assistant_with_pending_deliveries(
snapshot: &TurnSnapshot,
completion_status: CompletionStatus,
pending: Vec<PendingTurnDelivery>,
) -> Option<ChatMessage> {
let (fallback_content, media_refs) = collapse_pending_turn_deliveries(pending);
let mut message = partial_assistant_message(snapshot, completion_status).or_else(|| {
(!media_refs.is_empty()).then(|| {
let mut message = ChatMessage::assistant("");
message.id = snapshot.message_id.clone();
message.turn_id = Some(snapshot.id.0.clone());
message.completion_status = completion_status;
message
})
})?;
attach_pending_to_message(&mut message, &fallback_content, &media_refs);
Some(message)
}
/// Result of handling a message - either an AI response or a command output
@ -182,7 +293,11 @@ async fn fail_turn_with_partial(
error: String,
) {
let snapshot = controller.snapshot();
let partial = partial_assistant_message(&snapshot, CompletionStatus::Interrupted);
let partial = partial_assistant_with_pending_deliveries(
&snapshot,
CompletionStatus::Interrupted,
take_current_turn_deliveries(),
);
if let Some(partial) = partial {
controller.begin_finalizing();
if let Err(persistence_error) = append_persisted_messages(session, vec![partial]).await {
@ -369,6 +484,60 @@ mod cancelled_partial_tests {
.is_none()
);
}
#[test]
fn pending_same_turn_media_is_attached_to_the_final_response() {
let final_message = ChatMessage::assistant("截图已经准备好了");
let final_id = final_message.id.clone();
let mut result = crate::agent::AgentProcessResult {
final_response: final_message.clone(),
emitted_messages: vec![
ChatMessage::tool("call-1", "send_message", "附件已加入当前回复"),
final_message,
],
total_tokens: None,
usage: None,
};
attach_pending_turn_deliveries(
&mut result,
vec![PendingTurnDelivery {
content: "这是百度首页截图".to_string(),
media: vec![MediaItem::new("/tmp/baidu.png", "image")],
}],
);
assert_eq!(result.final_response.content, "截图已经准备好了");
assert_eq!(result.final_response.media_refs.len(), 1);
assert_eq!(result.final_response.media_refs[0].path, "/tmp/baidu.png");
let committed_final = result
.emitted_messages
.iter()
.find(|message| message.id == final_id)
.unwrap();
assert_eq!(committed_final.media_refs.len(), 1);
assert_eq!(committed_final.media_refs[0].path, "/tmp/baidu.png");
}
#[test]
fn pending_media_survives_a_turn_without_partial_text() {
let (controller, _emitter, _) = TurnController::start("session", "message-id");
let message = partial_assistant_with_pending_deliveries(
&controller.snapshot(),
CompletionStatus::Interrupted,
vec![PendingTurnDelivery {
content: "这是已生成的截图".to_string(),
media: vec![MediaItem::new("/tmp/baidu.png", "image")],
}],
)
.unwrap();
assert_eq!(message.id, "message-id");
assert_eq!(message.content, "这是已生成的截图");
assert_eq!(message.completion_status, CompletionStatus::Interrupted);
assert_eq!(message.media_refs.len(), 1);
}
}
use crate::bus::MessageBus;
use crate::providers::{LLMProvider, create_provider};
@ -710,10 +879,11 @@ impl Session {
self.id.to_string()
}
pub(super) fn add_message_in_memory(
pub(super) fn add_message_in_memory_with_version(
&mut self,
message: ChatMessage,
persist: bool,
advance_state_version: bool,
) -> Option<MessagePersistSnapshot> {
let is_user = message.role == "user";
let now = chrono::Utc::now().timestamp_millis();
@ -768,7 +938,9 @@ impl Session {
self.message_count += 1;
}
self.last_active_at = now;
self.state_version = self.state_version.wrapping_add(1);
if advance_state_version {
self.state_version = self.state_version.wrapping_add(1);
}
persist_snapshot.map(|(storage, session_id, msg_meta)| {
let session_meta = crate::storage::session::SessionMeta {
@ -797,7 +969,11 @@ impl Session {
/// Roll back messages that were appended in memory but whose atomic
/// persistence failed. This is only called while holding the session lock,
/// so the suffix check also protects against removing unrelated messages.
pub(super) fn rollback_message_suffix(&mut self, message_ids: &[String]) {
pub(super) fn rollback_message_suffix_with_version(
&mut self,
message_ids: &[String],
advance_state_version: bool,
) {
if message_ids.is_empty() || self.messages.len() < message_ids.len() {
return;
}
@ -819,7 +995,29 @@ impl Session {
self.seq_counter -= message_ids.len() as i64;
self.total_message_count -= message_ids.len() as i64;
self.message_count -= removed_user_messages;
self.state_version = self.state_version.wrapping_add(1);
if advance_state_version {
self.state_version = self.state_version.wrapping_add(1);
}
}
pub(super) fn owns_active_turn(&self, turn_id: &str) -> bool {
self.active_turn_emitter
.as_ref()
.is_some_and(|active| active.turn_id == turn_id)
}
#[cfg(test)]
pub(super) fn state_version_for_test(&self) -> u64 {
self.state_version
}
#[cfg(test)]
pub(super) fn set_active_turn_for_test(&mut self, turn_id: &str) {
let (_controller, emitter, _) = TurnController::start(self.id.to_string(), "test-message");
self.active_turn_emitter = Some(ActiveTurnEmitter {
turn_id: turn_id.to_string(),
emitter,
});
}
/// 获取消息历史
@ -2767,6 +2965,8 @@ fn spawn_agent_worker(
let commit_delivery = turn_delivery.clone();
let commit_target = turn_target.clone();
let turn_lifecycle = &turn_controller;
let pending_turn_deliveries = Arc::new(std::sync::Mutex::new(Vec::new()));
let scoped_turn_deliveries = pending_turn_deliveries.clone();
let process_future = async move {
let response_session_id = unified_str2.clone();
let process_result = crate::agent::sub_agent::DELEGATE_CONTEXT.scope(
@ -2777,7 +2977,7 @@ fn spawn_agent_worker(
},
agent.process_streaming(history_out.clone(), agent_turn.clone()),
).await;
let result = match process_result {
let mut result = match process_result {
Ok(r) => r,
Err(AgentError::LlmError(ref msg))
if is_context_overflow_error(msg) =>
@ -2921,6 +3121,8 @@ fn spawn_agent_worker(
}
};
let pending = take_current_turn_deliveries();
attach_pending_turn_deliveries(&mut result, pending);
let response_content = result.final_response.content;
let total_tokens = result.total_tokens;
let usage = result.usage;
@ -3005,15 +3207,20 @@ fn spawn_agent_worker(
let _ = bus2.publish_outbound(outbound).await;
}
};
let process_future =
CURRENT_TURN_ID.scope(Some(active_turn_id.clone()), process_future);
let process_future = CURRENT_TURN_DELIVERIES
.scope(Some(scoped_turn_deliveries), process_future);
tokio::select! {
() = process_future => {}
_ = cancel_rx => {
// cancelled — current_cancel already taken by /stop
let snapshot = turn_controller.snapshot();
if let Some(partial) = partial_assistant_message(
if let Some(partial) = partial_assistant_with_pending_deliveries(
&snapshot,
CompletionStatus::Cancelled,
take_pending_turn_deliveries(&pending_turn_deliveries),
) {
turn_controller.begin_finalizing();
match append_persisted_messages(&session, vec![partial]).await {

View File

@ -41,7 +41,7 @@ pub use pty::{PtyManager, PtyTool};
pub use registry::ToolRegistry;
pub use send_message::SendMessageTool;
pub use todo::TodoTool;
pub use traits::{OutboundMessenger, Tool, ToolResult, ToolResultWithMedia};
pub use traits::{OutboundDelivery, OutboundMessenger, Tool, ToolResult, ToolResultWithMedia};
pub use web_fetch::WebFetchTool;
use crate::agent::SubAgentManager;

View File

@ -6,7 +6,7 @@ use mime_guess::mime;
use crate::bus::{MediaItem, MessageSource, SourceKind};
use super::traits::{OutboundMessenger, Tool, ToolResult};
use super::traits::{OutboundDelivery, OutboundMessenger, Tool, ToolResult};
pub struct SendMessageTool {
messenger: Arc<dyn OutboundMessenger>,
@ -144,11 +144,16 @@ target_chat_id 支持两种格式:<channel>:<chat_id>(发送到该聊天下
.send_message(channel, chat_id, dialog_id, content, source, media)
.await
{
Ok(()) => Ok(ToolResult {
Ok(OutboundDelivery::Delivered) => Ok(ToolResult {
success: true,
output: "消息已发送".to_string(),
error: None,
}),
Ok(OutboundDelivery::AttachedToCurrentTurn) => Ok(ToolResult {
success: true,
output: "附件已加入当前回复".to_string(),
error: None,
}),
Err(e) => Ok(ToolResult {
success: false,
output: String::new(),

View File

@ -16,6 +16,12 @@ pub struct ToolResultWithMedia {
pub media_refs: Vec<MediaRef>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum OutboundDelivery {
Delivered,
AttachedToCurrentTurn,
}
impl From<ToolResult> for ToolResultWithMedia {
fn from(result: ToolResult) -> Self {
Self {
@ -67,5 +73,5 @@ pub trait OutboundMessenger: Send + Sync {
content: &str,
source: MessageSource,
media: Vec<MediaItem>,
) -> Result<(), String>;
) -> Result<OutboundDelivery, String>;
}

View File

@ -1,12 +1,12 @@
{
"name": "picobot-webui",
"version": "1.2.0",
"version": "1.2.2",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "picobot-webui",
"version": "1.2.0",
"version": "1.2.2",
"dependencies": {
"bits-ui": "^2.0.0",
"dompurify": "^3.4.12",

View File

@ -1,7 +1,7 @@
{
"name": "picobot-webui",
"private": true,
"version": "1.2.0",
"version": "1.2.2",
"type": "module",
"engines": {
"node": ">=20"

View File

@ -250,7 +250,7 @@
}
function canPreview(attachment) {
return ["image/png", "image/jpeg", "image/gif", "image/webp"].includes(attachment.mime_type);
return ["image/png", "image/jpeg", "image/gif", "image/webp", "image/bmp", "image/avif", "image/x-icon"].includes(attachment.mime_type);
}
function addFiles(files) {
@ -439,13 +439,20 @@
{#if message.attachments?.length}
<div class="message-attachments">
{#each message.attachments as attachment (`${message.id}:${attachment.index}`)}
<article class="attachment-card">
{#if canPreview(attachment)}
<img src={attachmentUrl(message, attachment, true)} alt={attachment.name} />
{:else}<span class="attachment-icon"></span>{/if}
<div><strong>{attachment.name}</strong><small>{attachment.mime_type || attachment.media_type}</small></div>
<a href={attachmentUrl(message, attachment)} download={attachment.name} aria-label={`下载 ${attachment.name}`}>↓</a>
</article>
{#if canPreview(attachment)}
<article class="image-attachment">
<a class="image-preview-link" href={attachmentUrl(message, attachment)} target="_blank" rel="noreferrer" aria-label={`查看原图 ${attachment.name}`}>
<img class="image-preview" src={attachmentUrl(message, attachment, true)} alt={attachment.name} />
</a>
<div class="attachment-meta"><div><strong>{attachment.name}</strong><small>{attachment.mime_type || attachment.media_type}</small></div><a href={attachmentUrl(message, attachment)} download={attachment.name} aria-label={`下载 ${attachment.name}`}>↓</a></div>
</article>
{:else}
<article class="attachment-card">
<span class="attachment-icon"></span>
<div><strong>{attachment.name}</strong><small>{attachment.mime_type || attachment.media_type}</small></div>
<a href={attachmentUrl(message, attachment)} download={attachment.name} aria-label={`下载 ${attachment.name}`}>↓</a>
</article>
{/if}
{/each}
</div>
{/if}

View File

@ -145,10 +145,17 @@ main { min-width: 0; height: 100vh; display: flex; flex-direction: column; }
.message.user .bubble { background: var(--user-bubble); border-color: var(--accent-border); }
.message-attachments { display: grid; gap: 7px; width: min(100%, 520px); }
.attachment-card { display: grid; grid-template-columns: 42px minmax(0, 1fr) 30px; gap: 9px; align-items: center; padding: 8px; border: 1px solid var(--line); border-radius: 10px; background: var(--panel); }
.attachment-card img, .attachment-icon { width: 42px; height: 42px; border-radius: 7px; object-fit: cover; background: var(--panel-2); display: grid; place-items: center; color: var(--accent); }
.attachment-icon { width: 42px; height: 42px; border-radius: 7px; background: var(--panel-2); display: grid; place-items: center; color: var(--accent); }
.attachment-card strong, .attachment-card small { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.attachment-card strong { font-size: 12px; }.attachment-card small { margin-top: 3px; color: var(--muted); font-size: 9px; }
.attachment-card a { width: 28px; height: 28px; display: grid; place-items: center; border: 1px solid var(--line); border-radius: 7px; color: var(--accent); text-decoration: none; }
.image-attachment { width: min(100%, 680px); overflow: hidden; border: 1px solid var(--line); border-radius: 12px; background: var(--panel); }
.image-preview-link { display: grid; max-height: 520px; overflow: hidden; place-items: center; background: var(--panel-2); }
.image-preview { display: block; max-width: 100%; max-height: 520px; object-fit: contain; }
.attachment-meta { display: grid; grid-template-columns: minmax(0, 1fr) 30px; gap: 9px; align-items: center; padding: 8px 10px; }
.attachment-meta strong, .attachment-meta small { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.attachment-meta strong { font-size: 12px; }.attachment-meta small { margin-top: 3px; color: var(--muted); font-size: 9px; }
.attachment-meta > a { width: 28px; height: 28px; display: grid; place-items: center; border: 1px solid var(--line); border-radius: 7px; color: var(--accent); text-decoration: none; }
.typing .bubble { color: var(--muted); }
.active-turn .message-content { width: min(82%, 760px); }
.streaming { border-color: var(--accent-border); }