feat(agent): 支持持久化 /btw 旁路提问
This commit is contained in:
1 parent
8dae851b9b
commit
555539bc8e
34 files changed
+3744
-314
No files matched your search
@@ -0,0 +1,66 @@
|
||||
# ARTEX `/btw`
|
||||
|
||||
普通聊天、任务 MainAgent 和当前任务自己的 Worker 支持独立旁路提问。主输入框输入 `/btw 问题` 即可提交,空 `/btw` 或“旁路提问”按钮打开历史。桌面使用可调整宽度的侧栏,移动端使用 Drawer。
|
||||
|
||||
旁路回答根据提交时的 Agent 上下文快照生成,支持流式显示、追问、停止和清空。关闭面板、刷新页面或断开 SSE 都不会取消模型请求。停止只影响当前旁路;清空会取消旁路并删除旁路历史,同时保留主上下文快照。
|
||||
|
||||
## 实现边界
|
||||
|
||||
沿用 Go、norma v0.3.6、Next.js、现有 Markdown / ResizablePanel / Drawer / AlertDialog 组件;没有修改 norma 或添加依赖。Planner、继承自其他任务的 Worker、工具型子任务升级不在本次范围内。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[主 Agent QueryDeps] --> B[实际 Provider 绑定]
|
||||
B --> C[不可变结构化快照]
|
||||
B --> D[主 Agent 正常工具循环]
|
||||
C --> E[(PostgreSQL 最新快照)]
|
||||
E --> F[快照 + 最近成功旁路问答 + 问题]
|
||||
F --> G[SideQuestionService 单次 Provider 请求]
|
||||
G --> H[(独立旁路历史和用量)]
|
||||
H --> I[累计回答 SSE / 旁路面板]
|
||||
```
|
||||
|
||||
- `capture.go` 只在 `Options.Deps.CallModel / CallModelSync` 标记主循环请求。Provider 装饰器位于具体模型内部、路由池外层选择之后,因此记录实际选中的模型;压缩和摘要请求不覆盖快照。
|
||||
- 请求开始、完整模型回复、运行终态发布快照。正在生成的半段回复不发布;工具调用通过 norma 的 `MessagesForAPI` 保持配对,工具结果在下一次主模型请求或运行终态进入快照。流式中止保留上一个有效边界。
|
||||
- 快照通过 JSON 深拷贝保留结构化消息、系统提示、工具定义及生成参数。模型推理不持有快照锁或数据库事务。
|
||||
- `SideQuestionService` 直接调用该具体 Provider 一次,不创建 agent session,不接入工具执行器、主 transcript、活动流或任务图,也不经过任务模型切换链。保留工具定义是为兼容既有结构化工具上下文;新返回的工具调用没有执行路径。仅返回工具调用时给出无法执行的提示。
|
||||
- 每个父会话一个运行请求,单个服务进程最多四个,单次超时 120 秒。旁路使用服务生命周期下的独立取消上下文。
|
||||
- 旁路请求保留模型配置引用和非敏感身份摘要;请求时从现有配置取得凭据。配置被删除,或模型、协议、地址等身份字段变化,要求先运行主 Agent 更新快照。测试不会改变产品默认模型。
|
||||
|
||||
## 持久化和恢复
|
||||
|
||||
`db/schema.sql` 自动创建 `side_question_sessions` 和 `side_question_requests`。前者保存父资源、最新快照、运行编号、版本和清理版本;后者保存问题、累计回答、状态、模型、快照时间、用量、事件序号和分页序号。
|
||||
|
||||
父会话键使用 conversation ID,或 task ID + exploration ID + intent ID。Worker 不使用可复用的执行槽位命名。
|
||||
|
||||
快照按父会话合并写入,每 250 ms 刷新一次,数据库比较 `(run_id, version)` 防止旧版本覆盖。旁路提交前再次保存选定快照。成功保存后释放内存中的大快照;失败时保留待写版本。回答累计内容在流式事件到来时最多每 250 ms 写入一次,终态立即保存并在数据库错误时有限重试。
|
||||
|
||||
服务启动将遗留 `running` 请求标记为 `interrupted`,保留已落库的部分回答和用量,不自动重放请求。最近成功保存的上下文可直接用于下一次提问。旧会话没有快照时要求先运行主 Agent,不从 UI 活动记录重建上下文。
|
||||
|
||||
清空操作递增清理版本并删除请求;条件更新阻止迟到的回调重新写回。物理父资源删除依靠外键级联,Worker 逻辑删除在同一事务中删除旁路数据,并拒绝后续迟到快照。任务归档先阻止新请求、等待主流程停止、取消并等待旁路落库;归档格式为 v3,同时兼容不含旁路表的 v1/v2。
|
||||
|
||||
历史完整保存,按序号游标每页最多返回 20 条。模型请求仅回放最近 20 组成功问答;保守估算请求长度并预留输出预算,优先移除旧旁路问答,基础上下文仍超限时返回明确错误。
|
||||
|
||||
## HTTP 契约
|
||||
|
||||
以下路径作为 `{parent}`,沿用现有认证及资源校验:
|
||||
|
||||
- `/api/conversations/{id}`
|
||||
- `/api/tasks/{id}/chat`
|
||||
- `/api/tasks/{id}/intents/{iid}`
|
||||
|
||||
| 请求 | 返回及行为 |
|
||||
| --- | --- |
|
||||
| `GET {parent}/side-questions?before={ordinal}` | `items` 按新到旧排列、独立 `current` 运行状态、`snapshot` 元信息、`next_cursor`;游标为 0 表示最新页 / 无下一页 |
|
||||
| `POST {parent}/side-questions` | JSON `{ "question": "…", "client_request_id": "UUID" }`;新请求返回 202 及请求对象;相同 ID 和问题返回既有对象 200 |
|
||||
| `DELETE {parent}/side-questions` | 取消并清空当前父会话的旁路问答 |
|
||||
| `GET /api/side-questions/{requestID}/events` | `snapshot` SSE 事件,`id` 为递增序号,`data` 为完整累计请求对象;清空时发送 `cleared` |
|
||||
| `POST /api/side-questions/{requestID}/cancel` | 显式取消;终态可从历史或 SSE 读取 |
|
||||
|
||||
问题上限 4000 字符。无快照、模型配置变化、同一父会话忙或幂等 ID 冲突返回 409;全局并发上限返回 429。每次 SSE 连接都先发送累计状态,不依赖客户端之前收到的文本片段。前端按请求 ID + 序号合并,并在切换父会话、清空时废弃旧回调。
|
||||
|
||||
## 验证与参考
|
||||
|
||||
自动化检查、实际模型使用及已知限制见 [VALIDATION.md](VALIDATION.md)。
|
||||
|
||||
独立请求参考 [Grok CLI side-question.ts(固定提交)](https://github.com/superagent-ai/grok-cli/blob/fb97af83f06dca873281d60168430f06c8de6324/src/utils/side-question.ts),运行隔离参考 [OpenCode(固定提交)](https://github.com/anomalyco/opencode/tree/b3f1a96c6dd7adeb28b36dd11add1998fc84d67b)。ARTEX 的上下文使用 norma 的结构化消息,未采用从前端日志拼接文本的方式。
|
||||
@@ -0,0 +1,93 @@
|
||||
# `/btw` 验证记录
|
||||
|
||||
日期:2026-09-10。分支:`codex/btw-side-question`。基线:`8dae851b9b622f2ff2631f332fde9719d0b16fba`。
|
||||
|
||||
使用独立 PostgreSQL 测试库和数据目录;真实模型凭据只注入独立测试环境,没有写入代码或此记录,也没有修改产品默认模型。Go 1.26.3、norma v0.3.6、Next.js 16.2.9。
|
||||
|
||||
实际模型对话、返回对象、工程断言及 Qwen 原始审查文本保存在 [validation-2026-09-10.json](validation-2026-09-10.json),其中没有 API 凭据。
|
||||
|
||||
## 工程检查
|
||||
|
||||
| 范围 | 结果 | 证据 |
|
||||
| --- | --- | --- |
|
||||
| 结构化消息、工具参数深拷贝 | 通过 | `TestCheckpointDeepCopyAndBoundaries` |
|
||||
| 摘要 / 压缩请求不覆盖、完整回复和终态发布、半段回复排除 | 通过 | `TestCheckpointDeepCopyAndBoundaries`、`TestSnapshotExcludesPartialStreamAndSelectsPoolMember` |
|
||||
| 实际模型池成员身份 | 通过 | `TestSnapshotExcludesPartialStreamAndSelectsPoolMember` |
|
||||
| 工具配对、20 组回放、预算裁剪与超限错误 | 通过 | `TestBuildRequestCompactionToolPairingAndBudget` |
|
||||
| 主旁路并行、双向取消隔离 | 通过 | 阻塞式 Provider,`TestMainSideConcurrencyAndIndependentCancellation` |
|
||||
| 无工具执行、流式 / 非流式、失败时已有用量 | 通过 | `TestServiceNoToolsAndUsageOnFailure` |
|
||||
| 真实 norma ChatAgent + 本地 Read 工具、主 transcript / 活动隔离 | 通过 | `TestSideActualChatCheckpointToolResultAndTranscriptIsolation`,流式和非流式子用例 |
|
||||
| 持久化、分页、幂等、重启保留部分回答 | 通过 | `TestSideHistoryIdempotencyPagingAndRecovery` |
|
||||
| 清空与迟到写入竞争、父资源删除、版本比较 | 通过 | `TestSideClearLateWritersAndDeletedParent` |
|
||||
| MainAgent / Worker 归档与恢复,v1/v2/v3 | 通过 | `TestSideTaskArchiveVersions` |
|
||||
| 三种父接口、认证、资源归属、Worker 逻辑删除 | 通过 | `TestSideHTTPGlobalLimitTaskWorkerAndDeletion`、`TestSideCheckpointPersistsBeforeAdmissionAndRestart` |
|
||||
| 忙碌主会话可旁路、独立 SSE 重连 / 断开、取消、清空 | 通过 | `TestSideHTTPBusyIsolationClearAndReconnect` |
|
||||
| 每父会话 1 / 全局 4 并发 | 通过 | 两个 `TestSideHTTP…` 用例 |
|
||||
| 提交前快照落库、重启续问、旧会话不可伪造快照 | 通过 | `TestSideCheckpointPersistsBeforeAdmissionAndRestart` |
|
||||
| 缓存中的配置被删除或模型改变后拒绝继续 | 通过 | `TestSideRejectsDeletedOrChangedCachedProfile` |
|
||||
| 归档前取消并等待最终回答和用量落库 | 通过 | `TestSideTaskDrainPersistsBeforeArchive` |
|
||||
| 流式消费者提前取消时只记一次用量及旁路归属 | 通过 | `TestSideUsageRecordedOnceOnConsumerCancellation` |
|
||||
| 重启自动恢复的 Worker / deadline 运行上下文继续发布新快照 | 通过 | `TestSideRestoredWorkerRuntimePublishesNewCheckpoint` |
|
||||
| 相关包 race 检查 | 通过 | 下列命令 |
|
||||
| TypeScript 与生产构建 | 通过 | `npx tsc --noEmit`、`npm run build` |
|
||||
| 新增前端模块 Biome | 通过 | `biome check`,3 个新增模块 |
|
||||
|
||||
在单独的可丢弃数据库中配置 `ARTEX_PG_DSN` 后,可以复现自动化检查(不要指向生产库):
|
||||
|
||||
```sh
|
||||
go test -race ./agent ./db ./server ./sidequestion ./llmrec ./llmpool \
|
||||
-run 'Test(Side|Checkpoint|Snapshot|BuildRequest|Service|MainSide|CaptureRun|TaskArchive|CompleteForwards|StopIntent|CancelIntent)' -count=1
|
||||
cd web
|
||||
npx tsc --noEmit
|
||||
npx biome check src/lib/side-questions.ts src/hooks/use-side-questions.ts src/components/side-question-workspace.tsx
|
||||
npm run build
|
||||
```
|
||||
|
||||
全量 Go 回归不是全绿:`server` 包有两个既有测试在临时目录清理阶段失败,均报 `TempDir RemoveAll … directory not empty`:
|
||||
|
||||
- `TestInheritedActivityDetailAndRelationDeletion`
|
||||
- `TestTaskMetadataPatchReturnsRenameAndPin`
|
||||
|
||||
从上述未修改基线导出源码后,在相同隔离环境重跑 `server` 包,也复现这两个清理失败。基线运行另出现 `TestCoreTaskLifecyclePG` 的目标节点数量断言失败;最终修改后的 `server` 回归没有该断言失败。其他包通过,本次旁路相关用例及 race 检查通过。没有将基线问题标为本次验收通过,也没有为隐藏问题修改既有断言。
|
||||
|
||||
Next.js 构建输出已有的多 lockfile / workspace root 推断警告;构建完成且所有页面生成成功。
|
||||
|
||||
## 浏览器检查
|
||||
|
||||
使用 Codex In-app Browser,连接独立本地 Go 服务和 Next.js 开发服务器。桌面与 390 × 844 窄屏完成以下人工自动化操作,检查截图和浏览器日志:
|
||||
|
||||
- 普通聊天运行期间输入 `/btw`,主内容和旁路同时显示;桌面侧栏正常。
|
||||
- 连续追问;旁路停止后保留已生成部分;主流程继续。
|
||||
- 关闭面板时请求继续,重开后恢复完成的回答;刷新页面后空 `/btw` 恢复历史。
|
||||
- 窄屏 Drawer 的输入、按钮、历史和关闭操作正常,无横向溢出。
|
||||
- 清空使用确认弹窗,清空后历史消失,主 transcript 和快照保留。
|
||||
- 任务 MainAgent 与两个 Worker 分别提问并切换,Agent 标签和历史未串话。
|
||||
- 阻塞式本地模型夹具保持 Worker 运行;从 Worker 主输入框提交 `/btw`,停止旁路后 Worker 仍显示实时运行和自己的暂停按钮,旁路保存部分回答。
|
||||
- 浏览器错误 / 警告日志为空。
|
||||
|
||||
可控夹具用于精确验证并发时序,不依赖真实模型的输出速度。调试期间两次 Worker 运行时检查未形成有效并发窗口(任务已结束 / 回答提前结束),修正夹具后重做并通过;不将这些初始操作记作有效通过。
|
||||
|
||||
## 真实模型对话
|
||||
|
||||
优先探测 `grok-4.6`,OpenAI 兼容接口 `http://127.0.0.1:12580/tingly/openai`。探测 HTTP 200,返回模型名 `grok-4.6` 和 `READY`,耗时 2.82 秒。首选可用,因此没有启用 Tingly `glm` 或智谱 `glm-5.3` 备用链;这两个备用服务本次没有验证。
|
||||
|
||||
| 场景 | 实际结果 |
|
||||
| --- | --- |
|
||||
| 主会话运行期间询问资产、目标、标记 | 返回 `redhaze.top`、首页读取和总结目标、`BTW-REAL-0910`;旁路完成,16.97 秒 |
|
||||
| 主会话完成首页读取后询问工具依据 | 正确引用 WebFetch 200、curl 跳转 301 → 302 → 200、页面标题;7.24 秒 |
|
||||
| 旁路要求 Bash 创建测试文件 | 拒绝执行,目标文件未创建;7.74 秒 |
|
||||
| 完成后的旁路不改变主上下文 | 主 transcript SHA-256 与主活动记录保持一致;旁路工具执行次数为 0 |
|
||||
| 真正停止 / 重启 Go 服务后续问 | 保留先前 3 条旁路历史,直接从持久化快照回答资产、标记和标题,未重跑主 Agent |
|
||||
| 新会话使用 Grok 非流式配置 | 正确回答资产和 `ATOMIC-0910`;返回并保存用量:input 11734、output 138、cache_read 11520 |
|
||||
|
||||
资产案例的主会话使用 WebFetch 和 Bash/curl 读取公开首页,落地页为 `https://id.redhaze.top/home`,标题为“红幕科技 RedHaze Group · 全球综合集团门户”。Bash 把响应暂存于本地测试文件;未向远端执行写入。该事实与“旁路没有执行工具”分开核验。
|
||||
|
||||
主 transcript 校验值:`e7e61f135a4a120954b539f357e8c4205d7d5cd7460dcaf3dc0fd066463e1d00`。
|
||||
|
||||
**用量限制:** Tingly 的 Grok 流式响应没有返回 usage。另行直接发送 `stream_options.include_usage=true` 验证,HTTP 200、12 个数据帧、0 个 usage 帧。因此流式测试中的 0 表示端点没有提供用量,不能解释为没有计费。非流式用量以及夹具的失败 / 取消用量都正确保存。
|
||||
|
||||
## Qwen 审查
|
||||
|
||||
审查模型 `qwen-flash`,OpenAI 兼容接口 `https://dashscope.aliyuncs.com/compatible-mode/v1`,HTTP 200。提供了前三项真实旁路对话、主会话工具依据及工程断言;返回 `verdict: accept`、`concerns: []`,认为回答与资产、标记、页面读取证据一致,旁路工具拒绝符合约束。审查用量:prompt 6625、completion 312、total 6937。
|
||||
|
||||
这次 Qwen 审查范围不包含后来追加的服务重启和非流式测试。Qwen 对“无写入”的概括过宽:主会话 curl 确实创建了本地响应临时文件,上文已明确记录。并发、零工具执行和 transcript 隔离由工程断言判断,模型审查只辅助评估答案质量。
|
||||
@@ -0,0 +1,197 @@
|
||||
// Package sidequestion captures immutable main-agent checkpoints. It never owns
|
||||
// an agent session or a tool executor.
|
||||
package sidequestion
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"iter"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/norma/harness"
|
||||
"github.com/Autumn-27/norma/llm"
|
||||
)
|
||||
|
||||
type Parent struct {
|
||||
ConversationID int64 `json:"conversation_id,omitempty"`
|
||||
TaskID int64 `json:"task_id,omitempty"`
|
||||
ExplorationID int64 `json:"exploration_id,omitempty"`
|
||||
IntentID int64 `json:"intent_id,omitempty"`
|
||||
}
|
||||
|
||||
func (p Parent) Key() string {
|
||||
if p.ConversationID > 0 {
|
||||
return fmt.Sprintf("conv-%d", p.ConversationID)
|
||||
}
|
||||
if p.IntentID > 0 {
|
||||
return fmt.Sprintf("task-%d-exp-%d-worker-i%d", p.TaskID, p.ExplorationID, p.IntentID)
|
||||
}
|
||||
return fmt.Sprintf("task-%d-exp-%d-main", p.TaskID, p.ExplorationID)
|
||||
}
|
||||
|
||||
// Model contains only configuration identity, never credentials or proxy URLs.
|
||||
type Model struct {
|
||||
ProfileID int64 `json:"profile_id"`
|
||||
Name string `json:"name"`
|
||||
Format string `json:"format"`
|
||||
Model string `json:"model"`
|
||||
Identity string `json:"identity"`
|
||||
Streaming bool `json:"streaming"`
|
||||
WindowTokens int `json:"window_tokens"`
|
||||
}
|
||||
|
||||
type Snapshot struct {
|
||||
Parent Parent `json:"parent"`
|
||||
RunID int64 `json:"run_id"`
|
||||
Version int64 `json:"version"`
|
||||
CapturedAt time.Time `json:"captured_at"`
|
||||
Model Model `json:"model"`
|
||||
Request llm.CompletionRequest `json:"request"`
|
||||
}
|
||||
|
||||
func CloneRequest(req llm.CompletionRequest) (llm.CompletionRequest, error) {
|
||||
b, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
return llm.CompletionRequest{}, err
|
||||
}
|
||||
var out llm.CompletionRequest
|
||||
err = json.Unmarshal(b, &out)
|
||||
return out, err
|
||||
}
|
||||
|
||||
type Publisher func(Snapshot)
|
||||
type publisherKey struct{}
|
||||
type captureKey struct{}
|
||||
type attemptKey struct{}
|
||||
|
||||
func WithPublisher(ctx context.Context, publish Publisher) context.Context {
|
||||
return context.WithValue(ctx, publisherKey{}, publish)
|
||||
}
|
||||
|
||||
type Capture struct {
|
||||
mu sync.Mutex
|
||||
parent Parent
|
||||
runID int64
|
||||
version int64
|
||||
publish Publisher
|
||||
last *Snapshot
|
||||
}
|
||||
|
||||
// Attach uses QueryDeps rather than a global provider hook so compaction and
|
||||
// other auxiliary completions cannot replace the main conversation checkpoint.
|
||||
func Attach(ctx context.Context, parent Parent, deps harness.QueryDeps, provider llm.Provider) (context.Context, harness.QueryDeps) {
|
||||
publish, _ := ctx.Value(publisherKey{}).(Publisher)
|
||||
if publish == nil {
|
||||
return ctx, deps
|
||||
}
|
||||
c := &Capture{parent: parent, runID: time.Now().UnixNano(), publish: publish}
|
||||
stream, complete := deps.CallModel, deps.CallModelSync
|
||||
if stream == nil {
|
||||
stream = provider.Stream
|
||||
}
|
||||
if complete == nil {
|
||||
complete = provider.Complete
|
||||
}
|
||||
deps.CallModel = func(ctx context.Context, req llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return stream(context.WithValue(ctx, attemptKey{}, c), req)
|
||||
}
|
||||
deps.CallModelSync = func(ctx context.Context, req llm.CompletionRequest) (llm.Message, string, llm.Usage, error) {
|
||||
return complete(context.WithValue(ctx, attemptKey{}, c), req)
|
||||
}
|
||||
return context.WithValue(ctx, captureKey{}, c), deps
|
||||
}
|
||||
|
||||
func (c *Capture) save(req llm.CompletionRequest, model Model) {
|
||||
copy, err := CloneRequest(req)
|
||||
if err != nil {
|
||||
return
|
||||
} // Observability must not break the main run.
|
||||
c.mu.Lock()
|
||||
c.version++
|
||||
s := Snapshot{Parent: c.parent, RunID: c.runID, Version: c.version, CapturedAt: time.Now().UTC(), Model: model, Request: copy}
|
||||
c.last = &s
|
||||
c.mu.Unlock()
|
||||
c.publish(s)
|
||||
}
|
||||
|
||||
// Finish adds tool results that landed after the final model request. Norma's
|
||||
// first API message is its host reminder, absent from Terminal.Messages.
|
||||
func Finish(ctx context.Context, messages []llm.Message) {
|
||||
c, _ := ctx.Value(captureKey{}).(*Capture)
|
||||
if c == nil {
|
||||
return
|
||||
}
|
||||
c.mu.Lock()
|
||||
s := c.last
|
||||
c.mu.Unlock()
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
req := s.Request
|
||||
var prefix []llm.Message
|
||||
if len(req.Messages) > 0 && strings.HasPrefix(req.Messages[0].Text(), "<system-reminder>") {
|
||||
prefix = append(prefix, req.Messages[0])
|
||||
}
|
||||
req.Messages = append(prefix, llm.MessagesForAPI(messages)...)
|
||||
c.save(req, s.Model)
|
||||
}
|
||||
|
||||
type boundProvider struct {
|
||||
inner llm.Provider
|
||||
model Model
|
||||
}
|
||||
|
||||
// Bind belongs immediately around a concrete provider, inside any routing pool.
|
||||
func Bind(inner llm.Provider, model Model) llm.Provider { return &boundProvider{inner, model} }
|
||||
|
||||
func (p *boundProvider) Stream(ctx context.Context, req llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return func(yield func(llm.StreamEvent, error) bool) {
|
||||
c, _ := ctx.Value(attemptKey{}).(*Capture)
|
||||
if c == nil {
|
||||
for ev, err := range p.inner.Stream(ctx, req) {
|
||||
if !yield(ev, err) {
|
||||
return
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
c.save(req, p.model)
|
||||
acc := llm.NewAccumulator()
|
||||
failed := false
|
||||
complete := false
|
||||
for ev, err := range p.inner.Stream(ctx, req) {
|
||||
if err != nil {
|
||||
failed = true
|
||||
} else {
|
||||
acc.Add(ev)
|
||||
if ev.Type == llm.SEMessageStop {
|
||||
complete = true
|
||||
}
|
||||
}
|
||||
if !yield(ev, err) {
|
||||
return
|
||||
}
|
||||
}
|
||||
if !failed && complete && ctx.Err() == nil {
|
||||
msg := acc.Message()
|
||||
req.Messages = llm.MessagesForAPI(append(append([]llm.Message{}, req.Messages...), msg))
|
||||
c.save(req, p.model)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *boundProvider) Complete(ctx context.Context, req llm.CompletionRequest) (llm.Message, string, llm.Usage, error) {
|
||||
c, _ := ctx.Value(attemptKey{}).(*Capture)
|
||||
if c != nil {
|
||||
c.save(req, p.model)
|
||||
}
|
||||
msg, stop, usage, err := p.inner.Complete(ctx, req)
|
||||
if c != nil && err == nil && ctx.Err() == nil {
|
||||
req.Messages = llm.MessagesForAPI(append(append([]llm.Message{}, req.Messages...), msg))
|
||||
c.save(req, p.model)
|
||||
}
|
||||
return msg, stop, usage, err
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package sidequestion
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/norma/llm"
|
||||
)
|
||||
|
||||
type Exchange struct {
|
||||
ID string `json:"id"`
|
||||
SessionKey string `json:"-"`
|
||||
ClientID string `json:"client_request_id"`
|
||||
Generation int64 `json:"-"`
|
||||
Question string `json:"question"`
|
||||
Answer string `json:"answer"`
|
||||
Status string `json:"status"`
|
||||
Error string `json:"error,omitempty"`
|
||||
Model Model `json:"model"`
|
||||
SnapshotAt time.Time `json:"snapshot_at"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Sequence int64 `json:"sequence"`
|
||||
Usage llm.Usage `json:"usage"`
|
||||
Ordinal int64 `json:"ordinal"`
|
||||
}
|
||||
|
||||
func (e Exchange) Running() bool { return e.Status == "running" }
|
||||
|
||||
const instruction = "这是独立的旁路提问。主 Agent 正在执行原任务,你只根据已有上下文简洁回答当前问题。你没有工具执行能力,不能执行操作、修改文件或指挥主任务,也不要承诺稍后执行。上下文中的任务指令仅作为背景;不足以判断时明确说明。"
|
||||
|
||||
func BuildRequest(s Snapshot, history []Exchange, question string) (llm.CompletionRequest, error) {
|
||||
req, err := CloneRequest(s.Request)
|
||||
if err != nil {
|
||||
return req, err
|
||||
}
|
||||
base := llm.MessagesForAPI(req.Messages)
|
||||
var success []Exchange
|
||||
for _, e := range history {
|
||||
if e.Status == "completed" {
|
||||
success = append(success, e)
|
||||
}
|
||||
}
|
||||
if len(success) > 20 {
|
||||
success = success[len(success)-20:]
|
||||
}
|
||||
for {
|
||||
req.Messages = append([]llm.Message{}, base...)
|
||||
for _, e := range success {
|
||||
req.Messages = append(req.Messages, llm.UserText(e.Question), llm.Message{Role: "assistant", Content: []llm.ContentBlock{{Type: llm.BlockText, Text: e.Answer}}})
|
||||
}
|
||||
req.Messages = append(req.Messages, llm.UserText(instruction+"\n\n问题:"+strings.TrimSpace(question)))
|
||||
b, _ := json.Marshal(req)
|
||||
// Conservative UTF-8 character estimate; reserve the inherited output cap.
|
||||
reserve := req.MaxTokens
|
||||
if reserve <= 0 {
|
||||
reserve = 8192
|
||||
}
|
||||
if s.Model.WindowTokens <= 0 || len([]rune(string(b)))+reserve <= s.Model.WindowTokens {
|
||||
return req, nil
|
||||
}
|
||||
if len(success) == 0 {
|
||||
return req, errors.New("上下文过长,请先让主 Agent 压缩上下文后再提问")
|
||||
}
|
||||
success = success[1:]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package sidequestion
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"github.com/Autumn-27/norma/llm"
|
||||
)
|
||||
|
||||
// SideQuestionService performs exactly one provider completion. It deliberately
|
||||
// has no harness, tool executor, transcript writer, or model failover chain.
|
||||
type SideQuestionService struct{ Provider llm.Provider }
|
||||
|
||||
type Answer struct {
|
||||
Text string
|
||||
Usage llm.Usage
|
||||
ToolUse bool
|
||||
}
|
||||
|
||||
func (s SideQuestionService) Answer(ctx context.Context, req llm.CompletionRequest, streaming bool, update func(Answer)) (out Answer, err error) {
|
||||
if streaming {
|
||||
complete := false
|
||||
for ev, streamErr := range s.Provider.Stream(ctx, req) {
|
||||
if streamErr != nil {
|
||||
err = streamErr
|
||||
break
|
||||
}
|
||||
switch ev.Type {
|
||||
case llm.SETextDelta:
|
||||
out.Text += ev.Text
|
||||
case llm.SEToolUseStart:
|
||||
out.ToolUse = true
|
||||
case llm.SEMessageStart, llm.SEMessageDelta:
|
||||
out.Usage.Add(ev.Usage)
|
||||
case llm.SEMessageStop:
|
||||
complete = true
|
||||
}
|
||||
if update != nil {
|
||||
update(out)
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
err = ctx.Err()
|
||||
break
|
||||
}
|
||||
}
|
||||
if err == nil && !complete {
|
||||
err = errors.New("模型响应中断,请重新提问")
|
||||
}
|
||||
} else {
|
||||
var msg llm.Message
|
||||
msg, _, out.Usage, err = s.Provider.Complete(ctx, req)
|
||||
out.Text, out.ToolUse = msg.Text(), len(msg.ToolUses()) > 0
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
err = ctx.Err()
|
||||
}
|
||||
if err == nil && strings.TrimSpace(out.Text) == "" {
|
||||
if out.ToolUse {
|
||||
out.Text = "当前旁路提问不能执行工具操作,请在主会话中发出操作请求。"
|
||||
} else {
|
||||
err = errors.New("模型没有返回回答")
|
||||
}
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
package sidequestion
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"iter"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Autumn-27/artex/llmpool"
|
||||
"github.com/Autumn-27/norma/harness"
|
||||
"github.com/Autumn-27/norma/llm"
|
||||
)
|
||||
|
||||
type fakeProvider struct {
|
||||
stream func(context.Context, llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error]
|
||||
complete func(context.Context, llm.CompletionRequest) (llm.Message, string, llm.Usage, error)
|
||||
}
|
||||
|
||||
func (p fakeProvider) Stream(ctx context.Context, r llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return p.stream(ctx, r)
|
||||
}
|
||||
func (p fakeProvider) Complete(ctx context.Context, r llm.CompletionRequest) (llm.Message, string, llm.Usage, error) {
|
||||
return p.complete(ctx, r)
|
||||
}
|
||||
func assistant(text string) llm.Message {
|
||||
return llm.Message{Role: llm.RoleAssistant, Content: []llm.ContentBlock{llm.TextBlock(text)}}
|
||||
}
|
||||
func fixture() llm.CompletionRequest {
|
||||
temp := 0.3
|
||||
return llm.CompletionRequest{System: []string{"system"}, Temperature: &temp, MaxTokens: 128, Stop: []string{"END"}, Messages: []llm.Message{
|
||||
llm.UserText("<system-reminder>date</system-reminder>"), llm.UserText("read asset"),
|
||||
{Role: llm.RoleAssistant, Content: []llm.ContentBlock{{Type: llm.BlockThinking, Thinking: "reason", Signature: "signed"}, {Type: llm.BlockToolUse, ID: "tool-1", Name: "read", Input: json.RawMessage(`{"url":"fixture"}`)}}},
|
||||
{Role: llm.RoleUser, Content: []llm.ContentBlock{llm.ToolResultText("tool-1", "verified asset", false)}},
|
||||
}, Tools: []llm.ToolSchema{{Name: "read", InputSchema: map[string]any{"properties": map[string]any{"url": map[string]any{"type": "string"}}}}}}
|
||||
}
|
||||
|
||||
func TestCheckpointDeepCopyAndBoundaries(t *testing.T) {
|
||||
var snapshots []Snapshot
|
||||
p := Bind(fakeProvider{complete: func(context.Context, llm.CompletionRequest) (llm.Message, string, llm.Usage, error) {
|
||||
return assistant("finished"), "end_turn", llm.Usage{}, nil
|
||||
}}, Model{Model: "actual"})
|
||||
ctx, deps := Attach(WithPublisher(t.Context(), func(s Snapshot) { snapshots = append(snapshots, s) }), Parent{ConversationID: 1}, harness.QueryDeps{}, p)
|
||||
req := fixture()
|
||||
if _, _, _, err := deps.CallModelSync(ctx, req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(snapshots) != 2 || snapshots[1].Request.Messages[len(snapshots[1].Request.Messages)-1].Text() != "finished" {
|
||||
t.Fatalf("checkpoints: %+v", snapshots)
|
||||
}
|
||||
before, _ := json.Marshal(snapshots)
|
||||
req.System[0] = "mutated"
|
||||
*req.Temperature = 2
|
||||
req.Stop[0] = "bad"
|
||||
req.Messages[2].Content[1].Input[2] = 'X'
|
||||
req.Messages[3].Content[0].Content[0].Text = "mutated"
|
||||
req.Tools[0].InputSchema["properties"].(map[string]any)["url"].(map[string]any)["type"] = "number"
|
||||
after, _ := json.Marshal(snapshots)
|
||||
if string(before) != string(after) {
|
||||
t.Fatal("published checkpoint aliases structured input")
|
||||
}
|
||||
// Auxiliary/compaction calls lack the model-attempt marker.
|
||||
_, _, _, _ = p.Complete(ctx, llm.CompletionRequest{Messages: []llm.Message{llm.UserText("summary request")}})
|
||||
if len(snapshots) != 2 {
|
||||
t.Fatal("auxiliary completion replaced snapshot")
|
||||
}
|
||||
terminal := fixture().Messages[1:]
|
||||
terminal = append(terminal, assistant("terminal answer"))
|
||||
Finish(ctx, terminal)
|
||||
last := snapshots[len(snapshots)-1]
|
||||
if last.Request.Messages[0].Text() != "<system-reminder>date</system-reminder>" || len(last.Request.Messages) != 5 {
|
||||
t.Fatalf("terminal/reminder duplication: %+v", last.Request.Messages)
|
||||
}
|
||||
if !strings.Contains(string(mustJSON(t, last)), "verified asset") {
|
||||
t.Fatal("terminal lost paired tool result")
|
||||
}
|
||||
}
|
||||
|
||||
func mustJSON(t *testing.T, v any) []byte {
|
||||
t.Helper()
|
||||
b, e := json.Marshal(v)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func TestSnapshotExcludesPartialStreamAndSelectsPoolMember(t *testing.T) {
|
||||
var snapshots []Snapshot
|
||||
failed := Bind(fakeProvider{stream: func(context.Context, llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return func(y func(llm.StreamEvent, error) bool) { y(llm.StreamEvent{}, errors.New("HTTP 503 unavailable")) }
|
||||
}}, Model{Model: "failed"})
|
||||
good := Bind(fakeProvider{stream: func(context.Context, llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return func(y func(llm.StreamEvent, error) bool) {
|
||||
if !y(llm.StreamEvent{Type: llm.SETextDelta, Text: "complete"}, nil) {
|
||||
return
|
||||
}
|
||||
y(llm.StreamEvent{Type: llm.SEMessageStop}, nil)
|
||||
}
|
||||
}}, Model{Model: "selected"})
|
||||
pool := llmpool.New([]*llmpool.Member{{ID: 1, Name: "failed", Rank: 2, Prov: failed}, {ID: 2, Name: "selected", Rank: 1, Prov: good}}, nil)
|
||||
ctx, deps := Attach(WithPublisher(t.Context(), func(s Snapshot) { snapshots = append(snapshots, s) }), Parent{ConversationID: 2}, harness.QueryDeps{}, pool)
|
||||
for ev, err := range deps.CallModel(ctx, fixture()) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ev.Type == llm.SETextDelta && strings.Contains(string(mustJSON(t, snapshots[len(snapshots)-1])), "complete") {
|
||||
t.Fatal("partial reply published")
|
||||
}
|
||||
}
|
||||
if last := snapshots[len(snapshots)-1]; last.Model.Model != "selected" || last.Request.Messages[len(last.Request.Messages)-1].Text() != "complete" {
|
||||
t.Fatalf("wrong selected model/checkpoint: %+v", last)
|
||||
}
|
||||
count := len(snapshots)
|
||||
for range deps.CallModel(ctx, fixture()) {
|
||||
break
|
||||
}
|
||||
if len(snapshots) != count+2 || len(snapshots[len(snapshots)-1].Request.Messages) != len(fixture().Messages) {
|
||||
t.Fatal("consumer stop published partial reply")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRequestCompactionToolPairingAndBudget(t *testing.T) {
|
||||
s := Snapshot{Request: fixture(), Model: Model{WindowTokens: 100000}}
|
||||
s.Request.Messages = append(s.Request.Messages, llm.BoundaryMessage(llm.BoundaryMeta{Trigger: "auto"}), llm.UserText("compressed summary"), s.Request.Messages[3])
|
||||
history := []Exchange{}
|
||||
for i := 0; i < 25; i++ {
|
||||
history = append(history, Exchange{Question: strings.Repeat("q", 100), Answer: "answer", Status: "completed"})
|
||||
}
|
||||
r, err := BuildRequest(s, history, "current question")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(r.Messages) != 42 || r.Messages[0].Text() != "compressed summary" {
|
||||
t.Fatalf("history cap/compaction: %d %+v", len(r.Messages), r.Messages[0])
|
||||
}
|
||||
for _, m := range r.Messages {
|
||||
for _, b := range m.Content {
|
||||
if b.Type == llm.BlockToolResult {
|
||||
t.Fatal("orphan tool result survived")
|
||||
}
|
||||
}
|
||||
}
|
||||
s.Model.WindowTokens = 1600
|
||||
r, err = BuildRequest(s, history, "current question")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(r.Messages) >= 42 || len(r.Messages) < 2 {
|
||||
t.Fatal("budget did not reduce side history")
|
||||
}
|
||||
s.Model.WindowTokens = 10
|
||||
if _, err = BuildRequest(s, nil, "question"); err == nil {
|
||||
t.Fatal("oversized base accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestServiceNoToolsAndUsageOnFailure(t *testing.T) {
|
||||
for _, streaming := range []bool{false, true} {
|
||||
t.Run(map[bool]string{true: "stream", false: "atomic"}[streaming], func(t *testing.T) {
|
||||
calls := 0
|
||||
p := fakeProvider{complete: func(context.Context, llm.CompletionRequest) (llm.Message, string, llm.Usage, error) {
|
||||
calls++
|
||||
return llm.Message{Role: llm.RoleAssistant, Content: []llm.ContentBlock{{Type: llm.BlockToolUse, ID: "new", Name: "run_shell", Input: json.RawMessage(`{"command":"touch forbidden"}`)}}}, "tool_use", llm.Usage{InputTokens: 11}, nil
|
||||
}, stream: func(context.Context, llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return func(y func(llm.StreamEvent, error) bool) {
|
||||
calls++
|
||||
y(llm.StreamEvent{Type: llm.SEMessageStart, Usage: llm.Usage{InputTokens: 11}}, nil)
|
||||
y(llm.StreamEvent{Type: llm.SEToolUseStart, ToolID: "new", ToolName: "run_shell"}, nil)
|
||||
y(llm.StreamEvent{Type: llm.SEMessageStop}, nil)
|
||||
}
|
||||
}}
|
||||
before := fixture()
|
||||
req, _ := CloneRequest(before)
|
||||
out, err := (SideQuestionService{p}).Answer(t.Context(), req, streaming, nil)
|
||||
if err != nil || calls != 1 || !out.ToolUse || !strings.Contains(out.Text, "不能执行工具") || out.Usage.InputTokens != 11 {
|
||||
t.Fatalf("answer %+v calls=%d err=%v", out, calls, err)
|
||||
}
|
||||
if !reflect.DeepEqual(before, req) {
|
||||
t.Fatal("service mutated main context")
|
||||
}
|
||||
})
|
||||
}
|
||||
p := fakeProvider{stream: func(context.Context, llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return func(y func(llm.StreamEvent, error) bool) {
|
||||
y(llm.StreamEvent{Type: llm.SEMessageStart, Usage: llm.Usage{InputTokens: 12}}, nil)
|
||||
y(llm.StreamEvent{Type: llm.SETextDelta, Text: "partial"}, nil)
|
||||
y(llm.StreamEvent{}, errors.New("broken"))
|
||||
}
|
||||
}}
|
||||
out, err := (SideQuestionService{p}).Answer(t.Context(), fixture(), true, nil)
|
||||
if err == nil || out.Text != "partial" || out.Usage.InputTokens != 12 {
|
||||
t.Fatalf("lost failure usage/partial: %+v %v", out, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMainSideConcurrencyAndIndependentCancellation(t *testing.T) {
|
||||
for _, cancelMain := range []bool{false, true} {
|
||||
t.Run(map[bool]string{true: "stop-main", false: "stop-side"}[cancelMain], func(t *testing.T) {
|
||||
started := make(chan struct{}, 2)
|
||||
p := fakeProvider{stream: func(ctx context.Context, _ llm.CompletionRequest) iter.Seq2[llm.StreamEvent, error] {
|
||||
return func(y func(llm.StreamEvent, error) bool) {
|
||||
started <- struct{}{}
|
||||
y(llm.StreamEvent{Type: llm.SEMessageStart, Usage: llm.Usage{InputTokens: 7}}, nil)
|
||||
<-ctx.Done()
|
||||
y(llm.StreamEvent{}, ctx.Err())
|
||||
}
|
||||
}}
|
||||
mainCtx, stopMain := context.WithCancel(t.Context())
|
||||
defer stopMain()
|
||||
sideCtx, stopSide := context.WithCancel(t.Context())
|
||||
defer stopSide()
|
||||
var mu sync.Mutex
|
||||
bound := Bind(p, Model{Model: "blocking"})
|
||||
mainCtx, deps := Attach(WithPublisher(mainCtx, func(Snapshot) { mu.Lock(); mu.Unlock() }), Parent{ConversationID: 3}, harness.QueryDeps{}, bound)
|
||||
mainDone, sideDone := make(chan struct{}), make(chan Answer, 1)
|
||||
go func() {
|
||||
defer close(mainDone)
|
||||
for range deps.CallModel(mainCtx, fixture()) {
|
||||
}
|
||||
}()
|
||||
go func() { a, _ := (SideQuestionService{bound}).Answer(sideCtx, fixture(), true, nil); sideDone <- a }()
|
||||
for i := 0; i < 2; i++ {
|
||||
select {
|
||||
case <-started:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("main and side did not execute concurrently")
|
||||
}
|
||||
}
|
||||
if cancelMain {
|
||||
stopMain()
|
||||
select {
|
||||
case <-mainDone:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("main cancellation stuck")
|
||||
}
|
||||
select {
|
||||
case <-sideDone:
|
||||
t.Fatal("main cancellation stopped side")
|
||||
default:
|
||||
}
|
||||
stopSide()
|
||||
} else {
|
||||
stopSide()
|
||||
select {
|
||||
case a := <-sideDone:
|
||||
if a.Usage.InputTokens != 7 {
|
||||
t.Fatal("cancelled usage lost")
|
||||
}
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("side cancellation stuck")
|
||||
}
|
||||
select {
|
||||
case <-mainDone:
|
||||
t.Fatal("side cancellation stopped main")
|
||||
default:
|
||||
}
|
||||
stopMain()
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
{
|
||||
"date": "2026-09-10",
|
||||
"branch": "codex/btw-side-question",
|
||||
"notes": [
|
||||
"Independent test database and profiles; product default unchanged.",
|
||||
"Streaming zero usage means no usage frames supplied by this endpoint, not zero billing.",
|
||||
"Qwen reviewed the initial streaming conversations; restart and non-streaming tests were added afterwards.",
|
||||
"The main curl command saved a local response file. No side question executed a tool."
|
||||
],
|
||||
"availability": [
|
||||
{
|
||||
"model": "grok-4.6",
|
||||
"protocol": "openai",
|
||||
"base_url": "http://127.0.0.1:12580/tingly/openai",
|
||||
"http_status": 200,
|
||||
"response_model": "grok-4.6",
|
||||
"answer": "READY",
|
||||
"available": true,
|
||||
"seconds": 2.82
|
||||
}
|
||||
],
|
||||
"streaming": {
|
||||
"model": "grok-4.6",
|
||||
"conversation_id": 2,
|
||||
"assertions": [
|
||||
{
|
||||
"name": "context_during_main",
|
||||
"status": "completed",
|
||||
"model": "grok-4.6",
|
||||
"seconds": 16.97
|
||||
},
|
||||
{
|
||||
"name": "completed_tool_context",
|
||||
"status": "completed",
|
||||
"model": "grok-4.6",
|
||||
"seconds": 7.24
|
||||
},
|
||||
{
|
||||
"name": "tools_denied",
|
||||
"status": "completed",
|
||||
"model": "grok-4.6",
|
||||
"seconds": 7.74
|
||||
},
|
||||
{
|
||||
"name": "main_transcript_unchanged_after_side",
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"name": "main_activity_unchanged_after_side",
|
||||
"passed": true
|
||||
},
|
||||
{
|
||||
"name": "side_tool_executions_zero",
|
||||
"passed": true
|
||||
}
|
||||
],
|
||||
"side_questions": [
|
||||
{
|
||||
"id": "ff082cda-1008-4fc8-a9a0-817a8afceb0d",
|
||||
"client_request_id": "09343105-666d-4763-a89d-2c09530b753a",
|
||||
"question": "本轮正在处理的资产、目标和测试标记分别是什么?只根据已有上下文回答。",
|
||||
"answer": "本轮正在处理的信息如下:\n\n- **资产**:redhaze.top \n- **目标**:读取 https://redhaze.top/ 首页,并总结页面标题、HTTP 状态及可见内容 \n- **测试标记**:BTW-REAL-0910",
|
||||
"status": "completed",
|
||||
"model": {
|
||||
"profile_id": 2,
|
||||
"name": "btw-real-grok-stream",
|
||||
"format": "openai",
|
||||
"model": "grok-4.6",
|
||||
"identity": "ac6708faaa1292cc2fa9b96f3bd2d0c355b3ff7832addd598db48b227c5c141b",
|
||||
"streaming": true,
|
||||
"window_tokens": 200000
|
||||
},
|
||||
"snapshot_at": "2026-09-10T21:06:40.374922+08:00",
|
||||
"created_at": "2026-09-10T21:06:40.794717+08:00",
|
||||
"sequence": 3,
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"cache_read_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"ordinal": 4
|
||||
},
|
||||
{
|
||||
"id": "c6c4cf8b-7d79-4e80-a477-c381f0277d2d",
|
||||
"client_request_id": "19fad20a-e79b-418f-a9ec-d6900440fbb5",
|
||||
"question": "已经完成的首页读取工具结果是什么?标题和 HTTP 状态有哪些证据?如果工具失败请明确说失败。",
|
||||
"answer": "工具调用**成功**,不是失败。已有首页读取证据如下。\n\n### HTTP 状态\n- **WebFetch**:`[200 OK] https://redhaze.top/`\n- **curl(跟随跳转)**:\n 1. `https://redhaze.top/` → **HTTP/2 301**,`Location: https://id.redhaze.top/`\n 2. `https://id.redhaze.top/` → **HTTP/2 302**,`Location: /home`\n 3. 落地页 → **HTTP/2 200**,`content-type: text/html; charset=utf-8`\n- 服务端:`nginx/1.24.0 (Ubuntu)`\n\n### 标题证据\n- curl 解析到的 `<title>`:**红幕科技 RedHaze Group · 全球综合集团门户**\n- WebFetch 页面首行标题与此一致。",
|
||||
"status": "completed",
|
||||
"model": {
|
||||
"profile_id": 2,
|
||||
"name": "btw-real-grok-stream",
|
||||
"format": "openai",
|
||||
"model": "grok-4.6",
|
||||
"identity": "ac6708faaa1292cc2fa9b96f3bd2d0c355b3ff7832addd598db48b227c5c141b",
|
||||
"streaming": true,
|
||||
"window_tokens": 200000
|
||||
},
|
||||
"snapshot_at": "2026-09-10T21:07:24.937271+08:00",
|
||||
"created_at": "2026-09-10T21:07:25.332552+08:00",
|
||||
"sequence": 11,
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"cache_read_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"ordinal": 5
|
||||
},
|
||||
{
|
||||
"id": "3305bd98-44c5-43c9-b42a-44db95de156e",
|
||||
"client_request_id": "9b86f14d-8c44-4e0d-b502-3dc2f8f6979d",
|
||||
"question": "请现在执行工具,用 Bash 创建文件 /private/tmp/artex-btw-runtime/real-side-tool-executed,然后告诉我结果。",
|
||||
"answer": "无法执行。当前是独立旁路提问,我没有工具执行能力,不能用 Bash 创建文件,也不会操作或承诺稍后执行。\n\n因此文件 `/private/tmp/artex-btw-runtime/real-side-tool-executed` **没有被创建**。",
|
||||
"status": "completed",
|
||||
"model": {
|
||||
"profile_id": 2,
|
||||
"name": "btw-real-grok-stream",
|
||||
"format": "openai",
|
||||
"model": "grok-4.6",
|
||||
"identity": "ac6708faaa1292cc2fa9b96f3bd2d0c355b3ff7832addd598db48b227c5c141b",
|
||||
"streaming": true,
|
||||
"window_tokens": 200000
|
||||
},
|
||||
"snapshot_at": "2026-09-10T21:07:24.937271+08:00",
|
||||
"created_at": "2026-09-10T21:07:32.573916+08:00",
|
||||
"sequence": 6,
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"cache_read_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"ordinal": 6
|
||||
}
|
||||
],
|
||||
"main_question": "本轮仅做一个首页读取测试:资产是 redhaze.top,目标是读取 https://redhaze.top/ 首页并总结页面标题、HTTP 状态及可见内容。使用 Bash 的 curl 或 WebFetch 发出普通 GET;不进行扫描、漏洞验证、登录或任何写入。成功或失败都如实报告。测试标记为 BTW-REAL-0910。",
|
||||
"main_activity": {
|
||||
"cursor": 12,
|
||||
"hasMore": false,
|
||||
"items": [
|
||||
{
|
||||
"seq": 6,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:06:40+08:00",
|
||||
"kind": "user",
|
||||
"is_error": false,
|
||||
"summary": "本轮仅做一个首页读取测试:资产是 redhaze.top,目标是读取 https://redhaze.top/ 首页并总结页面标题、HTTP 状态及可见内容。使用 Bash 的 curl 或 WebFetch 发出普通 GET;不进行扫描、漏洞验证、登录或任何写入。成功或失败都如实报告。测试标记为 BTW-REAL-0910。"
|
||||
},
|
||||
{
|
||||
"seq": 7,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:06:49+08:00",
|
||||
"kind": "text",
|
||||
"is_error": false,
|
||||
"summary": "按你的要求只做首页普通 GET 读取,先拉 https://redhaze.top/ 的状态和页面内容。"
|
||||
},
|
||||
{
|
||||
"seq": 8,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:06:49+08:00",
|
||||
"kind": "tool_use",
|
||||
"tool": "WebFetch",
|
||||
"tool_use_id": "call-90414eaa-3e0d-466d-8524-e878676a6868-0",
|
||||
"is_error": false,
|
||||
"summary": "WebFetch {\"url\":\"https://redhaze.top/\"}"
|
||||
},
|
||||
{
|
||||
"seq": 9,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:06:49+08:00",
|
||||
"kind": "tool_use",
|
||||
"tool": "Bash",
|
||||
"tool_use_id": "call-90414eaa-3e0d-466d-8524-e878676a6868-1",
|
||||
"is_error": false,
|
||||
"summary": "Bash {\"command\":\"curl -sS -D - -o /tmp/redhaze_body.html -L --max-time 20 \\\"https://redhaze.top/\\\" | head -n 40; echo \\\"----BODY_HEAD----\\\"; head -c 4000 /tmp/redhaze_body.html; echo; echo \\\"----TITLE----\\…"
|
||||
},
|
||||
{
|
||||
"seq": 10,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:07:01+08:00",
|
||||
"kind": "tool_result",
|
||||
"tool": "WebFetch",
|
||||
"tool_use_id": "call-90414eaa-3e0d-466d-8524-e878676a6868-0",
|
||||
"is_error": false,
|
||||
"summary": "[200 OK] https://redhaze.top/"
|
||||
},
|
||||
{
|
||||
"seq": 11,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:07:13+08:00",
|
||||
"kind": "tool_result",
|
||||
"tool": "Bash",
|
||||
"tool_use_id": "call-90414eaa-3e0d-466d-8524-e878676a6868-1",
|
||||
"is_error": false,
|
||||
"summary": "HTTP/1.1 200 Connection established\r"
|
||||
},
|
||||
{
|
||||
"seq": 12,
|
||||
"worker": "auto",
|
||||
"ts": "2026-09-10T21:07:24+08:00",
|
||||
"kind": "result",
|
||||
"is_error": false,
|
||||
"summary": "**BTW-REAL-0910 · 首页读取测试(仅 GET)**",
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"cache_read_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
}
|
||||
],
|
||||
"running": false
|
||||
},
|
||||
"transcript_sha256": "e7e61f135a4a120954b539f357e8c4205d7d5cd7460dcaf3dc0fd066463e1d00"
|
||||
},
|
||||
"restart": {
|
||||
"snapshot_before_new_question": "2026-09-10T13:07:24.937271Z",
|
||||
"request": {
|
||||
"id": "1c65b387-a172-4540-b6af-b7efe0245eb7",
|
||||
"client_request_id": "66ad3fd6-9c2e-41bc-807c-5ccb6c4b8162",
|
||||
"question": "服务刚重启。不要重新执行工具,告诉我此前的资产、测试标记和首页标题。",
|
||||
"answer": "此前结果如下:\n\n- **资产**:redhaze.top \n- **测试标记**:BTW-REAL-0910 \n- **首页标题**:红幕科技 RedHaze Group · 全球综合集团门户",
|
||||
"status": "completed",
|
||||
"model": {
|
||||
"profile_id": 2,
|
||||
"name": "btw-real-grok-stream",
|
||||
"format": "openai",
|
||||
"model": "grok-4.6",
|
||||
"identity": "ac6708faaa1292cc2fa9b96f3bd2d0c355b3ff7832addd598db48b227c5c141b",
|
||||
"streaming": true,
|
||||
"window_tokens": 200000
|
||||
},
|
||||
"snapshot_at": "2026-09-10T21:07:24.937271+08:00",
|
||||
"created_at": "2026-09-10T21:09:52.132635+08:00",
|
||||
"sequence": 4,
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"cache_read_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"ordinal": 7
|
||||
},
|
||||
"history_count": 3,
|
||||
"main_not_rerun": true
|
||||
},
|
||||
"non_streaming": {
|
||||
"model": "grok-4.6",
|
||||
"mode": "non-streaming",
|
||||
"conversation_id": 3,
|
||||
"request": {
|
||||
"id": "9dac734d-3935-476c-80e8-88f39f0bff9e",
|
||||
"client_request_id": "b327aeca-d87d-4365-93cb-9ece35ca5375",
|
||||
"question": "此前给出的资产和测试标记是什么?",
|
||||
"answer": "资产:redhaze.top;测试标记:ATOMIC-0910。",
|
||||
"status": "completed",
|
||||
"model": {
|
||||
"profile_id": 3,
|
||||
"name": "btw-real-grok-atomic",
|
||||
"format": "openai",
|
||||
"model": "grok-4.6",
|
||||
"identity": "cc8d68b191a4adf3751a029a750de63fa443d11e5591638fbc079b9d2a843418",
|
||||
"streaming": false,
|
||||
"window_tokens": 200000
|
||||
},
|
||||
"snapshot_at": "2026-09-10T21:13:03.766695+08:00",
|
||||
"created_at": "2026-09-10T21:13:04.156515+08:00",
|
||||
"sequence": 1,
|
||||
"usage": {
|
||||
"input_tokens": 11734,
|
||||
"output_tokens": 138,
|
||||
"cache_read_tokens": 11520,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"ordinal": 8
|
||||
},
|
||||
"passed": true
|
||||
},
|
||||
"stream_usage_probe": {
|
||||
"http_status": 200,
|
||||
"stream_include_usage_requested": true,
|
||||
"data_frames": 12,
|
||||
"usage_frames": []
|
||||
},
|
||||
"qwen_review": {
|
||||
"http_status": 200,
|
||||
"model": "qwen-flash",
|
||||
"review": "```json\n{\n \"verdict\": \"accept\",\n \"supported_findings\": [\n \"测试标记 BTW-REAL-0910 明确记录在上下文中,且与主对话一致。\",\n \"首页读取任务仅执行普通 GET,未进行扫描、漏洞验证或写入操作,符合指令要求。\",\n \"工具调用结果(WebFetch 与 curl)均成功返回,且证据链完整:包含跳转路径、最终状态码 200、服务端信息 nginx/1.24.0 (Ubuntu),以及页面标题和可见内容摘要。\",\n \"页面标题 '红幕科技 RedHaze Group · 全球综合集团门户' 由 WebFetch 和 curl 解析双重验证,一致性高。\",\n \"旁路问题的回答准确引用已有上下文,拒绝执行未授权的工具操作,符合工程断言中 'tools_denied' 与 'side_tool_executions_zero' 的要求。\",\n \"工程断言全部通过,包括 context_during_main、completed_tool_context、tools_denied 等,表明主流程未被干扰,侧问未引发意外执行。\"\n ],\n \"concerns\": [],\n \"acceptance_notes\": \"回答质量高,逻辑清晰,所有结论均有直接证据支持。对工具失败、旁路请求的拒绝处理得当,未编造已完成工具结果,未遗漏关键信息。工程断言满足验收条件,可接受。\"\n}\n```",
|
||||
"usage": {
|
||||
"completion_tokens": 312,
|
||||
"prompt_tokens": 6625,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
},
|
||||
"total_tokens": 6937
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user