From 379e4870a23475593fc945ca29a99ce069fc4018 Mon Sep 17 00:00:00 2001 From: liuzaisen Date: Sat, 19 Sep 2026 23:48:05 +0800 Subject: [PATCH 1/2] feat(mcp): reduce tool round trips and analyze sessions --- .../plans/2026-09-18-mcp-round-efficiency.md | 23 ++ .../2026-09-19-local-ai-session-analyzer.md | 25 ++ docs/superpowers/plans/README.md | 2 + .../2026-09-18-mcp-round-efficiency-design.md | 99 +++++ ...-09-19-local-ai-session-analyzer-design.md | 107 ++++++ docs/superpowers/specs/README.md | 2 + package.json | 3 +- .../extension/src/background/bg-tools/nav.ts | 70 +++- .../tests/background/meta-tool-router.test.ts | 34 +- packages/mcp-server/README.md | 16 +- packages/mcp-server/package.json | 3 +- packages/mcp-server/src/discover-tool.ts | 52 ++- packages/mcp-server/src/mcp-server.ts | 9 +- .../src/session-analysis/adapters.ts | 354 ++++++++++++++++++ .../src/session-analysis/analyzer.ts | 221 +++++++++++ .../src/session-analysis/cli-support.ts | 122 ++++++ .../mcp-server/src/session-analysis/cli.ts | 38 ++ .../src/session-analysis/scanner.ts | 107 ++++++ .../mcp-server/src/session-analysis/types.ts | 103 +++++ .../mcp-server/tests/discover-tool.test.ts | 53 ++- packages/mcp-server/tests/mcp-server.test.ts | 48 ++- .../mcp-server/tests/session-analysis.test.ts | 317 ++++++++++++++++ packages/shared/src/llm/builtin-tool-defs.ts | 2 +- skill/SKILL.md | 17 +- 24 files changed, 1788 insertions(+), 39 deletions(-) create mode 100644 docs/superpowers/plans/2026-09-18-mcp-round-efficiency.md create mode 100644 docs/superpowers/plans/2026-09-19-local-ai-session-analyzer.md create mode 100644 docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md create mode 100644 docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md create mode 100644 packages/mcp-server/src/session-analysis/adapters.ts create mode 100644 packages/mcp-server/src/session-analysis/analyzer.ts create mode 100644 packages/mcp-server/src/session-analysis/cli-support.ts create mode 100644 packages/mcp-server/src/session-analysis/cli.ts create mode 100644 packages/mcp-server/src/session-analysis/scanner.ts create mode 100644 packages/mcp-server/src/session-analysis/types.ts create mode 100644 packages/mcp-server/tests/session-analysis.test.ts diff --git a/docs/superpowers/plans/2026-09-18-mcp-round-efficiency.md b/docs/superpowers/plans/2026-09-18-mcp-round-efficiency.md new file mode 100644 index 0000000..47e55ea --- /dev/null +++ b/docs/superpowers/plans/2026-09-18-mcp-round-efficiency.md @@ -0,0 +1,23 @@ +# MCP Round Efficiency Implementation Plan + +**Goal:** Remove redundant model turns by making screenshot and resize results +self-describing and allowing one-call discovery group activation. + +**Spec:** `docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md` + +## Tasks + +- [x] Add failing MCP tests for screenshot metadata and `enableGroups`. +- [x] Add failing extension tests for actual viewport verification. +- [x] Preserve non-image screenshot metadata in MCP content. +- [x] Measure and report the actual viewport after resize. +- [x] Implement direct discovery-group activation and update tool guidance. +- [x] Run focused tests, package typechecks, full tests, and build. + +## Constraints + +- Do not duplicate base64 image data in a text content block. +- Do not change capability validation, dangerous accounting, or approval gates. +- Keep old discovery and resize calls backward compatible. +- Do not add dependencies. +- Do not enable behavioral trace collection without a separate privacy design. diff --git a/docs/superpowers/plans/2026-09-19-local-ai-session-analyzer.md b/docs/superpowers/plans/2026-09-19-local-ai-session-analyzer.md new file mode 100644 index 0000000..1cbe692 --- /dev/null +++ b/docs/superpowers/plans/2026-09-19-local-ai-session-analyzer.md @@ -0,0 +1,25 @@ +# Local AI Session Analyzer Implementation Plan + +**Goal:** Analyze local Claude Code and Codex histories for avoidable +AtWebPilot tool rounds without collecting or exposing conversation content. + +**Spec:** `docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md` + +## Tasks + +- [x] Define normalized privacy-preserving event/report types. +- [x] Add failing Claude/Codex adapter tests with synthetic JSONL fixtures. +- [x] Implement streaming adapters and result joining. +- [x] Add failing analyzer rule tests. +- [x] Implement aggregation, sequence rules and report rendering. +- [x] Add CLI argument tests and repository/package scripts. +- [x] Run the analyzer against local histories and inspect only aggregates. +- [x] Run package/full typecheck, tests and builds. + +## Constraints + +- No dependencies. +- No raw prompt, argument, result, credential or absolute session path in output. +- Read-only access to client history directories. +- Unknown and malformed records are skipped and counted, never fatal. +- Client file layouts are treated as versioned adapters, not stable public APIs. diff --git a/docs/superpowers/plans/README.md b/docs/superpowers/plans/README.md index 2b47ec1..269aac3 100644 --- a/docs/superpowers/plans/README.md +++ b/docs/superpowers/plans/README.md @@ -22,6 +22,8 @@ | 16 | [`2026-06-06-atwebpilot-rename.md`](./2026-06-06-atwebpilot-rename.md) | 7 | 0 (rename only) | ~492 | | 17 | [`2026-06-07-mcp-publish.md`](./2026-06-07-mcp-publish.md) | 9 | 0 (publish only) | ~492 | | 18 | [`2026-09-18-non-intrusive-screenshot.md`](./2026-09-18-non-intrusive-screenshot.md) | 4 | + focused MCP/CDP capture tests | current | +| 19 | [`2026-09-18-mcp-round-efficiency.md`](./2026-09-18-mcp-round-efficiency.md) | 6 | +7 focused MCP/extension tests | current | +| 20 | [`2026-09-19-local-ai-session-analyzer.md`](./2026-09-19-local-ai-session-analyzer.md) | 8 | synthetic adapter/analyzer/CLI tests | current | 各 plan 的 task 严格 TDD:写失败测试 → 实现 → 验证通过 → commit。每 task 一组 commit;plan 内的 commit 加起来构成一次完整 feature。 diff --git a/docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md b/docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md new file mode 100644 index 0000000..61ec8c2 --- /dev/null +++ b/docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md @@ -0,0 +1,99 @@ +# MCP Round Efficiency Design + +**Status:** implemented. + +## Problem + +Real MCP sessions show avoidable model turns around otherwise successful tool +calls. In one responsive-page workflow the agent called `resize`, took a +`screenshot`, then used `runJS` twice to recover viewport facts the tools had +already measured internally. It also called `browser_discoverTools({})` before +deciding which extra capability it needed. + +Each avoidable tool round is expensive beyond its small JSON arguments: the +default MCP surface is about 16k characters (roughly 4.5k tokens) and the +conversation plus prior tool results grows on every turn. + +## Goals + +- Make successful screenshot and resize calls self-verifying. +- Let an agent enable a known discovery group in one call. +- Preserve existing tool names, arguments, result fields, capability checks, + and approval behavior. +- Improve agent guidance so it trusts verified postconditions and avoids fixed + waits when a tool already returns the needed state. + +## Design + +### Screenshot metadata survives MCP conversion + +`screenshot` already returns `backend`, dimensions, target information and +capture details beside its base64 payload. MCP currently emits only the image +block. It will emit the image first, followed by a compact JSON text block that +contains every top-level field except `data` and `media_type`. + +This keeps image rendering compatible while exposing enough evidence to avoid +a follow-up viewport probe. The base64 payload is never duplicated into text. + +### Resize verifies the actual viewport + +After applying either CDP device metrics or a real window resize, `resize` +measures `window.innerWidth`, `window.innerHeight` and `devicePixelRatio`. The +main-world path polls briefly because `chrome.windows.update` may resolve before +layout catches up. + +The existing `width`, `height`, `backend` and `chromeInset` fields remain. New +fields are: + +```json +{ + "actualViewport": { "width": 390, "height": 844 }, + "devicePixelRatio": 1, + "verified": true +} +``` + +`verified` is false rather than an exception when the browser accepts the +resize but reports a different viewport. This is useful evidence for responsive +debugging and prevents blind retries. + +### Discovery enables known groups directly + +`browser_discoverTools` gains `enableGroups`, whose values are the existing +discovery groups. `enable` and `enableGroups` may be supplied together; names +are deduplicated before worker-support filtering. Unknown groups are returned +as `unknownGroups` without failing valid requests. + +Calling with no arguments remains a catalog operation for genuinely unknown +capabilities. Guidance changes from a mandatory catalog-then-enable sequence to +direct group activation when the group is known. + +## Finding The Next Inefficiencies + +A comprehensive follow-up should add opt-in MCP efficiency traces. Each trace +record should contain timestamps, tool name, redacted argument/result byte +counts, image dimensions, outcome, and a task/session correlation id. It must +not retain page text, images, credentials, or raw arguments by default. + +An offline analyzer can then flag: + +- repeated calls with equivalent arguments; +- `resize -> runJS(viewport)` and `screenshot -> runJS(viewport)` chains; +- fixed waits followed by a state-reading tool; +- no-argument discovery immediately followed by group activation; +- oversized text/image results and repeated full snapshots; +- error/retry loops and tools whose postconditions are routinely rechecked. + +Release validation should replay a small fixed scenario suite and compare tool +rounds, result bytes, wall time, and task success at median and p95. Trace +collection is deliberately not part of this change because retention, redaction +and user opt-in need an explicit privacy decision. + +## Verification + +- MCP tests cover image plus metadata content blocks. +- Background tests cover verified and mismatched resize results. +- Discovery tests cover direct group activation, deduplication, unknown groups, + worker support, and list-changed behavior. +- MCP token-budget tests, package typechecks, package tests, and extension build + remain green. diff --git a/docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md b/docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md new file mode 100644 index 0000000..5c083f4 --- /dev/null +++ b/docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md @@ -0,0 +1,107 @@ +# Local AI Session Analyzer Design + +**Status:** implemented. + +## Problem + +MCP inefficiencies are easiest to recognize from real agent behavior, but adding +runtime telemetry duplicates data already written by local AI clients and adds +privacy and maintenance costs. This machine has both Claude Code and Codex JSONL +histories. Their formats differ, and naive text search is inaccurate because +tool definitions are also embedded in session records. + +## Goals + +- Provide a read-only local CLI for Claude Code and Codex session histories. +- Parse actual tool-call events rather than matching tool names in arbitrary + text. +- Report tool frequency, model tool rounds, result bytes and likely avoidable + call chains. +- Never print or persist prompts, tool argument values, tool result bodies, + credentials, or raw session paths. +- Tolerate malformed lines and unknown client versions. + +## CLI + +From the repository root: + +```bash +pnpm analyze:sessions +pnpm analyze:sessions -- --since 7d --clients claude --format json +``` + +Defaults: + +- clients: `claude,codex` +- lookback: `30d` +- output: human-readable text +- roots: `~/.claude/projects` and `~/.codex/sessions` + +Optional `--claude-dir` and `--codex-dir` overrides support fixtures and custom +installations. `--format json` returns the same aggregated report as structured +JSON. The CLI exits successfully when a client directory is missing and reports +that client as unavailable. + +## Privacy Model + +Raw lines are processed as a stream. Tool inputs are reduced immediately to: + +- sorted top-level field names; +- a SHA-256 fingerprint for equality checks; +- boolean semantic signals such as `fixedWait`, `viewportProbe`, + `emptyDiscovery`, and `enablesDiscovery`. + +Tool results are reduced to byte length and error status. No cache is written in +this version. Reports contain client, aggregate counts, rule IDs, occurrence +counts, affected session counts and estimated avoidable model rounds. + +## Adapters + +### Claude Code + +- Calls: `assistant.message.content[*].type == "tool_use"`. +- Results: `user.message.content[*].type == "tool_result"`, joined by + `tool_use_id`. +- A single assistant record containing one or more tool uses is one model tool + round. + +### Codex + +- Direct calls: `response_item.payload.type == "function_call"` or + `custom_tool_call`, joined to their corresponding output by `call_id`. +- Nested calls made through the Codex `exec` orchestrator are identified only + when its source contains an actual `tools.mcp__atwebpilot__...(` invocation. + They are marked `confidence: "derived"`; argument shape and individual result + size are unavailable. +- Token-usage records are not attributed to individual Codex tool calls; the + report only totals usage attached directly to a parsed model tool round. + +Both adapters normalize `mcp__atwebpilot__browser_screenshot` to +`browser_screenshot`. Calls unrelated to AtWebPilot are ignored. + +## Rules + +The first rule set detects: + +- `viewport_recheck_after_resize`: resize followed shortly by viewport-only + runJS; +- `viewport_recheck_after_screenshot`: screenshot followed shortly by the same; +- `fixed_wait`: `waitFor` with only `ms` rather than a state condition; +- `discovery_round_trip`: empty discovery followed by an enabling discovery; +- `duplicate_call`: the same tool and argument fingerprint repeated nearby; +- `repeated_screenshot`: screenshots repeated nearby; +- `serial_form_fill`: two or more consecutive `fillInput` calls that could use + `fillForm`. + +Sequence rules inspect only AtWebPilot calls within the same local session and +a small call window. They are heuristics, so the report labels them as +opportunities rather than guaranteed waste. + +## Verification + +- Synthetic Claude and Codex JSONL fixtures cover extraction, result joining, + nested-call confidence, malformed lines and privacy guarantees. +- Analyzer tests cover every rule and ensure reports contain no raw values. +- CLI tests cover argument parsing and text/JSON rendering. +- Full monorepo typecheck and test remain green; MCP build includes no new + runtime dependency. diff --git a/docs/superpowers/specs/README.md b/docs/superpowers/specs/README.md index 5c0a158..df6d609 100644 --- a/docs/superpowers/specs/README.md +++ b/docs/superpowers/specs/README.md @@ -41,6 +41,8 @@ | 35 | URL 注入策略 | [`2026-08-19-url-injection-policy-design.md`](./2026-08-19-url-injection-policy-design.md) | 默认与逐 hostname 的禁用/只读/操作/诊断注入级别;网页助手独立继承;惰性 bootstrap 与按需 MAIN-world recorder;配对错误状态分流 | | 36 | 会话标签分组 | [`2026-08-20-session-tab-groups-design.md`](./2026-08-20-session-tab-groups-design.md) | 按逻辑 session id 和 window 创建原生 Chrome tab group;区分侧栏/MCP/远程来源;结束后恢复原分组;用户手动移出后不拉回;MV3 session 状态恢复 | | 37 | 无干扰 MCP 截图 | [`2026-09-18-non-intrusive-screenshot-design.md`](./2026-09-18-non-intrusive-screenshot-design.md) | `switchToTab` 退出默认 core;CDP `Page.captureScreenshot` 后台截图;visible-tab 兼容回退不覆盖用户中途切 tab | +| 38 | MCP 调用轮次优化 | [`2026-09-18-mcp-round-efficiency-design.md`](./2026-09-18-mcp-round-efficiency-design.md) | screenshot 保留元数据;resize 返回实测 viewport;discoverTools 可直接按组启用;定义 opt-in trace 后续方向 | +| 39 | 本地 AI 会话效率分析 | [`2026-09-19-local-ai-session-analyzer-design.md`](./2026-09-19-local-ai-session-analyzer-design.md) | 只读扫描 Claude/Codex JSONL;脱敏聚合 AtWebPilot 调用;识别重复验证、固定等待与可合并步骤 | ## 不在 spec 里的细节修复 diff --git a/package.json b/package.json index 12e3a6f..4f18321 100644 --- a/package.json +++ b/package.json @@ -10,7 +10,8 @@ "preview": "pnpm --filter @atwebpilot/extension preview", "typecheck": "pnpm -r typecheck", "test": "pnpm -r test", - "test:watch": "pnpm --filter @atwebpilot/extension test:watch" + "test:watch": "pnpm --filter @atwebpilot/extension test:watch", + "analyze:sessions": "pnpm --filter @attson/atwebpilot-mcp analyze:sessions" }, "packageManager": "pnpm@9.0.0" } diff --git a/packages/extension/src/background/bg-tools/nav.ts b/packages/extension/src/background/bg-tools/nav.ts index 5d63ceb..4eaf127 100644 --- a/packages/extension/src/background/bg-tools/nav.ts +++ b/packages/extension/src/background/bg-tools/nav.ts @@ -5,6 +5,51 @@ function asObj(raw: Json): Record { return (raw ?? {}) as Record; } +type ViewportMetrics = { + iw: number; + ih: number; + ow: number; + oh: number; + dpr: number; +}; + +async function measureViewport(tabId: number): Promise { + const [probe] = await chrome.scripting.executeScript({ + target: { tabId }, + func: () => ({ + iw: window.innerWidth, + ih: window.innerHeight, + ow: window.outerWidth, + oh: window.outerHeight, + dpr: window.devicePixelRatio + }) + }); + const metrics = probe?.result as ViewportMetrics | undefined; + if (!metrics) throw new Error("resize: could not measure the viewport"); + return metrics; +} + +async function measureSettledViewport( + tabId: number, + width: number, + height: number +): Promise { + let latest = await measureViewport(tabId); + for (let attempt = 0; attempt < 4 && (latest.iw !== width || latest.ih !== height); attempt++) { + await new Promise((resolve) => setTimeout(resolve, 50)); + latest = await measureViewport(tabId); + } + return latest; +} + +function verifiedViewportResult(metrics: ViewportMetrics, width: number, height: number) { + return { + actualViewport: { width: metrics.iw, height: metrics.ih }, + devicePixelRatio: metrics.dpr, + verified: metrics.iw === width && metrics.ih === height + }; +} + /** * History navigation. Running off the end of the history is an ordinary * outcome, not a failure — an agent walking back through pages should get a @@ -48,20 +93,17 @@ export async function resize(raw: Json, tabId: number): Promise { const viaCdp = getCdpResizer(tabId); if (viaCdp) { await viaCdp(width, height); - return { ok: true, width, height, backend: "cdp" } as unknown as Json; + const actual = await measureSettledViewport(tabId, width, height); + return { + ok: true, + width, + height, + backend: "cdp", + ...verifiedViewportResult(actual, width, height) + } as unknown as Json; } - const [probe] = await chrome.scripting.executeScript({ - target: { tabId }, - func: () => ({ - iw: window.innerWidth, - ih: window.innerHeight, - ow: window.outerWidth, - oh: window.outerHeight - }) - }); - const m = probe?.result as { iw: number; ih: number; ow: number; oh: number } | undefined; - if (!m) throw new Error("resize: could not measure the viewport"); + const m = await measureViewport(tabId); const tab = await chrome.tabs.get(tabId); if (tab.windowId == null) throw new Error("resize: tab has no window"); @@ -72,12 +114,14 @@ export async function resize(raw: Json, tabId: number): Promise { width: width + chromeW, height: height + chromeH }); + const actual = await measureSettledViewport(tabId, width, height); return { ok: true, width, height, backend: "main-world", - chromeInset: { w: chromeW, h: chromeH } + chromeInset: { w: chromeW, h: chromeH }, + ...verifiedViewportResult(actual, width, height) } as unknown as Json; } diff --git a/packages/extension/tests/background/meta-tool-router.test.ts b/packages/extension/tests/background/meta-tool-router.test.ts index 3c0fc93..3d3610c 100644 --- a/packages/extension/tests/background/meta-tool-router.test.ts +++ b/packages/extension/tests/background/meta-tool-router.test.ts @@ -132,18 +132,50 @@ describe("navigation", () => { it("compensates for browser chrome when resizing", async () => { const update = vi.fn(async () => undefined); + const executeScript = vi.fn() + .mockResolvedValueOnce([{ result: { iw: 1000, ih: 700, ow: 1016, oh: 790, dpr: 1 } }]) + .mockResolvedValue([{ result: { iw: 1280, ih: 800, ow: 1296, oh: 890, dpr: 2 } }]); fakeChrome({ scripting: { - executeScript: vi.fn(async () => [{ result: { iw: 1000, ih: 700, ow: 1016, oh: 790 } }]) + executeScript }, tabs: { get: vi.fn(async () => ({ windowId: 9 })) }, windows: { update } }); const out = (await META_TOOLS.resize({ width: 1280, height: 800 } as never, 1)) as unknown as { chromeInset: { w: number; h: number }; + actualViewport: { width: number; height: number }; + devicePixelRatio: number; + verified: boolean; }; expect(update).toHaveBeenCalledWith(9, { width: 1280 + 16, height: 800 + 90 }); expect(out.chromeInset).toEqual({ w: 16, h: 90 }); + expect(out.actualViewport).toEqual({ width: 1280, height: 800 }); + expect(out.devicePixelRatio).toBe(2); + expect(out.verified).toBe(true); + }); + + it("reports a viewport mismatch instead of pretending the requested size applied", async () => { + vi.useFakeTimers(); + try { + fakeChrome({ + scripting: { + executeScript: vi.fn(async () => [{ result: { iw: 1920, ih: 887, ow: 1920, oh: 1008, dpr: 1 } }]) + }, + tabs: { get: vi.fn(async () => ({ windowId: 9 })) }, + windows: { update: vi.fn(async () => undefined) } + }); + const pending = META_TOOLS.resize({ width: 390, height: 844 } as never, 1) as Promise; + await vi.runAllTimersAsync(); + const out = await pending as { + actualViewport: { width: number; height: number }; + verified: boolean; + }; + expect(out.actualViewport).toEqual({ width: 1920, height: 887 }); + expect(out.verified).toBe(false); + } finally { + vi.useRealTimers(); + } }); it("requires both dimensions", async () => { diff --git a/packages/mcp-server/README.md b/packages/mcp-server/README.md index 76de11f..b230662 100644 --- a/packages/mcp-server/README.md +++ b/packages/mcp-server/README.md @@ -49,7 +49,7 @@ listen 端口的能力,所以方向只能如此。这也是配对页存在的 ## 工具面 - 控制面 4 个:`list_tabs / open_session / close_session / get_quota` -- 发现 1 个:`browser_discoverTools` —— 不带参数返回未发布工具目录(按 `export / network / storage / browser-data / inspect / legacy-dom / form / tabs` 分组);带 `enable: [...]` 把它们加进本进程的 `tools/list`(发送 `tools/list_changed`),并直接返回完整 schema。 +- 发现 1 个:`browser_discoverTools` —— 已知工具时直接传 `enable: [...]`,已知分组时直接传 `enableGroups: ["export", ...]`;只有两者都未知时才不带参数读取目录。启用后会加入本进程的 `tools/list`(发送 `tools/list_changed`),并直接返回完整 schema。 - 执行面默认 **core 31 个** `browser_*`:浏览 / 采集 / 填表 / 导航 / 截图 闭环所需的工具。其余 20 个由 AI 按需 `discoverTools` 拉取,用户不用配置。 - 不暴露 `askUser`(MCP 会话没有人在侧边栏应答)和 `attachTab` / `detachTab`(目标 tab 已由 `open_session` 绑定)。 @@ -70,6 +70,20 @@ hover / uploadFile / scroll / waitFor`);`browser_storage` 也不会列出, 无法识别的值(包括已移除的 `parity`)按 `core` 处理,并往 stderr 打一条提示。 +## 本地会话效率分析 + +在仓库根目录运行: + +```bash +pnpm analyze:sessions +pnpm analyze:sessions -- --since 7d --clients claude --format json +``` + +分析器只读扫描 `~/.claude/projects` 与 `~/.codex/sessions` 中最近 30 天的 +JSONL,识别真实 AtWebPilot 工具调用及重复验证、固定等待、重复截图、可批量填表等 +模式。输出只包含聚合计数、参数字段名与不可逆指纹;不会输出或保存 prompt、参数值、 +工具结果正文、凭证或绝对会话路径。使用 `--help` 查看目录覆盖等选项。 + ### 从 0.0.70 及更早版本迁移 | 旧 | 新 | diff --git a/packages/mcp-server/package.json b/packages/mcp-server/package.json index 7e00311..9de4136 100644 --- a/packages/mcp-server/package.json +++ b/packages/mcp-server/package.json @@ -14,7 +14,8 @@ "typecheck": "tsc --noEmit", "test": "vitest run", "build": "tsup", - "start": "tsx src/index.ts" + "start": "tsx src/index.ts", + "analyze:sessions": "tsx src/session-analysis/cli.ts" }, "dependencies": { "@modelcontextprotocol/sdk": "^1.0.0", diff --git a/packages/mcp-server/src/discover-tool.ts b/packages/mcp-server/src/discover-tool.ts index 922019f..1a7f2cd 100644 --- a/packages/mcp-server/src/discover-tool.ts +++ b/packages/mcp-server/src/discover-tool.ts @@ -1,4 +1,12 @@ -import { discoveryCatalog, type CatalogEntry, type GeneratedTool } from "./tool-gen"; +import { + DISCOVERABLE_GROUPS, + discoveryCatalog, + type CatalogEntry, + type DiscoverGroup, + type GeneratedTool +} from "./tool-gen"; + +const DISCOVER_GROUPS = Object.keys(DISCOVERABLE_GROUPS) as DiscoverGroup[]; export const DISCOVER_TOOL: { name: "browser_discoverTools"; @@ -7,14 +15,19 @@ export const DISCOVER_TOOL: { } = { name: "browser_discoverTools", description: - "The default tool list is the core set. Call with no arguments to see the catalog of additional browser tools " + - "(export, network, storage, browser-data, inspect, legacy-dom, form). Call with enable=[names] to add them to " + - "tools/list for this server; the response includes their full schemas so you can call them immediately. " + + "The default tool list is the core set. Enable known extras directly with enable=[names] or enableGroups=[groups]. " + + "Omit both only when you need the catalog. Groups: export, network, storage, browser-data, inspect, legacy-dom, form, tabs. " + + "Enabled tools join tools/list and the response includes their full schemas so you can call them immediately. " + "Use this instead of rebuilding missing capabilities with runJS.", inputSchema: { type: "object", properties: { - enable: { type: "array", items: { type: "string" }, description: "Tool names from the catalog to advertise" } + enable: { type: "array", items: { type: "string" }, description: "Tool names from the catalog to advertise" }, + enableGroups: { + type: "array", + items: { type: "string", enum: DISCOVER_GROUPS }, + description: "Known groups to advertise directly, without a catalog round trip" + } }, additionalProperties: false } @@ -26,6 +39,8 @@ export type DiscoverResult = { unknown?: string[]; /** Requested names that exist but the connected worker cannot run. */ unsupported?: string[]; + /** Requested group values that do not exist. */ + unknownGroups?: string[]; /** True when `advertised` grew; the caller sends tools/list_changed. */ changed: boolean; }; @@ -43,20 +58,34 @@ export function handleDiscover(input: { }): DiscoverResult { const { all, advertised, args, supported } = input; const byName = new Map(all.map((t) => [t.name, t])); - const requested = Array.isArray(args.enable) ? (args.enable as unknown[]).map(String) : null; - if (!requested) { + const hasSelection = Array.isArray(args.enable) || Array.isArray(args.enableGroups); + if (!hasSelection) { const runnable = all.filter((t) => isRunnable(t, supported)); return { catalog: discoveryCatalog(runnable, advertised), changed: false }; } + const explicit = Array.isArray(args.enable) ? (args.enable as unknown[]).map(String) : []; + const groups = Array.isArray(args.enableGroups) ? (args.enableGroups as unknown[]).map(String) : []; + const unknownGroups = groups.filter((group) => !DISCOVER_GROUPS.includes(group as DiscoverGroup)); + const grouped = groups.flatMap((group) => + DISCOVERABLE_GROUPS[group as DiscoverGroup] ?? [] + ); + const requestedByName = new Map(); + for (const raw of explicit) { + const name = raw.startsWith("browser_") ? raw : `browser_${raw}`; + requestedByName.set(name, raw); + } + for (const name of grouped) { + if (!requestedByName.has(name)) requestedByName.set(name, name); + } + const enabled: DiscoverResult["enabled"] = []; const unknown: string[] = []; const unsupported: string[] = []; - for (const raw of requested) { - const name = raw.startsWith("browser_") ? raw : `browser_${raw}`; + for (const [name, reportAs] of requestedByName) { const t = byName.get(name); - if (!t) { unknown.push(raw); continue; } - if (!isRunnable(t, supported)) { unsupported.push(raw); continue; } + if (!t) { unknown.push(reportAs); continue; } + if (!isRunnable(t, supported)) { unsupported.push(reportAs); continue; } if (advertised.has(name)) continue; advertised.add(name); enabled.push({ name: t.name, description: t.description, inputSchema: t.inputSchema }); @@ -64,6 +93,7 @@ export function handleDiscover(input: { return { enabled, ...(unknown.length ? { unknown } : {}), + ...(unknownGroups.length ? { unknownGroups } : {}), ...(unsupported.length ? { unsupported } : {}), changed: enabled.length > 0 }; diff --git a/packages/mcp-server/src/mcp-server.ts b/packages/mcp-server/src/mcp-server.ts index 241227e..307c86b 100644 --- a/packages/mcp-server/src/mcp-server.ts +++ b/packages/mcp-server/src/mcp-server.ts @@ -84,10 +84,15 @@ const ok = (data: unknown): CallResult => ({ */ function toolResult(gen: GeneratedTool, data: unknown): CallResult { if (gen.resultKind !== "image") return ok(data); - const d = (data ?? {}) as { data?: unknown; media_type?: unknown }; + const d = (data ?? {}) as Record; if (typeof d.data !== "string") return ok(data); const mimeType = typeof d.media_type === "string" ? d.media_type : "image/png"; - return { content: [{ type: "image", data: d.data, mimeType }] }; + const { data: imageData, media_type: _mediaType, ...metadata } = d; + const content: ContentBlock[] = [{ type: "image", data: imageData as string, mimeType }]; + if (Object.keys(metadata).length > 0) { + content.push({ type: "text", text: JSON.stringify(metadata) }); + } + return { content }; } const fail = (message: string): CallResult => ({ content: [{ type: "text", text: message }], isError: true }); diff --git a/packages/mcp-server/src/session-analysis/adapters.ts b/packages/mcp-server/src/session-analysis/adapters.ts new file mode 100644 index 0000000..584c917 --- /dev/null +++ b/packages/mcp-server/src/session-analysis/adapters.ts @@ -0,0 +1,354 @@ +import { createHash } from "node:crypto"; +import type { + EventSignals, + ParsedSession, + SessionClient, + ToolEvent +} from "./types"; + +type JsonObject = Record; + +type ParseInput = { + client: SessionClient; + lines: AsyncIterable; + fallbackSessionKey: string; + sinceMs: number; +}; + +function isObject(value: unknown): value is JsonObject { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function asObject(value: unknown): JsonObject { + return isObject(value) ? value : {}; +} + +function hash(value: string): string { + return createHash("sha256").update(value).digest("hex"); +} + +function stableStringify(value: unknown): string { + if (Array.isArray(value)) return `[${value.map(stableStringify).join(",")}]`; + if (isObject(value)) { + return `{${Object.keys(value).sort().map((key) => `${JSON.stringify(key)}:${stableStringify(value[key])}`).join(",")}}`; + } + return JSON.stringify(value) ?? "null"; +} + +function normalizeToolName(name: string): string | null { + const prefixes = [/^mcp__atwebpilot__/, /^atwebpilot__/, /^atwebpilot\./]; + for (const prefix of prefixes) { + if (prefix.test(name)) return name.replace(prefix, ""); + } + return null; +} + +function parseArguments(value: unknown): JsonObject { + if (isObject(value)) return value; + if (typeof value !== "string") return {}; + try { + return asObject(JSON.parse(value)); + } catch { + return {}; + } +} + +function signalsFor(tool: string, args: JsonObject): EventSignals { + if (tool === "browser_waitFor") { + return { + fixedWait: + typeof args.ms === "number" && + args.selector == null && + args.text == null && + args.textGone == null + }; + } + if (tool === "browser_runJS") { + const source = typeof args.source === "string" ? args.source : ""; + return { + viewportProbe: /\b(?:innerWidth|innerHeight|outerWidth|outerHeight|devicePixelRatio|visualViewport)\b|\bscreen\s*\./.test(source) + }; + } + if (tool === "browser_discoverTools") { + const enable = Array.isArray(args.enable) ? args.enable : []; + const enableGroups = Array.isArray(args.enableGroups) ? args.enableGroups : []; + return { + emptyDiscovery: enable.length === 0 && enableGroups.length === 0, + enablesDiscovery: enable.length > 0 || enableGroups.length > 0 + }; + } + return {}; +} + +function timestampOf(record: JsonObject): number | undefined { + const raw = record.timestamp; + if (typeof raw !== "string" && typeof raw !== "number") return undefined; + const value = typeof raw === "number" ? raw : Date.parse(raw); + return Number.isFinite(value) ? value : undefined; +} + +function numeric(value: unknown): number { + return typeof value === "number" && Number.isFinite(value) ? value : 0; +} + +function claudeUsage(message: JsonObject): { input: number; output: number } { + const usage = asObject(message.usage); + return { + input: + numeric(usage.input_tokens) + + numeric(usage.cache_creation_input_tokens) + + numeric(usage.cache_read_input_tokens), + output: numeric(usage.output_tokens) + }; +} + +function contentBlocks(value: unknown): JsonObject[] { + if (Array.isArray(value)) return value.filter(isObject); + return isObject(value) ? [value] : []; +} + +function safeSessionKey(client: SessionClient, raw: unknown, fallback: string): string { + const source = typeof raw === "string" && raw ? raw : fallback; + return hash(`${client}:${source}`).slice(0, 24); +} + +function safeRoundKey(sessionKey: string, raw: unknown, sequence: number): string { + const source = typeof raw === "string" && raw ? raw : `sequence:${sequence}`; + return hash(`${sessionKey}:${source}`).slice(0, 24); +} + +function makeExactEvent(input: { + client: SessionClient; + sessionKey: string; + roundKey: string; + sequence: number; + timestampMs?: number; + callId: string; + rawTool: string; + args: JsonObject; + inputTokens?: number; + outputTokens?: number; +}): ToolEvent | null { + const tool = normalizeToolName(input.rawTool); + if (!tool) return null; + return { + id: hash(`${input.sessionKey}:${input.callId}`).slice(0, 24), + client: input.client, + sessionKey: input.sessionKey, + roundKey: input.roundKey, + sequence: input.sequence, + ...(input.timestampMs == null ? {} : { timestampMs: input.timestampMs }), + tool, + argFields: Object.keys(input.args).sort(), + argHash: hash(stableStringify(input.args)), + confidence: "exact", + signals: signalsFor(tool, input.args), + ...(input.inputTokens == null ? {} : { roundInputTokens: input.inputTokens }), + ...(input.outputTokens == null ? {} : { roundOutputTokens: input.outputTokens }) + }; +} + +function attachResult( + callId: unknown, + callById: Map, + line: string, + isError: boolean +): void { + if (typeof callId !== "string") return; + const event = callById.get(callId); + if (!event) return; + event.resultBytes = Buffer.byteLength(line, "utf8"); + event.isError = isError; +} + +function nestedCodexEvents(input: { + record: JsonObject; + payload: JsonObject; + sessionKey: string; + startSequence: number; +}): ToolEvent[] { + if (input.payload.name !== "exec" && input.payload.name !== "functions.exec") return []; + if (typeof input.payload.input !== "string") return []; + const events: ToolEvent[] = []; + for (const rawTool of executableNestedToolNames(input.payload.input)) { + const tool = normalizeToolName(rawTool); + if (!tool) continue; + const sequence = input.startSequence + events.length; + const callId = `${String(input.payload.call_id ?? input.payload.id ?? "exec")}:${events.length}`; + events.push({ + id: hash(`${input.sessionKey}:${callId}`).slice(0, 24), + client: "codex", + sessionKey: input.sessionKey, + roundKey: safeRoundKey(input.sessionKey, input.payload.id ?? input.payload.call_id, sequence), + sequence, + ...(timestampOf(input.record) == null ? {} : { timestampMs: timestampOf(input.record) }), + tool, + argFields: [], + confidence: "derived", + signals: {} + }); + } + return events; +} + +/** + * Finds nested MCP calls in executable JavaScript while ignoring quoted patch + * bodies, examples and comments. This is intentionally a small lexer rather + * than a JavaScript parser; generated orchestration calls use the simple + * `tools.(...)` form. + */ +function executableNestedToolNames(source: string): string[] { + const names: string[] = []; + const prefix = "tools.mcp__atwebpilot__"; + let quote: "'" | '"' | "`" | null = null; + let lineComment = false; + let blockComment = false; + let escaped = false; + + for (let index = 0; index < source.length; index++) { + const char = source[index]; + const next = source[index + 1]; + if (lineComment) { + if (char === "\n") lineComment = false; + continue; + } + if (blockComment) { + if (char === "*" && next === "/") { + blockComment = false; + index++; + } + continue; + } + if (quote) { + if (escaped) { + escaped = false; + } else if (char === "\\") { + escaped = true; + } else if (char === quote) { + quote = null; + } + continue; + } + if (char === "/" && next === "/") { + lineComment = true; + index++; + continue; + } + if (char === "/" && next === "*") { + blockComment = true; + index++; + continue; + } + if (char === "'" || char === '"' || char === "`") { + quote = char; + continue; + } + if (!source.startsWith(prefix, index)) continue; + + let end = index + "tools.".length; + while (end < source.length && /[A-Za-z0-9_]/.test(source[end])) end++; + let cursor = end; + while (cursor < source.length && /\s/.test(source[cursor])) cursor++; + if (source[cursor] === "(") names.push(source.slice(index + "tools.".length, end)); + index = end - 1; + } + return names; +} + +function candidateLine(client: SessionClient, line: string): boolean { + return client === "claude" + ? line.includes('"tool_use"') || line.includes('"tool_result"') + : line.includes("function_call") || line.includes("custom_tool_call"); +} + +export async function parseSessionLines(input: ParseInput): Promise { + const events: ToolEvent[] = []; + const callById = new Map(); + let malformedLines = 0; + let sequence = 0; + + for await (const chunk of input.lines) { + const line = chunk.toString().trim(); + if (!line) continue; + if (!candidateLine(input.client, line)) continue; + let record: JsonObject; + try { + record = asObject(JSON.parse(line)); + } catch { + malformedLines++; + continue; + } + + const timestampMs = timestampOf(record); + if (timestampMs != null && timestampMs < input.sinceMs) continue; + + if (input.client === "claude") { + const message = asObject(record.message); + if (record.type === "assistant") { + const usage = claudeUsage(message); + const sessionKey = safeSessionKey("claude", record.sessionId ?? record.session_id, input.fallbackSessionKey); + const roundKey = safeRoundKey(sessionKey, message.id ?? record.uuid, sequence); + for (const block of contentBlocks(message.content)) { + if (block.type !== "tool_use" || typeof block.name !== "string") continue; + const callId = typeof block.id === "string" ? block.id : `line:${sequence}`; + const event = makeExactEvent({ + client: "claude", + sessionKey, + roundKey, + sequence, + timestampMs, + callId, + rawTool: block.name, + args: asObject(block.input), + inputTokens: usage.input, + outputTokens: usage.output + }); + if (!event) continue; + sequence++; + events.push(event); + callById.set(callId, event); + } + } else if (record.type === "user") { + for (const block of contentBlocks(message.content)) { + if (block.type !== "tool_result") continue; + attachResult(block.tool_use_id, callById, line, block.is_error === true); + } + } + continue; + } + + if (record.type !== "response_item") continue; + const payload = asObject(record.payload); + if (payload.type === "function_call_output" || payload.type === "custom_tool_call_output") { + attachResult(payload.call_id, callById, line, false); + continue; + } + if (payload.type !== "function_call" && payload.type !== "custom_tool_call") continue; + + const sessionKey = safeSessionKey("codex", record.session_id ?? record.sessionId, input.fallbackSessionKey); + const derived = nestedCodexEvents({ record, payload, sessionKey, startSequence: sequence }); + if (derived.length > 0) { + events.push(...derived); + sequence += derived.length; + continue; + } + if (typeof payload.name !== "string") continue; + const callId = typeof payload.call_id === "string" ? payload.call_id : `line:${sequence}`; + const event = makeExactEvent({ + client: "codex", + sessionKey, + roundKey: safeRoundKey(sessionKey, payload.id ?? payload.call_id, sequence), + sequence, + timestampMs, + callId, + rawTool: payload.name, + args: parseArguments(payload.arguments ?? payload.input) + }); + if (!event) continue; + sequence++; + events.push(event); + callById.set(callId, event); + } + + return { events, malformedLines }; +} diff --git a/packages/mcp-server/src/session-analysis/analyzer.ts b/packages/mcp-server/src/session-analysis/analyzer.ts new file mode 100644 index 0000000..f3a9e2e --- /dev/null +++ b/packages/mcp-server/src/session-analysis/analyzer.ts @@ -0,0 +1,221 @@ +import type { + AnalysisContext, + AnalysisReport, + Finding, + FindingId, + ToolEvent +} from "./types"; + +const FINDING_COPY: Record = { + viewport_recheck_after_resize: { + description: "runJS re-read viewport data shortly after resize", + suggestion: "Trust resize.actualViewport/verified; use runJS only when verified is false or more state is needed." + }, + viewport_recheck_after_screenshot: { + description: "runJS re-read viewport data shortly after screenshot", + suggestion: "Use the screenshot metadata block for backend and dimensions." + }, + fixed_wait: { + description: "waitFor used a fixed delay instead of a page condition", + suggestion: "Prefer selector/text/textGone or an action-specific settle condition." + }, + discovery_round_trip: { + description: "an empty tool catalog lookup was followed by tool/group activation", + suggestion: "Enable a known tool or enableGroups directly." + }, + duplicate_call: { + description: "the same tool and argument fingerprint repeated nearby", + suggestion: "Reuse the prior result or return a stronger postcondition from the first call." + }, + repeated_screenshot: { + description: "screenshots repeated within a small call window", + suggestion: "Capture once after the page settles, or target the relevant selector/block." + }, + serial_form_fill: { + description: "multiple fillInput calls were issued serially", + suggestion: "Use fillForm for independent fields so one model round can fill them together." + } +}; + +type MutableFinding = { + id: FindingId; + occurrences: number; + sessions: Set; + avoidableEventIds: Set; +}; + +function finding(map: Map, id: FindingId): MutableFinding { + let current = map.get(id); + if (!current) { + current = { id, occurrences: 0, sessions: new Set(), avoidableEventIds: new Set() }; + map.set(id, current); + } + return current; +} + +function addFinding( + map: Map, + id: FindingId, + sessionKey: string, + eventIds: string[] +): void { + const current = finding(map, id); + current.occurrences++; + current.sessions.add(sessionKey); + for (const eventId of eventIds) current.avoidableEventIds.add(eventId); +} + +function bySession(events: ToolEvent[]): Map { + const sessions = new Map(); + for (const event of events) { + const current = sessions.get(event.sessionKey) ?? []; + current.push(event); + sessions.set(event.sessionKey, current); + } + for (const current of sessions.values()) { + current.sort((a, b) => + (a.timestampMs ?? 0) - (b.timestampMs ?? 0) || a.sequence - b.sequence + ); + } + return sessions; +} + +function isStateBoundary(tool: string): boolean { + return /^(?:browser_)?(?:click|clickByUid|fill|fillInput|fillByUid|fillForm|selectOption|setCheckbox|submitForm|navigate|openTab|closeTab|switchToTab|scroll|resize|pressKey|drag|drop|uploadFile|runJS|waitFor)$/.test(tool); +} + +function previousStableWindow(events: ToolEvent[], index: number): ToolEvent[] { + const previous: ToolEvent[] = []; + for (let cursor = index - 1; cursor >= 0 && previous.length < 3; cursor--) { + const candidate = events[cursor]; + previous.unshift(candidate); + if (isStateBoundary(candidate.tool)) break; + } + return previous; +} + +export function analyzeEvents(events: ToolEvent[], context: AnalysisContext): AnalysisReport { + const findings = new Map(); + const avoidableEventIds = new Set(); + + for (const current of bySession(events).values()) { + for (let index = 0; index < current.length; index++) { + const event = current[index]; + const previous = current.slice(Math.max(0, index - 3), index); + const stablePrevious = previousStableWindow(current, index); + + if (event.signals.fixedWait) { + addFinding(findings, "fixed_wait", event.sessionKey, [event.id]); + } + if (event.signals.viewportProbe) { + const trigger = [...stablePrevious].reverse().find((candidate) => + candidate.tool === "browser_resize" || candidate.tool === "browser_screenshot" + ); + if (trigger) { + addFinding( + findings, + trigger.tool === "browser_resize" + ? "viewport_recheck_after_resize" + : "viewport_recheck_after_screenshot", + event.sessionKey, + [event.id] + ); + } + } + if (event.signals.enablesDiscovery) { + const catalog = [...previous].reverse().find((candidate) => candidate.signals.emptyDiscovery); + if (catalog) addFinding(findings, "discovery_round_trip", event.sessionKey, [catalog.id]); + } + if ( + event.argHash && + stablePrevious.some((candidate) => candidate.tool === event.tool && candidate.argHash === event.argHash) + ) { + addFinding(findings, "duplicate_call", event.sessionKey, [event.id]); + } + if ( + event.tool === "browser_screenshot" && + stablePrevious.some((candidate) => candidate.tool === "browser_screenshot") + ) { + addFinding(findings, "repeated_screenshot", event.sessionKey, [event.id]); + } + } + + for (let start = 0; start < current.length;) { + if (current[start].tool !== "browser_fillInput") { + start++; + continue; + } + let end = start + 1; + while (end < current.length && current[end].tool === "browser_fillInput") end++; + if (end - start >= 2) { + addFinding( + findings, + "serial_form_fill", + current[start].sessionKey, + current.slice(start + 1, end).map((event) => event.id) + ); + } + start = end; + } + } + + for (const current of findings.values()) { + for (const eventId of current.avoidableEventIds) avoidableEventIds.add(eventId); + } + + const tools = new Map(); + const sessions = new Set(); + const rounds = new Map(); + let exactCalls = 0; + let derivedCalls = 0; + let resultBytes = 0; + for (const event of events) { + tools.set(event.tool, (tools.get(event.tool) ?? 0) + 1); + sessions.add(event.sessionKey); + if (event.confidence === "exact") exactCalls++; + else derivedCalls++; + resultBytes += event.resultBytes ?? 0; + const roundId = `${event.sessionKey}:${event.roundKey}`; + const prior = rounds.get(roundId) ?? { input: 0, output: 0 }; + rounds.set(roundId, { + input: Math.max(prior.input, event.roundInputTokens ?? 0), + output: Math.max(prior.output, event.roundOutputTokens ?? 0) + }); + } + + const publicFindings: Finding[] = [...findings.values()] + .map((current) => ({ + id: current.id, + ...FINDING_COPY[current.id], + occurrences: current.occurrences, + affectedSessions: current.sessions.size, + estimatedAvoidableRounds: current.avoidableEventIds.size + })) + .sort((a, b) => b.estimatedAvoidableRounds - a.estimatedAvoidableRounds || a.id.localeCompare(b.id)); + + return { + generatedAt: context.generatedAt, + since: context.since, + clients: context.clients, + summary: { + toolCalls: events.length, + exactCalls, + derivedCalls, + sessions: sessions.size, + modelToolRounds: rounds.size, + toolRoundInputTokens: [...rounds.values()].reduce((sum, usage) => sum + usage.input, 0), + toolRoundOutputTokens: [...rounds.values()].reduce((sum, usage) => sum + usage.output, 0), + resultBytes, + estimatedAvoidableRounds: avoidableEventIds.size + }, + tools: [...tools.entries()] + .map(([name, calls]) => ({ name, calls })) + .sort((a, b) => b.calls - a.calls || a.name.localeCompare(b.name)), + findings: publicFindings, + privacy: { + rawContentRetained: false, + rawPathsRetained: false, + argumentValuesRetained: false + } + }; +} diff --git a/packages/mcp-server/src/session-analysis/cli-support.ts b/packages/mcp-server/src/session-analysis/cli-support.ts new file mode 100644 index 0000000..ddc8d32 --- /dev/null +++ b/packages/mcp-server/src/session-analysis/cli-support.ts @@ -0,0 +1,122 @@ +import { join } from "node:path"; +import type { AnalysisReport, CliOptions, SessionClient } from "./types"; + +const CLIENTS = new Set(["claude", "codex"]); + +function optionValue(args: string[], index: number, flag: string): string { + const value = args[index + 1]; + if (!value || value.startsWith("--")) throw new Error(`${flag} requires a value`); + return value; +} + +function durationMs(raw: string): number { + const match = /^(\d+)(h|d|w)$/.exec(raw); + if (!match) throw new Error(`Invalid --since value ${JSON.stringify(raw)}; use e.g. 24h, 7d, or 4w`); + const amount = Number(match[1]); + const unit = match[2] === "h" ? 60 * 60_000 : match[2] === "d" ? 24 * 60 * 60_000 : 7 * 24 * 60 * 60_000; + return amount * unit; +} + +export function parseCliArgs(args: string[], homeDir: string, now = new Date()): CliOptions { + let clients: SessionClient[] = ["claude", "codex"]; + let since = "30d"; + let format: CliOptions["format"] = "text"; + let claudeDir = join(homeDir, ".claude", "projects"); + let codexDir = join(homeDir, ".codex", "sessions"); + let help = false; + + for (let index = 0; index < args.length; index++) { + const arg = args[index]; + if (arg === "--") continue; + if (arg === "--help" || arg === "-h") { + help = true; + continue; + } + if (arg === "--clients") { + const raw = optionValue(args, index, arg); + index++; + clients = raw.split(",").map((value) => value.trim()).filter(Boolean) as SessionClient[]; + if (clients.length === 0 || clients.some((client) => !CLIENTS.has(client))) { + throw new Error(`Invalid client list ${JSON.stringify(raw)}; use claude,codex`); + } + clients = [...new Set(clients)]; + continue; + } + if (arg === "--since") { + since = optionValue(args, index, arg); + index++; + continue; + } + if (arg === "--format") { + const raw = optionValue(args, index, arg); + index++; + if (raw !== "text" && raw !== "json") throw new Error(`Invalid --format ${JSON.stringify(raw)}; use text or json`); + format = raw; + continue; + } + if (arg === "--claude-dir" || arg === "--codex-dir") { + const raw = optionValue(args, index, arg); + index++; + if (arg === "--claude-dir") claudeDir = raw; + else codexDir = raw; + continue; + } + throw new Error(`Unknown argument: ${arg}`); + } + + const sinceMs = now.getTime() - durationMs(since); + return { clients, sinceMs, sinceLabel: new Date(sinceMs).toISOString(), format, claudeDir, codexDir, help }; +} + +function bytes(value: number): string { + if (value < 1024) return `${value} B`; + if (value < 1024 * 1024) return `${(value / 1024).toFixed(1)} KiB`; + return `${(value / (1024 * 1024)).toFixed(1)} MiB`; +} + +export function renderReport(report: AnalysisReport, format: "text" | "json"): string { + if (format === "json") return JSON.stringify(report, null, 2); + const lines = [ + "AtWebPilot local session analysis", + `Period: ${report.since} -> ${report.generatedAt}`, + "", + `Calls: ${report.summary.toolCalls} (${report.summary.exactCalls} exact, ${report.summary.derivedCalls} derived)`, + `Sessions: ${report.summary.sessions}; model tool rounds: ${report.summary.modelToolRounds}`, + `Known tool-round tokens: ${report.summary.toolRoundInputTokens} input / ${report.summary.toolRoundOutputTokens} output`, + `Recorded result bytes: ${bytes(report.summary.resultBytes)}`, + `Estimated avoidable rounds: ${report.summary.estimatedAvoidableRounds}`, + "", + "Clients:" + ]; + for (const client of ["claude", "codex"] as const) { + const stats = report.clients[client]; + lines.push( + ` ${client}: ${stats.available ? "available" : "unavailable"}; files ${stats.filesScanned}/${stats.filesSeen}; calls ${stats.toolCalls} (${stats.exactCalls} exact, ${stats.derivedCalls} derived); malformed candidate lines ${stats.malformedLines}` + ); + } + lines.push("", "Top tools:"); + for (const tool of report.tools.slice(0, 15)) lines.push(` ${tool.calls.toString().padStart(5)} ${tool.name}`); + lines.push("", "Opportunities:"); + if (report.findings.length === 0) lines.push(" No configured patterns found."); + for (const finding of report.findings) { + lines.push( + ` ${finding.id}: ${finding.occurrences} occurrence(s), ${finding.affectedSessions} session(s), ~${finding.estimatedAvoidableRounds} avoidable round(s)`, + ` ${finding.suggestion}` + ); + } + lines.push("", "Privacy: raw prompts, argument values, result bodies and session paths were not retained."); + return lines.join("\n"); +} + +export const HELP_TEXT = `Usage: pnpm analyze:sessions -- [options] + +Options: + --since Lookback such as 24h, 7d, 4w (default: 30d) + --clients claude,codex (default: both) + --format Output format (default: text) + --claude-dir Override ~/.claude/projects + --codex-dir Override ~/.codex/sessions + -h, --help Show this help + +The analyzer is read-only and never prints raw prompts, tool arguments, tool +results, credentials, or absolute session paths.`; diff --git a/packages/mcp-server/src/session-analysis/cli.ts b/packages/mcp-server/src/session-analysis/cli.ts new file mode 100644 index 0000000..19975b2 --- /dev/null +++ b/packages/mcp-server/src/session-analysis/cli.ts @@ -0,0 +1,38 @@ +import { homedir } from "node:os"; +import { pathToFileURL } from "node:url"; +import { analyzeEvents } from "./analyzer"; +import { HELP_TEXT, parseCliArgs, renderReport } from "./cli-support"; +import { scanHistories } from "./scanner"; + +export async function main(args = process.argv.slice(2)): Promise { + let options; + try { + options = parseCliArgs(args, homedir()); + } catch (error) { + process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n\n${HELP_TEXT}\n`); + return 2; + } + if (options.help) { + process.stdout.write(`${HELP_TEXT}\n`); + return 0; + } + + const scan = await scanHistories(options); + const report = analyzeEvents(scan.events, { + generatedAt: new Date().toISOString(), + since: options.sinceLabel, + clients: scan.clients + }); + process.stdout.write(`${renderReport(report, options.format)}\n`); + return 0; +} + +const invokedPath = process.argv[1] ? pathToFileURL(process.argv[1]).href : ""; +if (import.meta.url === invokedPath) { + main().then((code) => { + process.exitCode = code; + }).catch((error) => { + process.stderr.write(`Session analysis failed: ${error instanceof Error ? error.message : String(error)}\n`); + process.exitCode = 1; + }); +} diff --git a/packages/mcp-server/src/session-analysis/scanner.ts b/packages/mcp-server/src/session-analysis/scanner.ts new file mode 100644 index 0000000..e3e172a --- /dev/null +++ b/packages/mcp-server/src/session-analysis/scanner.ts @@ -0,0 +1,107 @@ +import { createReadStream } from "node:fs"; +import { opendir, stat } from "node:fs/promises"; +import { join } from "node:path"; +import { createInterface } from "node:readline"; +import { parseSessionLines } from "./adapters"; +import type { ClientScanStats, CliOptions, SessionClient, ToolEvent } from "./types"; + +type ScanResult = { + events: ToolEvent[]; + clients: Record; +}; + +function emptyStats(available: boolean): ClientScanStats { + return { + available, + filesSeen: 0, + filesScanned: 0, + malformedLines: 0, + toolCalls: 0, + exactCalls: 0, + derivedCalls: 0 + }; +} + +async function directoryExists(path: string): Promise { + try { + return (await stat(path)).isDirectory(); + } catch { + return false; + } +} + +async function* jsonlFiles(root: string): AsyncGenerator { + const pending = [root]; + while (pending.length > 0) { + const current = pending.pop()!; + let directory; + try { + directory = await opendir(current); + } catch { + continue; + } + for await (const entry of directory) { + const path = join(current, entry.name); + if (entry.isDirectory()) pending.push(path); + else if (entry.isFile() && entry.name.endsWith(".jsonl")) yield path; + } + } +} + +async function scanClient( + client: SessionClient, + root: string, + sinceMs: number +): Promise<{ events: ToolEvent[]; stats: ClientScanStats }> { + const available = await directoryExists(root); + const stats = emptyStats(available); + const events: ToolEvent[] = []; + if (!available) return { events, stats }; + + for await (const path of jsonlFiles(root)) { + stats.filesSeen++; + let fileStat; + try { + fileStat = await stat(path); + } catch { + continue; + } + if (fileStat.mtimeMs < sinceMs) continue; + stats.filesScanned++; + + try { + const stream = createReadStream(path, { encoding: "utf8" }); + const lines = createInterface({ input: stream, crlfDelay: Infinity }); + const parsed = await parseSessionLines({ + client, + lines, + fallbackSessionKey: path, + sinceMs + }); + events.push(...parsed.events); + stats.malformedLines += parsed.malformedLines; + stats.toolCalls += parsed.events.length; + stats.exactCalls += parsed.events.filter((event) => event.confidence === "exact").length; + stats.derivedCalls += parsed.events.filter((event) => event.confidence === "derived").length; + } catch { + // A concurrently deleted or unreadable file should not invalidate the scan. + stats.malformedLines++; + } + } + + return { events, stats }; +} + +export async function scanHistories(options: CliOptions): Promise { + const selected = new Set(options.clients); + const claude = selected.has("claude") + ? await scanClient("claude", options.claudeDir, options.sinceMs) + : { events: [], stats: emptyStats(false) }; + const codex = selected.has("codex") + ? await scanClient("codex", options.codexDir, options.sinceMs) + : { events: [], stats: emptyStats(false) }; + return { + events: [...claude.events, ...codex.events], + clients: { claude: claude.stats, codex: codex.stats } + }; +} diff --git a/packages/mcp-server/src/session-analysis/types.ts b/packages/mcp-server/src/session-analysis/types.ts new file mode 100644 index 0000000..c240b28 --- /dev/null +++ b/packages/mcp-server/src/session-analysis/types.ts @@ -0,0 +1,103 @@ +export type SessionClient = "claude" | "codex"; +export type EventConfidence = "exact" | "derived"; + +export type EventSignals = { + fixedWait?: boolean; + viewportProbe?: boolean; + emptyDiscovery?: boolean; + enablesDiscovery?: boolean; +}; + +export type ToolEvent = { + /** Non-reversible identifier used only to deduplicate findings. */ + id: string; + client: SessionClient; + /** SHA-256-derived session identifier; never the client's raw session id/path. */ + sessionKey: string; + /** Hashed or synthetic model response identifier. */ + roundKey: string; + sequence: number; + timestampMs?: number; + tool: string; + argFields: string[]; + argHash?: string; + resultBytes?: number; + isError?: boolean; + confidence: EventConfidence; + signals: EventSignals; + roundInputTokens?: number; + roundOutputTokens?: number; +}; + +export type ParsedSession = { + events: ToolEvent[]; + malformedLines: number; +}; + +export type ClientScanStats = { + available: boolean; + filesSeen: number; + filesScanned: number; + malformedLines: number; + toolCalls: number; + exactCalls: number; + derivedCalls: number; +}; + +export type FindingId = + | "viewport_recheck_after_resize" + | "viewport_recheck_after_screenshot" + | "fixed_wait" + | "discovery_round_trip" + | "duplicate_call" + | "repeated_screenshot" + | "serial_form_fill"; + +export type Finding = { + id: FindingId; + description: string; + suggestion: string; + occurrences: number; + affectedSessions: number; + estimatedAvoidableRounds: number; +}; + +export type AnalysisReport = { + generatedAt: string; + since: string; + clients: Record; + summary: { + toolCalls: number; + exactCalls: number; + derivedCalls: number; + sessions: number; + modelToolRounds: number; + toolRoundInputTokens: number; + toolRoundOutputTokens: number; + resultBytes: number; + estimatedAvoidableRounds: number; + }; + tools: Array<{ name: string; calls: number }>; + findings: Finding[]; + privacy: { + rawContentRetained: false; + rawPathsRetained: false; + argumentValuesRetained: false; + }; +}; + +export type AnalysisContext = { + generatedAt: string; + since: string; + clients: Record; +}; + +export type CliOptions = { + clients: SessionClient[]; + sinceMs: number; + sinceLabel: string; + format: "text" | "json"; + claudeDir: string; + codexDir: string; + help: boolean; +}; diff --git a/packages/mcp-server/tests/discover-tool.test.ts b/packages/mcp-server/tests/discover-tool.test.ts index bd13a66..4ec2594 100644 --- a/packages/mcp-server/tests/discover-tool.test.ts +++ b/packages/mcp-server/tests/discover-tool.test.ts @@ -6,11 +6,12 @@ const all = generateBrowserTools("full"); const coreNames = () => new Set(generateBrowserTools("core").map((t) => t.name)); describe("browser_discoverTools", () => { - it("has an English description and an optional enable array", () => { + it("has an English description and optional tool/group enable arrays", () => { expect(DISCOVER_TOOL.name).toBe("browser_discoverTools"); expect(DISCOVER_TOOL.description).not.toMatch(/[一-鿿]/); const props = DISCOVER_TOOL.inputSchema.properties as Record; expect(props.enable.type).toBe("array"); + expect(props.enableGroups.type).toBe("array"); expect(DISCOVER_TOOL.inputSchema.required ?? []).not.toContain("enable"); }); @@ -49,6 +50,56 @@ describe("browser_discoverTools", () => { expect(r.enabled!.map((t) => t.name)).toEqual(["browser_downloadSpreadsheet"]); }); + it("preserves bare names when reporting unknown explicit tools", () => { + const advertised = coreNames(); + const r = handleDiscover({ all, advertised, args: { enable: ["nope"] } }); + expect(r.unknown).toEqual(["nope"]); + }); + + it("enables a known group directly and deduplicates explicit names", () => { + const advertised = coreNames(); + const r = handleDiscover({ + all, + advertised, + args: { + enableGroups: ["browser-data"], + enable: ["browser_searchHistory"] + } + }); + expect(r.enabled!.map((t) => t.name)).toEqual([ + "browser_searchHistory", + "browser_searchBookmarks" + ]); + expect(r.unknownGroups).toBeUndefined(); + }); + + it("reports unknown groups without blocking valid groups", () => { + const advertised = coreNames(); + const r = handleDiscover({ + all, + advertised, + args: { enableGroups: ["browser-data", "nope"] } + }); + expect(r.enabled!.map((t) => t.name)).toEqual([ + "browser_searchBookmarks", + "browser_searchHistory" + ]); + expect(r.unknownGroups).toEqual(["nope"]); + expect(r.changed).toBe(true); + }); + + it("applies worker support filtering to group activation", () => { + const advertised = coreNames(); + const r = handleDiscover({ + all, + advertised, + args: { enableGroups: ["browser-data"] }, + supported: new Set(["searchBookmarks"]) + }); + expect(r.enabled!.map((t) => t.name)).toEqual(["browser_searchBookmarks"]); + expect(r.unsupported).toEqual(["browser_searchHistory"]); + }); + it("catalog omits tools the worker cannot run", () => { const advertised = coreNames(); const supported = new Set(["downloadImage"]); diff --git a/packages/mcp-server/tests/mcp-server.test.ts b/packages/mcp-server/tests/mcp-server.test.ts index 322197a..1ba3cda 100644 --- a/packages/mcp-server/tests/mcp-server.test.ts +++ b/packages/mcp-server/tests/mcp-server.test.ts @@ -200,7 +200,14 @@ describe("image results", () => { coordinator.registerWorker(fakeWorker()); const shot: Result = { type: "RESULT", nonce: "n", ts: 1, protocol_version: 1, req_id: "req_1", ok: true, - return: { data: "QUJD", media_type: "image/png", byteLen: 3 } + return: { + data: "QUJD", + media_type: "image/png", + byteLen: 3, + backend: "cdp", + width: 390, + height: 844 + } }; return staticDeps(coordinator, { exec: async () => shot } as any); } @@ -214,6 +221,19 @@ describe("image results", () => { expect(r.content[0]).toEqual({ type: "image", data: "QUJD", mimeType: "image/png" }); }); + it("keeps compact screenshot metadata without duplicating image data", async () => { + const d = imageDeps(); + const open = await dispatchCall(d, "open_session", { tab_id: "42" }); + const session_id = JSON.parse(textOf(open)).session_id; + const r = await dispatchCall(d, "browser_screenshot", { session_id }); + + expect(r.content).toHaveLength(2); + expect(r.content[1].type).toBe("text"); + const metadata = JSON.parse((r.content[1] as { type: "text"; text: string }).text); + expect(metadata).toEqual({ byteLen: 3, backend: "cdp", width: 390, height: 844 }); + expect((r.content[1] as { type: "text"; text: string }).text).not.toContain("QUJD"); + }); + it("falls back to text when the payload is not an image", async () => { const d = deps(); const open = await dispatchCall(d, "open_session", { tab_id: "42" }); @@ -265,6 +285,32 @@ describe("discovery over MCP", () => { await Promise.all([client.close(), server.close()]); }); + it("enableGroups adds a known group in one call and emits list_changed once", async () => { + const state = createToolState("core"); + const coordinator = new Coordinator({ hub: { send: async () => undefined } as any, clock: new FakeClock(0), idGen: new FakeIdGen() }); + coordinator.registerWorker({ ...fakeWorker(), supported_tools: new Set(["searchBookmarks", "searchHistory"]) }); + const d = staticDeps(coordinator, { exec: async () => okResult } as any); + const server = createMcpServer(d, state); + const client = new Client({ name: "t", version: "1" }); + const [ct, st] = InMemoryTransport.createLinkedPair(); + const changes: number[] = []; + client.setNotificationHandler(ToolListChangedNotificationSchema, async () => { changes.push(1); }); + await Promise.all([server.connect(st), client.connect(ct)]); + + const r = await client.callTool({ name: "browser_discoverTools", arguments: { enableGroups: ["browser-data"] } }); + const body = JSON.parse((r.content as Array<{ text: string }>)[0].text); + expect(body.enabled.map((t: { name: string }) => t.name)).toEqual([ + "browser_searchBookmarks", + "browser_searchHistory" + ]); + await vi.waitFor(() => expect(changes).toHaveLength(1)); + + const after = (await client.listTools()).tools.map((t) => t.name); + expect(after).toContain("browser_searchBookmarks"); + expect(after).toContain("browser_searchHistory"); + await Promise.all([client.close(), server.close()]); + }); + it("a non-advertised tool is still callable by name (client cache may lag)", async () => { const d = deps(); const state = createToolState("core"); diff --git a/packages/mcp-server/tests/session-analysis.test.ts b/packages/mcp-server/tests/session-analysis.test.ts new file mode 100644 index 0000000..ac06062 --- /dev/null +++ b/packages/mcp-server/tests/session-analysis.test.ts @@ -0,0 +1,317 @@ +import { Readable } from "node:stream"; +import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; +import { analyzeEvents } from "../src/session-analysis/analyzer"; +import { parseSessionLines } from "../src/session-analysis/adapters"; +import { parseCliArgs, renderReport } from "../src/session-analysis/cli-support"; +import { scanHistories } from "../src/session-analysis/scanner"; +import type { ToolEvent } from "../src/session-analysis/types"; + +function jsonLines(records: unknown[]): AsyncIterable { + return Readable.from(records.map((record) => `${typeof record === "string" ? record : JSON.stringify(record)}\n`)); +} + +describe("local session adapters", () => { + it("extracts Claude tool calls/results without retaining raw values", async () => { + const records = [ + { + type: "assistant", + sessionId: "claude-session", + timestamp: "2026-09-18T10:00:00.000Z", + uuid: "assistant-1", + message: { + id: "msg-1", + role: "assistant", + usage: { input_tokens: 100, cache_read_input_tokens: 50, output_tokens: 7 }, + content: [{ + type: "tool_use", + id: "call-1", + name: "mcp__atwebpilot__browser_resize", + input: { session_id: "secret-token", width: 390, height: 844 } + }] + } + }, + { + type: "user", + sessionId: "claude-session", + timestamp: "2026-09-18T10:00:01.000Z", + message: { + role: "user", + content: [{ + type: "tool_result", + tool_use_id: "call-1", + content: "secret-result-body" + }] + } + } + ]; + + const parsed = await parseSessionLines({ + client: "claude", + lines: jsonLines(records), + fallbackSessionKey: "fallback", + sinceMs: 0 + }); + + expect(parsed.events).toHaveLength(1); + expect(parsed.events[0]).toMatchObject({ + client: "claude", + tool: "browser_resize", + argFields: ["height", "session_id", "width"], + confidence: "exact", + roundInputTokens: 150, + roundOutputTokens: 7 + }); + expect(parsed.events[0].argHash).toMatch(/^[a-f0-9]{64}$/); + expect(parsed.events[0].resultBytes).toBeGreaterThan(0); + expect(JSON.stringify(parsed)).not.toContain("secret-token"); + expect(JSON.stringify(parsed)).not.toContain("secret-result-body"); + }); + + it("extracts direct and nested Codex calls and skips malformed lines", async () => { + const records = [ + '{"type":"response_item","payload":{"type":"function_call"', + { + type: "response_item", + timestamp: "2026-09-18T10:00:00.000Z", + payload: { + type: "function_call", + call_id: "direct-1", + id: "item-1", + name: "mcp__atwebpilot__browser_waitFor", + arguments: JSON.stringify({ session_id: "secret-token", ms: 500 }) + } + }, + { + type: "response_item", + timestamp: "2026-09-18T10:00:01.000Z", + payload: { + type: "function_call_output", + call_id: "direct-1", + output: "secret-output" + } + }, + { + type: "response_item", + timestamp: "2026-09-18T10:00:02.000Z", + payload: { + type: "custom_tool_call", + call_id: "exec-1", + id: "item-2", + name: "exec", + input: [ + "const fake = 'await tools.mcp__atwebpilot__browser_resize({width: 1})';", + "// await tools.mcp__atwebpilot__browser_click({selector: '.x'});", + "const a = await tools.mcp__atwebpilot__browser_screenshot({session_id: token}); text(a);" + ].join("\n") + } + } + ]; + + const parsed = await parseSessionLines({ + client: "codex", + lines: jsonLines(records), + fallbackSessionKey: "codex-session", + sinceMs: 0 + }); + + expect(parsed.malformedLines).toBe(1); + expect(parsed.events).toHaveLength(2); + expect(parsed.events[0]).toMatchObject({ + tool: "browser_waitFor", + confidence: "exact", + signals: { fixedWait: true } + }); + expect(parsed.events[0].resultBytes).toBeGreaterThan(0); + expect(parsed.events[1]).toMatchObject({ + tool: "browser_screenshot", + confidence: "derived", + argFields: [] + }); + expect(JSON.stringify(parsed)).not.toContain("secret-token"); + expect(JSON.stringify(parsed)).not.toContain("secret-output"); + }); +}); + +function event( + sequence: number, + tool: string, + signals: ToolEvent["signals"] = {}, + argHash = `${tool}-${sequence}` +): ToolEvent { + return { + id: `event-${sequence}`, + client: "claude", + sessionKey: "session-a", + roundKey: `round-${sequence}`, + sequence, + tool, + argFields: [], + argHash, + confidence: "exact", + signals + }; +} + +describe("session efficiency analyzer", () => { + it("detects redundant verification, waits, discovery, duplicates and serial fills", () => { + const sameClickHash = "same-click"; + const events = [ + event(1, "browser_resize"), + event(2, "browser_screenshot"), + event(3, "browser_runJS", { viewportProbe: true }), + event(4, "browser_waitFor", { fixedWait: true }), + event(5, "browser_discoverTools", { emptyDiscovery: true }), + event(6, "browser_discoverTools", { enablesDiscovery: true }), + event(7, "browser_click", {}, sameClickHash), + event(8, "browser_click", {}, sameClickHash), + event(9, "browser_screenshot"), + event(10, "browser_screenshot"), + event(11, "browser_fillInput"), + event(12, "browser_fillInput"), + event(13, "browser_fillInput") + ]; + + const report = analyzeEvents(events, { + since: "2026-08-20T00:00:00.000Z", + generatedAt: "2026-09-19T00:00:00.000Z", + clients: { + claude: { available: true, filesSeen: 1, filesScanned: 1, malformedLines: 0, toolCalls: 13, exactCalls: 13, derivedCalls: 0 }, + codex: { available: false, filesSeen: 0, filesScanned: 0, malformedLines: 0, toolCalls: 0, exactCalls: 0, derivedCalls: 0 } + } + }); + const byId = new Map(report.findings.map((finding) => [finding.id, finding])); + + expect(byId.get("viewport_recheck_after_screenshot")?.occurrences).toBe(1); + expect(byId.get("fixed_wait")?.occurrences).toBe(1); + expect(byId.get("discovery_round_trip")?.occurrences).toBe(1); + expect(byId.get("duplicate_call")?.occurrences).toBeGreaterThanOrEqual(1); + expect(byId.get("repeated_screenshot")?.occurrences).toBe(1); + expect(byId.get("serial_form_fill")?.estimatedAvoidableRounds).toBe(2); + expect(report.summary.estimatedAvoidableRounds).toBeGreaterThanOrEqual(6); + expect(report.summary.toolCalls).toBe(events.length); + }); + + it("renders only aggregate data", () => { + const report = analyzeEvents([event(1, "browser_waitFor", { fixedWait: true }, "secret-hash")], { + since: "2026-08-20T00:00:00.000Z", + generatedAt: "2026-09-19T00:00:00.000Z", + clients: { + claude: { available: true, filesSeen: 1, filesScanned: 1, malformedLines: 0, toolCalls: 1, exactCalls: 1, derivedCalls: 0 }, + codex: { available: false, filesSeen: 0, filesScanned: 0, malformedLines: 0, toolCalls: 0, exactCalls: 0, derivedCalls: 0 } + } + }); + const text = renderReport(report, "text"); + const json = renderReport(report, "json"); + expect(text).toContain("fixed_wait"); + expect(json).toContain('"toolCalls": 1'); + expect(text + json).not.toContain("secret-hash"); + expect(text + json).not.toContain("session-a"); + }); + + it("does not call observations duplicates across a page-state boundary", () => { + const events = [ + event(1, "browser_takeSnapshot", {}, "same-snapshot"), + event(2, "browser_click"), + event(3, "browser_takeSnapshot", {}, "same-snapshot"), + event(4, "browser_screenshot", {}, "same-shot"), + event(5, "browser_waitFor", { fixedWait: true }), + event(6, "browser_screenshot", {}, "same-shot") + ]; + const report = analyzeEvents(events, { + since: "2026-08-20T00:00:00.000Z", + generatedAt: "2026-09-19T00:00:00.000Z", + clients: { + claude: { available: true, filesSeen: 1, filesScanned: 1, malformedLines: 0, toolCalls: 6, exactCalls: 6, derivedCalls: 0 }, + codex: { available: false, filesSeen: 0, filesScanned: 0, malformedLines: 0, toolCalls: 0, exactCalls: 0, derivedCalls: 0 } + } + }); + expect(report.findings.find((finding) => finding.id === "duplicate_call")).toBeUndefined(); + expect(report.findings.find((finding) => finding.id === "repeated_screenshot")).toBeUndefined(); + }); +}); + +describe("session analyzer CLI", () => { + it("uses privacy-preserving defaults", () => { + const options = parseCliArgs([], "/home/tester", new Date("2026-09-19T00:00:00.000Z")); + expect(options.clients).toEqual(["claude", "codex"]); + expect(options.format).toBe("text"); + expect(options.sinceMs).toBe(Date.parse("2026-08-20T00:00:00.000Z")); + expect(options.claudeDir).toBe("/home/tester/.claude/projects"); + expect(options.codexDir).toBe("/home/tester/.codex/sessions"); + }); + + it("parses client, duration, format and directory overrides", () => { + const options = parseCliArgs([ + "--clients", "claude", + "--since", "7d", + "--format", "json", + "--claude-dir", "/tmp/claude" + ], "/home/tester", new Date("2026-09-19T00:00:00.000Z")); + expect(options.clients).toEqual(["claude"]); + expect(options.sinceMs).toBe(Date.parse("2026-09-12T00:00:00.000Z")); + expect(options.format).toBe("json"); + expect(options.claudeDir).toBe("/tmp/claude"); + }); + + it("rejects unknown clients and arguments", () => { + expect(() => parseCliArgs(["--clients", "other"], "/home/tester", new Date())).toThrow(/client/i); + expect(() => parseCliArgs(["--wat"], "/home/tester", new Date())).toThrow(/unknown argument/i); + }); + + it("ignores pnpm's forwarded argument separator", () => { + const options = parseCliArgs(["--", "--help"], "/home/tester", new Date()); + expect(options.help).toBe(true); + }); +}); + +describe("session history scanner", () => { + it("scans available clients and tolerates a missing client directory", async () => { + const root = await mkdtemp(join(tmpdir(), "atwebpilot-session-analysis-")); + try { + const claudeDir = join(root, "claude"); + await mkdir(claudeDir); + await writeFile(join(claudeDir, "session.jsonl"), `${JSON.stringify({ + type: "assistant", + sessionId: "private-session-id", + timestamp: "2026-09-18T10:00:00.000Z", + message: { + id: "round-1", + content: [{ + type: "tool_use", + id: "call-1", + name: "mcp__atwebpilot__browser_waitFor", + input: { ms: 100, session_id: "private-token" } + }] + } + })}\n`, "utf8"); + + const scan = await scanHistories({ + clients: ["claude", "codex"], + sinceMs: Date.parse("2026-09-01T00:00:00.000Z"), + sinceLabel: "2026-09-01T00:00:00.000Z", + format: "text", + claudeDir, + codexDir: join(root, "missing-codex"), + help: false + }); + + expect(scan.events).toHaveLength(1); + expect(scan.clients.claude).toMatchObject({ + available: true, + filesSeen: 1, + filesScanned: 1, + toolCalls: 1, + exactCalls: 1, + derivedCalls: 0 + }); + expect(scan.clients.codex.available).toBe(false); + expect(JSON.stringify(scan)).not.toContain("private-token"); + expect(JSON.stringify(scan)).not.toContain(root); + } finally { + await rm(root, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/shared/src/llm/builtin-tool-defs.ts b/packages/shared/src/llm/builtin-tool-defs.ts index 9071f05..08f5c78 100644 --- a/packages/shared/src/llm/builtin-tool-defs.ts +++ b/packages/shared/src/llm/builtin-tool-defs.ts @@ -984,7 +984,7 @@ export const TOOL_DEFS: LlmTool[] = [ name: "resize", description: "[FLOW] 把视口调整到指定尺寸。main-world 档量取 outerWidth-innerWidth 反推浏览器边框后改窗口外框,视口精确但**用户的窗口会真的变大小**;cdp 档用 Emulation 覆盖设备尺寸,不动真实窗口。", - mcp: { description: "Resize the viewport to width×height." }, + mcp: { description: "Resize the viewport to width×height and return the measured actualViewport plus verified status. Do not re-measure with runJS." }, input_schema: { type: "object", properties: { diff --git a/skill/SKILL.md b/skill/SKILL.md index f3790c9..f7048c0 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -35,7 +35,7 @@ All of these are prefixed `browser_` over MCP, e.g. `browser_takeSnapshot`. | navigation / tabs | `navigate`(`action` 取 `back` / `forward` / `reload` / `goto`), `listTabs`, `openTab`, `closeTab`, `resize`, `scroll` | | observation | `screenshot`, `waitFor`, `runJS`(static-scanned), `consoleMessages`, `networkRequests` | -**Discoverable — call `browser_discoverTools` first:** +**Discoverable — enable directly with `browser_discoverTools`:** | Group | Tools | |---|---| @@ -58,10 +58,13 @@ core — exporting a spreadsheet, calling an API with cookies, reading localStorage, inspecting a request body, searching history, debugging layout — do this, in order: -1. `browser_discoverTools({})` → catalog of everything not yet advertised. -2. `browser_discoverTools({ enable: ["browser_downloadSpreadsheet", ...] })` → - the tools join `tools/list` (a `tools/list_changed` notification is sent) - and the response carries their full schemas, so you can call them right away. +1. If you know the tool, enable it directly with + `browser_discoverTools({ enable: ["browser_downloadSpreadsheet", ...] })`. + If you know only the group, use + `browser_discoverTools({ enableGroups: ["export"] })`. +2. Call `browser_discoverTools({})` only when neither the tool nor group is + known. Enabled tools join `tools/list` (a `tools/list_changed` notification + is sent), and the response carries their full schemas for immediate use. **Do not** rebuild these capabilities with `runJS` (`fetch`, CSV blobs, `localStorage[...]`). It is slower, loses schema validation, and usually trips @@ -73,7 +76,7 @@ prefer to pay the context cost once. ## Recommended flow 1. **探查先于操作**:每次进入新页面,先 `getPageInfo` 确认位置,再 `takeSnapshot`(要点击/填表)或 `createPageIndex`(要读内容/抽字段);`snapshotDOM` / `querySelector` 属于 legacy-dom 组,只在需要分析整页结构时通过 `browser_discoverTools` 启用。 -2. **小步快跑**:每次只动一个元素,验证 DOM 变化后再继续,避免连点连填触发反爬。 +2. **最少充分验证**:危险或不可逆操作保持单步;多个独立字段用 `fillForm`。工具已返回 `verified` 或明确后置状态时,不要再用 `runJS` 重复验证。 3. **dangerous 工具会被人工审核**:调用前用 `extractText` 给用户看上下文,让审批更顺。 4. **跑不动了就停下来问**:候选不唯一、缺关键信息、需要二次确认时,把问题写在回复里交给用户,不要瞎猜(MCP 会话没有 `askUser`)。 5. **完成后给一个简洁的总结**:用户希望看到「做了 N 步,最终结果 X」,不希望看流水账。 @@ -91,6 +94,8 @@ prefer to pay the context cost once. - `pressKey`: `{ selector: 'input[name=q]', key: 'Enter' }` submits a form-less search; `{ key: 'Escape' }` closes a modal. - `navigate`: `{ action: 'back' }`, `{ action: 'goto', url: 'https://example.com/page' }`. - `switchToTab`: all tools already target the session-bound tab. Discover and call this only when the user explicitly asks to bring Chrome and that tab to the foreground; never use it as setup for click, wait, runJS, or screenshot. +- `resize`: returns measured `actualViewport`, `devicePixelRatio`, and `verified`; do not follow it with a viewport-only `runJS` probe. +- `screenshot`: the image block is followed by compact capture metadata (backend, dimensions and target when available); use that instead of probing the viewport again. - `httpRequest`: `{ url: '.../api/comments?page=2' }` (no cookies) vs `{ url, withCredentials: true }` (cookied, reviewed). - `consoleMessages`: `{ level: 'error', limit: 50 }`; incremental polling with `{ sinceId }`. - `networkRequestDetail`: arm bodies first with `recorderConfig({ bodies: true })` in main-world mode. From f76d64bef2f157779753650c69c6312454c31e50 Mon Sep 17 00:00:00 2001 From: liuzaisen Date: Sat, 19 Sep 2026 23:48:23 +0800 Subject: [PATCH 2/2] chore(release): v0.0.73 --- package.json | 2 +- packages/mcp-server/package.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/package.json b/package.json index 4f18321..90175b7 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "atwebpilot-monorepo", "private": true, - "version": "0.0.72", + "version": "0.0.73", "description": "AtWebPilot — AI 网页助手(侧边面板)monorepo root", "type": "module", "scripts": { diff --git a/packages/mcp-server/package.json b/packages/mcp-server/package.json index 9de4136..fb4ed1a 100644 --- a/packages/mcp-server/package.json +++ b/packages/mcp-server/package.json @@ -1,6 +1,6 @@ { "name": "@attson/atwebpilot-mcp", - "version": "0.0.72", + "version": "0.0.73", "license": "Apache-2.0", "description": "MCP server for the atwebpilot browser extension - let Codex or Claude Code drive your browser.", "homepage": "https://github.com/attson/atwebpilot#readme",