Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/superpowers/plans/2026-09-18-mcp-round-efficiency.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# MCP Round Efficiency Implementation Plan

**Goal:** Remove redundant model turns by making screenshot and resize results
self-describing and allowing one-call discovery group activation.

**Spec:** `docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md`

## Tasks

- [x] Add failing MCP tests for screenshot metadata and `enableGroups`.
- [x] Add failing extension tests for actual viewport verification.
- [x] Preserve non-image screenshot metadata in MCP content.
- [x] Measure and report the actual viewport after resize.
- [x] Implement direct discovery-group activation and update tool guidance.
- [x] Run focused tests, package typechecks, full tests, and build.

## Constraints

- Do not duplicate base64 image data in a text content block.
- Do not change capability validation, dangerous accounting, or approval gates.
- Keep old discovery and resize calls backward compatible.
- Do not add dependencies.
- Do not enable behavioral trace collection without a separate privacy design.
25 changes: 25 additions & 0 deletions docs/superpowers/plans/2026-09-19-local-ai-session-analyzer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Local AI Session Analyzer Implementation Plan

**Goal:** Analyze local Claude Code and Codex histories for avoidable
AtWebPilot tool rounds without collecting or exposing conversation content.

**Spec:** `docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md`

## Tasks

- [x] Define normalized privacy-preserving event/report types.
- [x] Add failing Claude/Codex adapter tests with synthetic JSONL fixtures.
- [x] Implement streaming adapters and result joining.
- [x] Add failing analyzer rule tests.
- [x] Implement aggregation, sequence rules and report rendering.
- [x] Add CLI argument tests and repository/package scripts.
- [x] Run the analyzer against local histories and inspect only aggregates.
- [x] Run package/full typecheck, tests and builds.

## Constraints

- No dependencies.
- No raw prompt, argument, result, credential or absolute session path in output.
- Read-only access to client history directories.
- Unknown and malformed records are skipped and counted, never fatal.
- Client file layouts are treated as versioned adapters, not stable public APIs.
2 changes: 2 additions & 0 deletions docs/superpowers/plans/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
| 16 | [`2026-06-06-atwebpilot-rename.md`](./2026-06-06-atwebpilot-rename.md) | 7 | 0 (rename only) | ~492 |
| 17 | [`2026-06-07-mcp-publish.md`](./2026-06-07-mcp-publish.md) | 9 | 0 (publish only) | ~492 |
| 18 | [`2026-09-18-non-intrusive-screenshot.md`](./2026-09-18-non-intrusive-screenshot.md) | 4 | + focused MCP/CDP capture tests | current |
| 19 | [`2026-09-18-mcp-round-efficiency.md`](./2026-09-18-mcp-round-efficiency.md) | 6 | +7 focused MCP/extension tests | current |
| 20 | [`2026-09-19-local-ai-session-analyzer.md`](./2026-09-19-local-ai-session-analyzer.md) | 8 | synthetic adapter/analyzer/CLI tests | current |

各 plan 的 task 严格 TDD:写失败测试 → 实现 → 验证通过 → commit。每 task 一组 commit;plan 内的 commit 加起来构成一次完整 feature。

Expand Down
99 changes: 99 additions & 0 deletions docs/superpowers/specs/2026-09-18-mcp-round-efficiency-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# MCP Round Efficiency Design

**Status:** implemented.

## Problem

Real MCP sessions show avoidable model turns around otherwise successful tool
calls. In one responsive-page workflow the agent called `resize`, took a
`screenshot`, then used `runJS` twice to recover viewport facts the tools had
already measured internally. It also called `browser_discoverTools({})` before
deciding which extra capability it needed.

Each avoidable tool round is expensive beyond its small JSON arguments: the
default MCP surface is about 16k characters (roughly 4.5k tokens) and the
conversation plus prior tool results grows on every turn.

## Goals

- Make successful screenshot and resize calls self-verifying.
- Let an agent enable a known discovery group in one call.
- Preserve existing tool names, arguments, result fields, capability checks,
and approval behavior.
- Improve agent guidance so it trusts verified postconditions and avoids fixed
waits when a tool already returns the needed state.

## Design

### Screenshot metadata survives MCP conversion

`screenshot` already returns `backend`, dimensions, target information and
capture details beside its base64 payload. MCP currently emits only the image
block. It will emit the image first, followed by a compact JSON text block that
contains every top-level field except `data` and `media_type`.

This keeps image rendering compatible while exposing enough evidence to avoid
a follow-up viewport probe. The base64 payload is never duplicated into text.

### Resize verifies the actual viewport

After applying either CDP device metrics or a real window resize, `resize`
measures `window.innerWidth`, `window.innerHeight` and `devicePixelRatio`. The
main-world path polls briefly because `chrome.windows.update` may resolve before
layout catches up.

The existing `width`, `height`, `backend` and `chromeInset` fields remain. New
fields are:

```json
{
"actualViewport": { "width": 390, "height": 844 },
"devicePixelRatio": 1,
"verified": true
}
```

`verified` is false rather than an exception when the browser accepts the
resize but reports a different viewport. This is useful evidence for responsive
debugging and prevents blind retries.

### Discovery enables known groups directly

`browser_discoverTools` gains `enableGroups`, whose values are the existing
discovery groups. `enable` and `enableGroups` may be supplied together; names
are deduplicated before worker-support filtering. Unknown groups are returned
as `unknownGroups` without failing valid requests.

Calling with no arguments remains a catalog operation for genuinely unknown
capabilities. Guidance changes from a mandatory catalog-then-enable sequence to
direct group activation when the group is known.

## Finding The Next Inefficiencies

A comprehensive follow-up should add opt-in MCP efficiency traces. Each trace
record should contain timestamps, tool name, redacted argument/result byte
counts, image dimensions, outcome, and a task/session correlation id. It must
not retain page text, images, credentials, or raw arguments by default.

An offline analyzer can then flag:

- repeated calls with equivalent arguments;
- `resize -> runJS(viewport)` and `screenshot -> runJS(viewport)` chains;
- fixed waits followed by a state-reading tool;
- no-argument discovery immediately followed by group activation;
- oversized text/image results and repeated full snapshots;
- error/retry loops and tools whose postconditions are routinely rechecked.

Release validation should replay a small fixed scenario suite and compare tool
rounds, result bytes, wall time, and task success at median and p95. Trace
collection is deliberately not part of this change because retention, redaction
and user opt-in need an explicit privacy decision.

## Verification

- MCP tests cover image plus metadata content blocks.
- Background tests cover verified and mismatched resize results.
- Discovery tests cover direct group activation, deduplication, unknown groups,
worker support, and list-changed behavior.
- MCP token-budget tests, package typechecks, package tests, and extension build
remain green.
107 changes: 107 additions & 0 deletions docs/superpowers/specs/2026-09-19-local-ai-session-analyzer-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Local AI Session Analyzer Design

**Status:** implemented.

## Problem

MCP inefficiencies are easiest to recognize from real agent behavior, but adding
runtime telemetry duplicates data already written by local AI clients and adds
privacy and maintenance costs. This machine has both Claude Code and Codex JSONL
histories. Their formats differ, and naive text search is inaccurate because
tool definitions are also embedded in session records.

## Goals

- Provide a read-only local CLI for Claude Code and Codex session histories.
- Parse actual tool-call events rather than matching tool names in arbitrary
text.
- Report tool frequency, model tool rounds, result bytes and likely avoidable
call chains.
- Never print or persist prompts, tool argument values, tool result bodies,
credentials, or raw session paths.
- Tolerate malformed lines and unknown client versions.

## CLI

From the repository root:

```bash
pnpm analyze:sessions
pnpm analyze:sessions -- --since 7d --clients claude --format json
```

Defaults:

- clients: `claude,codex`
- lookback: `30d`
- output: human-readable text
- roots: `~/.claude/projects` and `~/.codex/sessions`

Optional `--claude-dir` and `--codex-dir` overrides support fixtures and custom
installations. `--format json` returns the same aggregated report as structured
JSON. The CLI exits successfully when a client directory is missing and reports
that client as unavailable.

## Privacy Model

Raw lines are processed as a stream. Tool inputs are reduced immediately to:

- sorted top-level field names;
- a SHA-256 fingerprint for equality checks;
- boolean semantic signals such as `fixedWait`, `viewportProbe`,
`emptyDiscovery`, and `enablesDiscovery`.

Tool results are reduced to byte length and error status. No cache is written in
this version. Reports contain client, aggregate counts, rule IDs, occurrence
counts, affected session counts and estimated avoidable model rounds.

## Adapters

### Claude Code

- Calls: `assistant.message.content[*].type == "tool_use"`.
- Results: `user.message.content[*].type == "tool_result"`, joined by
`tool_use_id`.
- A single assistant record containing one or more tool uses is one model tool
round.

### Codex

- Direct calls: `response_item.payload.type == "function_call"` or
`custom_tool_call`, joined to their corresponding output by `call_id`.
- Nested calls made through the Codex `exec` orchestrator are identified only
when its source contains an actual `tools.mcp__atwebpilot__...(` invocation.
They are marked `confidence: "derived"`; argument shape and individual result
size are unavailable.
- Token-usage records are not attributed to individual Codex tool calls; the
report only totals usage attached directly to a parsed model tool round.

Both adapters normalize `mcp__atwebpilot__browser_screenshot` to
`browser_screenshot`. Calls unrelated to AtWebPilot are ignored.

## Rules

The first rule set detects:

- `viewport_recheck_after_resize`: resize followed shortly by viewport-only
runJS;
- `viewport_recheck_after_screenshot`: screenshot followed shortly by the same;
- `fixed_wait`: `waitFor` with only `ms` rather than a state condition;
- `discovery_round_trip`: empty discovery followed by an enabling discovery;
- `duplicate_call`: the same tool and argument fingerprint repeated nearby;
- `repeated_screenshot`: screenshots repeated nearby;
- `serial_form_fill`: two or more consecutive `fillInput` calls that could use
`fillForm`.

Sequence rules inspect only AtWebPilot calls within the same local session and
a small call window. They are heuristics, so the report labels them as
opportunities rather than guaranteed waste.

## Verification

- Synthetic Claude and Codex JSONL fixtures cover extraction, result joining,
nested-call confidence, malformed lines and privacy guarantees.
- Analyzer tests cover every rule and ensure reports contain no raw values.
- CLI tests cover argument parsing and text/JSON rendering.
- Full monorepo typecheck and test remain green; MCP build includes no new
runtime dependency.
2 changes: 2 additions & 0 deletions docs/superpowers/specs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
| 35 | URL 注入策略 | [`2026-08-19-url-injection-policy-design.md`](./2026-08-19-url-injection-policy-design.md) | 默认与逐 hostname 的禁用/只读/操作/诊断注入级别;网页助手独立继承;惰性 bootstrap 与按需 MAIN-world recorder;配对错误状态分流 |
| 36 | 会话标签分组 | [`2026-08-20-session-tab-groups-design.md`](./2026-08-20-session-tab-groups-design.md) | 按逻辑 session id 和 window 创建原生 Chrome tab group;区分侧栏/MCP/远程来源;结束后恢复原分组;用户手动移出后不拉回;MV3 session 状态恢复 |
| 37 | 无干扰 MCP 截图 | [`2026-09-18-non-intrusive-screenshot-design.md`](./2026-09-18-non-intrusive-screenshot-design.md) | `switchToTab` 退出默认 core;CDP `Page.captureScreenshot` 后台截图;visible-tab 兼容回退不覆盖用户中途切 tab |
| 38 | MCP 调用轮次优化 | [`2026-09-18-mcp-round-efficiency-design.md`](./2026-09-18-mcp-round-efficiency-design.md) | screenshot 保留元数据;resize 返回实测 viewport;discoverTools 可直接按组启用;定义 opt-in trace 后续方向 |
| 39 | 本地 AI 会话效率分析 | [`2026-09-19-local-ai-session-analyzer-design.md`](./2026-09-19-local-ai-session-analyzer-design.md) | 只读扫描 Claude/Codex JSONL;脱敏聚合 AtWebPilot 调用;识别重复验证、固定等待与可合并步骤 |

## 不在 spec 里的细节修复

Expand Down
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "atwebpilot-monorepo",
"private": true,
"version": "0.0.72",
"version": "0.0.73",
"description": "AtWebPilot — AI 网页助手(侧边面板)monorepo root",
"type": "module",
"scripts": {
Expand All @@ -10,7 +10,8 @@
"preview": "pnpm --filter @atwebpilot/extension preview",
"typecheck": "pnpm -r typecheck",
"test": "pnpm -r test",
"test:watch": "pnpm --filter @atwebpilot/extension test:watch"
"test:watch": "pnpm --filter @atwebpilot/extension test:watch",
"analyze:sessions": "pnpm --filter @attson/atwebpilot-mcp analyze:sessions"
},
"packageManager": "[email protected]"
}
70 changes: 57 additions & 13 deletions packages/extension/src/background/bg-tools/nav.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,51 @@ function asObj(raw: Json): Record<string, unknown> {
return (raw ?? {}) as Record<string, unknown>;
}

type ViewportMetrics = {
iw: number;
ih: number;
ow: number;
oh: number;
dpr: number;
};

async function measureViewport(tabId: number): Promise<ViewportMetrics> {
const [probe] = await chrome.scripting.executeScript({
target: { tabId },
func: () => ({
iw: window.innerWidth,
ih: window.innerHeight,
ow: window.outerWidth,
oh: window.outerHeight,
dpr: window.devicePixelRatio
})
});
const metrics = probe?.result as ViewportMetrics | undefined;
if (!metrics) throw new Error("resize: could not measure the viewport");
return metrics;
}

async function measureSettledViewport(
tabId: number,
width: number,
height: number
): Promise<ViewportMetrics> {
let latest = await measureViewport(tabId);
for (let attempt = 0; attempt < 4 && (latest.iw !== width || latest.ih !== height); attempt++) {
await new Promise((resolve) => setTimeout(resolve, 50));
latest = await measureViewport(tabId);
}
return latest;
}

function verifiedViewportResult(metrics: ViewportMetrics, width: number, height: number) {
return {
actualViewport: { width: metrics.iw, height: metrics.ih },
devicePixelRatio: metrics.dpr,
verified: metrics.iw === width && metrics.ih === height
};
}

/**
* History navigation. Running off the end of the history is an ordinary
* outcome, not a failure — an agent walking back through pages should get a
Expand Down Expand Up @@ -48,20 +93,17 @@ export async function resize(raw: Json, tabId: number): Promise<Json> {
const viaCdp = getCdpResizer(tabId);
if (viaCdp) {
await viaCdp(width, height);
return { ok: true, width, height, backend: "cdp" } as unknown as Json;
const actual = await measureSettledViewport(tabId, width, height);
return {
ok: true,
width,
height,
backend: "cdp",
...verifiedViewportResult(actual, width, height)
} as unknown as Json;
}

const [probe] = await chrome.scripting.executeScript({
target: { tabId },
func: () => ({
iw: window.innerWidth,
ih: window.innerHeight,
ow: window.outerWidth,
oh: window.outerHeight
})
});
const m = probe?.result as { iw: number; ih: number; ow: number; oh: number } | undefined;
if (!m) throw new Error("resize: could not measure the viewport");
const m = await measureViewport(tabId);

const tab = await chrome.tabs.get(tabId);
if (tab.windowId == null) throw new Error("resize: tab has no window");
Expand All @@ -72,12 +114,14 @@ export async function resize(raw: Json, tabId: number): Promise<Json> {
width: width + chromeW,
height: height + chromeH
});
const actual = await measureSettledViewport(tabId, width, height);

return {
ok: true,
width,
height,
backend: "main-world",
chromeInset: { w: chromeW, h: chromeH }
chromeInset: { w: chromeW, h: chromeH },
...verifiedViewportResult(actual, width, height)
} as unknown as Json;
}
Loading
Loading