Make context() read the page it reports, with refs attached

[DavertMik]

Two stacked defects made context() useless and clickRef almost always fail. Diagnosed from a Langfuse session on 2026-08-19.

What was wrong

context() never captured. It read stateManager.getCurrentState() and re-rendered it through ActionResult.fromState(...), so it handed back the snapshot the model already had, while its own description sells it as fresh state ("You suspect page changed externally"). verify in the same file has always done await explorer.capture().

getInteractiveARIA() carries no element refs. In that session, context() output contained [ref=...] in 0 of 38 calls, while the injected <page_aria> carried refs in 305 of 677 blocks. Refs appeared in the prompt, vanished the moment the model refreshed its context, and the model fell back on stale ones. clickRef failed 18 of 20 calls (90%) with Invalid frame in aria-ref selector and ref-not-found timeouts.

The fix

1. The live-snapshot helper (page.locator('body').ariaSnapshot({ mode: 'ai' })) moved out of Tester into src/utils/aria.ts as interactiveAriaWithRefs(explorer, state), next to compactAriaSnapshot. Both Tester call sites now use it, no copy left behind. The degradation is preserved exactly: when withPage fails, it falls back to the ref-free getInteractiveARIA(). The 305-of-677 split shows the live path fails routinely, so context() must never start throwing where it previously answered. Covered by two new tests in tests/unit/aria.test.ts.
2. context() captures fresh, then takes its ARIA from that helper. Both halves ship together on purpose — adding refs without the fresh capture would hand the model confident-looking stale refs, which is exactly what produces Invalid frame in aria-ref selector today, and clickRef would get worse rather than better.
3. clickRef's description taught that a ref looks like e14. Real refs are frame-scoped and longer. It now states the shape generally — an opaque id whose form varies with the page and the frame, to be copied character for character, never invented or rebuilt. click's description carried the same false [ref=e14] example one screen up and is fixed with it.

Behaviour change: context() now counts as state activity

explorer.capture() has a side effect the cached read did not: it reaches stateManager.updateState(result, codeBlock) (src/action.ts:175). So context() calls now participate in transition history and dead-loop bookkeeping. verify already does this. Verified rather than assumed:

• ActionResult.getStateHash() is built from url + h1 + h2 only, so an unchanged page re-captured yields the same hash.
• StateManager.updateState pushes a StateTransition only when the hash changed (or a dialog appeared), and isInDeadLoop() reads stateHistory. An unchanged page therefore adds nothing to stall detection.

What does change: currentState is replaced by a fresher ActionResult (so a later tool's pageDiff is measured from a fresher baseline — more accurate, not less), a state id is consumed, and a state html/aria file is written. If the page genuinely moved on since the last recorded state, the capture records that transition — which is the point of the tool.

Two smaller consequences of capturing, both matching what neighbouring tools already do: fatal browser errors now propagate out of context() instead of being flattened into a failed tool result (commit 754dffc exists so a dead browser stops the run), and a non-fatal capture failure returns a failed tool result rather than success: true with empty content.

Rejected: making capture globally produce ref-annotated snapshots

Changing src/action.ts:147 to ariaSnapshot({ mode: 'ai' }) looks like the simpler fix and breaks diffAriaSnapshots. Refs renumber on every snapshot, so an identical page re-snapshotted diffs as all-changed. Reproduced directly against src/utils/aria.ts — same content, refs e1..e6 vs f4e116..f4e121:

{ "count": 6, "text": "ariaDiff:\n  added:\n    - button \"New item\" [ref=f4e121]\n ... removed:\n    - button \"New item\" [ref=e6] ..." }

That is the ariaDiff the Tester and Pilot rely on for progress and stall detection, so action.ts:147 is left alone. The ref-carrying snapshot is taken separately, after the capture, only for what the model reads.

Optional, separable

xpathCheck had the same defect class — it matched XPath against ActionResult.fromState(currentState).html while checking visibility against the live browser, so the two halves could describe different pages. It was a genuine one-liner (const html = (await explorer.capture()).html;) and the failure path stays graceful: an error ActionResult returns '' from .html and the existing if (!html) guard fires. Drop this hunk if you disagree — nothing else depends on it. The state-activity note above applies to it too: it captures, so it updates state the same way.

The research tool (src/ai/tools.ts, its aria: field) still reports getInteractiveARIA() off cached state. Same defect class, deliberately left out of scope here.

Tests

• bun run format, bun run lint:fix — clean
• bun test tests/unit/ — 1024 pass, 0 fail
• bun test tests/integration/ — 80 pass, 1 skip, 0 fail
• tsc --noEmit — the repo has pre-existing errors; none added by this change, and one in tester.ts was removed (the helper's withRefs was inferred as {} and is now typed).

Not regression-tested — that is the user's call.

🤖 Generated with Claude Code

[DenysKuchma]

Passing refs to Tester here, but clickRef is still missing from ACTION_TOOLS. This means even a successful clickRef won’t count as an action, so Tester may stop the test as stalled despite making real progress. We should either add clickRef to all action/progress tracking or keep refs limited to Prima as in #130

[DavertMik]

We don't need refs for Tester at all