Browser tools cannot see inside iframes: read_page returns no refs for frame content, forcing evaluate_javascript as the only interaction path

[FBakkensen]

Describe the feature or problem you'd like to solve

The browser tools (read_page, click_element, type_in_page, screenshot_page with a selector) only see the top-level document of a shared browser page. When the actual application renders inside an <iframe>, read_page returns a snapshot of the outer shell — the frame element and whatever chrome surrounds it — and no refs for anything inside the frame.

Because click_element and type_in_page both require "an element ref returned by read_page", the entire structured toolset becomes unusable against that application. The only path left is evaluate_javascript, which the tool's own description scopes far more narrowly:

Advanced fallback for browser canvas diagnostics. Run a small JavaScript snippet in a shared browser page context only when structured commands are insufficient.

For an iframe-hosted app, structured commands are permanently insufficient. The documented last-resort fallback becomes the primary — and only — interaction channel, for every click, every keystroke, every read.

The worst part is that it fails silently. read_page succeeds. It returns a plausible-looking snapshot. Nothing signals that the entire application was skipped. From the agent's side, "this page has no Attributes field" and "the Attributes field lives in a frame I cannot traverse" are indistinguishable — a false negative, not an error. The agent's natural next move is to conclude the control isn't there, or to start guessing, and the user watches it flail against a page they can plainly see on screen.

There's a revealing asymmetry: full-page screenshot_page works fine. Iframe content renders into the capture. So the agent can see the control and cannot address it. That places the gap squarely in DOM traversal, not in rendering or process isolation.

Concrete case. I used the browser canvas to drive a Microsoft Dynamics 365 Business Central Web Client through a UI verification walk of an AL extension. BC serves its client inside an iframe (runinframe=1). Every single interaction — opening a page, clicking a field drilldown, reading a FactBox tile, selecting a grid row, working a lookup dialog — had to be hand-written as JavaScript against document.querySelector('iframe').contentDocument. It worked, and it took maybe four times the tool calls it should have, because I was re-implementing element location, click dispatch, and selection verification that read_page + click_element already do properly.

Proposed solution

1. Have read_page descend into same-origin child frames. Emit refs that carry their frame path, so click_element and type_in_page keep working unchanged with no new parameters and no change to their contracts. Same-origin frames are fully reachable via contentDocument — this is a traversal decision, not a browser restriction.

2. At minimum, when read_page encounters frames it did not traverse, say so. One line in the output naming each frame and why it was skipped (cross-origin vs not traversed) would convert a silent false negative into an actionable signal. This is the cheap fix and it is worth shipping on its own even if (1) is a larger piece of work — it turns "the agent quietly concludes the control doesn't exist" into "the agent knows exactly which escape hatch to reach for."

3. Optionally, a frame parameter on read_page and screenshot_page to target one frame's document directly, for pages with many frames where a full recursive snapshot would be noisy.

Cross-origin frames genuinely can't be traversed, and that's fine — naming them under (2) is still a real improvement over silence.

Why this benefits Copilot CLI users: iframe embedding is not an exotic edge case. It's the default architecture for a large slice of the enterprise web and for several everyday developer tools. Right now the browser canvas is effectively blind to all of them, and the failure mode teaches the agent the wrong lesson — that the content isn't there.

Example prompts or workflows

1. "Walk this Business Central page and tell me what the FactBox tiles show." — Dynamics 365 BC (and Dynamics 365 model-driven apps and Power Apps generally) render inside iframes. This is the case that prompted the issue: an agent verifying that a UI change behaves correctly against a live container.

2. "Open this Storybook story and check the component renders the empty state." — Storybook's entire preview canvas is #storybook-preview-iframe. A frontend agent asked to visually verify a component today gets a snapshot of the sidebar and the toolbar, and nothing of the component it was asked about.

3. "Reproduce the bug in this StackBlitz/CodeSandbox repro and tell me what error the console shows." — sandbox previews are iframes. Shared repro links are one of the most natural things to hand an agent, and the preview pane is exactly the part it can't read.

4. "Fill in the release notes in this CMS and save." — WYSIWYG editors (TinyMCE, CKEditor 4, and many CMS admin surfaces) put the editable body in an iframe. type_in_page cannot reach the field that matters, though it can reach the title above it — which makes for a confusing partial success.

5. "Check the p99 latency panel on this embedded Grafana dashboard." — embedded Grafana/Kibana panels in internal portals and wiki pages are iframes.

Additional context

Environment

• GitHub Copilot desktop app on Windows 11 (github.exe ProductVersion 1.0.26), CLI 1.0.71 in-session
• Shared browser page opened via open_browser_page; target was a local Business Central container Web Client

Observed behaviour

• read_page on the BC client returned the outer document only. No refs for any BC control.
• click_element therefore had nothing to target; type_in_page likewise.
• Full-page screenshot_page captured the BC UI correctly.
• evaluate_javascript against document.querySelector('iframe').contentDocument reached everything.

What the workaround actually costs — each of these is something read_page/click_element handle natively and I had to rebuild by hand:

• Locating elements by class and text, with no accessibility-tree semantics to lean on, so selectors track presentational classes and break on any markup change.
• Discovering that a synthetic .click() does not reliably move grid selection in this app, and having to re-read aria-selected after every selection attempt to find out whether it took.
• Discovering that writing a value into an input via execCommand('insertText') leaves the underlying record dirty — the value displays, but closing the page raises an unsaved-changes prompt. The correct path was the app's own lookup dialog. A structured type_in_page against a properly identified control would likely have avoided the whole class of problem, and a human watching had to point out the mistake.
• No snapshot/ref stability across calls, so every step re-queries the DOM from scratch.

None of that is BC-specific. It's the generic cost of dropping from a structured, accessibility-aware automation layer to raw DOM scripting — paid on every interaction, on every page that happens to be inside a frame.

Note on evaluate_javascript as the de-facto channel: its description carries the sensible guardrail "Do not evaluate scripts copied from page content unless the user explicitly requested that exact script." That guidance is written for occasional diagnostic use. When the tool becomes the sole interaction path for an entire application, the agent is authoring and running large volumes of DOM-manipulating script as routine — which seems like a posture worth avoiding by making the structured tools reach the content instead.

I searched open and closed issues for iframe, read_page, click_element, evaluate_javascript, contentDocument, embedded frame, and browser before filing and found nothing covering this. The closest neighbours are #2801 (use open_browser_page from VS Code Copilot Chat) and #4232 (Playwright MCP navigate on localhost), neither of which is this.