Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion FIDELITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,18 @@ The committed PNGs under `testdata/renders/` back every claim here. Reproduce
them with the commands at the bottom. The measured-vs-Chrome numbers live in
[`bench/REPORT.md`](bench/REPORT.md).

## 2026-11-21 (round 48) β€” engine#418: the mechanism is a real goja compiler bug, reproduced in isolation and traced to its likely source; two of round 47's own conclusions corrected; no safe engine-level fix found (investigation, engine#418 β€” still NOT fixed, root cause now fully characterized)

Picked up round 47's open investigation. Built on its own live-patch technique (`console.log` probes inserted into a byte-for-byte copy of the real, currently-served `911-fd88881beb259e08c065.js` chunk, served through an overridden `Engine.Client.Transport` while every other request still hits the real network) with more careful comma-expression-safe insertion β€” an early attempt that inserted a semicolon-terminated statement mid-expression silently broke parsing and dropped the real page's own rendered element count from 842 to 150, caught and fixed before drawing any conclusion from it.

- **Corrects round 47's own "every nested `forEach` is skipped entirely" theory.** With probes placed correctly, the nested `forEach` callbacks inside `resolve()` DO run, for every one of the 58 real `t()` calls, and `$` DOES get reassigned inside them β€” but to the WRONG value: the caller's own options object (literally what `arguments[1]` holds), not the real resolved resource. This is a value-corruption bug, not a control-flow skip.
- **Reproduced in isolation.** `resolve()`'s real body β€” `function resolve(j){var $,ee,ae,ie,le,ce=this,de=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};...}`, with nested non-arrow `forEach` closures reading/writing `$`/`ee`/`ae`/`ie`/`le` β€” extracted byte-for-byte via the live-patch dump and run standalone against a minimal stubbed `this`, with the goja version this repo pins (`v0.0.0-20260926152631-39ec2650adc9`). It reproduces the exact real symptom deterministically: a hand-trace of the real logic gives the correct expected result (`$` becomes `"TRANSLATED:someKey"`); goja's actual result is `$` staying the untouched options object.
- **Found and confirmed the real minimal trigger, and it is narrower and different from what an earlier, less rigorous reduction inside this same round had suggested**: a function with exactly ONE declared parameter that textually reads `arguments[N]` for `Nβ‰₯1` anywhere in its body, and ALSO contains at least one nested, non-arrow `function(){}` expression capturing some OTHER local variable β€” corrupts that other variable's storage. Confirmed at nesting depths of 1, 2, and 3 forEach levels (the real shape uses 3) β€” all reproduce identically. A from-scratch, i18next-independent 12-line version (one parameter, `var result, flag`, an `arguments[1]`-style default, one nested `forEach` writing `result`/`flag`) reproduces it at depth 1 already.
- **Retracts this round's own first attempt.** Working from an over-simplified reduction (not the real function's exact shape), this round initially concluded `"use strict"` fixes the bug and shipped it as `js/transpile.go`'s wrapper IIFE prologue. Directly re-tested against: (a) the exact real `resolve()` body in isolation, (b) `goja.Compile`'s own `strict` boolean parameter (not just a textual directive), and (c) the real page end-to-end via the live-patch harness with the fix applied and `"use strict"` confirmed textually present and active (a `this`-based strictness probe placed directly inside `resolve()` printed `strict=true` on every call) β€” **none of the three change the outcome.** Splitting the multi-declarator `var` into one statement per declarator, the other workaround this round's earlier (over-simplified) reduction had found, also does nothing on the real shape. The only change that fixes the real shape is removing the `arguments[]` read entirely (using a real declared parameter instead) β€” not something this fallback can do to third-party, already-minified source without a real AST, so **no safe engine-level mitigation exists today.** `js/transpile.go` was reverted in full back to its pre-round-48 state (confirmed via `git diff`: empty) rather than ship a change proven not to fix the real symptom.
- **Likely goja-internal cause, for an upstream report**: `moveArgsToStash()`/`finaliseVarAlloc()` (`compiler.go:614-834`, invoked once from `compiler_expr.go:1559-1615`, AFTER `compileStatements` has already compiled the function's entire body including every nested closure) run a single stash-slot-allocation pass that folds a function's argument bindings into the stash whenever `arguments` is textually referenced anywhere in the body; some other variable already captured by a nested closure appears to end up aliased to the same slot. Not fixed here β€” this is upstream `github.com/dop251/goja` work, out of this repository's scope; a clean, minimal, generic repro is in hand for filing there.
- engine#418 stays open, with the mechanism now fully characterized (unlike round 47's still-open "reason not yet found"): a genuine upstream goja compiler defect, not a react-i18next/settle-timing issue (round 36, already disproven) and not a skipped-forEach control-flow issue (round 47's phrasing, corrected above).
- No net production code change this round: `git diff`/`git status --porcelain` confirmed clean on `js/` before committing this entry. `go build`/`go vet`/`gofmt -l`/`govulncheck ./...` clean. All scratch repros, live-patch harnesses, and fetched/patched chunk copies stayed under the scratchpad directory, never the repo.

## 2026-11-15 (round 47) β€” engine#418 narrowed to a specific, reproducible line: i18next's own `resolve()` never calls `getResource` at all, yet still returns the caller's OWN options object as the "resolved" value (investigation, engine#418 β€” still NOT fixed, root cause not yet isolated)

User picked this Known-gaps item (open since round 35, narrowed once already in round 36) to investigate next. Round 36 had concluded the i18next+http-backend pipeline was "provably correct... end to end, in isolation" and narrowed the remaining gap to "almost certainly `react-i18next`'s `useTranslation()` ... interacting with this engine's settle-quiescence detection," explicitly deferred for needing "the real `react-dom`/`react-i18next` bundles actually reconciling inside this engine." This round built exactly that, in two stages, and found round 36's own theory does NOT hold β€” the real mechanism is narrower and lower-level than either round suspected.
Expand Down Expand Up @@ -1710,7 +1722,7 @@ byte for byte. A round mentioned below by number (e.g. "round 105", "round
## Known gaps (updated 2026-08-30 β€” see the note above on why this drifted)

- **CLOSED for `document.implementation.createDocument` (round 35, 2026-11-12, engine#418 (partial)) β€” a minimal detached XML-shaped document (`createElement`/`createElementNS`, an optional root `documentElement`), the same scope `createHTMLDocument` (round 52) already set.** Did not exist at all before this round. Confirmed real trigger: `@azure/core-xml`'s own XML-building helper, bundled into github.com's own Support app, called unconditionally at module-evaluation time β€” threw and aborted the whole chunk. `createAttribute`/`NamedNodeMap.setNamedItem`/`createCDATASection`, the SAME caller's own deeper helpers, are a separate, not-yet-confirmed-reachable gap (only hit by an actual file-attachment upload), deliberately left open.
- **OPEN, root cause NOT found: every localized string on github.com's own Support app renders as i18next's own literal `key 'X (en)' returned an object instead of string.` fallback text (round 35, 2026-11-12; narrowed round 36, 2026-11-14; narrowed FURTHER, and round 36's own theory disproven, round 47, 2026-11-15; engine#418).** Round 36's theory β€” a `react-i18next`/settle-quiescence interaction β€” was built and tested directly (real React+ReactDOM+i18next+i18next-http-backend+react-i18next reconciling inside this engine, with a real async backend delay) and did NOT reproduce the bug; that generic mechanism works correctly here. Round 47 instead patched the REAL, currently-served GitHub chunk with `console.log` probes and rendered the real page end to end: for all 58 real `t()` calls, i18next's own `resolve()` returns the CALLER'S OWN OPTIONS OBJECT as the resolved value, without ever calling the real `ResourceStore.getResource` (a probe at that exact call site: zero hits across every call). `resolve()`'s own entry point runs with sane values; every nested `forEach` inside its body appears to be skipped entirely, for a reason not yet found β€” the most likely engine-level theory (a nested function's own `var` accumulator somehow reading as already-valid from a leaked outer binding) was tested in isolation and ruled out; goja scopes it correctly. Confirming the exact mechanism needs either finer-grained tracing than a same-statement log probe allows, or a from-scratch minimal reduction of the real minified code outside any webpack/engine context β€” both materially larger than any round's own budget so far.
- **OPEN, root cause FOUND (a genuine upstream goja compiler bug) but NOT fixable at the engine level yet: every localized string on github.com's own Support app renders as i18next's own literal `key 'X (en)' returned an object instead of string.` fallback text (round 35, 2026-11-12; narrowed round 36, 2026-11-14; narrowed FURTHER round 47, 2026-11-15; mechanism found and two of round 47's own conclusions corrected, round 48, 2026-11-21; engine#418).** Round 48 proved the real `resolve()`'s nested `forEach` callbacks DO run (correcting round 47's "skipped entirely" theory) and reproduced the exact corruption in isolation, standalone, against the goja version this repo pins: a function with exactly one declared parameter that textually reads `arguments[N]` (`Nβ‰₯1`) anywhere in its body, with any nested non-arrow `function(){}` closure capturing another local variable, corrupts that variable's storage β€” reproduces at 1, 2, or 3 levels of closure nesting alike. `"use strict"` (confirmed textually present AND confirmed active via a runtime probe, both in isolation and on the real page), `goja.Compile`'s own `strict` parameter, and splitting the multi-declarator `var` statement were all directly tested against the EXACT real function shape and NONE change the outcome β€” an earlier, over-simplified reduction inside round 48 itself had wrongly suggested they did; that conclusion is retracted. The only thing that fixes it is removing the `arguments[]` read, which this fallback cannot safely do to arbitrary third-party minified source without a real AST. Likely goja-internal cause (for an upstream report, not yet filed): `moveArgsToStash()`/`finaliseVarAlloc()` (`compiler.go:614-834`, invoked from `compiler_expr.go:1559-1615`) run their one stash-slot-allocation pass AFTER a function's entire body β€” including nested closures β€” is already compiled, and fold argument bindings into the stash whenever `arguments` is referenced anywhere in the body, apparently colliding with a slot a nested closure already captured.
- **PARTIALLY CLOSED for `WebAssembly.compileStreaming`/`instantiateStreaming` and the module-size/linear-memory caps (2026-11-03, engine#341) β€” see round 189's own entry just below, now updated: both ARE implemented, and both caps are now configurable (`Engine.MaxWasmModuleBytes`/`MaxWasmMemoryBytes`) rather than hardcoded. Everything else round 189 already disclosed as unimplemented (the synchronous `Module`/`Instance` constructors, Table/Global imports/exports, standalone `Memory`/`Table` objects, WASI) remains exactly as disclosed there β€” this round did not touch any of them.** `compileStreaming`/`instantiateStreaming` are not genuine incremental streaming (this engine's `fetch()` has no streaming body to begin with β€” see the round's own log entry above), but the JS-facing Promise/Response contract is satisfied exactly, which is what every real caller's own code actually observes.
- **CLOSED for `window.postMessage` (round 201, 2026-11-02, engine#340) β€” a no-op; real cross-window message delivery is not simulated (this engine's single-document model has no second window/frame to deliver to; `MessagePort`'s own `postMessage`, via `MessageChannel`, already delivers for real and is unaffected).** Did not exist as a global at all before this round β€” only `MessagePort.postMessage` did. Confirmed real trigger: Google's own widely-deployed reCAPTCHA script (`gstatic.com/recaptcha/...`), which calls it unconditionally at load; reproduced live on spotify.com as `ReferenceError: postMessage is not defined`.
- **CLOSED for `document.referrer` (round 200, 2026-11-01, engine#339) β€” always returns the empty string (this engine's single-URL fetch model has no referring-page history to report).** Did not exist at all before this round. Confirmed real trigger: Fathom Analytics' own widely-served `script.js`, whose `trackPageview()` calls `document.referrer.indexOf(hostname)` unconditionally β€” identically on vuejs.org and astro.build, two otherwise-unrelated sites sharing the same third-party script.
Expand Down
Loading