From 3ece387088592746be8a838bb1171aaeb905ffde Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 20:08:36 -0700 Subject: [PATCH 01/29] PRD-752: run web games in a worker so the page cannot stall them Files Cycle v5 of PRD-573 as its own PRD (worker mode, Wasm threads), records the owner's 2026-10-26 worker HUD decision and clears the PRD-573 blocker. --- ...eal-class-performance-in-release-cycles.md | 4 +- ...-a-worker-so-the-page-cannot-stall-them.md | 86 +++++++++++++++++++ 2 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md diff --git a/docs/PRDs/performance/PRD-573-unreal-class-performance-in-release-cycles.md b/docs/PRDs/performance/PRD-573-unreal-class-performance-in-release-cycles.md index 5879adc8e9..4c73be3e1f 100644 --- a/docs/PRDs/performance/PRD-573-unreal-class-performance-in-release-cycles.md +++ b/docs/PRDs/performance/PRD-573-unreal-class-performance-in-release-cycles.md @@ -56,6 +56,7 @@ shares: animation about 13%, cull and project about 11%, `Renderer::render` 21%, when its web and desktop rows both pass on the engine ([PRD-617](PRD-617-every-template-runs-on-the-native-engine.md)). The engine exposes the upstream Three.js API, and a game uses the same source on every back end. This refines the previous decision: the default is not reverted while templates remain pinned. +- 2026-10-26 (owner): the worker HUD contract is confirmed (one state snapshot per frame out, intents back): "Yes, and v5.2 threads too". Cycle v5 moves to [PRD-752](./PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md). - 2026-10-15 (owner): housekeeping. Delete junk and reduce the file count; keep every behaviour. - Deferred, filed again only on scorecard evidence: transform compose skip (if `updateMatrixWorldSelf` is above 2% of the frame), projection reuse and SIMD world matrices (if @@ -311,6 +312,8 @@ TN today: HZB declined ([PRD-489](../done/PRD-489-gpu-scene-occlusion-culling.md ## Cycle v5 — web worker +**Owned by [PRD-752](./PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md)** (owner, 2026-10-26: worker HUD contract confirmed, "Yes, and v5.2 threads too"). The boxes below stay as the summary; PRD-752 carries the proof. + The game side reads the DOM directly (23 core files read `document.` or `window.`; audio, pointer lock, gamepads, `localStorage`). A worker keeps frame p50; only Wasm threads can cut it. #### Phase v5.1: Worker mode @@ -328,7 +331,6 @@ The game side reads the DOM directly (23 core files read `document.` or `window. - The v3 margin over Godot is an owner number, set before v3 starts. - Whether engine-only commits may skip `native-platforms` is an owner call (slice 0b). - `AnimationMixer.updateRate` adds a non-three property; João confirms the name before v2.3. -- The worker HUD contract (state out once per frame) needs João's confirmation before v5.1. - Android timing rows need the Pixel 8 attached; the emulator holds counter and memory rows. - Safari rows need a macOS machine. diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md new file mode 100644 index 0000000000..fdd55c9f71 --- /dev/null +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -0,0 +1,86 @@ +# PRD-752 — Web games run in a worker, so the page cannot stall them + +**Status:** NOT STARTED +**Priority:** P1 — on a loaded host every web game collapses (shooter kit: frame p50 33 ms, p95 100 ms, main thread 100% busy); no phase below is done yet. +**Complexity:** 8 (HIGH) — 10+ files across core, the web host, the Wasm engine and the harness (+3), a new host-capability layer (+2), worker and `OffscreenCanvas` boot (+2), Wasm threads and isolation headers (+1); risk override: none +**Owner:** João +**Parent:** [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md), Cycle v5 (v5.1 worker mode, v5.2 Wasm threads). This PRD carries the v5 boxes; PRD-573 points here. +**Depends on:** None. The waste cuts on the main thread are PR #574 and are not part of this PRD. + +**Update rule:** one PRD, one PR. Each phase lands as commits on that PR. Each box is ticked, with its result, in the commit that proves it. + +## Goal and route + +**Goal:** a web game keeps its frame time when the page, the browser or the host is busy. The game, its AI, the JS-to-Wasm glue and render submission leave the browser main thread. + +**Route:** + +```mermaid +flowchart LR + subgraph Main[Main thread] + DOM[DOM, React HUD, input, audio, pointer lock] + end + subgraph Worker[Worker] + G[Game and AI] --> E[Wasm engine] + E --> C[OffscreenCanvas] + end + DOM -- intents --> G + G -- one state snapshot per frame --> DOM +``` + +1. Core reaches the DOM only through one **host-capability layer**. Today 23 core files read `document.` or `window.` (audio, pointer lock, gamepads, `localStorage`). +2. Behind a flag, and without COOP/COEP, the game and engine boot in a worker on an `OffscreenCanvas`. +3. With cross-origin isolation, the engine adds Wasm threads (`TN_WEB_THREADS`) for its job pool. Without isolation it reports `threads: off`. + +## Evidence + +Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame median 33 ms, p95 100 ms, main thread 100% busy. The game, AI, React HUD, JS-to-Wasm glue and render submission all share the one main thread. + +## Honest expectation + +- A worker moves the game and its cost off the main thread. React, the DOM, extensions and GC stay behind: about 5-10% of the frame in a production build. +- Wasm threads help the C++ side only: about 15-20%. +- Neither fixes the fragility when every core is busy. PR #574 cuts waste separately. +- So the KPIs are **measured** main-thread busy and p95 under a **stated** CPU load, set from the phase 1 baseline. They are not guesses. The numbers in the boxes below are targets to confirm or revise against that baseline, in a `## Decisions` entry. + +## Decisions + +- 2026-10-26 (owner): the worker HUD contract is confirmed: the HUD gets one state snapshot per frame from the worker and sends intents back. "Yes, and v5.2 threads too." This clears the last v5.1 line under PRD-573 "Blocked on" and adds v5.2. +- 2026-10-26 (lead): this PRD has 4 phases and 14 boxes, above R5, because it carries the parent's v5.1 and v5.2 boxes plus the measurement phase. PRD-573 waived R5 for the same epic. Splitting it would cut the baseline from the KPI it sets. +- 2026-10-26 (owner): scale-down rule. Bounded implementation slices go to a cheaper arm (`opencode-go/deepseek-v4.1-flash`, variant `max`, at most 2 arms). The lead designs, writes the red tests and reviews every diff. + +### Phase 1: Loaded-host baseline +**Status:** NOT STARTED + +Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated load. + +- [ ] A repeatable loaded-host profile exists: it starts N busy-loop processes (N and load recorded), runs the shooter and the starter, and prints main-thread busy %, frame p50 and p95. proof: `sh scripts/xvfb.sh pnpm profile:wasm-page -- --url --cpu-work --seconds 10 --json`, with the command and output path in this box +- [ ] The baseline for the shooter and the starter is recorded in this PRD under `## Decisions`, with load, adapter name (not SwiftShader) and three runs per arm. The v5 KPI numbers are set from it. proof: the `## Decisions` entry, `adapter.info` captured in the run + +### Phase 2: Host-capability layer +**Status:** NOT STARTED + +- [ ] A red-green core spec scans `packages/core/src` and fails on any direct `document.` or `window.` read outside the host-capability layer (an allow-list of the one layer file). It is red on the 23 current files and green after the move. proof: `pnpm exec vitest run packages/core/__tests__/host-capability-boundary.spec.ts` +- [ ] The native host runs core unmodified through the same layer, with no dynamic `import()` in core. proof: `pnpm exec vitest run scripts/__tests__/check-core-boundary.spec.ts`; `pnpm native:verify:desktop` +- [ ] The starter plays through a loopback proxy (main thread talks to the layer over a message channel in the same thread) with unchanged behaviour. proof: `pnpm test:templates` on the starter, green + +### Phase 3: Worker mode and v5 KPI +**Status:** NOT STARTED + +- [ ] Behind a flag, without COOP/COEP, the starter boots in a worker on an `OffscreenCanvas`; input, pointer lock and audio forward from the main thread, and the HUD takes one state snapshot per frame and sends intents back. proof: `pnpm test:templates` with the flag, starter scenario green +- [ ] The playtest harness captures the worker canvas (screenshot not blank, frame counter advances). proof: `pnpm test:templates` with the flag; `node packages/playtest/dist/runner/cli.js doctor --url --text` +- [ ] Main-thread busy falls below the target set in phase 1 (first target: below 0.5x of the main-thread arm) and p50 is no worse, under the same stated load. proof: `profile:wasm-page` worker URL against the main-thread `--control` URL, three runs per arm +- [ ] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR +- [ ] The shooter kit plays through the worker with the same checks. proof: `pnpm test:templates` with the flag on the shooter scenario + +### Phase 4: Wasm threads and gate +**Status:** NOT STARTED + +- [ ] A `TN_WEB_THREADS` build costs at most 2% on one thread against the non-threaded build. proof: `profile:wasm-page` A/B, three runs per arm +- [ ] Without cross-origin isolation the engine reports `threads: off` and runs unchanged; with isolation (COOP/COEP) it reports the pool size. proof: `native_engine_wasm_browser_backend` for both isolation states +- [ ] The job pool cuts engine frame CPU on the threaded build. proof: `profile:wasm-page` A/B on the shooter, threaded against not +- [ ] Stability gate S is green on web with and without isolation headers. proof: the S commands in [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md) + +## Blocked on + +- None. The platform proof rule applies: a helper admitted for being unportable ships native proof in the same commit (`packages/runtime-native/AGENTS.md`). From 6cb7191982d63e564f13e24d6ed085d5d424135d Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 20:15:32 -0700 Subject: [PATCH 02/29] profile:wasm-page --load runs a page under a stated CPU load (PRD-752) Keeps N busy-loop processes alive for the whole page run and stamps the count on the report, so a loaded-host baseline is reproducible. --- scripts/__tests__/profile-wasm-page.spec.ts | 24 ++++++++++++- scripts/profile-wasm-page.ts | 38 +++++++++++++++++++-- 2 files changed, 59 insertions(+), 3 deletions(-) diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 912f1fe302..1ec25c606e 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -65,7 +65,9 @@ vi.mock("@playwright/test", () => ({ })); // Importing the command must not launch a profile. -const { cpuWork, frameStats, measure, options } = await import("../profile-wasm-page.js"); +const { cpuWork, frameStats, measure, options, startCpuLoad } = await import( + "../profile-wasm-page.js" +); afterEach(() => { state.load = 2; state.memory = {}; @@ -155,6 +157,26 @@ describe("profile:wasm-page startup, input and CPU work", () => { expect(options(["--url", "u"]).saveProfile).toBeUndefined(); }); + it("parses --load as a count of busy processes and refuses a bad count", () => { + expect(options(["--url", "u", "--load", "6"]).load).toBe(6); + expect(options(["--url", "u"]).load).toBe(0); + for (const bad of ["-1", "1.5", "x"]) + expect(() => options(["--url", "u", "--load", bad])).toThrow("--load"); + }); + + it("starts the busy processes, keeps them alive and stops them", async () => { + const load = startCpuLoad(2); + expect(load.pids).toHaveLength(2); + for (const pid of load.pids) expect(() => process.kill(pid, 0)).not.toThrow(); + await load.stop(); + for (const pid of load.pids) expect(() => process.kill(pid, 0)).toThrow(); + }); + + it("stamps the stated load on the run", async () => { + const run = await measure("http://test", options(["--url", "http://test", "--load", "1"])); + expect(run.cpuLoad).toEqual({ processes: 1 }); + }); + it("clicks in order before the warm-up, stamps startup, and reports CPU work per frame", async () => { state.presented = [100, 140]; const run = await measure( diff --git a/scripts/profile-wasm-page.ts b/scripts/profile-wasm-page.ts index 918170293e..d374fead20 100644 --- a/scripts/profile-wasm-page.ts +++ b/scripts/profile-wasm-page.ts @@ -1,5 +1,5 @@ /** - * `pnpm profile:wasm-page -- --url [--control ] [--seconds 5] [--calls] [--memory] [--json]` + * `pnpm profile:wasm-page -- --url [--control ] [--seconds 5] [--load 0] [--calls] [--memory] [--json]` * * Profiles a running game page the way a Wasm-engine performance question needs: frame time from * requestAnimationFrame, a Chrome CPU profile grouped into engine Wasm, JavaScript and WebGPU calls @@ -7,12 +7,14 @@ * (three-native's opt-in census, `__tnCallCounts`), and with `--gpu-calls` the WebGPU API calls per * frame on both pages (render passes, draws, bind groups, buffer writes). `--control` runs a second page, the same game on * three.js, through the same measurement, so the verdict is a ratio on one lane, never an absolute. + * `--load ` keeps `n` busy-loop processes alive for the whole page run: the stated CPU load. * * It refuses a software WebGPU adapter (`--allow-software` overrides) and needs a display: run it as * `sh scripts/xvfb.sh pnpm profile:wasm-page -- ...`. Wasm function names need a module linked with * `--profiling-funcs`; without it the Wasm rows are numbered. Diagnose with it; the steady-state * counter gates (render_database_test steady_state, wasm-engine-boot's renderer scenario) hold a fix. */ +import { spawn } from "node:child_process"; import { mkdirSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; @@ -40,6 +42,8 @@ interface IOptions { clicks: string[]; /** Directory that receives each page's raw Chrome .cpuprofile, named by the page's port. */ saveProfile?: string; + /** Busy-loop processes kept running for the whole page run: the stated CPU load. */ + load: number; } export function options(argv: readonly string[]): IOptions { @@ -50,12 +54,15 @@ export function options(argv: readonly string[]): IOptions { const url = value("url"); if (url === undefined) throw new Error( - "usage: profile:wasm-page -- --url [--control ] [--seconds 5] [--calls] [--memory] [--json]", + "usage: profile:wasm-page -- --url [--control ] [--seconds 5] [--load 0] [--calls] [--memory] [--json]", ); const seconds = Number(value("seconds") ?? 5); const warmupMs = Number(value("warmup-ms") ?? 12000); if (!(seconds > 0) || !(warmupMs >= 0)) throw new Error("--seconds and --warmup-ms must be positive numbers"); + const load = Number(value("load") ?? 0); + if (!(Number.isInteger(load) && load >= 0)) + throw new Error("--load must be a whole number of processes, 0 or more"); const control = value("control"); return { url, @@ -71,6 +78,7 @@ export function options(argv: readonly string[]): IOptions { ...(value("save-profile") === undefined ? {} : { saveProfile: value("save-profile") as string }), + load, clicks: argv.flatMap((arg, i) => { if (arg !== "--click") return []; const selector = argv[i + 1]; @@ -81,6 +89,27 @@ export function options(argv: readonly string[]): IOptions { }; } +export function startCpuLoad(processes: number): { pids: number[]; stop(): Promise } { + const children = Array.from({ length: processes }, () => + spawn(process.execPath, ["-e", "for(;;){}"], { stdio: "ignore" }), + ); + return { + pids: children.map((child) => child.pid as number), + async stop() { + await Promise.all( + children.map( + (child) => + new Promise((resolve) => { + if (child.exitCode !== null || child.signalCode !== null) return resolve(); + child.once("exit", () => resolve()); + child.kill("SIGKILL"); + }), + ), + ); + }, + }; +} + interface ICpuProfile { nodes: IProfileNode[]; samples: number[]; @@ -343,6 +372,8 @@ export async function measure(url: string, o: IOptions) { headless: false, args: [...WEBGPU_BROWSER_ARGS, ...PERFORMANCE_BROWSER_ARGS], }); + // Started after the launch, so a launch failure leaves no busy process behind. + const cpuLoad = startCpuLoad(o.load); try { const page = await browser.newPage({ viewport: { width: 1280, height: 720 } }); const errors: string[] = []; @@ -406,6 +437,7 @@ export async function measure(url: string, o: IOptions) { return { url, hostLoad, + cpuLoad: { processes: o.load }, ...(memory ? { memory: parseWebMetrics({ memory }).memory } : {}), adapter, startupMs, @@ -415,6 +447,7 @@ export async function measure(url: string, o: IOptions) { ...(gpu ? { gpu } : {}), }; } finally { + await cpuLoad.stop(); await browser.close(); } } @@ -424,6 +457,7 @@ function print(label: string, r: Awaited>) { console.log( `host load ${r.hostLoad.load1} on ${r.hostLoad.cores} cores, absolutes ${r.hostLoad.absolutes}`, ); + console.log(`cpu load ${r.cpuLoad.processes} busy processes`); if (r.memory) console.log( `memory: Wasm ${r.memory.wasmMemoryBytes} bytes, JS heap used ${r.memory.jsHeapUsedBytes} bytes`, From c4f4644b301aa099d64b05e7eff1be869d89d1de Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 20:33:30 -0700 Subject: [PATCH 03/29] Core reads window and document only through platform.ts; loaded-host baseline (PRD-752) - check-core-boundary type-checks packages/core/src against the webworker lib and fails on any value read of a global a worker lacks outside platform.ts (red on 19 reads in 5 files, green after the move). - platform.ts gains hostWindow/hostDocument/hostResizeObserver; audio, game, hot, renderer and viewport use them. The renderer now throws TN_RENDERER_NO_CANVAS instead of a ReferenceError with no document. - profile:wasm-page reports main-thread busy percent. - PRD-752 phase 1 baseline and v5 KPIs recorded; owner scope decision. --- ...-a-worker-so-the-page-cannot-stall-them.md | 28 +++++--- packages/core/src/audio.ts | 3 +- packages/core/src/game.ts | 19 ++--- packages/core/src/hot.ts | 8 ++- packages/core/src/platform.ts | 15 ++++ packages/core/src/renderer.ts | 17 ++++- packages/core/src/viewport.ts | 13 ++-- scripts/__tests__/check-core-boundary.spec.ts | 40 ++++++++++- scripts/__tests__/profile-wasm-page.spec.ts | 7 +- scripts/check-core-boundary.ts | 69 ++++++++++++++++++- scripts/profile-wasm-page.ts | 6 +- 11 files changed, 186 insertions(+), 39 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index fdd55c9f71..9cfc1cc7bb 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -1,7 +1,7 @@ # PRD-752 — Web games run in a worker, so the page cannot stall them -**Status:** NOT STARTED -**Priority:** P1 — on a loaded host every web game collapses (shooter kit: frame p50 33 ms, p95 100 ms, main thread 100% busy); no phase below is done yet. +**Status:** PARTIAL — phase 1 done; phase 2 box 1 done. +**Priority:** P1 — on a loaded host every web game collapses (shooter kit: frame p50 33 ms, p95 100 ms, main thread 100% busy); worker mode (phase 3) and Wasm threads (phase 4) are open. **Complexity:** 8 (HIGH) — 10+ files across core, the web host, the Wasm engine and the harness (+3), a new host-capability layer (+2), worker and `OffscreenCanvas` boot (+2), Wasm threads and isolation headers (+1); risk override: none **Owner:** João **Parent:** [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md), Cycle v5 (v5.1 worker mode, v5.2 Wasm threads). This PRD carries the v5 boxes; PRD-573 points here. @@ -28,7 +28,7 @@ flowchart LR G -- one state snapshot per frame --> DOM ``` -1. Core reaches the DOM only through one **host-capability layer**. Today 23 core files read `document.` or `window.` (audio, pointer lock, gamepads, `localStorage`). +1. Core reaches the DOM only through one **host-capability layer**. PRD-573 counted 23 files by text search; most hits are local variables named `window`. A type check against the worker lib found 19 real reads in 5 files (`audio`, `game`, `hot`, `renderer`, `viewport`). The layer is `packages/core/src/platform.ts`. 2. Behind a flag, and without COOP/COEP, the game and engine boot in a worker on an `OffscreenCanvas`. 3. With cross-origin isolation, the engine adds Wasm threads (`TN_WEB_THREADS`) for its job pool. Without isolation it reports `threads: off`. @@ -46,21 +46,33 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me ## Decisions - 2026-10-26 (owner): the worker HUD contract is confirmed: the HUD gets one state snapshot per frame from the worker and sends intents back. "Yes, and v5.2 threads too." This clears the last v5.1 line under PRD-573 "Blocked on" and adds v5.2. +- 2026-10-26 (owner, "ok go 1"): scope stays web-only. Native has no browser main thread; its render thread and job pool stay in PRD-573 Cycle v3. Native is a regression gate here only (`pnpm native:verify:desktop` in phase 2). - 2026-10-26 (lead): this PRD has 4 phases and 14 boxes, above R5, because it carries the parent's v5.1 and v5.2 boxes plus the measurement phase. PRD-573 waived R5 for the same epic. Splitting it would cut the baseline from the KPI it sets. +- 2026-10-26 (lead): **baseline** (phase 1). Desktop, RTX 2080 (`adapter.info` "nvidia turing"), Xvfb, production `vite preview` builds of the scaffolded kits, `--seconds 10 --warmup-ms 8000 --cpu-work`, 3 runs per arm, medians. Load 24 = 24 busy-loop processes on 24 logical cores (`--load 24`). + + | Kit | `--load` | load1 | p50 | p95 | main thread busy | + |---|---|---|---|---|---| + | shooter | 0 | 8.3 | 11.3 ms | 17.6 ms | 69% (7.9 ms/frame) | + | shooter | 24 | 31.1 | 18.6 ms | 116.3 ms | 97% | + | starter | 0 | 31.2 | 17.0 ms | 17.9 ms | 33% (5.1 ms/frame) | + | starter | 24 | 34.1 | 17.6 ms | 35.4 ms | 55% | + + The starter load-0 runs followed the shooter load-24 runs, so their load1 is the lagging one-minute average, not live load. The loaded shooter reproduces the reported collapse. +- 2026-10-26 (lead): **v5 KPIs**, set from that baseline, measured on the same lane at `--load 24`. Shooter: main thread busy at or below 48% (0.5x of 97%), p50 no worse than 18.6 ms, p95 at or below 116.3 ms with 58 ms (0.5x) as the target. Starter: main thread busy at or below 28% (0.5x of 55%), p50 no worse than 17.6 ms. A worker run that is not on this lane re-measures its own main-thread arm. - 2026-10-26 (owner): scale-down rule. Bounded implementation slices go to a cheaper arm (`opencode-go/deepseek-v4.1-flash`, variant `max`, at most 2 arms). The lead designs, writes the red tests and reviews every diff. ### Phase 1: Loaded-host baseline -**Status:** NOT STARTED +**Status:** DONE Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated load. -- [ ] A repeatable loaded-host profile exists: it starts N busy-loop processes (N and load recorded), runs the shooter and the starter, and prints main-thread busy %, frame p50 and p95. proof: `sh scripts/xvfb.sh pnpm profile:wasm-page -- --url --cpu-work --seconds 10 --json`, with the command and output path in this box -- [ ] The baseline for the shooter and the starter is recorded in this PRD under `## Decisions`, with load, adapter name (not SwiftShader) and three runs per arm. The v5 KPI numbers are set from it. proof: the `## Decisions` entry, `adapter.info` captured in the run +- [x] A repeatable loaded-host profile exists: it starts N busy-loop processes (N and load recorded), runs the shooter and the starter, and prints main-thread busy %, frame p50 and p95. proof: `sh scripts/xvfb.sh pnpm profile:wasm-page -- --url --seconds 10 --warmup-ms 8000 --cpu-work --load 24 --json`. Result: `--load` and `busyPercent` added, `scripts/__tests__/profile-wasm-page.spec.ts` 15/15 green, 12 runs recorded in Decisions +- [x] The baseline for the shooter and the starter is recorded in this PRD under `## Decisions`, with load, adapter name (not SwiftShader) and three runs per arm. The v5 KPI numbers are set from it. proof: the `## Decisions` entry, `adapter.info` captured in the run. Result: baseline and KPI entries above, adapter "nvidia turing" in all 12 runs ### Phase 2: Host-capability layer -**Status:** NOT STARTED +**Status:** PARTIAL — box 1 done -- [ ] A red-green core spec scans `packages/core/src` and fails on any direct `document.` or `window.` read outside the host-capability layer (an allow-list of the one layer file). It is red on the 23 current files and green after the move. proof: `pnpm exec vitest run packages/core/__tests__/host-capability-boundary.spec.ts` +- [x] A red-green spec scans `packages/core/src` and fails on any read of a global a dedicated worker lacks (type check against the `webworker` lib, value positions only) outside the host-capability layer `packages/core/src/platform.ts`. It is red on the current reads and green after the move. proof: `pnpm exec vitest run scripts/__tests__/check-core-boundary.spec.ts`. Result: red with 19 reads in 5 files, green 32/32 after the move to `hostWindow`/`hostDocument`/`hostResizeObserver`; `pnpm exec tsx scripts/check-core-boundary.ts` passes; 11 related core and boundary spec files 248/248; core typecheck exit 0 - [ ] The native host runs core unmodified through the same layer, with no dynamic `import()` in core. proof: `pnpm exec vitest run scripts/__tests__/check-core-boundary.spec.ts`; `pnpm native:verify:desktop` - [ ] The starter plays through a loopback proxy (main thread talks to the layer over a message channel in the same thread) with unchanged behaviour. proof: `pnpm test:templates` on the starter, green diff --git a/packages/core/src/audio.ts b/packages/core/src/audio.ts index 639692d510..92d19eeea2 100644 --- a/packages/core/src/audio.ts +++ b/packages/core/src/audio.ts @@ -1,4 +1,5 @@ import { AudioListener, Object3D, PositionalAudio, Audio as ThreeAudio, type Vector3 } from "three"; +import { hostWindow } from "./platform.js"; /** * Bus-level dynamics compression, every value the game's. @@ -255,7 +256,7 @@ export class AudioBus { this.#camera = options.camera; this.listener = options.listener ?? new AudioListener(); if (options.compressor !== undefined) this.#installCompressor(options.compressor); - const source = options.source ?? (() => (typeof window === "undefined" ? undefined : window)); + const source = options.source ?? (() => hostWindow()); this.#gestureTarget = options.gestureTarget ?? source(); this.setCamera(options.camera); this.#gesture = () => { diff --git a/packages/core/src/game.ts b/packages/core/src/game.ts index bd2c472225..ce7f24eafa 100644 --- a/packages/core/src/game.ts +++ b/packages/core/src/game.ts @@ -33,7 +33,7 @@ import { MatrixWorldPass } from "./matrix-world.js"; import { updateModelLods } from "./model-lod.js"; import { ScenePicker } from "./picking.js"; import type { IPipelineCensus } from "./pipeline-census.js"; -import { getPlatform } from "./platform.js"; +import { getPlatform, hostDocument, hostWindow } from "./platform.js"; import { PointerEvents3D } from "./pointer-events.js"; import { FrameCounters, counterDeviceOf } from "./profiling/FrameCounters.js"; import { @@ -1104,22 +1104,18 @@ class GameImpl, TPhysics> this.#renderer = renderer; const canvas = renderer.domElement; const platform = this.#config.platform; + const page = hostDocument(); if (platform !== undefined) platform.mountCanvas(canvas, this.#config.container); else if ( this.#config.container !== undefined && canvas.parentElement !== this.#config.container ) this.#config.container.append(canvas); - else if (canvas.parentElement === null && typeof document !== "undefined") - document.body.append(canvas); + else if (canvas.parentElement === null && page !== undefined) page.body.append(canvas); const inputTarget = this.#config.inputTarget ?? - (platform === undefined - ? typeof window === "undefined" - ? canvas - : window - : (platform.inputTarget ?? canvas)); + (platform === undefined ? (hostWindow() ?? canvas) : (platform.inputTarget ?? canvas)); this.#input = new InputMap( this.#config.input, inputTarget, @@ -1555,12 +1551,7 @@ class GameImpl, TPhysics> this.#entities = entities; this.#scheduler = scheduler; this.#afterPhysicsPhase = afterPhysicsPhase; - const devToolsHost = - platform === undefined - ? typeof window === "undefined" - ? undefined - : window - : platform.devToolsHost; + const devToolsHost = platform === undefined ? hostWindow() : platform.devToolsHost; this.#cleanup.push( installDevTools( entities, diff --git a/packages/core/src/hot.ts b/packages/core/src/hot.ts index c69b4e4178..01fd9ffcc9 100644 --- a/packages/core/src/hot.ts +++ b/packages/core/src/hot.ts @@ -2,6 +2,7 @@ import { audioRuntimeSnapshot } from "./audio.js"; import type { IGame } from "./game.js"; +import { hostDocument, hostWindow } from "./platform.js"; export interface IHotDiagnostics { readonly reloads: number; readonly entities: number; @@ -56,8 +57,9 @@ export function acceptHotUpdate, TPhysics // the game boots at its start scene, which is what it would have done anyway. } } - if (isDev && typeof window !== "undefined") { - const host = window as unknown as Record & + const page = hostWindow(); + if (isDev && page !== undefined) { + const host = page as unknown as Record & Partial & { hot?: () => IHotDiagnostics }>>; host.__THREENATIVE__ = { ...host.__THREENATIVE__, @@ -115,7 +117,7 @@ function diagnostics, TPhysics>( reloads, entities: ctx === undefined ? 0 : Object.keys(ctx.entities.snapshot()).length, sceneObjects, - canvases: typeof document === "undefined" ? 0 : document.querySelectorAll("canvas").length, + canvases: hostDocument()?.querySelectorAll("canvas").length ?? 0, audio: audioRuntimeSnapshot(), physics: typeof physics?.numBodies === "function" ? physics.numBodies() : null, }; diff --git a/packages/core/src/platform.ts b/packages/core/src/platform.ts index fdf469f45a..1b0c2c209b 100644 --- a/packages/core/src/platform.ts +++ b/packages/core/src/platform.ts @@ -208,3 +208,18 @@ export function isMobile(): boolean { export function isTouchscreenAvailable(): boolean { return getPlatform().maxTouchPoints > 0; } + +/** The page's `window`, or undefined in a worker or on a host without one. Core reads it only here (PRD-752). */ +export function hostWindow(): (Window & typeof globalThis) | undefined { + return typeof window === "undefined" ? undefined : window; +} + +/** The page's `document`, or undefined in a worker or on a host without one. */ +export function hostDocument(): Document | undefined { + return typeof document === "undefined" ? undefined : document; +} + +/** `ResizeObserver`, or undefined where the host has none. */ +export function hostResizeObserver(): typeof ResizeObserver | undefined { + return typeof ResizeObserver === "undefined" ? undefined : ResizeObserver; +} diff --git a/packages/core/src/renderer.ts b/packages/core/src/renderer.ts index 9e993aec01..0ecc5d015e 100644 --- a/packages/core/src/renderer.ts +++ b/packages/core/src/renderer.ts @@ -28,6 +28,7 @@ import { type PipelineCensus, createPipelineCensus, } from "./pipeline-census.js"; +import { hostDocument, hostResizeObserver } from "./platform.js"; import type { ITimestampQueryPool } from "./render-pass-budget.js"; import { AlphaAntialiasing, type IAlphaAntialiasingReport } from "./render/alpha-antialiasing.js"; import { @@ -560,8 +561,9 @@ export function readCanvasSize(canvas: HTMLCanvasElement): readonly [number, num } export function observeCanvasResize(canvas: HTMLCanvasElement, resize: () => void): () => void { - if (typeof ResizeObserver !== "undefined") { - const observer = new ResizeObserver(resize); + const Observer = hostResizeObserver(); + if (Observer !== undefined) { + const observer = new Observer(resize); observer.observe(canvas); return () => observer.disconnect(); } @@ -1835,6 +1837,15 @@ export async function adapterTextureLimits(): Promise<{ requiredLimits?: Record< } } +function newPageCanvas(): HTMLCanvasElement { + const page = hostDocument(); + if (page === undefined) + throw new Error( + "TN_RENDERER_NO_CANVAS: no canvas was given and this host has no document; pass options.canvas", + ); + return page.createElement("canvas"); +} + export async function createRenderer(options: IRendererOptions = {}): Promise { const source = options.source; const gpuTimestampFrameInterval = @@ -1854,7 +1865,7 @@ export async function createRenderer(options: IRendererOptions = {}): Promise { await expect(checkCoreBoundary(root)).resolves.toEqual([]); }); }); + +describe("core reaches worker-missing globals only through platform.ts (PRD-752)", () => { + const core = (root: string, file: string) => path.join(root, "packages/core/src", file); + + it("names a core file that reads document or window directly", async () => { + const root = await fixtureRoot(); + await writeFile(core(root, "entities.ts"), "export const ok = true;\n"); + await writeFile( + core(root, "hud.ts"), + 'export const live = typeof window !== "undefined";\nexport const body = document.body;\n', + ); + const findings = await checkCoreBoundary(root); + expect(findings).toContain( + "packages/core/src/hud.ts:1 reads 'window', which a worker does not have; reach it through packages/core/src/platform.ts", + ); + expect(findings.join("\n")).toContain("packages/core/src/hud.ts:2 reads 'document'"); + }); + + it("allows the host layer, type-only DOM names and a local named window", async () => { + const root = await fixtureRoot(); + await writeFile(core(root, "platform.ts"), "export const body = () => document.body;\n"); + await writeFile( + core(root, "frames.ts"), + [ + "export function size(canvas: HTMLCanvasElement): number { return canvas.width; }", + "const window = { fps: 60 };", + "export const fps = window.fps;", + ].join("\n"), + ); + await expect(checkWorkerGlobals(root)).resolves.toEqual([]); + }); + + it("finds no worker-missing global in the real core source", { timeout: 120_000 }, async () => { + await expect(checkWorkerGlobals(path.resolve(import.meta.dirname, "../.."))).resolves.toEqual( + [], + ); + }); +}); diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 1ec25c606e..6071866624 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -119,7 +119,12 @@ describe("profile:wasm-page startup, input and CPU work", () => { samples: [1, 2, 2, 1], timeDeltas: [1000, 1000, 1000, 1000], }; - expect(cpuWork(profile, 2)).toEqual({ busyMsPerFrame: 1, frames: 2, wallMs: 4 }); + expect(cpuWork(profile, 2)).toEqual({ + busyMsPerFrame: 1, + busyPercent: 50, + frames: 2, + wallMs: 4, + }); expect(() => cpuWork(profile, 0)).toThrow("TN_PROFILE_CPU_WORK_NO_FRAMES"); expect(() => cpuWork({ ...profile, timeDeltas: [1] }, 2)).toThrow("TN_PROFILE_CPU_WORK_DELTAS"); }); diff --git a/scripts/check-core-boundary.ts b/scripts/check-core-boundary.ts index bd492eeac1..8bddc1656c 100644 --- a/scripts/check-core-boundary.ts +++ b/scripts/check-core-boundary.ts @@ -1,8 +1,11 @@ import { existsSync } from "node:fs"; import { readFile, readdir } from "node:fs/promises"; import path from "node:path"; +import ts from "typescript"; export const ENTITY_LINE_LIMIT = 80; +// The one core file that may read globals a dedicated worker lacks (PRD-752). +export const HOST_LAYER = "packages/core/src/platform.ts"; const BANNED_ENTITY_TOKENS = /\b(?:archetype|createQuery|defineComponent|System|Component)\b/gu; // The Academy's licence text, which the ACES curve in this file must keep, uses the English words "System" and "Components". @@ -179,11 +182,75 @@ async function checkBuiltScaffold(root: string): Promise { return findings; } +function deepestNodeAt(sourceFile: ts.SourceFile, position: number): ts.Node { + let node: ts.Node = sourceFile; + for (;;) { + let next: ts.Node | undefined; + for (const child of node.getChildren(sourceFile)) { + if (child.pos <= position && position < child.end) { + next = child; + break; + } + } + if (!next) return node; + node = next; + } +} + +function isTypePosition(node: ts.Node): boolean { + for (let current: ts.Node | undefined = node; current; current = current.parent) { + if (ts.isTypeNode(current)) return true; + if (ts.isStatement(current)) return false; + } + return false; +} + +export async function checkWorkerGlobals(root: string): Promise { + const directory = path.join(root, "packages/core/src"); + if (!existsSync(directory)) return []; + const files = (await filesUnder(directory)).filter((file) => + CORE_SOURCE_EXTENSIONS.has(path.extname(file)), + ); + const program = ts.createProgram(files, { + target: ts.ScriptTarget.ES2023, + module: ts.ModuleKind.ESNext, + moduleResolution: ts.ModuleResolutionKind.Bundler, + lib: ["lib.es2023.d.ts", "lib.webworker.d.ts"], + types: [], + noEmit: true, + skipLibCheck: true, + strict: true, + }); + const findings: string[] = []; + const seen = new Set(); + for (const file of files) { + const sourceFile = program.getSourceFile(file); + if (!sourceFile) continue; + const relativePath = relative(root, file); + if (relativePath === HOST_LAYER) continue; + for (const diagnostic of program.getSemanticDiagnostics(sourceFile)) { + if (![2304, 2552, 2584].includes(diagnostic.code) || diagnostic.start === undefined) continue; + if (isTypePosition(deepestNodeAt(sourceFile, diagnostic.start))) continue; + const name = sourceFile.text.slice( + diagnostic.start, + diagnostic.start + (diagnostic.length ?? 0), + ); + const line = sourceFile.getLineAndCharacterOfPosition(diagnostic.start).line + 1; + const finding = `${relativePath}:${line} reads '${name}', which a worker does not have; reach it through ${HOST_LAYER}`; + if (seen.has(finding)) continue; + seen.add(finding); + findings.push(finding); + } + } + return findings; +} + export async function checkCoreBoundary(root = process.cwd()): Promise { return [ ...(await checkEntityRegistry(root)), ...(await checkScaffoldHygiene(root)), ...(await checkBuiltScaffold(root)), + ...(await checkWorkerGlobals(root)), ]; } @@ -196,7 +263,7 @@ async function main(): Promise { process.exitCode = 1; return; } - console.log("core boundary and scaffold hygiene passed"); + console.log("core boundary, scaffold hygiene and worker globals passed"); } if ( diff --git a/scripts/profile-wasm-page.ts b/scripts/profile-wasm-page.ts index d374fead20..efa09a4b2e 100644 --- a/scripts/profile-wasm-page.ts +++ b/scripts/profile-wasm-page.ts @@ -132,7 +132,9 @@ export function cpuWork(profile: ICpuProfile, frames: number) { wallUs += delta; if (!idle.has(id)) busyUs += delta; }); - return { busyMsPerFrame: busyUs / 1000 / frames, frames, wallMs: wallUs / 1000 }; + // The share of the profile window the main thread was busy: the worker KPI (PRD-752). + const busyPercent = wallUs > 0 ? (100 * busyUs) / wallUs : 0; + return { busyMsPerFrame: busyUs / 1000 / frames, busyPercent, frames, wallMs: wallUs / 1000 }; } /** @@ -467,7 +469,7 @@ function print(label: string, r: Awaited>) { ); if (r.profile.work) console.log( - `cpu work ${r.profile.work.busyMsPerFrame.toFixed(2)} ms per frame (thread CPU ${r.profile.work.threadMsPerFrame.toFixed(2)} ms) over ${r.profile.work.frames} frames`, + `cpu work ${r.profile.work.busyMsPerFrame.toFixed(2)} ms per frame, main thread ${r.profile.work.busyPercent.toFixed(0)}% busy (thread CPU ${r.profile.work.threadMsPerFrame.toFixed(2)} ms) over ${r.profile.work.frames} frames`, ); console.log( `cpu (${r.profile.samples} samples): ${r.profile.groups.map((g) => `${g.name} ${g.percent}%`).join(", ")}`, From 3795190852cdbe6d1de5845dd82782b6c06695f5 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 20:38:02 -0700 Subject: [PATCH 04/29] UI bridge loopback over a MessageChannel behind ?tnUiLoopback=1 (PRD-752) Both ends stay in one realm but every frame arrives a task later, the hop a game in a worker adds. Installed as the host globals by whichever end connects first, so the game and the HUD do not change. --- .../core/__tests__/ui-bridge-inbound.spec.ts | 33 +++++++++++++++++++ packages/core/src/ui-bridge.ts | 31 +++++++++++++++++ 2 files changed, 64 insertions(+) diff --git a/packages/core/__tests__/ui-bridge-inbound.spec.ts b/packages/core/__tests__/ui-bridge-inbound.spec.ts index 7f739f15b4..d378dc0a84 100644 --- a/packages/core/__tests__/ui-bridge-inbound.spec.ts +++ b/packages/core/__tests__/ui-bridge-inbound.spec.ts @@ -61,3 +61,36 @@ describe("ui bridge inbound global", () => { expect(scope[UI_BRIDGE_GLOBALS.gameReceive]).toBeUndefined(); }); }); + +describe("ui bridge loopback over a message port (PRD-752)", () => { + const tick = () => new Promise((resolve) => setTimeout(resolve, 0)); + + it("carries state and intents over an async port when the page asks for ?tnUiLoopback=1", async () => { + const scope: Record = { location: { search: "?tnUiLoopback=1" } }; + const ui = connectUiBridge({ end: "ui", scope }); + const game = connectUiBridge({ end: "game", scope }); + const seenByUi: unknown[] = []; + const seenByGame: unknown[] = []; + ui.onMessage((message) => seenByUi.push(message)); + game.onMessage((message) => seenByGame.push(message)); + + expect([ui.transport, game.transport]).toEqual(["host", "host"]); + expect(game.hasPeer()).toBe(true); + game.post({ type: "tn:state", state: { score: 3 } }); + ui.post({ type: "tn:intent", name: "pause" }); + // A worker delivers a task later, never inside the post: the loopback must too. + expect([seenByUi, seenByGame]).toEqual([[], []]); + await tick(); + expect(seenByUi).toEqual([{ type: "tn:state", state: { score: 3 } }]); + expect(seenByGame).toEqual([{ type: "tn:intent", name: "pause" }]); + + ui.close(); + game.close(); + (scope.__tnUiLoopback as { close(): void }).close(); + }); + + it("keeps the in-process broker without the flag", () => { + const scope: Record = { location: { search: "" } }; + expect(connectUiBridge({ end: "ui", scope }).transport).toBe("in-process"); + }); +}); diff --git a/packages/core/src/ui-bridge.ts b/packages/core/src/ui-bridge.ts index 0ad875e8eb..5e00c81dde 100644 --- a/packages/core/src/ui-bridge.ts +++ b/packages/core/src/ui-bridge.ts @@ -270,6 +270,36 @@ function installInbound(scope: IScope, key: string, deliver: (frame: unknown) => }; } +const LOOPBACK_KEY = "__tnUiLoopback"; + +/** + * `?tnUiLoopback=1`: both ends stay in this realm but talk over a `MessageChannel`, so every + * frame arrives a task later, the hop a game in a worker adds (PRD-752). Installed once, as the + * host globals, by whichever end connects first; the game and the UI never know. + */ +function installLoopback(scope: IScope): void { + const search = (scope.location as { search?: unknown } | undefined)?.search; + if (typeof search !== "string" || new URLSearchParams(search).get("tnUiLoopback") !== "1") return; + if (scope[LOOPBACK_KEY] !== undefined) return; + const { port1: gamePort, port2: uiPort } = new MessageChannel(); + const receive = (key: string) => (event: MessageEvent) => + (scope[key] as ((frame: unknown) => void) | undefined)?.(event.data); + gamePort.onmessage = receive(UI_BRIDGE_GLOBALS.gameReceive); + uiPort.onmessage = receive(UI_BRIDGE_GLOBALS.uiReceive); + scope[UI_BRIDGE_GLOBALS.gamePost] = (frame: string) => gamePort.postMessage(frame); + scope[UI_BRIDGE_GLOBALS.uiHost] = { postMessage: (frame: string) => uiPort.postMessage(frame) }; + scope.__tnUiOverlayAttached = () => true; + scope[LOOPBACK_KEY] = { + close() { + gamePort.close(); + uiPort.close(); + for (const key of [UI_BRIDGE_GLOBALS.gamePost, UI_BRIDGE_GLOBALS.uiHost, LOOPBACK_KEY]) + scope[key] = undefined; + scope.__tnUiOverlayAttached = undefined; + }, + }; +} + /** * Connect one end of the bridge. * @@ -286,6 +316,7 @@ export function connectUiBridge(options: IConnectOptions): IUiBridge { throw new Error(`TN_UI_BRIDGE_END_INVALID: expected 'game' or 'ui', got '${String(end)}'.`); } const scope: IScope = options.scope ?? globalThis; + installLoopback(scope); const listeners = new Set<(message: IUiMessage) => void>(); const deliver = (frame: unknown): void => { const message = parseFrame(frame); From 69166e555da2fa9ed452ddea163fe7e32caa1e72 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 20:49:05 -0700 Subject: [PATCH 05/29] PRD-752 phase 2 done: native desktop gate and starter loopback proof --- ...games-run-in-a-worker-so-the-page-cannot-stall-them.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 9cfc1cc7bb..d611928bac 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -1,6 +1,6 @@ # PRD-752 — Web games run in a worker, so the page cannot stall them -**Status:** PARTIAL — phase 1 done; phase 2 box 1 done. +**Status:** PARTIAL — phases 1 and 2 done; phase 3 next. **Priority:** P1 — on a loaded host every web game collapses (shooter kit: frame p50 33 ms, p95 100 ms, main thread 100% busy); worker mode (phase 3) and Wasm threads (phase 4) are open. **Complexity:** 8 (HIGH) — 10+ files across core, the web host, the Wasm engine and the harness (+3), a new host-capability layer (+2), worker and `OffscreenCanvas` boot (+2), Wasm threads and isolation headers (+1); risk override: none **Owner:** João @@ -70,11 +70,11 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo - [x] The baseline for the shooter and the starter is recorded in this PRD under `## Decisions`, with load, adapter name (not SwiftShader) and three runs per arm. The v5 KPI numbers are set from it. proof: the `## Decisions` entry, `adapter.info` captured in the run. Result: baseline and KPI entries above, adapter "nvidia turing" in all 12 runs ### Phase 2: Host-capability layer -**Status:** PARTIAL — box 1 done +**Status:** DONE - [x] A red-green spec scans `packages/core/src` and fails on any read of a global a dedicated worker lacks (type check against the `webworker` lib, value positions only) outside the host-capability layer `packages/core/src/platform.ts`. It is red on the current reads and green after the move. proof: `pnpm exec vitest run scripts/__tests__/check-core-boundary.spec.ts`. Result: red with 19 reads in 5 files, green 32/32 after the move to `hostWindow`/`hostDocument`/`hostResizeObserver`; `pnpm exec tsx scripts/check-core-boundary.ts` passes; 11 related core and boundary spec files 248/248; core typecheck exit 0 -- [ ] The native host runs core unmodified through the same layer, with no dynamic `import()` in core. proof: `pnpm exec vitest run scripts/__tests__/check-core-boundary.spec.ts`; `pnpm native:verify:desktop` -- [ ] The starter plays through a loopback proxy (main thread talks to the layer over a message channel in the same thread) with unchanged behaviour. proof: `pnpm test:templates` on the starter, green +- [x] The native host runs core unmodified through the same layer, with no dynamic `import()` in core. proof: `pnpm exec vitest run scripts/__tests__/check-core-boundary.spec.ts`; `pnpm native:verify:desktop`. Result: 32/32; `native:verify:desktop` exit 0 ("desktop core gate passed: 300 frames, 1280x720", UI frame gate passed) after `pnpm native:build` in this worktree +- [x] The starter plays through a loopback proxy (main thread talks to the layer over a message channel in the same thread) with unchanged behaviour. proof: `packages/core/__tests__/ui-bridge-inbound.spec.ts`; a pause round-trip scenario (click `button[aria-pressed]`, `hud` asserts `pause` -> `resume`) on a scaffolded starter at `/` and at `/?tnUiLoopback=1`; the starter's own `pnpm test` (`test:templates` skips the starter: it runs in the golden-path lane). Result: spec 6/6; round trip exit 0 on both URLs; a Playwright probe reads `__tnUiLoopback` false at `/` and true with the flag; kit `movement`, `survives`, `production-readiness` pass. `play` fails `performance.maxFrameMsP95` on GPU time on this host, and the scaffold from before this change fails it too (before 2/2 runs at GPU p95 20.3/17.3 ms, after 1/2 at 18.1 ms; CPU about 7.5 ms in both) ### Phase 3: Worker mode and v5 KPI **Status:** NOT STARTED From 8f8fcbc2b4efa3fb4e28e1ea6cd8fca677303206 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 22:29:54 -0700 Subject: [PATCH 06/29] Engine side of the worker boot: platform hook, worker engine bundle, named WebGPU failure (PRD-752) A scratch starter booted in a worker on an OffscreenCanvas on the Wasm engine. These are the engine-layer fixes that spike needed: - core: defineGame takes a platform source a worker host installs (__tnGamePlatform) when the portable entry names none. - core: TN_NO_RENDERER names the WebGPU failure instead of swallowing it. - create-threenative: worker bundles get the engine resolver and ES output. - runtime-native: the web module is built for web and worker environments. - three-native: the renderer tolerates an OffscreenCanvas (no setAttribute). - PRD: the spike result and the phase 3 design. --- ...-a-worker-so-the-page-cannot-stall-them.md | 9 ++++ packages/core/__tests__/game.spec.ts | 52 +++++++++++++++++++ packages/core/__tests__/renderer.spec.ts | 26 ++++++++++ packages/core/src/game.ts | 20 +++++-- packages/core/src/renderer.ts | 6 ++- .../__tests__/web-brand.spec.ts | 27 ++++++++++ packages/create-threenative/src/web-brand.ts | 14 +++++ .../cmake/NativeEngineCore.cmake | 4 +- packages/three-native/src/browser-renderer.ts | 3 +- 9 files changed, 154 insertions(+), 7 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index d611928bac..54ff1e51cd 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -59,6 +59,15 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me The starter load-0 runs followed the shooter load-24 runs, so their load1 is the lagging one-minute average, not live load. The loaded shooter reproduces the reported collapse. - 2026-10-26 (lead): **v5 KPIs**, set from that baseline, measured on the same lane at `--load 24`. Shooter: main thread busy at or below 48% (0.5x of 97%), p50 no worse than 18.6 ms, p95 at or below 116.3 ms with 58 ms (0.5x) as the target. Starter: main thread busy at or below 28% (0.5x of 55%), p50 no worse than 17.6 ms. A worker run that is not on this lane re-measures its own main-thread arm. +- 2026-10-26 (lead): **phase 3 spike and design.** A scratch starter booted in a dedicated worker on a transferred `OffscreenCanvas`, on the Wasm engine (`TN_RENDERER` present, adapter "nvidia turing"). It drew the same frame as the main-thread arm. The spike found 6 gaps, each fixed at its own layer: + 1. The Emscripten glue read `document` at load. Fix: `-sENVIRONMENT=web,worker` on `tn-native-engine-web`. + 2. The Wasm renderer called `setAttribute` on its canvas, and an `OffscreenCanvas` has none. Fix: optional call in `browser-renderer.ts`. + 3. Vite builds a worker bundle with `worker.plugins` only, so the worker got upstream three, and skin weights failed as `unorm32x4`. Fix: the web brand plugin also registers the engine resolver for workers, as ES output. + 4. A message sent while the worker module is in its top-level await is dispatched before any handler exists, so it is lost. Fix: the worker posts `ready`, and the page sends the canvas after it. + 5. A relative `fetch("sky.jpg")` or `new Request("grade.cube")` resolves against the worker script URL (`/assets/`). Fix: the worker host resolves relative URLs against the page URL. + 6. A worker has no `AudioContext`. Fix: a Web Audio proxy. Each node is a handle, calls go over the port, and the page runs the real graph. + + Core already has the seam: `defineGame({ platform })` (`IGamePlatformSource`: canvas, size, input target, gamepads). No production code supplied it before. Worker mode supplies it from the worker host. The starter's `GameUi` already talks to the game only through the UI bridge, so the page mounts `GameUi` and the game runs in `src/worker.ts`. The UI bridge host globals carry the HUD over the worker port. A swallowed WebGPU error also hid gap 3 behind `TN_NO_RENDERER`, so `TN_NO_RENDERER` now names the WebGPU cause. - 2026-10-26 (owner): scale-down rule. Bounded implementation slices go to a cheaper arm (`opencode-go/deepseek-v4.1-flash`, variant `max`, at most 2 arms). The lead designs, writes the red tests and reviews every diff. ### Phase 1: Loaded-host baseline diff --git a/packages/core/__tests__/game.spec.ts b/packages/core/__tests__/game.spec.ts index 312de01ac3..06d7c3501d 100644 --- a/packages/core/__tests__/game.spec.ts +++ b/packages/core/__tests__/game.spec.ts @@ -3519,6 +3519,58 @@ describe("IGame", () => { } }); + it("takes the platform source a worker host installs when the game names none (PRD-752)", async () => { + // Worker mode: game.ts is the unchanged portable entry, so the worker host cannot pass + // `platform` to defineGame. It installs the source on the global scope before start(). + const canvas = new EventTarget() as HTMLCanvasElement; + let mounted = 0; + let unmounted = 0; + const platform: IGamePlatformSource = { + input: () => [], + inputTarget: new EventTarget(), + mountCanvas: () => { + mounted += 1; + }, + renderer: { + createCanvas: () => canvas, + hasWebGPU: () => false, + observeResize: () => () => undefined, + readSize: () => [320, 180], + }, + unmountCanvas: () => { + unmounted += 1; + }, + viewport: { + observeResize: () => () => undefined, + readSize: () => ({ aspect: 16 / 9, height: 180, width: 320 }), + }, + }; + vi.stubGlobal("__tnGamePlatform", platform); + const game = defineGame({ + renderer: { + preferWebGPU: false, + webgl2Factory: () => ({ + dispose: () => undefined, + domElement: canvas, + render: () => undefined, + setSize: () => undefined, + }), + }, + scenes: { test: EmptyScene }, + start: "test", + }); + try { + await game.start(); + expect(mounted).toBe(1); + expect(game.ctx?.viewport.size).toEqual({ aspect: 16 / 9, height: 180, width: 320 }); + game.stop(); + expect(unmounted).toBe(1); + } finally { + game.stop(); + vi.unstubAllGlobals(); + } + }); + it.each([ { opaque: false, worldDrawn: false }, { opaque: true, worldDrawn: true }, diff --git a/packages/core/__tests__/renderer.spec.ts b/packages/core/__tests__/renderer.spec.ts index 4f9bbd2417..2b8b2ec8cc 100644 --- a/packages/core/__tests__/renderer.spec.ts +++ b/packages/core/__tests__/renderer.spec.ts @@ -953,6 +953,32 @@ describe("createRenderer", () => { } }); + it("names the WebGPU failure in TN_NO_RENDERER instead of swallowing it (PRD-752)", async () => { + // A worker boot fell back because the Wasm renderer called a DOM-only canvas method; the + // boot error said only "WebGPU is not available", which hid the fix. + const descriptor = Object.getOwnPropertyDescriptor(globalThis, "navigator"); + Object.defineProperty(globalThis, "navigator", { configurable: true, value: { gpu: {} } }); + try { + await expect( + createRenderer({ + canvas: testCanvas(), + preferWebGPU: true, + webgl2Factory: () => { + throw new Error("no WebGL"); + }, + webgpuFactory: () => { + throw new TypeError("this.domElement.setAttribute is not a function"); + }, + }), + ).rejects.toThrow( + /^TN_NO_RENDERER: .* WebGPU failed: this\.domElement\.setAttribute is not a function$/u, + ); + } finally { + if (descriptor === undefined) Reflect.deleteProperty(globalThis, "navigator"); + else Object.defineProperty(globalThis, "navigator", descriptor); + } + }); + it("reports a facade and Wasm engine of two web host ABIs as that, not as a device without WebGPU", async () => { const descriptor = Object.getOwnPropertyDescriptor(globalThis, "navigator"); Object.defineProperty(globalThis, "navigator", { configurable: true, value: { gpu: {} } }); diff --git a/packages/core/src/game.ts b/packages/core/src/game.ts index ce7f24eafa..20f47ef2d1 100644 --- a/packages/core/src/game.ts +++ b/packages/core/src/game.ts @@ -1063,6 +1063,17 @@ class GameImpl, TPhysics> return pendingStart; } + /** + * The game's own platform source, else the one a worker host installed on the global scope: + * worker mode runs the unchanged portable entry, so it cannot pass `platform` here (PRD-752). + */ + get #platform(): IGamePlatformSource | undefined { + return ( + this.#config.platform ?? + (globalThis as { __tnGamePlatform?: IGamePlatformSource }).__tnGamePlatform + ); + } + async #boot(): Promise { const bootSceneName = this.#resumeScene ?? this.#config.start; this.#resumeScene = undefined; @@ -1095,7 +1106,7 @@ class GameImpl, TPhysics> this.#config.renderer?.resolutionScale, getPlatform().os, ), - source: this.#config.renderer?.source ?? this.#config.platform?.renderer, + source: this.#config.renderer?.source ?? this.#platform?.renderer, }); if (this.#aborted) { renderer.dispose(); @@ -1103,7 +1114,7 @@ class GameImpl, TPhysics> } this.#renderer = renderer; const canvas = renderer.domElement; - const platform = this.#config.platform; + const platform = this.#platform; const page = hostDocument(); if (platform !== undefined) platform.mountCanvas(canvas, this.#config.container); else if ( @@ -2498,8 +2509,9 @@ class GameImpl, TPhysics> const renderer = this.#renderer; renderer?.dispose(); if (renderer !== undefined) { - if (this.#config.platform === undefined) renderer.domElement.remove?.(); - else this.#config.platform.unmountCanvas(renderer.domElement); + const platform = this.#platform; + if (platform === undefined) renderer.domElement.remove?.(); + else platform.unmountCanvas(renderer.domElement); } this.#renderer = undefined; this.#viewport = undefined; diff --git a/packages/core/src/renderer.ts b/packages/core/src/renderer.ts index 0ecc5d015e..fab674306d 100644 --- a/packages/core/src/renderer.ts +++ b/packages/core/src/renderer.ts @@ -1891,6 +1891,9 @@ export async function createRenderer(options: IRendererOptions = {}): Promise resolveSampleCount(raw), }); let renderer: IRendererLike | undefined; + // Why WebGPU did not start, when it was tried: the boot failure must name it, or a fault in the + // WebGPU path (PRD-752: a DOM-only canvas call in a worker) reads as a device without WebGPU. + let webgpuFailure: string | undefined; if (preferWebGPU && (source?.hasWebGPU() ?? "gpu" in (globalThis.navigator ?? {}))) { try { @@ -1933,6 +1936,7 @@ export async function createRenderer(options: IRendererOptions = {}): Promise { delete process.env.THREENATIVE_NATIVE_BUNDLE; } }); + it("routes three to the Wasm engine inside a worker bundle too (PRD-752)", async () => { + // Vite runs only `worker.plugins` on a worker bundle. Without the resolver there, a game booted + // in a worker ran upstream three, which rejected the engine's skin weights as `unorm32x4`. + const root = await makeTempDir("threenative-web-brand-worker-"); + roots.push(root); + await writeFile( + path.join(root, "package.json"), + JSON.stringify({ name: "worker-game", type: "module" }), + ); + await writeFile( + path.join(root, "threenative.config.ts"), + 'export default { app: { name: "Worker Game" }, engine: "threenative" };\n', + ); + const plugin = createWebBrandPlugin(); + const config = (await plugin.config({ root })) as { + worker?: { + format?: string; + plugins?: () => { resolveId(source: string, importer?: string): unknown }[]; + }; + }; + await plugin.configResolved({ root }); + // ES output: the engine module awaits its Wasm at the top level. + expect(config.worker?.format).toBe("es"); + const workerPlugins = config.worker?.plugins?.() ?? []; + const routed = await Promise.all(workerPlugins.map((each) => each.resolveId("three"))); + expect(routed).toContain("\0threenative:web-engine"); + }); it("keeps an authored favicon when config has no replacement", async () => { const root = await makeTempDir("threenative-web-brand-"); roots.push(root); diff --git a/packages/create-threenative/src/web-brand.ts b/packages/create-threenative/src/web-brand.ts index 8fb5c3d17c..8fea7c2a97 100644 --- a/packages/create-threenative/src/web-brand.ts +++ b/packages/create-threenative/src/web-brand.ts @@ -132,6 +132,20 @@ export function createWebBrandPlugin(): IWebBrandVitePlugin { if (loaded.engine === "threenative" && process.env.THREENATIVE_NATIVE_BUNDLE === undefined) return { optimizeDeps: { exclude: ["three", "three/webgpu", "three/tsl", "three-mesh-bvh"] }, + // Vite runs only `worker.plugins` on a worker bundle, so a game booted in a worker + // (PRD-752) needs the same resolver, as ES output for the engine's top-level await. + worker: { + format: "es", + plugins: () => [ + { + name: "threenative-web-engine-worker", + enforce: "pre", + resolveId: (source: string, importer?: string) => + engine?.resolveId(source, importer) ?? null, + load: async (id: string) => engine?.load(id) ?? null, + }, + ], + }, }; return undefined; }, diff --git a/packages/runtime-native/cmake/NativeEngineCore.cmake b/packages/runtime-native/cmake/NativeEngineCore.cmake index 89427adb21..7a0c1642d5 100644 --- a/packages/runtime-native/cmake/NativeEngineCore.cmake +++ b/packages/runtime-native/cmake/NativeEngineCore.cmake @@ -977,7 +977,9 @@ if(EMSCRIPTEN) target_link_options(tn-native-engine-web PRIVATE --no-entry -sMODULARIZE=1 -sEXPORT_ES6=1 # The whole 32-bit address space: a game the size of Midway (161 MB of GLBs, decoded on the # engine side) ran past emscripten's 2 GB default maximum and failed with std::bad_alloc. - -sASSERTIONS=0 -sEXPORT_NAME=createTnWeb -sENVIRONMENT=web -sMAXIMUM_MEMORY=4GB -sALLOW_TABLE_GROWTH=1 + # `worker`: a game can run in a worker on an OffscreenCanvas (PRD-752); without it the glue + # reads `document` while the module loads. + -sASSERTIONS=0 -sEXPORT_NAME=createTnWeb -sENVIRONMENT=web,worker -sMAXIMUM_MEMORY=4GB -sALLOW_TABLE_GROWTH=1 "-sEXPORTED_FUNCTIONS=_tn_engine_version,_tn_context_create,_tn_context_destroy,_tn_type_id,_tn_object_release,_tn_object_engine_references,_tn_construct,_tn_invoke,_tn_get,_tn_set,_tn_set_callback,_tn_diagnostic_release,_tnw_attribute_view,_tnw_attribute_view_release,_tnw_web_init,_tnw_web_host_abi,_tnw_web_poll,_tnw_web_error,_tnw_web_adapter,_tnw_web_resize,_tnw_web_render,_tnw_web_compile_async,_tnw_web_compile_status,_tnw_web_frame,_tn_tsl_call,_tn_tsl_release,_tn_tsl_set,_tn_tsl_set_uniform,_tn_tsl_effect_parameter,_tnw_web_set_post,_malloc,_free,_tnw_web_renderer_state,_tnw_web_load_gltf,_tnw_web_load_error,_tnw_web_gltf_file,_tnw_web_gltf_files_clear,_tnw_web_render_target,_tnw_web_read_target,_tnw_web_read_target_take,_tnw_web_capture_face,_tnw_web_compute,_tnw_web_compute_flush,_tnw_web_read_storage,_tnw_web_read_storage_size,_tnw_web_read_storage_take,_tnw_web_delete_storage" "-sEXPORTED_RUNTIME_METHODS=wasmMemory,HEAPU8,HEAPU32,HEAPF64,UTF8ToString,stringToUTF8,lengthBytesUTF8,addFunction,specialHTMLTargets") endif() diff --git a/packages/three-native/src/browser-renderer.ts b/packages/three-native/src/browser-renderer.ts index 16304f2319..0fd04fb908 100644 --- a/packages/three-native/src/browser-renderer.ts +++ b/packages/three-native/src/browser-renderer.ts @@ -274,7 +274,8 @@ export function defineWebRenderer( (parameters.canvas as HTMLCanvasElement | undefined) ?? document.createElement("canvas"); // three's WebGPURenderer marks its canvas `three.js r webgpu`; tools read the backend there. - this.domElement.setAttribute("data-engine", "threenative wasm webgpu"); + // A worker's OffscreenCanvas has no attributes (PRD-752). + this.domElement.setAttribute?.("data-engine", "threenative wasm webgpu"); const samples = parameters.samples ?? (parameters.antialias ? 4 : 1); this.samples = Math.max(1, Number(samples)); this.#width = this.domElement.width || 300; From 929d0d93d40a6a79a4f184fd93e2f496339304df Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 22:40:46 -0700 Subject: [PATCH 07/29] Worker game host: @threenative/core/worker runs a game in a worker on an OffscreenCanvas (PRD-752) serveWorkerGame (worker) and startWorkerGame (page) speak a small message contract: a ready handshake before the canvas transfer, the platform source the game boots on, page-relative URLs, forwarded keyboard/pointer/wheel input in canvas coordinates, the UI bridge on its own port, and the playtest bridge mirrored onto the page as async calls. 9 linked-pair tests. --- packages/core/__tests__/worker.spec.ts | 295 ++++++++++++ packages/core/capabilities.json | 28 ++ packages/core/package.json | 4 + packages/core/src/worker.ts | 450 ++++++++++++++++++ packages/core/tsup.config.ts | 1 + .../references/capability-reference.md | 30 ++ packages/create-threenative/capabilities.json | 28 ++ scripts/api-surface.json | 3 + 8 files changed, 839 insertions(+) create mode 100644 packages/core/__tests__/worker.spec.ts create mode 100644 packages/core/src/worker.ts diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts new file mode 100644 index 0000000000..9658cc6e5a --- /dev/null +++ b/packages/core/__tests__/worker.spec.ts @@ -0,0 +1,295 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import type { IGamePlatformSource } from "../src/game.js"; +import { serveWorkerGame, startWorkerGame } from "../src/worker.js"; + +// PRD-752: a web game runs in a dedicated worker on an OffscreenCanvas. The page keeps the DOM, the +// HUD and the input; the worker runs the unchanged portable `game.ts`. These tests link the two ends +// in one process: what the page posts reaches the worker scope and back, one task later. + +type Scope = EventTarget & Record & { postMessage(data: unknown): void }; +type FakeWorker = EventTarget & { + postMessage(data: unknown, transfer?: unknown[]): void; + terminate(): void; +}; + +const tick = (ms = 5) => new Promise((resolve) => setTimeout(resolve, ms)); + +function link() { + const scope = new EventTarget() as Scope; + const worker = new EventTarget() as FakeWorker; + const toWorker: { type?: string }[] = []; + worker.postMessage = (data) => { + toWorker.push(data as { type?: string }); + setTimeout(() => scope.dispatchEvent(new MessageEvent("message", { data }))); + }; + worker.terminate = vi.fn(); + scope.postMessage = (data) => + setTimeout(() => worker.dispatchEvent(new MessageEvent("message", { data }))); + scope.fetch = vi.fn(async () => new Response("ok")); + scope.Request = Request; + return { scope, toWorker, worker }; +} + +function page() { + const offscreen = new EventTarget(); + const canvas = Object.assign(new EventTarget(), { + clientHeight: 180, + clientWidth: 320, + getBoundingClientRect: () => ({ height: 180, left: 10, top: 20, width: 320 }), + style: {} as Record, + transferControlToOffscreen: () => offscreen, + }); + const container = { append: vi.fn() }; + const view = Object.assign(new EventTarget(), { + devicePixelRatio: 2, + location: { href: "http://127.0.0.1:5173/game/index.html?tnWorker=1" }, + }) as unknown as EventTarget & Record; + let resize: (() => void) | undefined; + vi.stubGlobal("window", view); + vi.stubGlobal("document", Object.assign(new EventTarget(), { createElement: () => canvas })); + vi.stubGlobal( + "ResizeObserver", + class { + constructor(callback: () => void) { + resize = callback; + } + observe() {} + disconnect() {} + }, + ); + return { canvas, container, offscreen, resize: () => resize?.(), view }; +} + +function keyEvent(type: string, code: string): Event { + return Object.assign(new Event(type), { code, key: code.replace("Key", "").toLowerCase() }); +} + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +describe("worker game host (PRD-752)", () => { + it("sends the canvas only after the worker reports ready", async () => { + // A message sent while the worker module is still in its top-level await reaches no handler + // and is lost, so the page waits for the worker's own ready message. + const { scope, toWorker, worker } = link(); + const { container, offscreen } = page(); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await tick(); + expect(toWorker.some((message) => message.type === "tn-worker-init")).toBe(false); + serveWorkerGame({ start: async () => undefined }, scope); + await handle.ready; + const init = toWorker.find((message) => message.type === "tn-worker-init") as Record< + string, + unknown + >; + expect(init).toMatchObject({ canvas: offscreen, dpr: 2, height: 180, width: 320 }); + expect(container.append).toHaveBeenCalledTimes(1); + handle.stop(); + }); + + it("installs the platform source before start and keeps its size current", async () => { + const { scope, worker } = link(); + const { canvas, container, offscreen, resize } = page(); + let platform: IGamePlatformSource | undefined; + serveWorkerGame( + { + start: async () => { + platform = scope.__tnGamePlatform as IGamePlatformSource; + }, + }, + scope, + ); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + expect(platform?.renderer.createCanvas()).toBe(offscreen); + expect(platform?.renderer.readSize(offscreen as HTMLCanvasElement)).toEqual([320, 180]); + expect(scope.devicePixelRatio).toBe(2); + + const resized = vi.fn(); + platform?.viewport.observeResize(offscreen as HTMLCanvasElement, resized); + canvas.clientWidth = 640; + canvas.clientHeight = 360; + resize(); + await tick(); + expect(resized).toHaveBeenCalled(); + expect(platform?.viewport.readSize(offscreen as HTMLCanvasElement)).toEqual({ + aspect: 640 / 360, + height: 360, + width: 640, + }); + handle.stop(); + }); + + it("forwards keys to the input target and pointers to the canvas, in canvas coordinates", async () => { + const { scope, worker } = link(); + const { canvas, container, offscreen, view } = page(); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const platform = scope.__tnGamePlatform as IGamePlatformSource; + const keys: string[] = []; + platform.inputTarget?.addEventListener("keydown", (event) => + keys.push((event as KeyboardEvent).code), + ); + const pointers: number[][] = []; + offscreen.addEventListener("pointerdown", (event) => { + const pointer = event as PointerEvent; + pointers.push([pointer.clientX, pointer.clientY, pointer.button, pointer.pointerId]); + }); + + view.dispatchEvent(keyEvent("keydown", "KeyW")); + canvas.dispatchEvent( + Object.assign(new Event("pointerdown"), { + button: 0, + buttons: 1, + clientX: 110, + clientY: 70, + pointerId: 1, + pointerType: "mouse", + }), + ); + await tick(); + expect(keys).toEqual(["KeyW"]); + expect(pointers).toEqual([[100, 50, 0, 1]]); + handle.stop(); + }); + + it("resolves the game's relative URLs against the page, not the worker script", async () => { + // A worker's base URL is its own script (`/assets/worker-.js`), so `sky.jpg` and + // three's `new Request("grade.cube")` fetched the wrong path. + const { scope, worker } = link(); + const { container } = page(); + const fetched = scope.fetch as ReturnType; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + await (scope.fetch as typeof fetch)("sky.jpg"); + expect(String(fetched.mock.calls.at(-1)?.[0])).toBe("http://127.0.0.1:5173/game/sky.jpg"); + const request = new (scope.Request as typeof Request)("grade.cube"); + expect(request.url).toBe("http://127.0.0.1:5173/game/grade.cube"); + handle.stop(); + }); + + it("carries the HUD bridge both ways over its own port", async () => { + const { scope, worker } = link(); + const { container, view } = page(); + const gameReceive = vi.fn(); + const uiReceive = vi.fn(); + scope.__tnUiGameReceive = gameReceive; + view.__tnUiReceive = uiReceive; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + // The UI side connects as soon as React mounts, before the worker is ready: the host globals + // must exist from the first call. + (view.tnHost as { postMessage(frame: string): void }).postMessage('{"intent":"pause"}'); + await handle.ready; + (scope.__tnUiPost as (frame: string) => void)('{"state":{"score":1}}'); + expect((scope.__tnUiOverlayAttached as () => boolean)()).toBe(true); + await tick(20); + expect(gameReceive).toHaveBeenCalledWith('{"intent":"pause"}'); + expect(uiReceive).toHaveBeenCalledWith('{"state":{"score":1}}'); + handle.stop(); + }); + + it("copies the harness's page globals into the worker before start", async () => { + const { scope, worker } = link(); + const { container, view } = page(); + view.__THREENATIVE_PLAYTEST_CLOCK__ = "wall-clock"; + let clock: unknown; + serveWorkerGame( + { + start: async () => { + clock = scope.__THREENATIVE_PLAYTEST_CLOCK__; + }, + }, + scope, + ); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + expect(clock).toBe("wall-clock"); + handle.stop(); + }); + + it("mirrors the playtest bridge onto the page as async calls", async () => { + const { scope, worker } = link(); + const { container, view } = page(); + serveWorkerGame( + { + start: async () => { + // The playtest plugin installs its bridge while the game starts. + scope.__THREENATIVE_PLAYTEST_BRIDGE__ = { + describe: () => ({ name: "worker" }), + sample: async (request: unknown) => ({ echo: request }), + }; + }, + }, + scope, + ); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const bridge = view.__THREENATIVE_PLAYTEST_BRIDGE__ as { + describe(): Promise; + sample(request: unknown): Promise; + }; + await expect(bridge.describe()).resolves.toEqual({ name: "worker" }); + await expect(bridge.sample({ ticks: 3 })).resolves.toEqual({ echo: { ticks: 3 } }); + handle.stop(); + }); + + it("rejects ready with the worker's start failure", async () => { + const { scope, worker } = link(); + const { container } = page(); + serveWorkerGame( + { + start: async () => { + throw new Error("TN_NO_RENDERER: boom"); + }, + }, + scope, + ); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await expect(handle.ready).rejects.toThrow("TN_NO_RENDERER: boom"); + handle.stop(); + }); + + it("stop terminates the worker and forwards nothing more", async () => { + const { scope, toWorker, worker } = link(); + const { container, view } = page(); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + handle.stop(); + const sent = toWorker.length; + view.dispatchEvent(keyEvent("keydown", "KeyW")); + expect(worker.terminate).toHaveBeenCalledTimes(1); + expect(toWorker.length).toBe(sent); + }); +}); diff --git a/packages/core/capabilities.json b/packages/core/capabilities.json index 134640b9fa..d4ef6a0d5b 100644 --- a/packages/core/capabilities.json +++ b/packages/core/capabilities.json @@ -4368,6 +4368,34 @@ "supersedes": [], "symbol": "subscribeUiState" }, + { + "aliases": [], + "constraints": [], + "example": "serveWorkerGame(game);", + "importPath": "@threenative/core/worker", + "kind": "function", + "overrides": [], + "package": "@threenative/core", + "signature": "export function serveWorkerGame( game: { … }", + "situations": ["run the game loop in a web worker so a busy page cannot stall its frames"], + "summary": "Runs inside the worker: wire the page's messages to the unchanged portable game. Call it last in the worker entry, after the game module evaluated: it posts the ready message the page waits for before it sends the canvas.", + "supersedes": [], + "symbol": "serveWorkerGame" + }, + { + "aliases": [], + "constraints": [], + "example": "const handle = startWorkerGame(new Worker(new URL(\"./worker.ts\", import.meta.url), { type: \"module\" }), root);", + "importPath": "@threenative/core/worker", + "kind": "function", + "overrides": [], + "package": "@threenative/core", + "signature": "export function startWorkerGame( worker: Worker, container: HTMLElement, ): { … }", + "situations": ["host a worker-run game's canvas, input and HUD on the page"], + "summary": "Runs on the page: own the canvas and the DOM, and feed the worker game. Call it before the UI mounts: the HUD's bridge host globals exist from this call on.", + "supersedes": [], + "symbol": "startWorkerGame" + }, { "aliases": [], "constraints": [ diff --git a/packages/core/package.json b/packages/core/package.json index a8a2bde97c..d5ebfbac83 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -64,6 +64,10 @@ "types": "./dist/ui-layer.d.ts", "import": "./dist/ui-layer.js" }, + "./worker": { + "types": "./dist/worker.d.ts", + "import": "./dist/worker.js" + }, "./world": { "types": "./dist/world.d.ts", "import": "./dist/world.js" diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts new file mode 100644 index 0000000000..5e66d5e519 --- /dev/null +++ b/packages/core/src/worker.ts @@ -0,0 +1,450 @@ +/** + * The worker game host (PRD-752): the page keeps the DOM, the canvas and the input, while the + * unchanged portable game runs in a dedicated worker on an OffscreenCanvas. + * + * A worker cannot reach `window`, `document` or `ResizeObserver`, so the page forwards what the + * game would have read, one plain message at a time. The canvas frame and the UI bridge travel on + * their own ports: the game's start is slow, and a HUD that mounts first must not wait for it. + */ + +import type { IGamePlatformSource } from "./game.js"; +import { hostDocument, hostResizeObserver, hostWindow } from "./platform.js"; + +/** The realm `serveWorkerGame` runs in. Narrowed, because a test cannot build a real worker scope. */ +export type WorkerScope = EventTarget & + Record & { + postMessage(data: unknown): void; + }; + +interface IWorkerInitMessage { + type: "tn-worker-init"; + canvas: OffscreenCanvas; + width: number; + height: number; + dpr: number; + pageUrl: string; + globals: Record; + ui: MessagePort; +} + +interface IWorkerResizeMessage { + type: "tn-worker-resize"; + width: number; + height: number; + dpr: number; +} + +interface IWorkerEventMessage { + type: "tn-worker-event"; + target: "window" | "canvas"; + event: Record; +} + +interface IWorkerRpcMessage { + type: "tn-worker-rpc"; + id: number; + name: string; + method: string; + args: unknown[]; +} + +type WorkerInbound = + | IWorkerInitMessage + | IWorkerResizeMessage + | IWorkerEventMessage + | IWorkerRpcMessage; + +type WorkerOutbound = + | { type: "tn-worker-ready" } + | { type: "tn-worker-mirror"; name: string; methods: string[] } + | { type: "tn-worker-started" } + | { type: "tn-worker-failed"; message: string } + | { type: "tn-worker-rpc-result"; id: number; result?: unknown; error?: string }; + +type PageOutbound = WorkerInbound; + +interface IPendingCall { + resolve(value: unknown): void; + reject(error: Error): void; +} + +/** Debug surfaces the playtest harness drives from the page; they exist inside the worker only. */ +const MIRRORED_GLOBALS = ["__THREENATIVE_PLAYTEST_BRIDGE__", "__THREENATIVE__"] as const; + +/** The fields an input event carries across the boundary. `type` rides along as the event's name. */ +const EVENT_FIELDS = [ + "type", + "key", + "code", + "repeat", + "altKey", + "ctrlKey", + "metaKey", + "shiftKey", + "button", + "buttons", + "movementX", + "movementY", + "pointerId", + "pointerType", + "isPrimary", + "pressure", + "deltaX", + "deltaY", + "deltaZ", + "deltaMode", + "clientX", + "clientY", +] as const; + +const WINDOW_EVENTS = ["keydown", "keyup", "mousemove", "wheel", "blur"] as const; +const CANVAS_EVENTS = [ + "pointerdown", + "pointermove", + "pointerup", + "pointercancel", + "click", +] as const; + +/** + * Runs inside the worker: wire the page's messages to the unchanged portable game. + * + * Call it last in the worker entry, after the game module evaluated: it posts the ready message + * the page waits for before it sends the canvas. + * @situation run the game loop in a web worker so a busy page cannot stall its frames + * @example serveWorkerGame(game); + */ +export function serveWorkerGame( + game: { start(): Promise }, + scope: WorkerScope = globalThis as unknown as WorkerScope, +): void { + const inputTarget = new EventTarget(); + const observers = new Set<() => void>(); + const size = { dpr: 1, height: 1, width: 1 }; + let canvas: OffscreenCanvas | undefined; + + const post = (message: WorkerOutbound): void => scope.postMessage(message); + + const initialise = async (message: IWorkerInitMessage): Promise => { + try { + canvas = message.canvas; + size.width = message.width; + size.height = message.height; + size.dpr = message.dpr; + scope.devicePixelRatio = message.dpr; + for (const [key, value] of Object.entries(message.globals)) scope[key] = value; + + // A worker resolves relative URLs against its own script, so the page's URL is the base. + const pageUrl = message.pageUrl; + const resolve = (input: RequestInfo | URL): RequestInfo | URL => + typeof input === "string" || input instanceof URL ? new URL(input, pageUrl) : input; + const originalFetch = scope.fetch as typeof fetch; + scope.fetch = ((input: RequestInfo | URL, init?: RequestInit) => + originalFetch(resolve(input), init)) as typeof fetch; + const OriginalRequest = scope.Request as typeof Request; + scope.Request = class extends OriginalRequest { + constructor(input: RequestInfo | URL, init?: RequestInit) { + super(resolve(input), init); + } + }; + + const observe = (_target: HTMLCanvasElement, resize: () => void): (() => void) => { + observers.add(resize); + return () => { + observers.delete(resize); + }; + }; + const platform: IGamePlatformSource = { + input: () => [], + inputTarget, + mountCanvas: () => undefined, + renderer: { + createCanvas: () => message.canvas as unknown as HTMLCanvasElement, + hasWebGPU: () => "gpu" in navigator, + observeResize: observe, + readSize: () => [size.width, size.height], + }, + unmountCanvas: () => undefined, + viewport: { + observeResize: observe, + readSize: () => ({ + aspect: size.width / size.height, + height: size.height, + width: size.width, + }), + }, + }; + scope.__tnGamePlatform = platform; + + const ui = message.ui; + scope.__tnUiPost = (frame: string) => ui.postMessage(frame); + ui.onmessage = (event: MessageEvent) => + (scope.__tnUiGameReceive as ((frame: unknown) => void) | undefined)?.(event.data); + scope.__tnUiOverlayAttached = () => true; + + await game.start(); + + for (const name of MIRRORED_GLOBALS) { + const value = scope[name]; + if (typeof value !== "object" || value === null) continue; + const methods = Object.keys(value).filter( + (key) => typeof (value as Record)[key] === "function", + ); + post({ methods, name, type: "tn-worker-mirror" }); + } + post({ type: "tn-worker-started" }); + } catch (error) { + post({ + message: error instanceof Error ? error.message : String(error), + type: "tn-worker-failed", + }); + } + }; + + const dispatchForwarded = (message: IWorkerEventMessage): void => { + const forwarded = new Event(String(message.event.type), { cancelable: true }); + for (const [key, value] of Object.entries(message.event)) { + Object.defineProperty(forwarded, key, { enumerable: true, value }); + } + if (message.target === "window") inputTarget.dispatchEvent(forwarded); + else canvas?.dispatchEvent(forwarded); + }; + + const handleRpc = async (message: IWorkerRpcMessage): Promise => { + const id = message.id; + try { + const receiver = scope[message.name] as + | Record unknown> + | undefined; + const method = receiver?.[message.method]; + if (typeof method !== "function") { + throw new Error( + `TN_WORKER_RPC_UNKNOWN: '${message.name}.${message.method}' is not a function.`, + ); + } + const result = await method.apply(receiver, message.args); + post({ id, result, type: "tn-worker-rpc-result" }); + } catch (error) { + post({ + error: error instanceof Error ? error.message : String(error), + id, + type: "tn-worker-rpc-result", + }); + } + }; + + const onMessage = (event: Event): void => { + const message = (event as MessageEvent).data as WorkerInbound; + switch (message.type) { + case "tn-worker-init": + void initialise(message); + break; + case "tn-worker-resize": + size.width = message.width; + size.height = message.height; + size.dpr = message.dpr; + scope.devicePixelRatio = message.dpr; + for (const observer of [...observers]) observer(); + break; + case "tn-worker-event": + dispatchForwarded(message); + break; + case "tn-worker-rpc": + void handleRpc(message); + break; + } + }; + + scope.addEventListener("message", onMessage); + post({ type: "tn-worker-ready" }); +} + +/** + * Runs on the page: own the canvas and the DOM, and feed the worker game. + * + * Call it before the UI mounts: the HUD's bridge host globals exist from this call on. + * @situation host a worker-run game's canvas, input and HUD on the page + * @example const handle = startWorkerGame(new Worker(new URL("./worker.ts", import.meta.url), { type: "module" }), root); + */ +export function startWorkerGame( + worker: Worker, + container: HTMLElement, +): { ready: Promise; stop(): void } { + const documentHost = hostDocument(); + const windowHost = hostWindow(); + if (documentHost === undefined || windowHost === undefined) { + throw new Error("TN_WORKER_HOST_REQUIRED: startWorkerGame runs on a page."); + } + const page = windowHost as unknown as Record; + + const canvas = documentHost.createElement("canvas"); + canvas.style.cssText = "display:block;width:100%;height:100%;touch-action:none"; + container.append(canvas); + const offscreen = canvas.transferControlToOffscreen(); + const { port1: uiPort, port2: gamePort } = new MessageChannel(); + + let stopped = false; + let nextRpcId = 0; + const pending = new Map(); + const installed = new Set(); + const listeners: Array<[EventTarget, string, EventListener]> = []; + + const send = (message: PageOutbound, transfer?: Transferable[]): void => { + if (stopped) return; + if (transfer === undefined) worker.postMessage(message); + else worker.postMessage(message, transfer); + }; + + const postResize = (): void => { + send({ + dpr: windowHost.devicePixelRatio, + height: canvas.clientHeight, + type: "tn-worker-resize", + width: canvas.clientWidth, + }); + }; + + const listen = (target: EventTarget, type: string, listener: EventListener): void => { + target.addEventListener(type, listener); + listeners.push([target, type, listener]); + }; + + const eventFields = (event: Event): Record => { + const source = event as unknown as Record; + const fields: Record = {}; + for (const field of EVENT_FIELDS) { + if (field in event) fields[field] = source[field]; + } + return fields; + }; + + const forward = (event: Event, target: "window" | "canvas"): void => { + if (stopped) return; + const fields = eventFields(event); + // The worker draws on the canvas, so its pointer coordinates must be canvas-relative. + if (target === "canvas" || event.type === "wheel") { + const rect = canvas.getBoundingClientRect(); + const { clientX, clientY } = fields; + if (typeof clientX === "number") fields.clientX = clientX - rect.left; + if (typeof clientY === "number") fields.clientY = clientY - rect.top; + } + send({ event: fields, target, type: "tn-worker-event" }); + }; + + for (const type of WINDOW_EVENTS) listen(windowHost, type, (event) => forward(event, "window")); + for (const type of CANVAS_EVENTS) listen(canvas, type, (event) => forward(event, "canvas")); + listen(canvas, "contextmenu", (event) => event.preventDefault()); + + const ResizeObserverCtor = hostResizeObserver(); + const observer = + ResizeObserverCtor === undefined ? undefined : new ResizeObserverCtor(() => postResize()); + observer?.observe(canvas); + + const pageGlobals = (): Record => { + const globals: Record = {}; + for (const key of Object.keys(page)) { + if (!key.startsWith("__THREENATIVE_")) continue; + const value = page[key]; + if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") { + globals[key] = value; + } + } + return globals; + }; + + // The UI mounts before the game is ready, so its host globals exist from the first call. + page.tnHost = { postMessage: (frame: string) => uiPort.postMessage(frame) }; + page.__tnUiOverlayAttached = () => true; + installed.add("tnHost"); + installed.add("__tnUiOverlayAttached"); + uiPort.onmessage = (event: MessageEvent) => { + const receive = page.__tnUiReceive; + if (typeof receive === "function") (receive as (frame: unknown) => void)(event.data); + }; + + let resolveReady: () => void = () => undefined; + let rejectReady: (error: Error) => void = () => undefined; + const ready = new Promise((resolve, reject) => { + resolveReady = resolve; + rejectReady = reject; + }); + + const installMirror = (name: string, methods: string[]): void => { + const host: Record Promise> = {}; + for (const method of methods) { + host[method] = (...args: unknown[]) => { + if (stopped) { + return Promise.reject(new Error("TN_WORKER_STOPPED: the worker game stopped.")); + } + const id = nextRpcId++; + return new Promise((resolve, reject) => { + pending.set(id, { reject, resolve }); + send({ args, id, method, name, type: "tn-worker-rpc" }); + }); + }; + } + page[name] = host; + installed.add(name); + }; + + const onWorkerMessage = (event: Event): void => { + if (stopped) return; + const message = (event as MessageEvent).data as WorkerOutbound; + switch (message.type) { + case "tn-worker-ready": + send( + { + canvas: offscreen, + dpr: windowHost.devicePixelRatio ?? 1, + globals: pageGlobals(), + height: canvas.clientHeight, + pageUrl: windowHost.location.href, + type: "tn-worker-init", + ui: gamePort, + width: canvas.clientWidth, + }, + [offscreen, gamePort], + ); + break; + case "tn-worker-mirror": + installMirror(message.name, message.methods); + break; + case "tn-worker-started": + resolveReady(); + break; + case "tn-worker-failed": + rejectReady(new Error(message.message)); + break; + case "tn-worker-rpc-result": { + const call = pending.get(message.id); + if (call === undefined) break; + pending.delete(message.id); + if (message.error === undefined) call.resolve(message.result); + else call.reject(new Error(message.error)); + break; + } + } + }; + + const stop = (): void => { + if (stopped) return; + stopped = true; + for (const [target, type, listener] of listeners) target.removeEventListener(type, listener); + listeners.length = 0; + observer?.disconnect(); + worker.removeEventListener("message", onWorkerMessage); + for (const call of pending.values()) { + call.reject(new Error("TN_WORKER_STOPPED: the worker game stopped.")); + } + pending.clear(); + uiPort.onmessage = null; + uiPort.close(); + gamePort.close(); + for (const key of installed) delete page[key]; + installed.clear(); + worker.terminate(); + }; + + worker.addEventListener("message", onWorkerMessage); + return { ready, stop }; +} diff --git a/packages/core/tsup.config.ts b/packages/core/tsup.config.ts index 3428669edf..c983ae3066 100644 --- a/packages/core/tsup.config.ts +++ b/packages/core/tsup.config.ts @@ -8,6 +8,7 @@ export default defineConfig({ "src/react.ts", "src/react-css.ts", "src/ui-layer.ts", + "src/worker.ts", "src/world.ts", "src/net.ts", // The terrain jobs' worker entry, emitted as its own file so `new Worker(new URL(...))` in diff --git a/packages/create-threenative/agent-docs/references/capability-reference.md b/packages/create-threenative/agent-docs/references/capability-reference.md index 1ed6c36f89..5b0c597433 100644 --- a/packages/create-threenative/agent-docs/references/capability-reference.md +++ b/packages/create-threenative/agent-docs/references/capability-reference.md @@ -3402,6 +3402,36 @@ export function subscribeUiState(bridge: IUiBridge): IUiStateMirror { … const mirror = subscribeUiState(bridge); ``` +## `@threenative/core/worker` + +### `serveWorkerGame` + +`function` — Runs inside the worker: wire the page's messages to the unchanged portable game. Call it last in the worker entry, after the game module evaluated: it posts the ready message the page waits for before it sends the canvas. + +```ts +export function serveWorkerGame( game: { … } +``` + +- **Use when:** run the game loop in a web worker so a busy page cannot stall its frames + +```ts +serveWorkerGame(game); +``` + +### `startWorkerGame` + +`function` — Runs on the page: own the canvas and the DOM, and feed the worker game. Call it before the UI mounts: the HUD's bridge host globals exist from this call on. + +```ts +export function startWorkerGame( worker: Worker, container: HTMLElement, ): { … } +``` + +- **Use when:** host a worker-run game's canvas, input and HUD on the page + +```ts +const handle = startWorkerGame(new Worker(new URL("./worker.ts", import.meta.url), { type: "module" }), root); +``` + ## `@threenative/core/world` ### `cellPlacements` diff --git a/packages/create-threenative/capabilities.json b/packages/create-threenative/capabilities.json index 134640b9fa..d4ef6a0d5b 100644 --- a/packages/create-threenative/capabilities.json +++ b/packages/create-threenative/capabilities.json @@ -4368,6 +4368,34 @@ "supersedes": [], "symbol": "subscribeUiState" }, + { + "aliases": [], + "constraints": [], + "example": "serveWorkerGame(game);", + "importPath": "@threenative/core/worker", + "kind": "function", + "overrides": [], + "package": "@threenative/core", + "signature": "export function serveWorkerGame( game: { … }", + "situations": ["run the game loop in a web worker so a busy page cannot stall its frames"], + "summary": "Runs inside the worker: wire the page's messages to the unchanged portable game. Call it last in the worker entry, after the game module evaluated: it posts the ready message the page waits for before it sends the canvas.", + "supersedes": [], + "symbol": "serveWorkerGame" + }, + { + "aliases": [], + "constraints": [], + "example": "const handle = startWorkerGame(new Worker(new URL(\"./worker.ts\", import.meta.url), { type: \"module\" }), root);", + "importPath": "@threenative/core/worker", + "kind": "function", + "overrides": [], + "package": "@threenative/core", + "signature": "export function startWorkerGame( worker: Worker, container: HTMLElement, ): { … }", + "situations": ["host a worker-run game's canvas, input and HUD on the page"], + "summary": "Runs on the page: own the canvas and the DOM, and feed the worker game. Call it before the UI mounts: the HUD's bridge host globals exist from this call on.", + "supersedes": [], + "symbol": "startWorkerGame" + }, { "aliases": [], "constraints": [ diff --git a/scripts/api-surface.json b/scripts/api-surface.json index d7a02a81b4..f5b90a2128 100644 --- a/scripts/api-surface.json +++ b/scripts/api-surface.json @@ -42,6 +42,7 @@ "./react", "./react-css", "./ui-layer", + "./worker", "./world" ], "symbols": { @@ -62,6 +63,8 @@ "@threenative/core/ui-layer#publishUiState": "export function publishUiState( bridge: IUiBridge, store: IPublishableStore, options: IPublishOptions = { … }", "@threenative/core/ui-layer#sendUiIntent": "export function sendUiIntent(bridge: IUiBridge, intent: string, payload?: unknown): void { … }", "@threenative/core/ui-layer#subscribeUiState": "export function subscribeUiState(bridge: IUiBridge): IUiStateMirror { … }", + "@threenative/core/worker#serveWorkerGame": "export function serveWorkerGame( game: { … }", + "@threenative/core/worker#startWorkerGame": "export function startWorkerGame( worker: Worker, container: HTMLElement, ): { … }", "@threenative/core/world#cellPlacements": "export function cellPlacements(placements: ArrayBuffer, run: IWorldRun): Float32Array { … }", "@threenative/core/world#getWorldCapabilities": "export function getWorldCapabilities(options: IWorldCapabilitiesOptions = { … }", "@threenative/core/world#Heightfield": "export class Heightfield extends Group implements IComputeDriven { … }", From e8f494f07f07248474aabe3347adc89835c1f6c9 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 22:49:03 -0700 Subject: [PATCH 08/29] Worker game audio: the page's AudioContext plays a worker game's sound (PRD-752) A worker has no Web Audio API. The worker gets a stand-in AudioContext whose nodes are handles; calls and property writes cross a MessagePort in one batch per task and run on the page's real graph, so three's audio classes and AudioBus work unchanged. AudioBus finds its gesture target in a worker, canvas gestures reach it as they would by bubbling, and a page gesture resumes the page's context. 7 remote-audio tests, 2 worker-host tests, 1 platform test. --- packages/core/__tests__/platform.spec.ts | 21 +- packages/core/__tests__/worker-audio.spec.ts | 171 +++++++++ packages/core/__tests__/worker.spec.ts | 55 +++ packages/core/src/audio.ts | 4 +- packages/core/src/platform.ts | 9 + packages/core/src/worker-audio.ts | 356 +++++++++++++++++++ packages/core/src/worker.ts | 74 +++- 7 files changed, 673 insertions(+), 17 deletions(-) create mode 100644 packages/core/__tests__/worker-audio.spec.ts create mode 100644 packages/core/src/worker-audio.ts diff --git a/packages/core/__tests__/platform.spec.ts b/packages/core/__tests__/platform.spec.ts index e273454887..ef3dbe1fe7 100644 --- a/packages/core/__tests__/platform.spec.ts +++ b/packages/core/__tests__/platform.spec.ts @@ -1,7 +1,8 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import { detectPlatform, getPlatform, + hostGestureTarget, isMobile, isNative, isTouchscreenAvailable, @@ -133,3 +134,21 @@ describe("platform detection", () => { expect(isWeb()).not.toBe(isNative()); }); }); + +describe("host gesture target (PRD-752)", () => { + it("is the page window, else the input target a worker host installed", () => { + // AudioBus waits for a gesture before it plays. A worker has no window, so without the + // platform's input target every cue would stay queued. + const inputTarget = new EventTarget(); + const view = new EventTarget(); + vi.stubGlobal("__tnGamePlatform", { inputTarget }); + try { + vi.stubGlobal("window", undefined); + expect(hostGestureTarget()).toBe(inputTarget); + vi.stubGlobal("window", view); + expect(hostGestureTarget()).toBe(view); + } finally { + vi.unstubAllGlobals(); + } + }); +}); diff --git a/packages/core/__tests__/worker-audio.spec.ts b/packages/core/__tests__/worker-audio.spec.ts new file mode 100644 index 0000000000..0eed5a9504 --- /dev/null +++ b/packages/core/__tests__/worker-audio.spec.ts @@ -0,0 +1,171 @@ +import { describe, expect, it, vi } from "vitest"; +import { createRemoteAudioContext, serveRemoteAudio } from "../src/worker-audio.js"; + +// PRD-752: a worker has no AudioContext. The worker gets a stand-in whose nodes are handles; each +// call and property write crosses a port, and the page runs the real Web Audio graph. These tests +// put a recording fake where the page's AudioContext would be. + +const tick = (ms = 10) => new Promise((resolve) => setTimeout(resolve, ms)); + +function param(value = 1) { + return { + linearRampToValueAtTime: vi.fn(), + setTargetAtTime: vi.fn(), + setValueAtTime: vi.fn(), + value, + }; +} + +function fakePage() { + const destination = { name: "destination" }; + const node = () => ({ + connect: vi.fn((target: unknown) => target), + disconnect: vi.fn(), + }); + const created: Record[] = []; + const context = { + createBufferSource: vi.fn(() => { + const source = { + ...node(), + buffer: null as unknown, + detune: param(0), + loop: false, + onended: null as null | (() => void), + playbackRate: param(1), + start: vi.fn(), + stop: vi.fn(), + }; + created.push(source); + return source; + }), + createGain: vi.fn(() => { + const gain = { ...node(), gain: param(1) }; + created.push(gain); + return gain; + }), + currentTime: 4, + decodeAudioData: vi.fn(async (bytes: ArrayBuffer) => ({ + duration: bytes.byteLength / 1000, + length: bytes.byteLength, + numberOfChannels: 2, + sampleRate: 48_000, + })), + destination, + listener: { positionX: param(0) }, + resume: vi.fn(async () => undefined), + sampleRate: 48_000, + state: "suspended", + }; + return { context, created, destination }; +} + +function connect() { + const page = fakePage(); + const { port1, port2 } = new MessageChannel(); + const stop = serveRemoteAudio(port2, page.context as unknown as BaseAudioContext); + const remote = createRemoteAudioContext(port1, { sampleRate: 48_000, currentTime: 4 }); + return { + ...page, + remote, + stop: () => { + stop(); + port1.close(); + }, + }; +} + +describe("remote audio for a worker game (PRD-752)", () => { + it("runs a gain node's calls on the page's real node", async () => { + const { created, remote, stop } = connect(); + const gain = remote.createGain(); + gain.gain.setTargetAtTime(0.5, 0, 0.1); + await tick(); + const real = created[0] as { gain: ReturnType }; + expect(real.gain.setTargetAtTime).toHaveBeenCalledWith(0.5, 0, 0.1); + stop(); + }); + + it("connects remote nodes to the page's real nodes, the destination included", async () => { + const { created, destination, remote, stop } = connect(); + const source = remote.createBufferSource(); + const gain = remote.createGain(); + source.connect(gain); + gain.connect(remote.destination); + await tick(); + const [realSource, realGain] = created as { connect: ReturnType }[]; + expect(realSource?.connect).toHaveBeenCalledWith(realGain); + expect(realGain?.connect).toHaveBeenCalledWith(destination); + stop(); + }); + + it("decodes on the page and hands the worker a buffer it can play and measure", async () => { + // three's AudioLoader calls decodeAudioData(data, onSuccess) and also chains the promise. + const { created, remote, stop } = connect(); + const onSuccess = vi.fn(); + const buffer = await remote.decodeAudioData(new ArrayBuffer(2000), onSuccess); + expect(buffer.duration).toBe(2); + expect(buffer.numberOfChannels).toBe(2); + expect(onSuccess).toHaveBeenCalledWith(buffer); + const source = remote.createBufferSource(); + source.buffer = buffer; + source.loop = true; + source.start(0, 0.5); + await tick(); + const real = created[0] as { buffer: { duration: number }; loop: boolean; start: unknown }; + expect(real.buffer.duration).toBe(2); + expect(real.loop).toBe(true); + expect(real.start).toHaveBeenCalledWith(0, 0.5); + expect(source.loop).toBe(true); + stop(); + }); + + it("calls the worker's onended when the page's source ends", async () => { + const { created, remote, stop } = connect(); + const source = remote.createBufferSource(); + const ended = vi.fn(); + source.onended = ended; + await tick(); + (created[0] as { onended: () => void }).onended(); + await tick(); + expect(ended).toHaveBeenCalledTimes(1); + stop(); + }); + + it("moves the listener through the page's real listener", async () => { + const { context, remote, stop } = connect(); + remote.listener.positionX.linearRampToValueAtTime(3, 4.1); + await tick(); + expect(context.listener.positionX.linearRampToValueAtTime).toHaveBeenCalledWith(3, 4.1); + stop(); + }); + + it("sends one message per task, however many calls the frame made", async () => { + // AudioListener moves nine AudioParams every frame; nine messages per frame would cost more + // than the work they carry. + const page = fakePage(); + const { port1, port2 } = new MessageChannel(); + const sent = vi.spyOn(port1, "postMessage"); + const stop = serveRemoteAudio(port2, page.context as unknown as BaseAudioContext); + const remote = createRemoteAudioContext(port1, { sampleRate: 48_000, currentTime: 4 }); + for (let index = 0; index < 9; index += 1) + remote.listener.positionX.linearRampToValueAtTime(index, 4.1); + await tick(); + expect(sent).toHaveBeenCalledTimes(1); + expect(page.context.listener.positionX.linearRampToValueAtTime).toHaveBeenCalledTimes(9); + stop(); + port1.close(); + }); + + it("keeps its own clock from the page's start time and resumes the page's context", async () => { + const { context, remote, stop } = connect(); + const first = remote.currentTime; + expect(first).toBeGreaterThanOrEqual(4); + await tick(30); + expect(remote.currentTime).toBeGreaterThan(first); + expect(remote.sampleRate).toBe(48_000); + await remote.resume(); + await tick(); + expect(context.resume).toHaveBeenCalledTimes(1); + stop(); + }); +}); diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts index 9658cc6e5a..a6826e4307 100644 --- a/packages/core/__tests__/worker.spec.ts +++ b/packages/core/__tests__/worker.spec.ts @@ -292,4 +292,59 @@ describe("worker game host (PRD-752)", () => { expect(worker.terminate).toHaveBeenCalledTimes(1); expect(toWorker.length).toBe(sent); }); + + it("lets a canvas gesture reach the input target too, as it bubbles to the window on a page", async () => { + // AudioBus unlocks on a pointerdown it hears on its gesture target, the input target here. + const { scope, worker } = link(); + const { canvas, container } = page(); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const platform = scope.__tnGamePlatform as IGamePlatformSource; + const heard = vi.fn(); + platform.inputTarget?.addEventListener("pointerdown", heard); + canvas.dispatchEvent(Object.assign(new Event("pointerdown"), { clientX: 10, clientY: 20 })); + await tick(); + expect(heard).toHaveBeenCalledTimes(1); + handle.stop(); + }); + + it("gives the worker an AudioContext that plays through the page's own", async () => { + // The Wasm engine's AudioListener asks `globalThis.AudioContext`; a worker has none. + const { scope, worker } = link(); + const { container, view } = page(); + const created: unknown[] = []; + view.AudioContext = class { + currentTime = 0; + destination = {}; + listener = {}; + sampleRate = 44_100; + state = "suspended"; + createGain() { + const gain = { connect: vi.fn(), gain: { setTargetAtTime: vi.fn(), value: 1 } }; + created.push(gain); + return gain; + } + resume = vi.fn(async () => undefined); + }; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const Context = scope.AudioContext as new () => AudioContext; + const context = new Context(); + expect(context.sampleRate).toBe(44_100); + context.createGain().gain.setTargetAtTime(0.25, 0, 0.1); + await tick(20); + expect(created).toHaveLength(1); + expect( + (created[0] as { gain: { setTargetAtTime: ReturnType } }).gain.setTargetAtTime, + ).toHaveBeenCalledWith(0.25, 0, 0.1); + handle.stop(); + }); }); diff --git a/packages/core/src/audio.ts b/packages/core/src/audio.ts index 92d19eeea2..25d24c70c4 100644 --- a/packages/core/src/audio.ts +++ b/packages/core/src/audio.ts @@ -1,5 +1,5 @@ import { AudioListener, Object3D, PositionalAudio, Audio as ThreeAudio, type Vector3 } from "three"; -import { hostWindow } from "./platform.js"; +import { hostGestureTarget } from "./platform.js"; /** * Bus-level dynamics compression, every value the game's. @@ -256,7 +256,7 @@ export class AudioBus { this.#camera = options.camera; this.listener = options.listener ?? new AudioListener(); if (options.compressor !== undefined) this.#installCompressor(options.compressor); - const source = options.source ?? (() => hostWindow()); + const source = options.source ?? hostGestureTarget; this.#gestureTarget = options.gestureTarget ?? source(); this.setCamera(options.camera); this.#gesture = () => { diff --git a/packages/core/src/platform.ts b/packages/core/src/platform.ts index 1b0c2c209b..4468210845 100644 --- a/packages/core/src/platform.ts +++ b/packages/core/src/platform.ts @@ -214,6 +214,15 @@ export function hostWindow(): (Window & typeof globalThis) | undefined { return typeof window === "undefined" ? undefined : window; } +/** Where a user gesture lands: the page's window, or in a worker the input target its host installed. */ +export function hostGestureTarget(): EventTarget | undefined { + return ( + hostWindow() ?? + (globalThis as { __tnGamePlatform?: { inputTarget?: EventTarget } }).__tnGamePlatform + ?.inputTarget + ); +} + /** The page's `document`, or undefined in a worker or on a host without one. */ export function hostDocument(): Document | undefined { return typeof document === "undefined" ? undefined : document; diff --git a/packages/core/src/worker-audio.ts b/packages/core/src/worker-audio.ts new file mode 100644 index 0000000000..336a7fdd1d --- /dev/null +++ b/packages/core/src/worker-audio.ts @@ -0,0 +1,356 @@ +/** + * A stand-in `AudioContext` for a game running in a dedicated worker, where no Web Audio API + * exists (PRD-752). Every node is a handle; each call and property write crosses a `MessagePort` + * and runs on the page's real graph, so three's audio classes and `AudioBus` work unchanged. + */ + +interface IRemoteAudioInit { + readonly sampleRate: number; + readonly currentTime: number; +} + +interface IGetOp { + readonly op: "get"; + readonly id: number; + readonly key: string; + readonly out: number; +} + +interface ICallOp { + readonly op: "call"; + readonly id: number; + readonly method: string; + readonly args: readonly unknown[]; + readonly out: number; +} + +interface ISetOp { + readonly op: "set"; + readonly id: number; + readonly key: string; + readonly value: unknown; +} + +interface IDecodeOp { + readonly op: "decode"; + readonly req: number; + readonly out: number; + readonly bytes: ArrayBuffer; +} + +interface IFireOp { + readonly op: "fire"; + readonly fn: number; +} + +type PageOp = IGetOp | ICallOp | ISetOp | IDecodeOp; + +interface IDecodedReply { + readonly op: "decoded"; + readonly req: number; + readonly duration: number; + readonly length: number; + readonly numberOfChannels: number; + readonly sampleRate: number; +} + +interface IDecodeFailedReply { + readonly op: "decodeFailed"; + readonly req: number; + readonly message: string; +} + +interface IDecodeSlot { + readonly out: number; + readonly resolve: (buffer: unknown) => void; + readonly reject: (error: Error) => void; + readonly onSuccess: ((buffer: unknown) => void) | undefined; + readonly onError: ((error: Error) => void) | undefined; +} + +/** + * Keys whose page value is a node or an `AudioParam`. Reading one yields a child handle instead of + * a call, because `gain.gain` and `source.buffer` are objects to drive, not functions to run. + */ +const OBJECT_KEYS: ReadonlySet = new Set([ + "destination", + "listener", + "gain", + "playbackRate", + "detune", + "positionX", + "positionY", + "positionZ", + "orientationX", + "orientationY", + "orientationZ", + "forwardX", + "forwardY", + "forwardZ", + "upX", + "upY", + "upZ", + "threshold", + "knee", + "ratio", + "attack", + "release", + "frequency", + "Q", +]); + +/** Worker side: a page-side `AudioContext`, with every node replaced by a proxy handle. */ +export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioInit): AudioContext { + const createdAt = performance.now(); + const functions = new Map unknown>(); + const handleIds = new WeakMap(); + const slots = new Map(); + let nextId = 1; + let nextFnId = 1; + let running = false; + + // One message per task: AudioListener alone moves nine AudioParams a frame. + let queue: PageOp[] = []; + let transfers: Transferable[] = []; + const flush = (): void => { + port.postMessage({ op: "batch", ops: queue }, transfers); + queue = []; + transfers = []; + }; + const post = (message: PageOp, transfer?: Transferable[]): void => { + if (queue.length === 0) queueMicrotask(flush); + queue.push(message); + if (transfer !== undefined) transfers.push(...transfer); + }; + + function encode(value: unknown): unknown { + if (typeof value === "function") { + const fnId = nextFnId++; + functions.set(fnId, value as (...args: unknown[]) => unknown); + return { $fn: fnId }; + } + if (Array.isArray(value)) return value.map(encode); + if (value !== null && typeof value === "object") { + const id = handleIds.get(value); + if (id !== undefined) return { $h: id }; + } + return value; + } + + function readKey(id: number, cache: Map, key: string): unknown { + const override = id === 0 ? contextValue(key) : undefined; + if (override !== undefined) return override.value; + if (cache.has(key)) return cache.get(key); + if (OBJECT_KEYS.has(key)) { + const out = nextId++; + const child = makeHandle(out); + cache.set(key, child); + post({ key, op: "get", id, out }); + return child; + } + // ponytail: a property the worker never wrote reads as a function, not the page's value. + return (...args: unknown[]) => { + const out = nextId++; + post({ args: args.map(encode), method: key, op: "call", id, out }); + return makeHandle(out); + }; + } + + function makeHandle(id: number, seed?: Record): Record { + const cache = new Map(Object.entries(seed ?? {})); + const proxy = new Proxy({} as Record, { + get: (_target, key) => { + if (typeof key === "symbol" || key === "then") return undefined; + return readKey(id, cache, key); + }, + has: (_target, key) => typeof key === "string" && cache.has(key), + set: (_target, key, value) => { + if (typeof key === "symbol") return true; + cache.set(key, value); + post({ key, op: "set", id, value: encode(value) }); + return true; + }, + }); + handleIds.set(proxy, id); + return proxy; + } + + function control(method: string): Promise { + post({ args: [], method, op: "call", id: 0, out: nextId++ }); + return Promise.resolve(); + } + + function decodeAudioData( + data: ArrayBuffer, + onSuccess?: (buffer: unknown) => void, + onError?: (error: Error) => void, + ): Promise { + const req = nextId++; + const out = nextId++; + return new Promise((resolve, reject) => { + slots.set(req, { onError, onSuccess, out, reject, resolve }); + const bytes = data.slice(0); + post({ bytes, op: "decode", out, req }, [bytes]); + }); + } + + function contextValue(key: string): { readonly value: unknown } | undefined { + switch (key) { + case "currentTime": + return { value: init.currentTime + (performance.now() - createdAt) / 1000 }; + case "sampleRate": + return { value: init.sampleRate }; + case "state": + return { value: running ? "running" : "suspended" }; + case "resume": + return { + value: () => { + running = true; + return control("resume"); + }, + }; + case "suspend": + return { + value: () => { + running = false; + return control("suspend"); + }, + }; + case "close": + return { + value: () => { + running = false; + return control("close"); + }, + }; + case "decodeAudioData": + return { value: decodeAudioData }; + default: + return undefined; + } + } + + port.onmessage = (event: MessageEvent) => { + const reply = event.data as IDecodedReply | IDecodeFailedReply | IFireOp | undefined; + if (reply?.op === "decoded") { + const slot = slots.get(reply.req); + if (slot === undefined) return; + slots.delete(reply.req); + const buffer = makeHandle(slot.out, { + duration: reply.duration, + length: reply.length, + numberOfChannels: reply.numberOfChannels, + sampleRate: reply.sampleRate, + }); + slot.resolve(buffer); + slot.onSuccess?.(buffer); + return; + } + if (reply?.op === "decodeFailed") { + const slot = slots.get(reply.req); + if (slot === undefined) return; + slots.delete(reply.req); + const error = new Error(reply.message); + slot.onError?.(error); + slot.reject(error); + return; + } + if (reply !== undefined && "fn" in reply) { + const fired = reply as IFireOp; + functions.get(fired.fn)?.(); + } + }; + + return makeHandle(0) as unknown as AudioContext; +} + +/** Page side: runs the real graph for a worker's handles until the returned stop is called. */ +export function serveRemoteAudio(port: MessagePort, context: BaseAudioContext): () => void { + const objects = new Map([[0, context]]); + const warned = new Set(); + let stopped = false; + + function decodeValue(value: unknown): unknown { + if (Array.isArray(value)) return value.map(decodeValue); + if (value !== null && typeof value === "object") { + const handle = (value as { $h?: number }).$h; + if (typeof handle === "number") return objects.get(handle); + const fn = (value as { $fn?: number }).$fn; + if (typeof fn === "number") return () => port.postMessage({ fn, op: "fire" }); + } + return value; + } + + function warn(op: string | undefined, error: unknown): void { + const name = op ?? "unknown"; + if (warned.has(name)) return; + warned.add(name); + console.warn(`serveRemoteAudio: "${name}" failed; later failures stay quiet.`, error); + } + + async function serveDecode(op: IDecodeOp): Promise { + try { + const buffer = await context.decodeAudioData(op.bytes); + if (stopped) return; + objects.set(op.out, buffer); + port.postMessage({ + duration: buffer.duration, + length: buffer.length, + numberOfChannels: buffer.numberOfChannels, + op: "decoded", + req: op.req, + sampleRate: buffer.sampleRate, + }); + } catch (error) { + if (stopped) return; + port.postMessage({ + message: error instanceof Error ? error.message : String(error), + op: "decodeFailed", + req: op.req, + }); + } + } + + port.onmessage = (event: MessageEvent) => { + if (stopped) return; + for (const op of (event.data as { ops: PageOp[] }).ops) run(op); + }; + + function run(op: PageOp): void { + try { + switch (op?.op) { + case "get": { + const source = objects.get(op.id) as Record | undefined; + objects.set(op.out, source?.[op.key]); + break; + } + case "call": { + const target = objects.get(op.id) as Record; + const method = target[op.method] as (...args: unknown[]) => unknown; + const result = method.apply(target, op.args.map(decodeValue)); + if (result !== null && typeof result === "object") objects.set(op.out, result); + break; + } + case "set": { + const target = objects.get(op.id) as Record; + target[op.key] = decodeValue(op.value); + break; + } + case "decode": { + void serveDecode(op); + break; + } + default: + break; + } + } catch (error) { + warn(op.op, error); + } + } + + return () => { + stopped = true; + port.onmessage = null; + port.close(); + }; +} diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts index 5e66d5e519..5bc942e3b7 100644 --- a/packages/core/src/worker.ts +++ b/packages/core/src/worker.ts @@ -9,6 +9,7 @@ import type { IGamePlatformSource } from "./game.js"; import { hostDocument, hostResizeObserver, hostWindow } from "./platform.js"; +import { createRemoteAudioContext, serveRemoteAudio } from "./worker-audio.js"; /** The realm `serveWorkerGame` runs in. Narrowed, because a test cannot build a real worker scope. */ export type WorkerScope = EventTarget & @@ -25,6 +26,8 @@ interface IWorkerInitMessage { pageUrl: string; globals: Record; ui: MessagePort; + /** The page's AudioContext, served over its own port; absent where the page has none. */ + audio?: { port: MessagePort; sampleRate: number; currentTime: number }; } interface IWorkerResizeMessage { @@ -176,6 +179,16 @@ export function serveWorkerGame( }; scope.__tnGamePlatform = platform; + const audio = message.audio; + if (audio !== undefined) { + // Created on first use, so a silent game opens no page AudioContext traffic. + let remote: AudioContext | undefined; + scope.AudioContext = function RemoteAudioContext() { + remote ??= createRemoteAudioContext(audio.port, audio); + return remote; + }; + } + const ui = message.ui; scope.__tnUiPost = (frame: string) => ui.postMessage(frame); ui.onmessage = (event: MessageEvent) => @@ -207,7 +220,14 @@ export function serveWorkerGame( Object.defineProperty(forwarded, key, { enumerable: true, value }); } if (message.target === "window") inputTarget.dispatchEvent(forwarded); - else canvas?.dispatchEvent(forwarded); + else { + canvas?.dispatchEvent(forwarded); + // On a page a canvas event bubbles to the window, where AudioBus waits for its gesture. + const bubbled = new Event(forwarded.type); + for (const [key, value] of Object.entries(message.event)) + Object.defineProperty(bubbled, key, { enumerable: true, value }); + inputTarget.dispatchEvent(bubbled); + } }; const handleRpc = async (message: IWorkerRpcMessage): Promise => { @@ -335,6 +355,19 @@ export function startWorkerGame( for (const type of CANVAS_EVENTS) listen(canvas, type, (event) => forward(event, "canvas")); listen(canvas, "contextmenu", (event) => event.preventDefault()); + // The worker's game plays through the page's own AudioContext, resumed by a page gesture. + const AudioContextCtor = page.AudioContext as (new () => AudioContext) | undefined; + const audioContext = AudioContextCtor === undefined ? undefined : new AudioContextCtor(); + const audioChannel = audioContext === undefined ? undefined : new MessageChannel(); + const stopAudio = + audioContext === undefined || audioChannel === undefined + ? undefined + : serveRemoteAudio(audioChannel.port1, audioContext); + if (audioContext !== undefined) { + const resume = (): void => void audioContext.resume?.().catch(() => undefined); + for (const type of ["keydown", "pointerdown", "touchstart"]) listen(windowHost, type, resume); + } + const ResizeObserverCtor = hostResizeObserver(); const observer = ResizeObserverCtor === undefined ? undefined : new ResizeObserverCtor(() => postResize()); @@ -387,24 +420,35 @@ export function startWorkerGame( installed.add(name); }; + const sendInit = (): void => { + const init: IWorkerInitMessage = { + canvas: offscreen, + dpr: windowHost.devicePixelRatio ?? 1, + globals: pageGlobals(), + height: canvas.clientHeight, + pageUrl: windowHost.location.href, + type: "tn-worker-init", + ui: gamePort, + width: canvas.clientWidth, + }; + const transfer: Transferable[] = [offscreen, gamePort]; + if (audioContext !== undefined && audioChannel !== undefined) { + init.audio = { + currentTime: audioContext.currentTime, + port: audioChannel.port2, + sampleRate: audioContext.sampleRate, + }; + transfer.push(audioChannel.port2); + } + send(init, transfer); + }; + const onWorkerMessage = (event: Event): void => { if (stopped) return; const message = (event as MessageEvent).data as WorkerOutbound; switch (message.type) { case "tn-worker-ready": - send( - { - canvas: offscreen, - dpr: windowHost.devicePixelRatio ?? 1, - globals: pageGlobals(), - height: canvas.clientHeight, - pageUrl: windowHost.location.href, - type: "tn-worker-init", - ui: gamePort, - width: canvas.clientWidth, - }, - [offscreen, gamePort], - ); + sendInit(); break; case "tn-worker-mirror": installMirror(message.name, message.methods); @@ -440,6 +484,8 @@ export function startWorkerGame( uiPort.onmessage = null; uiPort.close(); gamePort.close(); + stopAudio?.(); + void audioContext?.close?.().catch(() => undefined); for (const key of installed) delete page[key]; installed.clear(); worker.terminate(); From 94a0f7a493572f09f53df2ba7399fe2240d30ed6 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:00:31 -0700 Subject: [PATCH 09/29] Worker game HUD: the UI bridge works when the game connects while its module loads (PRD-752) The starter's game.ts calls game.ui.onIntent at top level, so in a worker the game end connected before the page's port arrived and stayed on the in-process broker: a blank HUD. @threenative/core/worker now installs the game end's host globals when it loads in a worker, before the game module evaluates, and holds frames each way until init attaches the port. 3 tests; headed GPU probe shows the full HUD on the worker page. --- packages/core/__tests__/worker.spec.ts | 60 ++++++++++++++++++++++++++ packages/core/src/worker.ts | 53 ++++++++++++++++++++--- 2 files changed, 108 insertions(+), 5 deletions(-) diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts index a6826e4307..e6854e7e41 100644 --- a/packages/core/__tests__/worker.spec.ts +++ b/packages/core/__tests__/worker.spec.ts @@ -207,6 +207,66 @@ describe("worker game host (PRD-752)", () => { handle.stop(); }); + it("holds a UI frame sent before the game connects, then delivers it", async () => { + // UiLayer announces itself at mount; the worker game connects its bridge partway through a + // slow start. A dropped ready intent leaves the HUD blank. + const { scope, worker } = link(); + const { container, view } = page(); + const gameReceive = vi.fn(); + const game = { + start: async () => { + await tick(20); + scope.__tnUiGameReceive = gameReceive; + }, + }; + serveWorkerGame(game, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + (view.tnHost as { postMessage(frame: string): void }).postMessage('{"intent":"ready"}'); + await handle.ready; + await tick(); + expect(gameReceive).toHaveBeenCalledWith('{"intent":"ready"}'); + handle.stop(); + }); + + it("delivers game frames posted before the page's port arrived", async () => { + // The starter's `game.ui.onIntent` connects the bridge while game.ts evaluates, before init. + const { scope, worker } = link(); + const { container, view } = page(); + const uiReceive = vi.fn(); + view.__tnUiReceive = uiReceive; + serveWorkerGame({ start: async () => undefined }, scope); + (scope.__tnUiPost as (frame: string) => void)('{"state":{"lives":3}}'); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + await tick(); + expect(uiReceive).toHaveBeenCalledWith('{"state":{"lives":3}}'); + handle.stop(); + }); + + it("installs the UI host globals when it loads in a worker, before the game module runs", async () => { + vi.resetModules(); + vi.stubGlobal("WorkerGlobalScope", class {}); + try { + await import("../src/worker.js"); + const scope = globalThis as Record; + expect(typeof scope.__tnUiPost).toBe("function"); + expect((scope.__tnUiOverlayAttached as () => boolean)()).toBe(true); + } finally { + vi.unstubAllGlobals(); + const scope = globalThis as Record; + scope.__tnUiPost = undefined; + scope.__tnUiOverlayAttached = undefined; + scope.__tnUiGameReceive = undefined; + scope.__tnWorkerUi = undefined; + } + }); + it("copies the harness's page globals into the worker before start", async () => { const { scope, worker } = link(); const { container, view } = page(); diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts index 5bc942e3b7..b753284fde 100644 --- a/packages/core/src/worker.ts +++ b/packages/core/src/worker.ts @@ -109,6 +109,52 @@ const CANVAS_EVENTS = [ "click", ] as const; +/** + * The game end of the UI bridge, live before the page's port arrives. + * + * A game module may connect its bridge while it evaluates (`game.ui.onIntent` at top level), and + * the bridge picks its transport once, at connect. So the host globals exist from the moment this + * module loads, frames each way wait in order, and `attach` joins them to the page's port. + */ +function holdUiBridge(scope: Record): { attach(port: MessagePort): void } { + const existing = scope.__tnWorkerUi as { attach(port: MessagePort): void } | undefined; + if (existing !== undefined) return existing; + let port: MessagePort | undefined; + const outbound: unknown[] = []; + const inbound: unknown[] = []; + let receive = scope.__tnUiGameReceive as ((frame: unknown) => void) | undefined; + Object.defineProperty(scope, "__tnUiGameReceive", { + configurable: true, + get: () => receive, + set: (value: ((frame: unknown) => void) | undefined) => { + receive = value; + if (typeof value === "function") for (const frame of inbound.splice(0)) value(frame); + }, + }); + scope.__tnUiPost = (frame: string) => { + if (port === undefined) outbound.push(frame); + else port.postMessage(frame); + }; + scope.__tnUiOverlayAttached = () => true; + const held = { + attach(next: MessagePort): void { + port = next; + next.onmessage = (event: MessageEvent) => { + if (typeof receive === "function") receive(event.data); + else inbound.push(event.data); + }; + for (const frame of outbound.splice(0)) next.postMessage(frame); + }, + }; + scope.__tnWorkerUi = held; + return held; +} + +// In a dedicated worker, before the game module that imports this one after it evaluates. +if (hostWindow() === undefined && "WorkerGlobalScope" in globalThis) { + holdUiBridge(globalThis as unknown as Record); +} + /** * Runs inside the worker: wire the page's messages to the unchanged portable game. * @@ -127,6 +173,7 @@ export function serveWorkerGame( let canvas: OffscreenCanvas | undefined; const post = (message: WorkerOutbound): void => scope.postMessage(message); + const ui = holdUiBridge(scope); const initialise = async (message: IWorkerInitMessage): Promise => { try { @@ -189,11 +236,7 @@ export function serveWorkerGame( }; } - const ui = message.ui; - scope.__tnUiPost = (frame: string) => ui.postMessage(frame); - ui.onmessage = (event: MessageEvent) => - (scope.__tnUiGameReceive as ((frame: unknown) => void) | undefined)?.(event.data); - scope.__tnUiOverlayAttached = () => true; + ui.attach(message.ui); await game.start(); From e575927133f37335d4452f76553a4b55b8535630 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:01:57 -0700 Subject: [PATCH 10/29] Starter: worker.html runs the same game in a web worker (PRD-752) src/worker.ts serves the unchanged game.ts; src/worker-page.ts keeps the canvas, input and the same GameUi on the page. index.html stays the default, so one build carries both arms of the busy-page A/B. Headed GPU probe: same frame and HUD as index.html. --- .../__tests__/scaffold.spec.ts | 7 ++++- .../templates/starter/AGENTS.md | 3 +- .../templates/starter/CLAUDE.md | 3 +- .../templates/starter/src/worker-page.ts | 29 +++++++++++++++++++ .../templates/starter/src/worker.ts | 5 ++++ .../templates/starter/vite.config.ts | 2 ++ .../templates/starter/worker.html | 12 ++++++++ 7 files changed, 58 insertions(+), 3 deletions(-) create mode 100644 packages/create-threenative/templates/starter/src/worker-page.ts create mode 100644 packages/create-threenative/templates/starter/src/worker.ts create mode 100644 packages/create-threenative/templates/starter/worker.html diff --git a/packages/create-threenative/__tests__/scaffold.spec.ts b/packages/create-threenative/__tests__/scaffold.spec.ts index 0757ba674b..a0a40ac68e 100644 --- a/packages/create-threenative/__tests__/scaffold.spec.ts +++ b/packages/create-threenative/__tests__/scaffold.spec.ts @@ -329,7 +329,8 @@ const PRD_201_PARENT_SCAFFOLD_HASHES: Readonly> = { // to stay under the 200-line render-file cap; only starter's tree moves. // Recomputed 2026-10-23 on the merge of origin/develop into the PRD-570 lane: measured by this test on the merged tree. // Recomputed 2026-10-23: the native render-chain scenario expects the Unreal finishing chain (grade and grain off). - starter: "603cce83c0b7da17f1ed2eba72e317f940840327e95081555191c22849041a73", + // Recomputed 2026-10-26 (PRD-752): worker.html, src/worker.ts, src/worker-page.ts and an AGENTS.md line run the game in a worker. + starter: "84da9469c321538b7b2518b92e778e004cfdad2a772bb64e0222df5a2b9ad896", "tower-defense": "c2874bdeedfb61095c23374f4e9716aacedfc801e0966648b067bbf267b1cb11", }; @@ -428,6 +429,10 @@ const STARTER_PATHS = [ "vite.config.ts", "src/game.ts", "src/main.ts", + // PRD-752: the same game in a worker, at `worker.html`. + "worker.html", + "src/worker.ts", + "src/worker-page.ts", "src/scenes/Play.ts", "src/render/lighting.ts", "src/render/postprocessing.ts", diff --git a/packages/create-threenative/templates/starter/AGENTS.md b/packages/create-threenative/templates/starter/AGENTS.md index e45adf9347..e953b3d487 100644 --- a/packages/create-threenative/templates/starter/AGENTS.md +++ b/packages/create-threenative/templates/starter/AGENTS.md @@ -45,7 +45,8 @@ a flag, which together produce `won` or `lost`; `R` and the React restart intent `initialState`. Keep the packaged `assets/native-proof.glb` load and its console marker for the desktop asset gate. `Play.ts` owns gameplay, `src/render/` owns the look, `src/ui/Hud.tsx`/`Menu.tsx` own UI, and `state.ts` publishes JSON-safe values. Keep `playtests/survives.playtest.json` as smoke -proof and update outcome tests. +proof and update outcome tests. `/worker.html` runs the same `game.ts` in a web worker +(`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. On a touch-primary device (`isMobile() && isTouchscreenAvailable()`), local `src/render/touch-controls.ts` adds a left movement stick and right jump button; the scene passes its input to `Player`, with keyboard mapping as the desktop fallback. ## Portable authoring contracts diff --git a/packages/create-threenative/templates/starter/CLAUDE.md b/packages/create-threenative/templates/starter/CLAUDE.md index 3e3aabecdb..2b635315c3 100644 --- a/packages/create-threenative/templates/starter/CLAUDE.md +++ b/packages/create-threenative/templates/starter/CLAUDE.md @@ -47,7 +47,8 @@ a flag, which together produce `won` or `lost`; `R` and the React restart intent `initialState`. Keep the packaged `assets/native-proof.glb` load and its console marker for the desktop asset gate. `Play.ts` owns gameplay, `src/render/` owns the look, `src/ui/Hud.tsx`/`Menu.tsx` own UI, and `state.ts` publishes JSON-safe values. Keep `playtests/survives.playtest.json` as smoke -proof and update outcome tests. +proof and update outcome tests. `/worker.html` runs the same `game.ts` in a web worker +(`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. On a touch-primary device (`isMobile() && isTouchscreenAvailable()`), local `src/render/touch-controls.ts` adds a left movement stick and right jump button; the scene passes its input to `Player`, with keyboard mapping as the desktop fallback. ## Portable authoring contracts diff --git a/packages/create-threenative/templates/starter/src/worker-page.ts b/packages/create-threenative/templates/starter/src/worker-page.ts new file mode 100644 index 0000000000..fd1f295978 --- /dev/null +++ b/packages/create-threenative/templates/starter/src/worker-page.ts @@ -0,0 +1,29 @@ +import { startWorkerGame } from "@threenative/core/worker"; +import { createElement } from "react"; +import { createRoot } from "react-dom/client"; +import { GameUi } from "./ui/GameUi.js"; +import "./style.css"; + +/** + * `worker.html`: the game runs in `src/worker.ts`, off the page's thread, so a busy page cannot + * stall its frames. This page keeps the canvas, the input and the HUD; `index.html` runs the same + * game on the page itself. + */ +const root = document.getElementById("root"); +if (root === null) throw new Error("Missing #root element."); +root.className = "relative h-screen w-screen overflow-hidden bg-ink"; +const stage = document.createElement("div"); +stage.className = "absolute inset-0"; +root.append(stage); +const ui = document.createElement("div"); +root.append(ui); +// Before the UI mounts: the HUD finds the worker's bridge from its first render. +const game = startWorkerGame( + new Worker(new URL("./worker.ts", import.meta.url), { type: "module" }), + stage, +); +createRoot(ui).render(createElement(GameUi)); +game.ready.catch((error: unknown) => { + stage.textContent = error instanceof Error ? error.message : String(error); + stage.setAttribute("role", "alert"); +}); diff --git a/packages/create-threenative/templates/starter/src/worker.ts b/packages/create-threenative/templates/starter/src/worker.ts new file mode 100644 index 0000000000..e6d9f57a6f --- /dev/null +++ b/packages/create-threenative/templates/starter/src/worker.ts @@ -0,0 +1,5 @@ +import { serveWorkerGame } from "@threenative/core/worker"; +import game from "./game.js"; + +// The worker side of `worker.html`: the same portable game, on its own thread. +serveWorkerGame(game); diff --git a/packages/create-threenative/templates/starter/vite.config.ts b/packages/create-threenative/templates/starter/vite.config.ts index b0ddb9d9fc..a6ce2affa0 100644 --- a/packages/create-threenative/templates/starter/vite.config.ts +++ b/packages/create-threenative/templates/starter/vite.config.ts @@ -40,6 +40,8 @@ export default defineConfig({ tailwindcss(), assetsWatchPlugin(), ], + // `worker.html` runs the same game in a worker; `index.html` stays the default page. + build: { rollupOptions: { input: { index: "index.html", worker: "worker.html" } } }, server: { watch: { ignored: ["**/artifacts/**", "**/screenshots/**", "**/playtests/**"], diff --git a/packages/create-threenative/templates/starter/worker.html b/packages/create-threenative/templates/starter/worker.html new file mode 100644 index 0000000000..4cfc3edbf5 --- /dev/null +++ b/packages/create-threenative/templates/starter/worker.html @@ -0,0 +1,12 @@ + + + + + + __PROJECT_NAME__ + + +
+ + + From 150a391968346cbcea938574b5b13d52a43b8a1c Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:04:05 -0700 Subject: [PATCH 11/29] profile:wasm-page times frames on the game's own thread (PRD-752) A worker page's requestAnimationFrame times the page, not the game. The probe now reads frame intervals in the worker that holds the game's platform source, else on the page, and names that thread in the result (gameThread). Main-thread busy stays the page's CPU profile. 1 test. --- scripts/__tests__/profile-wasm-page.spec.ts | 18 +++++++++++++++++ scripts/profile-wasm-page.ts | 22 +++++++++++++++++---- 2 files changed, 36 insertions(+), 4 deletions(-) diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 6071866624..93affde462 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -6,6 +6,7 @@ const state = vi.hoisted(() => ({ presented: [0, 0] as number[], thread: [1, 1.002] as number[], clicks: [] as string[], + workers: [] as { evaluate(fn: (() => unknown) | string): Promise }[], })); vi.mock("node:os", () => ({ default: { availableParallelism: () => 8, loadavg: () => [state.load, 0, 0] }, @@ -21,6 +22,7 @@ vi.mock("@playwright/test", () => ({ click: async (selector: string) => { state.clicks.push(selector); }, + workers: () => state.workers, context: () => ({ newCDPSession: async () => ({ send: async (method: string) => @@ -74,6 +76,7 @@ afterEach(() => { state.presented = [0, 0]; state.thread = [1, 1.002]; state.clicks = []; + state.workers = []; vi.unstubAllGlobals(); }); describe("profile:wasm-page scorecard", () => { @@ -96,6 +99,21 @@ describe("profile:wasm-page scorecard", () => { (await measure("http://test", options(["--url", "http://test"]))).memory, ).toBeUndefined(); }); + it("reads frame times on the game's own thread: the page, or the worker that runs it (PRD-752)", async () => { + const page = await measure("http://test", options(["--url", "http://test"])); + expect(page.gameThread).toBe("page"); + expect(page.frame.p50).toBe(10); + // An idle helper worker, then the worker holding the game's platform source. + const idle = { evaluate: async () => "undefined" }; + const game = { + evaluate: async (fn: (() => unknown) | string) => + typeof fn === "string" ? "object" : Array.from({ length: 600 }, () => 20), + }; + state.workers = [idle, game]; + const worker = await measure("http://test", options(["--url", "http://test"])); + expect(worker.gameThread).toBe("worker"); + expect(worker.frame.p50).toBe(20); + }); it.each(["wasmMemoryBytes", "jsHeapUsedBytes"])( "fails closed when %s is unavailable", async (field) => { diff --git a/scripts/profile-wasm-page.ts b/scripts/profile-wasm-page.ts index efa09a4b2e..c0c3c63789 100644 --- a/scripts/profile-wasm-page.ts +++ b/scripts/profile-wasm-page.ts @@ -165,9 +165,21 @@ const SOFTWARE = /swiftshader|llvmpipe|lavapipe|softwarerasterizer|software adap const FRAME_SAMPLES = 601; -/** Presented-frame intervals in ms, one per requestAnimationFrame, for frameStats. */ -async function frames(page: Page) { - const intervals = await page.evaluate( +/** + * The thread the game draws on: the worker holding the platform source a worker host installs + * (PRD-752), else the page. A worker page's own requestAnimationFrame times the page, not the game. + */ +async function gameThread(page: Page) { + for (const worker of page.workers()) { + if ((await worker.evaluate("typeof globalThis.__tnGamePlatform")) === "object") + return { kind: "worker" as const, realm: worker }; + } + return { kind: "page" as const, realm: page }; +} + +/** Presented-frame intervals in ms, one per requestAnimationFrame on the game's thread, for frameStats. */ +async function frames(realm: Pick) { + const intervals = await realm.evaluate( (count) => new Promise((resolve) => { const times: number[] = []; @@ -410,7 +422,8 @@ export async function measure(url: string, o: IOptions) { await page.waitForTimeout(o.warmupMs); const reading = readHostLoad(); const hostLoad = { ...reading, absolutes: absolutesVerdict(reading) }; - const frame = await frames(page); + const thread = await gameThread(page); + const frame = await frames(thread.realm); if (o.saveProfile !== undefined) mkdirSync(o.saveProfile, { recursive: true }); const saveAs = o.saveProfile === undefined @@ -443,6 +456,7 @@ export async function measure(url: string, o: IOptions) { ...(memory ? { memory: parseWebMetrics({ memory }).memory } : {}), adapter, startupMs, + gameThread: thread.kind, frame, profile, ...(census ? { calls: census } : {}), From 2013a6f60fc843f5751632ee1bca9a17d7c687df Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:13:30 -0700 Subject: [PATCH 12/29] Worker game pointer lock: mouse look works for a game in a worker (PRD-752) An OffscreenCanvas cannot lock the pointer. In a worker its requestPointerLock and exitPointerLock ask the page, which locks its own canvas within the click's activation window and forwards each pointerlockchange with the page's verdict as captured; a refused lock is reported as captured false. InputMap takes that flag where no document exists, unchanged on a page. 2 tests. --- packages/core/__tests__/input.spec.ts | 11 ++++++++ packages/core/__tests__/worker.spec.ts | 36 ++++++++++++++++++++++++++ packages/core/src/input.ts | 5 +++- packages/core/src/worker.ts | 31 ++++++++++++++++++++++ 4 files changed, 82 insertions(+), 1 deletion(-) diff --git a/packages/core/__tests__/input.spec.ts b/packages/core/__tests__/input.spec.ts index 3a6b975b16..51b07d1f1d 100644 --- a/packages/core/__tests__/input.spec.ts +++ b/packages/core/__tests__/input.spec.ts @@ -572,6 +572,17 @@ describe("InputMap", () => { input.dispose(); }); + it("takes the lock state a worker host forwards with pointerlockchange, where no document exists (PRD-752)", () => { + const canvas = Object.assign(new EventTarget(), { requestPointerLock: () => undefined }); + const input = new InputMap({ look: { pointerRelative: true } }, canvas, canvas); + input.captureMouse(); + expect(input.raw.pointer.captured).toBe(true); + // Escape on the page: the host forwards the change with the page's verdict. + canvas.dispatchEvent(Object.assign(new Event("pointerlockchange"), { captured: false })); + expect(input.raw.pointer.captured).toBe(false); + input.dispose(); + }); + it("releases a held mouse button on a button-less move while the pointer is free", () => { // Released outside the window, no pointerup arrives: the next move is what clears it. const target = new EventTarget(); diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts index e6854e7e41..6253bc7352 100644 --- a/packages/core/__tests__/worker.spec.ts +++ b/packages/core/__tests__/worker.spec.ts @@ -267,6 +267,42 @@ describe("worker game host (PRD-752)", () => { } }); + it("locks the page's pointer when the worker game asks, and reports the lock back", async () => { + // The shooter's mouse look: InputMap asks its canvas, an OffscreenCanvas with no pointer lock. + const { scope, worker } = link(); + const { canvas, container, offscreen } = page(); + const requestPointerLock = vi.fn(); + Object.assign(canvas, { requestPointerLock }); + const pageDocument = globalThis.document as unknown as EventTarget & Record; + const exitPointerLock = vi.fn(); + pageDocument.exitPointerLock = exitPointerLock; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const target = offscreen as EventTarget & { + requestPointerLock(): void; + exitPointerLock(): void; + }; + target.requestPointerLock(); + await tick(); + expect(requestPointerLock).toHaveBeenCalledTimes(1); + const changes: unknown[] = []; + offscreen.addEventListener("pointerlockchange", (event) => + changes.push((event as Event & { captured?: boolean }).captured), + ); + pageDocument.pointerLockElement = canvas; + pageDocument.dispatchEvent(new Event("pointerlockchange")); + await tick(); + expect(changes).toEqual([true]); + target.exitPointerLock(); + await tick(); + expect(exitPointerLock).toHaveBeenCalledTimes(1); + handle.stop(); + }); + it("copies the harness's page globals into the worker before start", async () => { const { scope, worker } = link(); const { container, view } = page(); diff --git a/packages/core/src/input.ts b/packages/core/src/input.ts index 9a03bfa4ea..fc23441f46 100644 --- a/packages/core/src/input.ts +++ b/packages/core/src/input.ts @@ -288,7 +288,10 @@ export class InputMap { const lockTargets = new Set([this.#target, this.#pointerTarget]); if (lockDocument !== undefined) lockTargets.add(lockDocument); for (const target of lockTargets) { - this.#listen(target, "pointerlockchange", () => this.#syncCaptured()); + // A worker has no document to ask; its host forwards the page's verdict as `captured`. + this.#listen(target, "pointerlockchange", (event) => + this.#syncCaptured((event as Event & { captured?: boolean }).captured), + ); } } diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts index b753284fde..ed7bba5a13 100644 --- a/packages/core/src/worker.ts +++ b/packages/core/src/worker.ts @@ -62,6 +62,7 @@ type WorkerOutbound = | { type: "tn-worker-mirror"; name: string; methods: string[] } | { type: "tn-worker-started" } | { type: "tn-worker-failed"; message: string } + | { type: "tn-worker-pointer-lock"; lock: boolean } | { type: "tn-worker-rpc-result"; id: number; result?: unknown; error?: string }; type PageOutbound = WorkerInbound; @@ -178,6 +179,12 @@ export function serveWorkerGame( const initialise = async (message: IWorkerInitMessage): Promise => { try { canvas = message.canvas; + // An OffscreenCanvas cannot lock the pointer: the page locks its own canvas, inside the + // click's activation window, and forwards each pointerlockchange with `captured`. + Object.assign(canvas, { + exitPointerLock: () => post({ lock: false, type: "tn-worker-pointer-lock" }), + requestPointerLock: () => post({ lock: true, type: "tn-worker-pointer-lock" }), + }); size.width = message.width; size.height = message.height; size.dpr = message.dpr; @@ -397,6 +404,27 @@ export function startWorkerGame( for (const type of WINDOW_EVENTS) listen(windowHost, type, (event) => forward(event, "window")); for (const type of CANVAS_EVENTS) listen(canvas, type, (event) => forward(event, "canvas")); listen(canvas, "contextmenu", (event) => event.preventDefault()); + const pageLock = documentHost as Document & { pointerLockElement?: Element | null }; + const sendLock = (captured: boolean): void => + send({ + event: { captured, type: "pointerlockchange" }, + target: "canvas", + type: "tn-worker-event", + }); + listen(pageLock, "pointerlockchange", () => sendLock(pageLock.pointerLockElement === canvas)); + const pointerLock = (lock: boolean): void => { + if (!lock) { + pageLock.exitPointerLock?.(); + return; + } + // A refused lock fires no pointerlockchange: report it, or the game believes it captured. + const refused = (): void => sendLock(false); + try { + void (canvas.requestPointerLock?.() as Promise | undefined)?.catch(refused); + } catch { + refused(); + } + }; // The worker's game plays through the page's own AudioContext, resumed by a page gesture. const AudioContextCtor = page.AudioContext as (new () => AudioContext) | undefined; @@ -493,6 +521,9 @@ export function startWorkerGame( case "tn-worker-ready": sendInit(); break; + case "tn-worker-pointer-lock": + pointerLock(message.lock); + break; case "tn-worker-mirror": installMirror(message.name, message.methods); break; From 497030381381041313132ffe4032c166524aa757 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:31:03 -0700 Subject: [PATCH 13/29] profile:wasm-page times the frames the game presents (PRD-752) A worker's requestAnimationFrame does not follow what the game presents, and on a loaded page rAF ticks at 60 Hz while the game presents far fewer frames. The probe now also times the game's getCurrentTexture calls (the gpuCalls divisor) on the game's own thread, in both arms, as presented; it shims tsx's __name in the game worker, and fails closed with a timeout. 1 test. --- scripts/__tests__/profile-wasm-page.spec.ts | 16 +++++++ scripts/profile-wasm-page.ts | 52 ++++++++++++++++++++- 2 files changed, 66 insertions(+), 2 deletions(-) diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 93affde462..66c33249a2 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -47,6 +47,8 @@ vi.mock("@playwright/test", () => ({ evaluate: async (fn: (() => unknown) | string) => { if (typeof fn === "string") return state.presented.shift() ?? 0; if (fn.toString().includes("requestAdapter")) return "hardware"; + if (fn.toString().includes("getCurrentTexture")) + return Array.from({ length: 300 }, () => 10); if (fn.toString().includes("requestAnimationFrame")) return Array.from({ length: 600 }, (_, i) => (i === 599 ? 40 : 10)); vi.stubGlobal("performance", { @@ -114,6 +116,20 @@ describe("profile:wasm-page scorecard", () => { expect(worker.gameThread).toBe("worker"); expect(worker.frame.p50).toBe(20); }); + it("times presented frames by the game's getCurrentTexture calls, the same divisor in both arms (PRD-752)", async () => { + // A worker's requestAnimationFrame does not follow what the game presents. + const game = { + evaluate: async (fn: (() => unknown) | string) => + typeof fn === "string" + ? "object" + : fn.toString().includes("getCurrentTexture") + ? Array.from({ length: 300 }, () => 16) + : Array.from({ length: 600 }, () => 0.1), + }; + state.workers = [game]; + const run = await measure("http://test", options(["--url", "http://test"])); + expect(run.presented.p50).toBe(16); + }); it.each(["wasmMemoryBytes", "jsHeapUsedBytes"])( "fails closed when %s is unavailable", async (field) => { diff --git a/scripts/profile-wasm-page.ts b/scripts/profile-wasm-page.ts index c0c3c63789..ed2b4c68bf 100644 --- a/scripts/profile-wasm-page.ts +++ b/scripts/profile-wasm-page.ts @@ -171,8 +171,10 @@ const FRAME_SAMPLES = 601; */ async function gameThread(page: Page) { for (const worker of page.workers()) { - if ((await worker.evaluate("typeof globalThis.__tnGamePlatform")) === "object") - return { kind: "worker" as const, realm: worker }; + if ((await worker.evaluate("typeof globalThis.__tnGamePlatform")) !== "object") continue; + // The page's init script shims tsx's `__name`; a worker gets no init script. + await worker.evaluate("globalThis.__name = (fn) => fn;"); + return { kind: "worker" as const, realm: worker }; } return { kind: "page" as const, realm: page }; } @@ -195,6 +197,50 @@ async function frames(realm: Pick) { return frameStats(intervals); } +const PRESENTED_SAMPLES = 301; + +/** + * Intervals between the frames the game presents: one `getCurrentTexture` call each, on either back + * end (the `gpuCalls` divisor). Unlike requestAnimationFrame it follows the game in a worker too. + */ +async function presented(realm: Pick) { + const intervals = await realm.evaluate( + (count) => + new Promise((resolve, reject) => { + const proto = ( + globalThis as { + GPUCanvasContext?: { prototype: { getCurrentTexture(...args: unknown[]): unknown } }; + } + ).GPUCanvasContext?.prototype; + if (proto === undefined) { + reject( + new Error("TN_PROFILE_NO_WEBGPU_CANVAS: the game's thread has no GPUCanvasContext"), + ); + return; + } + const original = proto.getCurrentTexture; + const times: number[] = []; + const restore = setTimeout(() => { + proto.getCurrentTexture = original; + reject( + new Error(`TN_PROFILE_PRESENTED_TIMEOUT: ${times.length} of ${count} frames in 60 s`), + ); + }, 60_000); + proto.getCurrentTexture = function (this: unknown, ...args: unknown[]) { + times.push(performance.now()); + if (times.length === count) { + proto.getCurrentTexture = original; + clearTimeout(restore); + resolve(times.slice(1).map((x, i) => x - (times[i] as number))); + } + return original.apply(this, args); + }; + }), + PRESENTED_SAMPLES, + ); + return frameStats(intervals); +} + interface IProfileNode { id: number; callFrame: { functionName: string; url: string }; @@ -424,6 +470,7 @@ export async function measure(url: string, o: IOptions) { const hostLoad = { ...reading, absolutes: absolutesVerdict(reading) }; const thread = await gameThread(page); const frame = await frames(thread.realm); + const presentedFrame = await presented(thread.realm); if (o.saveProfile !== undefined) mkdirSync(o.saveProfile, { recursive: true }); const saveAs = o.saveProfile === undefined @@ -458,6 +505,7 @@ export async function measure(url: string, o: IOptions) { startupMs, gameThread: thread.kind, frame, + presented: presentedFrame, profile, ...(census ? { calls: census } : {}), ...(gpu ? { gpu } : {}), From 25518c3a44a8889e0d5acbd67e45cf1760c9419c Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:34:38 -0700 Subject: [PATCH 14/29] Shooter: worker.html runs the same game in a web worker; PRD-752 decisions (PRD-752) The shooter kit gets the starter's worker page: src/worker.ts serves the unchanged game.ts, src/worker-page.ts keeps the canvas, input and GameUi. Headed probe on the RTX 2080: town, soldiers, rifle and the live HUD with minimap and clock draw from the worker. The PRD logs gaps 7 to 9, the presented-frame timing, the loaded-host rule and the measured busy share (starter 0.9 to 1.9% on worker.html against 45.7 to 86.9% on the page). --- ...-a-worker-so-the-page-cannot-stall-them.md | 8 +++++ .../__tests__/scaffold.spec.ts | 3 +- .../templates/shooter/AGENTS.md | 3 +- .../templates/shooter/CLAUDE.md | 3 +- .../templates/shooter/src/worker-page.ts | 29 +++++++++++++++++++ .../templates/shooter/src/worker.ts | 5 ++++ .../templates/shooter/vite.config.ts | 2 ++ .../templates/shooter/worker.html | 13 +++++++++ 8 files changed, 63 insertions(+), 3 deletions(-) create mode 100644 packages/create-threenative/templates/shooter/src/worker-page.ts create mode 100644 packages/create-threenative/templates/shooter/src/worker.ts create mode 100644 packages/create-threenative/templates/shooter/worker.html diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 54ff1e51cd..8772cc642a 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -69,6 +69,13 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me Core already has the seam: `defineGame({ platform })` (`IGamePlatformSource`: canvas, size, input target, gamepads). No production code supplied it before. Worker mode supplies it from the worker host. The starter's `GameUi` already talks to the game only through the UI bridge, so the page mounts `GameUi` and the game runs in `src/worker.ts`. The UI bridge host globals carry the HUD over the worker port. A swallowed WebGPU error also hid gap 3 behind `TN_NO_RENDERER`, so `TN_NO_RENDERER` now names the WebGPU cause. - 2026-10-26 (owner): scale-down rule. Bounded implementation slices go to a cheaper arm (`opencode-go/deepseek-v4.1-flash`, variant `max`, at most 2 arms). The lead designs, writes the red tests and reviews every diff. +- 2026-10-26 (lead): **the flag is a page.** Each kit ships `worker.html` (`src/worker.ts` serves the unchanged `game.ts`, `src/worker-page.ts` keeps the canvas, input and `GameUi`) next to the default `index.html`. One build carries both arms of every A/B, and no game code branches on a flag. +- 2026-10-26 (lead): **three more gaps found on the real starter**, each with a red test: + 7. The starter calls `game.ui.onIntent` while `game.ts` evaluates, so the game end of the UI bridge connected before the page port existed and stayed on the in-process broker (a blank HUD). Fix: `@threenative/core/worker` installs the host globals when it loads in a worker, before the game module, and holds frames until the port arrives. + 8. An `OffscreenCanvas` cannot lock the pointer. Fix: in a worker it asks the page, which locks its own canvas inside the click's activation window and forwards each `pointerlockchange` with `captured`; `InputMap` takes that flag where no document exists. + 9. Web Audio calls cost one message each (`AudioListener` moves 9 params a frame). Fix: the proxy batches one message per task. +- 2026-10-26 (lead): **frame time is read from presented frames.** `profile:wasm-page` frame p50 was the page's `requestAnimationFrame`. In a worker that does not follow the game, and on a loaded page rAF ticks at 60 Hz while the game presents far fewer frames. The probe now also reports `presented`: intervals between the game's `getCurrentTexture` calls, on the game's own thread, in both arms. The baseline's p50/p95 above are rAF numbers, so the KPI compares the two arms of the same run on `presented`. +- 2026-10-26 (lead, owner AFK): **loaded host.** The desktop ran at `load1` 35 to 74 from other lanes on top of the 24 loaders, and the laptop's CPU package sat at 97 to 100 °C under other jobs (`offload.sh` exit 75). Main-thread busy is a share of the page thread, so load cannot fake it; frame time can. Per the owner, a frame-time red counts as load until a rerun inside `slot.sh` repeats it, so a watcher runs both kits in GPU slot 1 when `load1` drops under 14. ### Phase 1: Loaded-host baseline **Status:** DONE @@ -91,6 +98,7 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo - [ ] Behind a flag, without COOP/COEP, the starter boots in a worker on an `OffscreenCanvas`; input, pointer lock and audio forward from the main thread, and the HUD takes one state snapshot per frame and sends intents back. proof: `pnpm test:templates` with the flag, starter scenario green - [ ] The playtest harness captures the worker canvas (screenshot not blank, frame counter advances). proof: `pnpm test:templates` with the flag; `node packages/playtest/dist/runner/cli.js doctor --url --text` - [ ] Main-thread busy falls below the target set in phase 1 (first target: below 0.5x of the main-thread arm) and p50 is no worse, under the same stated load. proof: `profile:wasm-page` worker URL against the main-thread `--control` URL, three runs per arm + Open: busy is measured, p50 is not. Starter, desktop RTX 2080, `--load 24`, `load1` 28 to 52: main-thread busy 1.1 / 1.1 / 1.9% on `worker.html` against 57.5 / 52.8 / 45.7% on `/` (target at or below 28%). A second set at `load1` 35 to 74: 0.9 / 1.4% against 57.6 / 86.9%. Presented p50 was 80 to 101 ms in both arms at that load, and one worker run presented 268 frames in 60 s, so frame time waits for the quiet-window rerun in `slot.sh`. - [ ] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR - [ ] The shooter kit plays through the worker with the same checks. proof: `pnpm test:templates` with the flag on the shooter scenario diff --git a/packages/create-threenative/__tests__/scaffold.spec.ts b/packages/create-threenative/__tests__/scaffold.spec.ts index a0a40ac68e..89146c57a5 100644 --- a/packages/create-threenative/__tests__/scaffold.spec.ts +++ b/packages/create-threenative/__tests__/scaffold.spec.ts @@ -322,7 +322,8 @@ const PRD_201_PARENT_SCAFFOLD_HASHES: Readonly> = { runner: "cda53296a601c2dac944f169afc60d94d3fa03f049f5a3eaf88889819f1ec55d", // Finite-height readiness plus hull, sail, and buoy interpolation from the current swell pose. sailing: "5383db14b9c32f6185f406113a83054b465a869a36e171eb2a9308d544a5522d", - shooter: "07eb04cfbb07bab2321795b66c0a81b372a0e801df4120da74dce22e570f4b2e", + // Recomputed 2026-10-26 (PRD-752): worker.html, src/worker.ts, src/worker-page.ts and an AGENTS.md line run the game in a worker. + shooter: "1fdfdcda1de8fe7a097e80e003db56a8b236fc348845911fcfea86cd6df93fbc", snow: "183b80ada822f22f25c9f7e6ca0f45aef2f5789a71f51f8cc3048965ddda8d2b", // Measured 2026-10-16 through createProject after starter's completed-work quality fallback fix. // Recomputed 2026-10-16: starter's adaptiveQuality split its policy resolution into qualityPolicy.ts diff --git a/packages/create-threenative/templates/shooter/AGENTS.md b/packages/create-threenative/templates/shooter/AGENTS.md index f02da009ea..0255cc9802 100644 --- a/packages/create-threenative/templates/shooter/AGENTS.md +++ b/packages/create-threenative/templates/shooter/AGENTS.md @@ -53,7 +53,8 @@ click fires, F or right click aims, R reloads, shift sprints, ctrl/C crouches. ` owns the layout — blocks, quay, pier, raised decks, routes, spawns — and every solid in it is both a collider and a raycast target. `src/entities/` owns the fight: `FpsPlayer` moves, `Rifle` fires and zeroes, `Enemy` patrols, hears, sees and shoots back, `Target` scores the plates. `src/scenes/Play.ts` -wires them and `src/ui/Hud.tsx` is the only HUD. `playtests/survives.playtest.json` is the durable +wires them and `src/ui/Hud.tsx` is the only HUD. `/worker.html` runs the same `game.ts` in a web +worker (`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. `playtests/survives.playtest.json` is the durable smoke proof. The soldiers are `assets/mannequin-combat.glb` (Quaternius UAL, CC0); the hands in `assets/player-viewmodel.glb` are "Animated FPS hands (rifle animation pack)" by Cransh, CC-BY-4.0 — https://sketchfab.com/3d-models/animated-fps-hands-rifle-animation-pack-5f2d0ed780a94724b36ab505f7564057. diff --git a/packages/create-threenative/templates/shooter/CLAUDE.md b/packages/create-threenative/templates/shooter/CLAUDE.md index df7e5b44d1..290112ac8c 100644 --- a/packages/create-threenative/templates/shooter/CLAUDE.md +++ b/packages/create-threenative/templates/shooter/CLAUDE.md @@ -55,7 +55,8 @@ click fires, F or right click aims, R reloads, shift sprints, ctrl/C crouches. ` owns the layout — blocks, quay, pier, raised decks, routes, spawns — and every solid in it is both a collider and a raycast target. `src/entities/` owns the fight: `FpsPlayer` moves, `Rifle` fires and zeroes, `Enemy` patrols, hears, sees and shoots back, `Target` scores the plates. `src/scenes/Play.ts` -wires them and `src/ui/Hud.tsx` is the only HUD. `playtests/survives.playtest.json` is the durable +wires them and `src/ui/Hud.tsx` is the only HUD. `/worker.html` runs the same `game.ts` in a web +worker (`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. `playtests/survives.playtest.json` is the durable smoke proof. The soldiers are `assets/mannequin-combat.glb` (Quaternius UAL, CC0); the hands in `assets/player-viewmodel.glb` are "Animated FPS hands (rifle animation pack)" by Cransh, CC-BY-4.0 — https://sketchfab.com/3d-models/animated-fps-hands-rifle-animation-pack-5f2d0ed780a94724b36ab505f7564057. diff --git a/packages/create-threenative/templates/shooter/src/worker-page.ts b/packages/create-threenative/templates/shooter/src/worker-page.ts new file mode 100644 index 0000000000..b8c89ead95 --- /dev/null +++ b/packages/create-threenative/templates/shooter/src/worker-page.ts @@ -0,0 +1,29 @@ +import { startWorkerGame } from "@threenative/core/worker"; +import { createElement } from "react"; +import { createRoot } from "react-dom/client"; +import { GameUi } from "./ui/GameUi.js"; +import "./style.css"; + +/** + * `worker.html`: the game runs in `src/worker.ts`, off the page's thread, so a busy page cannot + * stall its frames. This page keeps the canvas, the input and the HUD; `index.html` runs the same + * game on the page itself. + */ +const root = document.getElementById("root"); +if (root === null) throw new Error("Missing #root element."); +root.className = "relative h-full w-full overflow-hidden bg-ink"; +const stage = document.createElement("div"); +stage.className = "absolute inset-0"; +root.append(stage); +const ui = document.createElement("div"); +root.append(ui); +// Before the UI mounts: the HUD finds the worker's bridge from its first render. +const game = startWorkerGame( + new Worker(new URL("./worker.ts", import.meta.url), { type: "module" }), + stage, +); +createRoot(ui).render(createElement(GameUi)); +game.ready.catch((error: unknown) => { + stage.textContent = error instanceof Error ? error.message : String(error); + stage.setAttribute("role", "alert"); +}); diff --git a/packages/create-threenative/templates/shooter/src/worker.ts b/packages/create-threenative/templates/shooter/src/worker.ts new file mode 100644 index 0000000000..e6d9f57a6f --- /dev/null +++ b/packages/create-threenative/templates/shooter/src/worker.ts @@ -0,0 +1,5 @@ +import { serveWorkerGame } from "@threenative/core/worker"; +import game from "./game.js"; + +// The worker side of `worker.html`: the same portable game, on its own thread. +serveWorkerGame(game); diff --git a/packages/create-threenative/templates/shooter/vite.config.ts b/packages/create-threenative/templates/shooter/vite.config.ts index b0ddb9d9fc..a6ce2affa0 100644 --- a/packages/create-threenative/templates/shooter/vite.config.ts +++ b/packages/create-threenative/templates/shooter/vite.config.ts @@ -40,6 +40,8 @@ export default defineConfig({ tailwindcss(), assetsWatchPlugin(), ], + // `worker.html` runs the same game in a worker; `index.html` stays the default page. + build: { rollupOptions: { input: { index: "index.html", worker: "worker.html" } } }, server: { watch: { ignored: ["**/artifacts/**", "**/screenshots/**", "**/playtests/**"], diff --git a/packages/create-threenative/templates/shooter/worker.html b/packages/create-threenative/templates/shooter/worker.html new file mode 100644 index 0000000000..254d549d9a --- /dev/null +++ b/packages/create-threenative/templates/shooter/worker.html @@ -0,0 +1,13 @@ + + + + + + + __PROJECT_NAME__ + + +
+ + + From ea25ac5b8902ab71eb298d7aff709ec683999909 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:40:57 -0700 Subject: [PATCH 15/29] Worker game playtest bridge: mirror it while start is held for the runner (PRD-752) With a runner announced, the playtest plugin holds the boot until the runner calls the bridge, so a mirror sent after start never reached the page and the harness reported TN_PLAYTEST_BRIDGE_MISSING. The worker now posts each mirror the moment the game installs the global. 1 test. --- packages/core/__tests__/worker.spec.ts | 29 ++++++++++++++++++++++++++ packages/core/src/worker.ts | 23 +++++++++++++++----- 2 files changed, 47 insertions(+), 5 deletions(-) diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts index 6253bc7352..5167d5ca96 100644 --- a/packages/core/__tests__/worker.spec.ts +++ b/packages/core/__tests__/worker.spec.ts @@ -303,6 +303,35 @@ describe("worker game host (PRD-752)", () => { handle.stop(); }); + it("mirrors the playtest bridge while start is still held for the runner to attach", async () => { + // With a runner announced, the playtest plugin holds the boot until the runner calls the + // bridge: a mirror sent only after start never reaches the page. + const { scope, worker } = link(); + const { container, view } = page(); + const game = { + start: () => + new Promise((resolve) => { + scope.__THREENATIVE_PLAYTEST_BRIDGE__ = { + describe: async () => { + resolve(); + return { protocolVersion: 1 }; + }, + }; + }), + }; + serveWorkerGame(game, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + for (let wait = 0; wait < 20 && view.__THREENATIVE_PLAYTEST_BRIDGE__ === undefined; wait += 1) + await tick(); + const bridge = view.__THREENATIVE_PLAYTEST_BRIDGE__ as { describe(): Promise }; + await expect(bridge.describe()).resolves.toEqual({ protocolVersion: 1 }); + await handle.ready; + handle.stop(); + }); + it("copies the harness's page globals into the worker before start", async () => { const { scope, worker } = link(); const { container, view } = page(); diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts index ed7bba5a13..5accf0809e 100644 --- a/packages/core/src/worker.ts +++ b/packages/core/src/worker.ts @@ -245,16 +245,29 @@ export function serveWorkerGame( ui.attach(message.ui); - await game.start(); - - for (const name of MIRRORED_GLOBALS) { - const value = scope[name]; - if (typeof value !== "object" || value === null) continue; + // Mirrored the moment the game installs one: with a runner announced, the playtest plugin + // holds start until the runner calls the bridge, so a mirror sent after start never arrives. + const mirror = (name: string, value: unknown): void => { + if (typeof value !== "object" || value === null) return; const methods = Object.keys(value).filter( (key) => typeof (value as Record)[key] === "function", ); post({ methods, name, type: "tn-worker-mirror" }); + }; + for (const name of MIRRORED_GLOBALS) { + let value = scope[name]; + mirror(name, value); + Object.defineProperty(scope, name, { + configurable: true, + get: () => value, + set: (next: unknown) => { + value = next; + mirror(name, next); + }, + }); } + + await game.start(); post({ type: "tn-worker-started" }); } catch (error) { post({ From 21f8a386ad78113013c1b0bef398d5e90c7fa757 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Mon, 26 Oct 2026 23:49:34 -0700 Subject: [PATCH 16/29] Worker is the default page for the starter and shooter kits (PRD-752) index.html now runs the game in a web worker; main.html is the named override that runs it on the page thread. The worker canvas forwards setAttribute to the page canvas, so the harness reads the renderer's data-engine for capture provenance (was TN_PLAYTEST_CAPTURE_PROVENANCE_MISSING). 1 test. --- packages/core/__tests__/worker.spec.ts | 19 +++++++++++++++++++ packages/core/src/worker.ts | 9 ++++++++- .../__tests__/scaffold.spec.ts | 12 ++++++------ .../templates/shooter/AGENTS.md | 4 ++-- .../templates/shooter/CLAUDE.md | 4 ++-- .../templates/shooter/index.html | 2 +- .../shooter/{worker.html => main.html} | 2 +- .../templates/shooter/src/worker-page.ts | 6 +++--- .../templates/shooter/src/worker.ts | 2 +- .../templates/shooter/vite.config.ts | 4 ++-- .../templates/starter/AGENTS.md | 4 ++-- .../templates/starter/CLAUDE.md | 4 ++-- .../templates/starter/index.html | 2 +- .../starter/{worker.html => main.html} | 2 +- .../templates/starter/src/worker-page.ts | 6 +++--- .../templates/starter/src/worker.ts | 2 +- .../templates/starter/vite.config.ts | 4 ++-- 17 files changed, 57 insertions(+), 31 deletions(-) rename packages/create-threenative/templates/shooter/{worker.html => main.html} (82%) rename packages/create-threenative/templates/starter/{worker.html => main.html} (79%) diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts index 5167d5ca96..d0e760acb3 100644 --- a/packages/core/__tests__/worker.spec.ts +++ b/packages/core/__tests__/worker.spec.ts @@ -303,6 +303,25 @@ describe("worker game host (PRD-752)", () => { handle.stop(); }); + it("copies the renderer's canvas attributes onto the page canvas", async () => { + // The harness reads `data-engine` on the page's canvas for capture provenance; the Wasm + // renderer sets it on its own canvas, which in a worker is the OffscreenCanvas. + const { scope, worker } = link(); + const { canvas, container, offscreen } = page(); + const setAttribute = vi.fn(); + Object.assign(canvas, { setAttribute }); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + (offscreen as unknown as Element).setAttribute("data-engine", "threenative wasm webgpu"); + await tick(); + expect(setAttribute).toHaveBeenCalledWith("data-engine", "threenative wasm webgpu"); + handle.stop(); + }); + it("mirrors the playtest bridge while start is still held for the runner to attach", async () => { // With a runner announced, the playtest plugin holds the boot until the runner calls the // bridge: a mirror sent only after start never reaches the page. diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts index 5accf0809e..ff74791d3a 100644 --- a/packages/core/src/worker.ts +++ b/packages/core/src/worker.ts @@ -63,6 +63,7 @@ type WorkerOutbound = | { type: "tn-worker-started" } | { type: "tn-worker-failed"; message: string } | { type: "tn-worker-pointer-lock"; lock: boolean } + | { type: "tn-worker-attribute"; name: string; value: string } | { type: "tn-worker-rpc-result"; id: number; result?: unknown; error?: string }; type PageOutbound = WorkerInbound; @@ -180,10 +181,13 @@ export function serveWorkerGame( try { canvas = message.canvas; // An OffscreenCanvas cannot lock the pointer: the page locks its own canvas, inside the - // click's activation window, and forwards each pointerlockchange with `captured`. + // click's activation window, and forwards each pointerlockchange with `captured`. Attributes + // (the renderer's `data-engine`) land on the page canvas, where the harness reads them. Object.assign(canvas, { exitPointerLock: () => post({ lock: false, type: "tn-worker-pointer-lock" }), requestPointerLock: () => post({ lock: true, type: "tn-worker-pointer-lock" }), + setAttribute: (name: string, value: string) => + post({ name, type: "tn-worker-attribute", value: String(value) }), }); size.width = message.width; size.height = message.height; @@ -537,6 +541,9 @@ export function startWorkerGame( case "tn-worker-pointer-lock": pointerLock(message.lock); break; + case "tn-worker-attribute": + canvas.setAttribute(message.name, message.value); + break; case "tn-worker-mirror": installMirror(message.name, message.methods); break; diff --git a/packages/create-threenative/__tests__/scaffold.spec.ts b/packages/create-threenative/__tests__/scaffold.spec.ts index 89146c57a5..4263a9e32b 100644 --- a/packages/create-threenative/__tests__/scaffold.spec.ts +++ b/packages/create-threenative/__tests__/scaffold.spec.ts @@ -322,16 +322,16 @@ const PRD_201_PARENT_SCAFFOLD_HASHES: Readonly> = { runner: "cda53296a601c2dac944f169afc60d94d3fa03f049f5a3eaf88889819f1ec55d", // Finite-height readiness plus hull, sail, and buoy interpolation from the current swell pose. sailing: "5383db14b9c32f6185f406113a83054b465a869a36e171eb2a9308d544a5522d", - // Recomputed 2026-10-26 (PRD-752): worker.html, src/worker.ts, src/worker-page.ts and an AGENTS.md line run the game in a worker. - shooter: "1fdfdcda1de8fe7a097e80e003db56a8b236fc348845911fcfea86cd6df93fbc", + // Recomputed 2026-10-26 (PRD-752): index.html runs the worker, main.html the page thread; src/worker.ts, src/worker-page.ts and an AGENTS.md line run the game in a worker. + shooter: "1053e6fa8282ab028fec295287e2eb011ddd3dc4c29a870c79b7c39efe7a898d", snow: "183b80ada822f22f25c9f7e6ca0f45aef2f5789a71f51f8cc3048965ddda8d2b", // Measured 2026-10-16 through createProject after starter's completed-work quality fallback fix. // Recomputed 2026-10-16: starter's adaptiveQuality split its policy resolution into qualityPolicy.ts // to stay under the 200-line render-file cap; only starter's tree moves. // Recomputed 2026-10-23 on the merge of origin/develop into the PRD-570 lane: measured by this test on the merged tree. // Recomputed 2026-10-23: the native render-chain scenario expects the Unreal finishing chain (grade and grain off). - // Recomputed 2026-10-26 (PRD-752): worker.html, src/worker.ts, src/worker-page.ts and an AGENTS.md line run the game in a worker. - starter: "84da9469c321538b7b2518b92e778e004cfdad2a772bb64e0222df5a2b9ad896", + // Recomputed 2026-10-26 (PRD-752): index.html runs the worker, main.html the page thread; src/worker.ts, src/worker-page.ts and an AGENTS.md line run the game in a worker. + starter: "6286c303cd4ad04ec9bbe3478e9f5fddaae6365e0e0a3a5baeaa51ecb5f9b488", "tower-defense": "c2874bdeedfb61095c23374f4e9716aacedfc801e0966648b067bbf267b1cb11", }; @@ -430,8 +430,8 @@ const STARTER_PATHS = [ "vite.config.ts", "src/game.ts", "src/main.ts", - // PRD-752: the same game in a worker, at `worker.html`. - "worker.html", + // PRD-752: `index.html` runs the game in a worker; `main.html` runs it on the page thread. + "main.html", "src/worker.ts", "src/worker-page.ts", "src/scenes/Play.ts", diff --git a/packages/create-threenative/templates/shooter/AGENTS.md b/packages/create-threenative/templates/shooter/AGENTS.md index 0255cc9802..1c99f5dfb8 100644 --- a/packages/create-threenative/templates/shooter/AGENTS.md +++ b/packages/create-threenative/templates/shooter/AGENTS.md @@ -53,8 +53,8 @@ click fires, F or right click aims, R reloads, shift sprints, ctrl/C crouches. ` owns the layout — blocks, quay, pier, raised decks, routes, spawns — and every solid in it is both a collider and a raycast target. `src/entities/` owns the fight: `FpsPlayer` moves, `Rifle` fires and zeroes, `Enemy` patrols, hears, sees and shoots back, `Target` scores the plates. `src/scenes/Play.ts` -wires them and `src/ui/Hud.tsx` is the only HUD. `/worker.html` runs the same `game.ts` in a web -worker (`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. `playtests/survives.playtest.json` is the durable +wires them and `src/ui/Hud.tsx` is the only HUD. `index.html` runs `game.ts` in a web +worker (`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames; `main.html` is the named override that runs it on the page thread. `playtests/survives.playtest.json` is the durable smoke proof. The soldiers are `assets/mannequin-combat.glb` (Quaternius UAL, CC0); the hands in `assets/player-viewmodel.glb` are "Animated FPS hands (rifle animation pack)" by Cransh, CC-BY-4.0 — https://sketchfab.com/3d-models/animated-fps-hands-rifle-animation-pack-5f2d0ed780a94724b36ab505f7564057. diff --git a/packages/create-threenative/templates/shooter/CLAUDE.md b/packages/create-threenative/templates/shooter/CLAUDE.md index 290112ac8c..861ad2f46d 100644 --- a/packages/create-threenative/templates/shooter/CLAUDE.md +++ b/packages/create-threenative/templates/shooter/CLAUDE.md @@ -55,8 +55,8 @@ click fires, F or right click aims, R reloads, shift sprints, ctrl/C crouches. ` owns the layout — blocks, quay, pier, raised decks, routes, spawns — and every solid in it is both a collider and a raycast target. `src/entities/` owns the fight: `FpsPlayer` moves, `Rifle` fires and zeroes, `Enemy` patrols, hears, sees and shoots back, `Target` scores the plates. `src/scenes/Play.ts` -wires them and `src/ui/Hud.tsx` is the only HUD. `/worker.html` runs the same `game.ts` in a web -worker (`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. `playtests/survives.playtest.json` is the durable +wires them and `src/ui/Hud.tsx` is the only HUD. `index.html` runs `game.ts` in a web +worker (`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames; `main.html` is the named override that runs it on the page thread. `playtests/survives.playtest.json` is the durable smoke proof. The soldiers are `assets/mannequin-combat.glb` (Quaternius UAL, CC0); the hands in `assets/player-viewmodel.glb` are "Animated FPS hands (rifle animation pack)" by Cransh, CC-BY-4.0 — https://sketchfab.com/3d-models/animated-fps-hands-rifle-animation-pack-5f2d0ed780a94724b36ab505f7564057. diff --git a/packages/create-threenative/templates/shooter/index.html b/packages/create-threenative/templates/shooter/index.html index bcebc52129..254d549d9a 100644 --- a/packages/create-threenative/templates/shooter/index.html +++ b/packages/create-threenative/templates/shooter/index.html @@ -8,6 +8,6 @@
- + diff --git a/packages/create-threenative/templates/shooter/worker.html b/packages/create-threenative/templates/shooter/main.html similarity index 82% rename from packages/create-threenative/templates/shooter/worker.html rename to packages/create-threenative/templates/shooter/main.html index 254d549d9a..bcebc52129 100644 --- a/packages/create-threenative/templates/shooter/worker.html +++ b/packages/create-threenative/templates/shooter/main.html @@ -8,6 +8,6 @@
- + diff --git a/packages/create-threenative/templates/shooter/src/worker-page.ts b/packages/create-threenative/templates/shooter/src/worker-page.ts index b8c89ead95..187ad63d08 100644 --- a/packages/create-threenative/templates/shooter/src/worker-page.ts +++ b/packages/create-threenative/templates/shooter/src/worker-page.ts @@ -5,9 +5,9 @@ import { GameUi } from "./ui/GameUi.js"; import "./style.css"; /** - * `worker.html`: the game runs in `src/worker.ts`, off the page's thread, so a busy page cannot - * stall its frames. This page keeps the canvas, the input and the HUD; `index.html` runs the same - * game on the page itself. + * `index.html`, the default: the game runs in `src/worker.ts`, off the page's thread, so a busy + * page cannot stall its frames. This page keeps the canvas, the input and the HUD; `main.html` runs + * the same game on the page itself. */ const root = document.getElementById("root"); if (root === null) throw new Error("Missing #root element."); diff --git a/packages/create-threenative/templates/shooter/src/worker.ts b/packages/create-threenative/templates/shooter/src/worker.ts index e6d9f57a6f..f7708549fb 100644 --- a/packages/create-threenative/templates/shooter/src/worker.ts +++ b/packages/create-threenative/templates/shooter/src/worker.ts @@ -1,5 +1,5 @@ import { serveWorkerGame } from "@threenative/core/worker"; import game from "./game.js"; -// The worker side of `worker.html`: the same portable game, on its own thread. +// The worker side of `index.html`: the same portable game, on its own thread. serveWorkerGame(game); diff --git a/packages/create-threenative/templates/shooter/vite.config.ts b/packages/create-threenative/templates/shooter/vite.config.ts index a6ce2affa0..bd824d249f 100644 --- a/packages/create-threenative/templates/shooter/vite.config.ts +++ b/packages/create-threenative/templates/shooter/vite.config.ts @@ -40,8 +40,8 @@ export default defineConfig({ tailwindcss(), assetsWatchPlugin(), ], - // `worker.html` runs the same game in a worker; `index.html` stays the default page. - build: { rollupOptions: { input: { index: "index.html", worker: "worker.html" } } }, + // `index.html` runs the game in a worker (the default); `main.html` runs it on the page thread. + build: { rollupOptions: { input: { index: "index.html", main: "main.html" } } }, server: { watch: { ignored: ["**/artifacts/**", "**/screenshots/**", "**/playtests/**"], diff --git a/packages/create-threenative/templates/starter/AGENTS.md b/packages/create-threenative/templates/starter/AGENTS.md index e953b3d487..12abcf31e9 100644 --- a/packages/create-threenative/templates/starter/AGENTS.md +++ b/packages/create-threenative/templates/starter/AGENTS.md @@ -45,8 +45,8 @@ a flag, which together produce `won` or `lost`; `R` and the React restart intent `initialState`. Keep the packaged `assets/native-proof.glb` load and its console marker for the desktop asset gate. `Play.ts` owns gameplay, `src/render/` owns the look, `src/ui/Hud.tsx`/`Menu.tsx` own UI, and `state.ts` publishes JSON-safe values. Keep `playtests/survives.playtest.json` as smoke -proof and update outcome tests. `/worker.html` runs the same `game.ts` in a web worker -(`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. +proof and update outcome tests. `index.html` runs `game.ts` in a web worker +(`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames; `main.html` is the named override that runs it on the page thread. On a touch-primary device (`isMobile() && isTouchscreenAvailable()`), local `src/render/touch-controls.ts` adds a left movement stick and right jump button; the scene passes its input to `Player`, with keyboard mapping as the desktop fallback. ## Portable authoring contracts diff --git a/packages/create-threenative/templates/starter/CLAUDE.md b/packages/create-threenative/templates/starter/CLAUDE.md index 2b635315c3..748931ef33 100644 --- a/packages/create-threenative/templates/starter/CLAUDE.md +++ b/packages/create-threenative/templates/starter/CLAUDE.md @@ -47,8 +47,8 @@ a flag, which together produce `won` or `lost`; `R` and the React restart intent `initialState`. Keep the packaged `assets/native-proof.glb` load and its console marker for the desktop asset gate. `Play.ts` owns gameplay, `src/render/` owns the look, `src/ui/Hud.tsx`/`Menu.tsx` own UI, and `state.ts` publishes JSON-safe values. Keep `playtests/survives.playtest.json` as smoke -proof and update outcome tests. `/worker.html` runs the same `game.ts` in a web worker -(`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames. +proof and update outcome tests. `index.html` runs `game.ts` in a web worker +(`src/worker.ts`, page side `src/worker-page.ts`), so a busy page cannot stall its frames; `main.html` is the named override that runs it on the page thread. On a touch-primary device (`isMobile() && isTouchscreenAvailable()`), local `src/render/touch-controls.ts` adds a left movement stick and right jump button; the scene passes its input to `Player`, with keyboard mapping as the desktop fallback. ## Portable authoring contracts diff --git a/packages/create-threenative/templates/starter/index.html b/packages/create-threenative/templates/starter/index.html index f59c7647a1..4cfc3edbf5 100644 --- a/packages/create-threenative/templates/starter/index.html +++ b/packages/create-threenative/templates/starter/index.html @@ -7,6 +7,6 @@
- + diff --git a/packages/create-threenative/templates/starter/worker.html b/packages/create-threenative/templates/starter/main.html similarity index 79% rename from packages/create-threenative/templates/starter/worker.html rename to packages/create-threenative/templates/starter/main.html index 4cfc3edbf5..f59c7647a1 100644 --- a/packages/create-threenative/templates/starter/worker.html +++ b/packages/create-threenative/templates/starter/main.html @@ -7,6 +7,6 @@
- + diff --git a/packages/create-threenative/templates/starter/src/worker-page.ts b/packages/create-threenative/templates/starter/src/worker-page.ts index fd1f295978..3740554c08 100644 --- a/packages/create-threenative/templates/starter/src/worker-page.ts +++ b/packages/create-threenative/templates/starter/src/worker-page.ts @@ -5,9 +5,9 @@ import { GameUi } from "./ui/GameUi.js"; import "./style.css"; /** - * `worker.html`: the game runs in `src/worker.ts`, off the page's thread, so a busy page cannot - * stall its frames. This page keeps the canvas, the input and the HUD; `index.html` runs the same - * game on the page itself. + * `index.html`, the default: the game runs in `src/worker.ts`, off the page's thread, so a busy + * page cannot stall its frames. This page keeps the canvas, the input and the HUD; `main.html` runs + * the same game on the page itself. */ const root = document.getElementById("root"); if (root === null) throw new Error("Missing #root element."); diff --git a/packages/create-threenative/templates/starter/src/worker.ts b/packages/create-threenative/templates/starter/src/worker.ts index e6d9f57a6f..f7708549fb 100644 --- a/packages/create-threenative/templates/starter/src/worker.ts +++ b/packages/create-threenative/templates/starter/src/worker.ts @@ -1,5 +1,5 @@ import { serveWorkerGame } from "@threenative/core/worker"; import game from "./game.js"; -// The worker side of `worker.html`: the same portable game, on its own thread. +// The worker side of `index.html`: the same portable game, on its own thread. serveWorkerGame(game); diff --git a/packages/create-threenative/templates/starter/vite.config.ts b/packages/create-threenative/templates/starter/vite.config.ts index a6ce2affa0..bd824d249f 100644 --- a/packages/create-threenative/templates/starter/vite.config.ts +++ b/packages/create-threenative/templates/starter/vite.config.ts @@ -40,8 +40,8 @@ export default defineConfig({ tailwindcss(), assetsWatchPlugin(), ], - // `worker.html` runs the same game in a worker; `index.html` stays the default page. - build: { rollupOptions: { input: { index: "index.html", worker: "worker.html" } } }, + // `index.html` runs the game in a worker (the default); `main.html` runs it on the page thread. + build: { rollupOptions: { input: { index: "index.html", main: "main.html" } } }, server: { watch: { ignored: ["**/artifacts/**", "**/screenshots/**", "**/playtests/**"], From 7bc76001efb91189e99d1108a0c4c8d19705fa02 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 00:06:38 -0700 Subject: [PATCH 17/29] Engine raycast: a cached triangle BVH per geometry (PRD-752) Mesh::raycast tested every triangle of the mesh. The shooter's merged town meshes hold about 430k triangles, so a few soldier line-of-sight rays cost about 90 ms in one update, which was the stutter left on the worker page. The tree is built on first use, rebuilt when positions, index or drawRange change, and skipped for morphed meshes and meshes that pose their vertices (SkinnedMesh). Candidates run the unchanged hit code in triangle order, so results stay identical: 2,048 oracle rays, 0 differ. 1,000 rays on a 20k mesh: 8,471 ms -> 6.0 ms; a focused ray tests 4 of 20,000 triangles. --- .../src/engine/animation/skinning/skeleton.h | 1 + .../src/engine/scene/geometry.h | 2 + .../runtime-native/src/engine/scene/nodes.h | 3 + .../src/engine/scene/raycaster.cpp | 138 ++++++++++++++- .../src/engine/scene/raycaster.h | 5 + .../native-engine/animation/skeleton_test.cpp | 40 ++++- .../native-engine/scene/raycast_test.cpp | 165 +++++++++++++++++- 7 files changed, 351 insertions(+), 3 deletions(-) diff --git a/packages/runtime-native/src/engine/animation/skinning/skeleton.h b/packages/runtime-native/src/engine/animation/skinning/skeleton.h index b6e7ecd3b0..f4989588fb 100644 --- a/packages/runtime-native/src/engine/animation/skinning/skeleton.h +++ b/packages/runtime-native/src/engine/animation/skinning/skeleton.h @@ -128,6 +128,7 @@ class SkinnedMesh : public Mesh { void updateMatrixWorld(bool force = false) override; void afterMatrixWorld() override; Vector3& getVertexPosition(uint64_t index, Vector3& target) const override; + [[nodiscard]] bool posesVertices() const override { return true; } Vector3& applyBoneTransform(uint64_t index, Vector3& target) const; void computeBoundingBox(); std::shared_ptr boundingBox; diff --git a/packages/runtime-native/src/engine/scene/geometry.h b/packages/runtime-native/src/engine/scene/geometry.h index c12794b12c..37b032947e 100644 --- a/packages/runtime-native/src/engine/scene/geometry.h +++ b/packages/runtime-native/src/engine/scene/geometry.h @@ -129,6 +129,7 @@ struct DrawRange { }; struct GeometryContentClass; // engine/renderer/projection/plan.h +struct TriangleBVH; // engine/scene/raycaster.cpp /** three's BufferGeometry: attributes, index, groups, bounds and the mutation methods on them. */ class BufferGeometry { @@ -149,6 +150,7 @@ class BufferGeometry { DrawRange drawRange; std::shared_ptr boundingBox; std::shared_ptr boundingSphere; + std::shared_ptr raycastBVH; std::map parameters; // already-JSON values, keyed by parameter name std::string name; std::string uuid = generateUuid(); // three's: unique per geometry diff --git a/packages/runtime-native/src/engine/scene/nodes.h b/packages/runtime-native/src/engine/scene/nodes.h index 4e59260629..aa88aaa9fc 100644 --- a/packages/runtime-native/src/engine/scene/nodes.h +++ b/packages/runtime-native/src/engine/scene/nodes.h @@ -89,6 +89,9 @@ class Mesh : public Object3D { } std::vector morphTargetInfluences; virtual Vector3& getVertexPosition(uint64_t index, Vector3& target) const; + /** Whether getVertexPosition moves stored positions past morphs (a SkinnedMesh's bones); raycast then + * tests every triangle, since a triangle tree built from stored positions would cull posed ones. */ + [[nodiscard]] virtual bool posesVertices() const { return false; } bool raycast(const Raycaster& raycaster, std::vector& intersects) override; [[nodiscard]] std::string_view type() const override { return "Mesh"; } diff --git a/packages/runtime-native/src/engine/scene/raycaster.cpp b/packages/runtime-native/src/engine/scene/raycaster.cpp index 9783d2bed6..cd050d5acd 100644 --- a/packages/runtime-native/src/engine/scene/raycaster.cpp +++ b/packages/runtime-native/src/engine/scene/raycaster.cpp @@ -5,7 +5,107 @@ #include namespace tn::engine { +struct TriangleBVH { + struct AttributeState { + std::shared_ptr attribute; + std::shared_ptr store; + uint64_t count = 0, writes = 0; + uint32_t version = 0; + int itemSize = 0; + bool normalized = false; + explicit AttributeState(const std::shared_ptr& a) : attribute(a) { + if (!a) return; + store = a->store; count = a->count(); writes = store->writes(); version = a->version(); + itemSize = a->itemSize; normalized = a->normalized; + } + bool matches(const std::shared_ptr& a) const { + return attribute == a && (!a || (store == a->store && count == a->count() && + writes == store->writes() && version == a->version() && + itemSize == a->itemSize && normalized == a->normalized)); + } + }; + struct Node { + Box3 bounds; + size_t begin = 0, count = 0, left = 0, right = 0; + }; + AttributeState position, index; + DrawRange drawRange; + std::vector nodes; + std::vector starts; + + TriangleBVH(const std::shared_ptr& p, const std::shared_ptr& ix, + const DrawRange& range, double start, double end) : position(p), index(ix), drawRange(range) { + struct Triangle { double start; Box3 bounds; }; + std::vector triangles; + for (double i = start; i < end; i += 3) { + Triangle triangle{i, {}}; + for (int j = 0; j < 3; ++j) { + const auto at = uint64_t(ix ? ix->getX(uint64_t(i + j)) : i + j); + const Vector3 v{p->getX(at), p->getY(at), p->getZ(at)}; + if (!std::isfinite(v.x) || !std::isfinite(v.y) || !std::isfinite(v.z)) { + triangle.bounds.min.setScalar(-std::numeric_limits::infinity()); + triangle.bounds.max.setScalar(std::numeric_limits::infinity()); + break; // Malformed triangles retain the original hit math instead of being culled. + } + triangle.bounds.expandByPoint(v); + } + triangles.push_back(triangle); + } + const auto component = [](const Vector3& v, int axis) { return axis == 0 ? v.x : axis == 1 ? v.y : v.z; }; + const auto build = [&](auto&& self, size_t begin, size_t end) -> size_t { + const size_t node = nodes.size(); + nodes.emplace_back(); + Box3 bounds; + for (size_t i = begin; i < end; ++i) bounds.unionWith(triangles[i].bounds); + nodes[node].bounds = bounds; + if (end - begin <= 8) { + nodes[node].begin = begin; nodes[node].count = end - begin; + } else { + Vector3 extent; bounds.getSize(extent); + const int axis = extent.x >= extent.y && extent.x >= extent.z ? 0 : extent.y >= extent.z ? 1 : 2; + const size_t middle = begin + (end - begin) / 2; + const auto center = [&](const Triangle& t) { + const double value = component(t.bounds.min, axis) * 0.5 + component(t.bounds.max, axis) * 0.5; + return std::isnan(value) ? 0 : value; + }; + std::nth_element(triangles.begin() + begin, triangles.begin() + middle, triangles.begin() + end, + [&](const Triangle& a, const Triangle& b) { return center(a) < center(b); }); + const size_t left = self(self, begin, middle), right = self(self, middle, end); + nodes[node].left = left; nodes[node].right = right; + } + return node; + }; + if (!triangles.empty()) build(build, 0, triangles.size()); + starts.reserve(triangles.size()); + for (const auto& triangle : triangles) starts.push_back(triangle.start); + // A deferred store may have filled its bytes while the bounds were read. + position = AttributeState(p); index = AttributeState(ix); + } +}; namespace { +thread_local uint64_t trianglesTested = 0; +bool intersectsBounds(const Ray& ray, const Box3& bounds, double near, double far) { + const double origins[] = {ray.origin.x, ray.origin.y, ray.origin.z}; + const double directions[] = {ray.direction.x, ray.direction.y, ray.direction.z}; + const double minima[] = {bounds.min.x, bounds.min.y, bounds.min.z}; + const double maxima[] = {bounds.max.x, bounds.max.y, bounds.max.z}; + for (int axis = 0; axis < 3; ++axis) { + // Pad both the bounds and slab distances: edge, vertex and zero-thickness hits must survive rounding. + const double padding = 64 * std::numeric_limits::epsilon() * + std::max({1.0, std::abs(origins[axis]), std::abs(minima[axis]), std::abs(maxima[axis])}); + const double low = minima[axis] - padding, high = maxima[axis] + padding; + if (directions[axis] == 0) { + if (origins[axis] < low || origins[axis] > high) return false; + continue; + } + double a = (low - origins[axis]) / directions[axis], b = (high - origins[axis]) / directions[axis]; + if (a > b) std::swap(a, b); + near = std::max(near, std::nextafter(a, -std::numeric_limits::infinity())); + far = std::min(far, std::nextafter(b, std::numeric_limits::infinity())); + if (near > far) return false; + } + return true; +} void intersect(Object3D& object, const Raycaster& caster, std::vector& hits, bool recursive) { bool propagate = true; if (object.layers().test(caster.layers)) propagate = object.raycast(caster, hits); @@ -41,6 +141,9 @@ template V interpolate(const BufferAttribute& attr, uint64_t a, uint64_ return target; } } +uint64_t raycastTrianglesTested() { return trianglesTested; } +void resetRaycastTrianglesTested() { trianglesTested = 0; } + bool Raycaster::setFromCamera(const Vector2& coords, Camera& source) { if (dynamic_cast(&source)) { ray.origin.setFromMatrixPosition(source.matrixWorld); @@ -111,7 +214,40 @@ bool Mesh::raycast(const Raycaster& caster, std::vector& hits) { const auto index = geometry->index; const double start = jsMax(0, geometry->drawRange.start); const double end = jsMin(double(index ? index->count() : position->count()), geometry->drawRange.start + geometry->drawRange.count); - for (double i = start; i < end; i += 3) { + const bool accelerated = !caster.meshBruteForceForTesting && !posesVertices() && + std::none_of(morphTargetInfluences.begin(), morphTargetInfluences.end(), [](double v) { return v != 0; }); + std::vector candidates; + if (accelerated) { + auto& tree = geometry->raycastBVH; + if (!tree || !tree->position.matches(position) || !tree->index.matches(index) || + tree->drawRange.start != geometry->drawRange.start || tree->drawRange.count != geometry->drawRange.count) + tree = std::make_shared(position, index, geometry->drawRange, start, end); + // The inverse linear transform's Frobenius norm overestimates every local/world distance ratio. + // Zero is a conservative lower bound for near; the unchanged hit code enforces the exact interval. + const auto& e = inverse.elements; + const double localScale = std::hypot(std::hypot(e[0], e[1], e[2]), + std::hypot(e[4], e[5], e[6]), std::hypot(e[8], e[9], e[10])); + const auto& world = matrixWorld.elements; + const double rounding = 64 * std::numeric_limits::epsilon() * std::max({1.0, + std::abs(caster.ray.origin.x), std::abs(caster.ray.origin.y), std::abs(caster.ray.origin.z), + std::abs(world[12]), std::abs(world[13]), std::abs(world[14])}); + const double localFar = std::isfinite(caster.far) && caster.far >= 0 && std::isfinite(localScale) && localScale > 0 + ? std::nextafter((caster.far + rounding) * localScale, std::numeric_limits::infinity()) + : std::numeric_limits::infinity(); + const auto visit = [&](auto&& self, size_t at) -> void { + const auto& node = tree->nodes[at]; + if (!intersectsBounds(local, node.bounds, 0, localFar)) return; + if (node.count) candidates.insert(candidates.end(), tree->starts.begin() + node.begin, + tree->starts.begin() + node.begin + node.count); + else { self(self, node.left); self(self, node.right); } + }; + if (!tree->nodes.empty()) visit(visit, 0); + std::sort(candidates.begin(), candidates.end()); + } + size_t candidate = 0; + for (double i = accelerated ? (candidates.empty() ? end : candidates[0]) : start; i < end; + i = accelerated ? (++candidate < candidates.size() ? candidates[candidate] : end) : i + 3) { + ++trianglesTested; const auto a = uint64_t(index ? index->getX(uint64_t(i)) : i); const auto b = uint64_t(index ? index->getX(uint64_t(i + 1)) : i + 1); const auto c = uint64_t(index ? index->getX(uint64_t(i + 2)) : i + 2); diff --git a/packages/runtime-native/src/engine/scene/raycaster.h b/packages/runtime-native/src/engine/scene/raycaster.h index 0f5b7a027c..662297d0b1 100644 --- a/packages/runtime-native/src/engine/scene/raycaster.h +++ b/packages/runtime-native/src/engine/scene/raycaster.h @@ -25,11 +25,16 @@ struct Intersection { /** A Line hit: the segment's first vertex index; its face, faceIndex and barycoord are null. */ std::optional index; }; +// CPU-test diagnostics; the counter is local to the calling thread. +uint64_t raycastTrianglesTested(); +void resetRaycastTrianglesTested(); + class Raycaster { public: Raycaster(const Vector3& origin = {}, const Vector3& direction = {0, 0, -1}, double near = 0, double far = std::numeric_limits::infinity()) : ray(origin, direction), near(near), far(far) {} Ray ray; + bool meshBruteForceForTesting = false; // Oracle uses the same triangle hit code. double near = 0, far = std::numeric_limits::infinity(); double lineThreshold = 1; // three's params.Line.threshold Camera* camera = nullptr; diff --git a/packages/runtime-native/tests/native-engine/animation/skeleton_test.cpp b/packages/runtime-native/tests/native-engine/animation/skeleton_test.cpp index 7f81f157b2..50394969f3 100644 --- a/packages/runtime-native/tests/native-engine/animation/skeleton_test.cpp +++ b/packages/runtime-native/tests/native-engine/animation/skeleton_test.cpp @@ -7,6 +7,8 @@ #include "check.h" #include "engine/animation/skinning/skeleton.h" #include "engine/scene/lights.h" +#include "engine/scene/material.h" +#include "engine/scene/raycaster.h" #include #include @@ -174,6 +176,42 @@ void cloneLights() { CHECK(std::dynamic_pointer_cast(cloneObject(ambient, true, error)) != nullptr); } +// A raycast's triangle tree is built from stored positions; a SkinnedMesh poses them through its bones, +// so a tree would cull the posed triangle. A ray at the posed triangle hits it, as the brute-force path does. +void raycastPosedSkin() { + // 16 triangles in a row along x (k spans [k-8, k-7]), enough that the tree splits them at x = 0. The bone + // moves every vertex +4 in x, so triangle 4 (bind pose x [-4, -3]) lands under x = 0.5, inside the bind + // pose bounds but in the other half of the tree. + std::vector positions, skinIndex, skinWeight; + for (int k = 0; k < 16; ++k) { + positions.insert(positions.end(), {k - 8.0, -1, 0, k - 7.0, -1, 0, k - 7.5, 1, 0}); + for (int v = 0; v < 3; ++v) { + skinIndex.insert(skinIndex.end(), {0, 0, 0, 0}); + skinWeight.insert(skinWeight.end(), {1, 0, 0, 0}); + } + } + auto geometry = std::make_shared(); + geometry->setAttribute("position", BufferAttribute::fromFloats(positions, 3)); + geometry->setAttribute("skinIndex", BufferAttribute::fromFloats(skinIndex, 4)); + geometry->setAttribute("skinWeight", BufferAttribute::fromFloats(skinWeight, 4)); + SkinnedMesh mesh(geometry, std::make_shared(MaterialType::Basic)); + auto bone = std::make_shared(); + mesh.add(*bone); + mesh.updateMatrixWorld(true); + mesh.bind(std::make_shared(std::vector>{bone}, std::vector{})); + bone->position.set(4, 0, 0); + mesh.updateMatrixWorld(true); + mesh.skeleton->update(); + for (const bool brute : {true, false}) { + Raycaster caster({0.5, 0, 5}, {0, 0, -1}); + caster.meshBruteForceForTesting = brute; + const auto hits = caster.intersectObject(mesh, false); + std::printf("posed skin raycast (%s): %zu hits\n", brute ? "brute" : "tree", hits.size()); + CHECK(hits.size() == 1); + } +} + } // namespace -TN_TEST_MAIN({"skeleton_pose", skeletonPose}, {"clone_bounds", cloneBounds}, {"clone_lights", cloneLights}) +TN_TEST_MAIN({"skeleton_pose", skeletonPose}, {"clone_bounds", cloneBounds}, {"clone_lights", cloneLights}, + {"raycast_posed_skin", raycastPosedSkin}) diff --git a/packages/runtime-native/tests/native-engine/scene/raycast_test.cpp b/packages/runtime-native/tests/native-engine/scene/raycast_test.cpp index e8f7ed3822..6f4edc1c88 100644 --- a/packages/runtime-native/tests/native-engine/scene/raycast_test.cpp +++ b/packages/runtime-native/tests/native-engine/scene/raycast_test.cpp @@ -4,6 +4,7 @@ #include "engine/scene/nodes.h" #include "engine/scene/material.h" #include +#include #include #include #include @@ -116,6 +117,165 @@ J read(const char* path) { if (!json::parse(text,value,error)) throw std::runtime_error(std::string("cannot parse ")+path); return value; } + +bool sameFloat(double a, double b) { return std::bit_cast(a) == std::bit_cast(b); } +template bool sameVector(const V& a, const V& b) { + bool same = sameFloat(a.x,b.x) && sameFloat(a.y,b.y); + if constexpr (std::is_same_v) same = same && sameFloat(a.z,b.z); + return same; +} +template bool sameOptional(const std::optional& a, const std::optional& b) { + return a.has_value() == b.has_value() && (!a || sameVector(*a,*b)); +} +void sameHits(const std::vector& a, const std::vector& b, const std::string& where) { + check(a.size() == b.size(),where+" count"); + for (size_t i=0;i gridGeometry() { + auto g=std::make_shared(); + std::vector positions, uvs, normals; + std::vector indices; + for (int y=0;y<=100;++y) for (int x=0;x<=100;++x) { + positions.insert(positions.end(),{double(x),double(y),0}); + uvs.insert(uvs.end(),{x/100.0,y/100.0}); + normals.insert(normals.end(),{x/1000.0,y/1000.0,1}); + } + for (uint32_t y=0;y<100;++y) for (uint32_t x=0;x<100;++x) { + const auto a=y*101+x; + indices.insert(indices.end(),{a,a+1,a+102,a,a+102,a+101}); + } + g->setAttribute("position",BufferAttribute::fromFloats(positions,3)); + g->setAttribute("uv",BufferAttribute::fromFloats(uvs,2)); + g->setAttribute("uv1",BufferAttribute::fromFloats(uvs,2)); + g->setAttribute("normal",BufferAttribute::fromFloats(normals,3)); + g->setIndexFromArray(indices); + return g; +} +std::vector seededRays(size_t count) { + uint32_t seed=0x573b00; + const auto random=[&]() { seed=1664525*seed+1013904223; return double(seed)/4294967296.0; }; + std::vector rays; + for (size_t i=0;i(); + std::vector positions; + std::vector indices; + for (uint32_t i=0;i<20;++i) { + const double x=i*10; + positions.insert(positions.end(),{x,0,0,x+1,0,0,x,1,0}); + indices.insert(indices.end(),{i*3,i*3+1,i*3+2}); + } + g->setAttribute("position",BufferAttribute::fromFloats(positions,3)); g->setIndexFromArray(indices); + auto material=std::make_shared(MaterialType::Basic); material->side=Side::Double; + Mesh mesh(g,material); mesh.updateMatrixWorld(true); + Raycaster ray({0.25,0.125,10},{0,0,-1}); + const auto compare=[&](const char* name, size_t count) { + auto brute=ray; brute.meshBruteForceForTesting=true; + const auto actual=ray.intersectObject(mesh,false), expected=brute.intersectObject(mesh,false); + sameHits(actual,expected,name); check(expected.size()==count,std::string(name)+" expected hits"); + }; + compare("BVH initial",1); auto cached=g->raycastBVH; + compare("BVH reuse",1); check(g->raycastBVH==cached,"BVH reused across rays"); + auto p=g->getAttribute("position"); + p->setXYZ(57,0,0,0); p->setXYZ(58,1,0,0); p->setXYZ(59,0,1,0); + compare("BVH position write",2); check(g->raycastBVH!=cached,"BVH position writes invalidate"); cached=g->raycastBVH; + p->setNeedsUpdate(); compare("BVH position version",2); + check(g->raycastBVH!=cached,"BVH position version invalidates"); cached=g->raycastBVH; + p=p->clone(); p->setXYZ(57,190,0,0); p->setXYZ(58,191,0,0); p->setXYZ(59,190,1,0); + g->setAttribute("position",p); compare("BVH position replacement",1); + check(g->raycastBVH!=cached,"BVH position identity invalidates"); cached=g->raycastBVH; + p->store=p->clone()->store; compare("BVH store replacement",1); + check(g->raycastBVH!=cached,"BVH store identity invalidates"); cached=g->raycastBVH; + g->index->setX(0,57); g->index->setX(1,58); g->index->setX(2,59); + compare("BVH index write",0); check(g->raycastBVH!=cached,"BVH index writes invalidate"); cached=g->raycastBVH; + g->index->setNeedsUpdate(); compare("BVH index version",0); + check(g->raycastBVH!=cached,"BVH index version invalidates"); cached=g->raycastBVH; + g->setIndexFromArray(indices); compare("BVH index replacement",1); + check(g->raycastBVH!=cached,"BVH index identity invalidates"); cached=g->raycastBVH; + g->setDrawRange(3,57); compare("BVH draw start",0); + check(g->raycastBVH!=cached,"BVH draw start invalidates"); cached=g->raycastBVH; + g->setDrawRange(0,0); compare("BVH empty draw",0); + check(g->raycastBVH!=cached,"BVH draw count invalidates"); + g->setDrawRange(0,60); compare("BVH restored draw",1); + auto morph=p->clone(); morph->setXYZ(3,0,0,0); morph->setXYZ(4,1,0,0); morph->setXYZ(5,0,1,0); + g->morphPositions.push_back(morph); mesh.morphTargetInfluences={1}; cached=g->raycastBVH; + compare("BVH morph fallback",2); resetRaycastTrianglesTested(); ray.intersectObject(mesh,false); + check(raycastTrianglesTested()==20 && g->raycastBVH==cached,"BVH morph scans original triangles"); + mesh.morphTargetInfluences={0}; compare("BVH zero morph reuses cache",1); + check(g->raycastBVH==cached,"BVH zero morph cache retained"); + g->index=nullptr; compare("BVH index removed",1); + p->store->resize(57*3); compare("BVH position count",1); + g->deleteAttribute("position"); compare("BVH missing position",0); +} +int bvh() { + const int oracleStart=differ; + const auto rays=seededRays(256); + size_t hitRays=0,missRays=0; + for (int variant=0;variant<8;++variant) { + auto g=gridGeometry(); + if (variant==1) g=g->toNonIndexed(); + if (variant==2) g->setDrawRange(15003,30000); + auto material=std::make_shared(MaterialType::Basic); + material->side=variant==3 ? Side::Back : variant>=4 ? Side::Double : Side::Front; + Mesh mesh(g,material); + if (variant>=5) { + mesh.scale.set(variant==6 ? -2 : 2,0.25,3); + mesh.rotation.set(0.3,-0.6,0.2); mesh.position.set(1e6,-2e6,3e6); + } + mesh.updateMatrixWorld(true); + if (variant==7) mesh.matrixWorld.elements[4]+=0.4; // Shear, not just TRS. + for (size_t i=0;i actual, expected; + if (variant>=5) { + r.ray.origin.applyMatrix4(mesh.matrixWorld); + r.ray.direction.transformDirection(mesh.matrixWorld); + } + r.intersectObject(mesh,false,actual); + r.meshBruteForceForTesting=true; r.intersectObject(mesh,false,expected); + sameHits(actual,expected,"bvh variant "+std::to_string(variant)+" ray "+std::to_string(i)); + if (expected.empty()) ++missRays; else ++hitRays; + } + } + check(hitRays>0 && missRays>0,"BVH oracle hits and misses"); + std::printf("BVH oracle: 2048 rays, %zu hit rays, %zu miss rays, %d differ\n",hitRays,missRays,differ-oracleStart); + const int invalidationStart=differ; bvhInvalidation(); + std::printf("BVH invalidation and morph fallback: %d differ\n",differ-invalidationStart); + auto g=gridGeometry(); auto material=std::make_shared(MaterialType::Basic); + material->side=Side::Double; Mesh mesh(g,material); mesh.updateMatrixWorld(true); + Raycaster focused({50.25,50.125,10},{0,0,-1}); + resetRaycastTrianglesTested(); const auto focusHits=focused.intersectObject(mesh,false); + const auto tested=raycastTrianglesTested(); + check(!focusHits.empty(),"BVH cost ray hits"); + check(tested<400,"BVH cost: triangles tested < 400 (2% of 20000)"); + std::printf("BVH cost: %llu / 20000 triangles (%.4f%%)\n",(unsigned long long)tested,100.0*tested/20000); + const auto benchmark=seededRays(1000); + const auto measure=[&](bool brute) { + resetRaycastTrianglesTested(); + const auto start=std::chrono::steady_clock::now(); size_t hits=0; + for (auto r:benchmark) { r.meshBruteForceForTesting=brute; hits+=r.intersectObject(mesh,false).size(); } + const double ms=std::chrono::duration(std::chrono::steady_clock::now()-start).count(); + std::printf("BVH benchmark %s: 1000 rays, %.3f ms, %llu triangles, %zu hits\n",brute ? "brute" : "cached",ms,(unsigned long long)raycastTrianglesTested(),hits); + }; + measure(true); measure(false); + return differ ? 1 : 0; +} int raycaster() { const auto table=read(TN_RAYCASTER_REFERENCE); auto scene=build(field(table,"scene")); scene->updateMatrixWorld(true); int count=0, observations=0; @@ -188,6 +348,9 @@ int lod() { } } int main(int argc,char** argv) { - try { return argc>1 && std::string(argv[1])=="lod" ? lod() : raycaster(); } + try { + if (argc>1 && std::string(argv[1])=="lod") return lod(); + const int reference=raycaster(); return bvh() || reference; + } catch(const std::exception& e) { std::printf("FAIL: %s\n",e.what()); return 1; } } From 105f49228a7871c4e63884114dc42a70d5b10e37 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 00:24:16 -0700 Subject: [PATCH 18/29] Worker mode: fix the review findings in the worker host, remote audio and profiler (PRD-752) Worker: startup errors reject ready, stop removes the canvas and releases pointer lock and capture, pointerlockerror, 0x0 clamp, HUD wheel stays on the HUD, motion coalesced per page frame, one HUD snapshot in flight, gamepads forwarded, a lazy playtest mirror with _clockEpoch. Audio: the clock freezes while suspended, fluent AudioParam calls reuse their handle, ended one-shots release theirs, numeric param defaults, control ops settle from the page. Profiler: probes run in the game realm. Each item has a red-green spec. --- ...-a-worker-so-the-page-cannot-stall-them.md | 9 +- packages/core/__tests__/worker-audio.spec.ts | 215 ++++++++- packages/core/__tests__/worker.spec.ts | 427 +++++++++++++++++- packages/core/src/worker-audio.ts | 326 ++++++++++--- packages/core/src/worker.ts | 264 ++++++++--- scripts/__tests__/profile-wasm-page.spec.ts | 55 +++ scripts/profile-wasm-page.ts | 52 ++- 7 files changed, 1203 insertions(+), 145 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 8772cc642a..9c8653ae91 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -76,6 +76,11 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me 9. Web Audio calls cost one message each (`AudioListener` moves 9 params a frame). Fix: the proxy batches one message per task. - 2026-10-26 (lead): **frame time is read from presented frames.** `profile:wasm-page` frame p50 was the page's `requestAnimationFrame`. In a worker that does not follow the game, and on a loaded page rAF ticks at 60 Hz while the game presents far fewer frames. The probe now also reports `presented`: intervals between the game's `getCurrentTexture` calls, on the game's own thread, in both arms. The baseline's p50/p95 above are rAF numbers, so the KPI compares the two arms of the same run on `presented`. - 2026-10-26 (lead, owner AFK): **loaded host.** The desktop ran at `load1` 35 to 74 from other lanes on top of the 24 loaders, and the laptop's CPU package sat at 97 to 100 °C under other jobs (`offload.sh` exit 75). Main-thread busy is a share of the page thread, so load cannot fake it; frame time can. Per the owner, a frame-time red counts as load until a rerun inside `slot.sh` repeats it, so a watcher runs both kits in GPU slot 1 when `load1` drops under 14. +- 2026-10-27 (owner): "do it the flip, worker will be the default". The starter and shooter `index.html` run the worker page; `main.html` is the named override that runs the game on the page thread, documented in each kit's `AGENTS.md`. A query flag was rejected: one static page cannot pick its entry without a dynamic `import()`. The other kits follow once these two pass their scenarios through the worker. +- 2026-10-27 (lead): **the stutter left on the worker page was the engine raycast.** A DevTools trace of the shooter's worker showed 93% busy, 25% of it in `ctx.raycastAll` from the soldiers' line of sight, and `TN_FRAME_BUDGET` update p95 about 90 ms. `Mesh::raycast` tested every triangle; the merged town meshes hold about 430k. Engine layer, because every game that raycasts pays it on web and desktop, so the fix is a cached triangle tree in C++, not a game workaround. Owner rule: one PR per session, so it lands here as a phase 3 box. +- 2026-10-27 (lead): **a skinned mesh keeps the brute-force scan.** `SkinnedMesh::getVertexPosition` poses vertices through bones, so a tree of stored positions would cull a posed triangle (the red test: 1 hit brute, 0 through the tree). Out of scope and logged: the engine `SkinnedMesh` raycast still rejects rays against the bind-pose bounding sphere, which three does not, so a limb posed far from its rest pose can miss. +- 2026-10-27 (lead): **a fresh review (Codex sol 6.1, max effort) found 15 items**; each confirmed one got a red-green spec. Worker: startup errors reject `ready`, stop removes the canvas and releases lock and capture, pointer capture, `pointerlockerror`, a 0x0 container, HUD wheel, motion coalesced per page frame, one HUD snapshot in flight, gamepads forwarded, a lazy playtest mirror with `_clockEpoch`. Audio and profiler items in the next entry. +- 2026-10-27 (lead): **review, audio and profiler.** The worker clock freezes while the page context is suspended and re-anchors on each page reply. A fluent `AudioParam` call returns its own handle, so 10,000 calls keep the table constant, and an ended one-shot releases its handles. Unwritten `AudioParam` scalars read the standard default (`AudioBus` fades start from a number), and an unknown property throws instead of reading as a function. `resume`/`suspend`/`close` settle from the page's reply. `profile:wasm-page` runs census and GPU probes in the game's realm. The lead added `addEventListener`, `removeEventListener` and `createStereoPanner` to the proxy's method list, with a red-green spec on an `ended` listener. ### Phase 1: Loaded-host baseline **Status:** DONE @@ -93,14 +98,16 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo - [x] The starter plays through a loopback proxy (main thread talks to the layer over a message channel in the same thread) with unchanged behaviour. proof: `packages/core/__tests__/ui-bridge-inbound.spec.ts`; a pause round-trip scenario (click `button[aria-pressed]`, `hud` asserts `pause` -> `resume`) on a scaffolded starter at `/` and at `/?tnUiLoopback=1`; the starter's own `pnpm test` (`test:templates` skips the starter: it runs in the golden-path lane). Result: spec 6/6; round trip exit 0 on both URLs; a Playwright probe reads `__tnUiLoopback` false at `/` and true with the flag; kit `movement`, `survives`, `production-readiness` pass. `play` fails `performance.maxFrameMsP95` on GPU time on this host, and the scaffold from before this change fails it too (before 2/2 runs at GPU p95 20.3/17.3 ms, after 1/2 at 18.1 ms; CPU about 7.5 ms in both) ### Phase 3: Worker mode and v5 KPI -**Status:** NOT STARTED +**Status:** PARTIAL - [ ] Behind a flag, without COOP/COEP, the starter boots in a worker on an `OffscreenCanvas`; input, pointer lock and audio forward from the main thread, and the HUD takes one state snapshot per frame and sends intents back. proof: `pnpm test:templates` with the flag, starter scenario green - [ ] The playtest harness captures the worker canvas (screenshot not blank, frame counter advances). proof: `pnpm test:templates` with the flag; `node packages/playtest/dist/runner/cli.js doctor --url --text` - [ ] Main-thread busy falls below the target set in phase 1 (first target: below 0.5x of the main-thread arm) and p50 is no worse, under the same stated load. proof: `profile:wasm-page` worker URL against the main-thread `--control` URL, three runs per arm Open: busy is measured, p50 is not. Starter, desktop RTX 2080, `--load 24`, `load1` 28 to 52: main-thread busy 1.1 / 1.1 / 1.9% on `worker.html` against 57.5 / 52.8 / 45.7% on `/` (target at or below 28%). A second set at `load1` 35 to 74: 0.9 / 1.4% against 57.6 / 86.9%. Presented p50 was 80 to 101 ms in both arms at that load, and one worker run presented 268 frames in 60 s, so frame time waits for the quiet-window rerun in `slot.sh`. + Shooter (current #574 look, before the raycast tree), private Xvfb, three pairs at host load (`load1` 9 to 28) and three at `--load 24` (`load1` 16 to 45): page-thread busy 3.3 / 4.2 / 3.6% and 2.2 / 1.7 / 1.1% on the worker against 65.9 / 88.0 / 86.7% and 95.6 / 99.1 / 92.2% on the page. Presented p50 45 to 69 ms (worker) against 38 to 50 ms (page) at host load, and p95 worse in 4 of 6 pairs. The game's own 90 ms update spikes and a concurrent C++ build dominated; rerun on the raycast-tree build. - [ ] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR - [ ] The shooter kit plays through the worker with the same checks. proof: `pnpm test:templates` with the flag on the shooter scenario +- [x] The shooter's frame on the worker page has no update spikes from scene ray queries: the engine raycast reads a cached triangle tree with hits identical to the brute-force scan. proof: `tn-native-engine-raycast-test` and `tn-native-engine-animation-skeleton-test raycast_posed_skin` in `packages/runtime-native/build/tn-engine-core` (`-DTN_ENGINE_CORE_ONLY=ON`); the shooter's `TN_FRAME_BUDGET` in Chrome on the RTX 2080. Result: 2,048 oracle rays 0 differ; 1,000 rays on a 20k-triangle mesh 8,471 ms -> 6.0 ms; a focused ray tests 4 of 20,000 triangles; posed skin hits 1/1 on both paths. Shooter on `/` (worker), 8 windows of 5 s: update p95 about 90 ms -> 5 to 10 ms, presented p95 100 ms -> 16.8 ms, 28 to 37 fps -> 56 to 60 fps ### Phase 4: Wasm threads and gate **Status:** NOT STARTED diff --git a/packages/core/__tests__/worker-audio.spec.ts b/packages/core/__tests__/worker-audio.spec.ts index 0eed5a9504..b31b5ac90f 100644 --- a/packages/core/__tests__/worker-audio.spec.ts +++ b/packages/core/__tests__/worker-audio.spec.ts @@ -1,4 +1,6 @@ -import { describe, expect, it, vi } from "vitest"; +import { Object3D, AudioContext as ThreeAudioContext } from "three"; +import { afterEach, describe, expect, it, onTestFinished, vi } from "vitest"; +import { AudioBus } from "../src/audio.js"; import { createRemoteAudioContext, serveRemoteAudio } from "../src/worker-audio.js"; // PRD-752: a worker has no AudioContext. The worker gets a stand-in whose nodes are handles; each @@ -9,10 +11,20 @@ const tick = (ms = 10) => new Promise((resolve) => setTimeout(resolve, ms)); function param(value = 1) { return { - linearRampToValueAtTime: vi.fn(), - setTargetAtTime: vi.fn(), - setValueAtTime: vi.fn(), + linearRampToValueAtTime: vi.fn(function (this: unknown) { + return this; + }), + setTargetAtTime: vi.fn(function (this: unknown) { + return this; + }), + setValueAtTime: vi.fn(function (this: unknown) { + return this; + }), value, + defaultValue: value, + minValue: -2, + maxValue: 2, + cancelScheduledValues: vi.fn(), }; } @@ -23,9 +35,19 @@ function fakePage() { disconnect: vi.fn(), }); const created: Record[] = []; + const events = new EventTarget(); const context = { + addEventListener: events.addEventListener.bind(events), + removeEventListener: events.removeEventListener.bind(events), + dispatchEvent: events.dispatchEvent.bind(events), createBufferSource: vi.fn(() => { + const sourceEvents = new EventTarget(); const source = { + addEventListener: sourceEvents.addEventListener.bind(sourceEvents), + end: () => { + source.onended?.(); + sourceEvents.dispatchEvent(new Event("ended")); + }, ...node(), buffer: null as unknown, detune: param(0), @@ -52,7 +74,18 @@ function fakePage() { })), destination, listener: { positionX: param(0) }, - resume: vi.fn(async () => undefined), + resume: vi.fn(async () => { + context.state = "running"; + events.dispatchEvent(new Event("statechange")); + }), + suspend: vi.fn(async () => { + context.state = "suspended"; + events.dispatchEvent(new Event("statechange")); + }), + close: vi.fn(async () => { + context.state = "closed"; + events.dispatchEvent(new Event("statechange")); + }), sampleRate: 48_000, state: "suspended", }; @@ -62,10 +95,20 @@ function fakePage() { function connect() { const page = fakePage(); const { port1, port2 } = new MessageChannel(); + const set = vi.spyOn(Map.prototype, "set"); const stop = serveRemoteAudio(port2, page.context as unknown as BaseAudioContext); + const objects = set.mock.contexts[ + set.mock.calls.findIndex(([id, value]) => id === 0 && value === page.context) + ] as Map; + set.mockRestore(); const remote = createRemoteAudioContext(port1, { sampleRate: 48_000, currentTime: 4 }); + onTestFinished(() => { + stop(); + port1.close(); + }); return { ...page, + objects, remote, stop: () => { stop(); @@ -74,6 +117,8 @@ function connect() { }; } +afterEach(() => vi.restoreAllMocks()); + describe("remote audio for a worker game (PRD-752)", () => { it("runs a gain node's calls on the page's real node", async () => { const { created, remote, stop } = connect(); @@ -131,6 +176,17 @@ describe("remote audio for a worker game (PRD-752)", () => { stop(); }); + it("runs a worker 'ended' listener when the page's source ends", async () => { + const { created, remote, stop } = connect(); + const ended = vi.fn(); + remote.createBufferSource().addEventListener("ended", ended); + await tick(); + (created[0] as { end: () => void }).end(); + await tick(); + expect(ended).toHaveBeenCalledTimes(1); + stop(); + }); + it("moves the listener through the page's real listener", async () => { const { context, remote, stop } = connect(); remote.listener.positionX.linearRampToValueAtTime(3, 4.1); @@ -156,16 +212,149 @@ describe("remote audio for a worker game (PRD-752)", () => { port1.close(); }); - it("keeps its own clock from the page's start time and resumes the page's context", async () => { - const { context, remote, stop } = connect(); - const first = remote.currentTime; - expect(first).toBeGreaterThanOrEqual(4); + it("rejects a failed page resume and keeps the actual suspended state", async () => { + const { context, remote } = connect(); + const rejected = Promise.reject(new Error("gesture required")); + void rejected.catch(() => undefined); + context.resume.mockImplementation(() => rejected); + await expect(remote.resume()).rejects.toThrow("gesture required"); + expect(remote.state).toBe("suspended"); + }); + + it.each(["resume", "suspend", "close"] as const)( + "waits for the page's %s result and actual state", + async (method) => { + const { context, remote } = connect(); + await tick(); + let finish: (() => void) | undefined; + context[method].mockImplementation( + () => + new Promise((resolve) => { + finish = () => { + context.state = + method === "resume" ? "running" : method === "close" ? "closed" : "suspended"; + resolve(); + }; + }), + ); + const settled = vi.fn(); + const pending = remote[method]().then(settled); + await tick(); + expect(settled).not.toHaveBeenCalled(); + expect(remote.state).toBe("suspended"); + finish?.(); + await pending; + expect(remote.state).toBe(context.state); + }, + ); + + it("reads numeric AudioParam defaults, preserves writes, and rejects unknown properties", async () => { + const { remote } = connect(); + const gain = remote.createGain().gain; + expect(gain.value).toBe(1); + expect(gain.defaultValue).toBe(1); + gain.value = 0.75; + await tick(); + expect(gain.value).toBe(0.75); + expect(gain.minValue).toBe(-2); + expect(gain.maxValue).toBe(2); + expect(() => Reflect.get(gain, "missingScalar")).toThrow(/unknown.*missingScalar/i); + }); + + it("runs AudioBus fades through the proxy with a numeric starting gain", async () => { + const { remote, created } = connect(); + vi.spyOn(ThreeAudioContext, "getContext").mockReturnValue(remote); + const bus = new AudioBus({ camera: new Object3D(), source: () => undefined }); + onTestFinished(() => bus.dispose()); + bus.setVolume(0.25, 0.5); + await tick(); + const gain = (created[0] as { gain: ReturnType }).gain; + expect(gain.setValueAtTime).toHaveBeenCalledWith(1, 4); + expect(gain.linearRampToValueAtTime).toHaveBeenCalledWith(0.25, 4.5); + }); + + it("keeps the handle table constant across 10,000 fluent AudioParam calls", async () => { + const { remote, objects } = connect(); + const gain = remote.createGain().gain; + await tick(); + const before = objects.size; + let returned: AudioParam | undefined; + for (let i = 0; i < 10_000; i += 1) returned = gain.setValueAtTime(i, 4); await tick(30); - expect(remote.currentTime).toBeGreaterThan(first); - expect(remote.sampleRate).toBe(48_000); + expect(objects.size).toBe(before); + expect(returned).toBe(gain); + }); + + it("releases overwritten, cleared and fired one-shot callback handles", async () => { + const { remote, created } = connect(); + const source = remote.createBufferSource(); + const first = vi.fn(); + const set = vi.spyOn(Map.prototype, "set"); + source.onended = first; + const functions = set.mock.contexts[ + set.mock.calls.findIndex(([id, value]) => typeof id === "number" && value === first) + ] as Map; + set.mockRestore(); + source.onended = vi.fn(); + expect(functions.size).toBe(1); + source.onended = null; + expect(functions.size).toBe(0); + const ended = vi.fn(); + source.onended = ended; + await tick(); + (created[0] as { end(): void }).end(); + await tick(); + expect(ended).toHaveBeenCalledTimes(1); + expect(functions.size).toBe(0); + }); + + it("releases ended source handles and their params without dropping reusable gains", async () => { + const { remote, created, objects } = connect(); + const gain = remote.createGain(); + gain.gain.setValueAtTime(1, 4); + await tick(); + const before = objects.size; + const source = remote.createBufferSource(); + source.playbackRate.setValueAtTime(1, 4); + source.connect(gain); + await tick(); + const real = created[1] as { end(): void; disconnect: ReturnType }; + expect([...objects.values()]).toContain(real); + real.end(); + await tick(); + expect([...objects.values()]).not.toContain(real); + expect(real.disconnect).toHaveBeenCalledTimes(1); + expect(objects.size).toBe(before); + gain.disconnect(); + gain.connect(remote.destination); + await tick(); + expect([...objects.values()]).toContain(created[0]); + }); + + it("freezes while suspended and re-anchors to page state changes and batches", async () => { + let now = 1000; + vi.spyOn(performance, "now").mockImplementation(() => now); + const { context, remote } = connect(); + await tick(); + expect(remote.currentTime).toBe(4); + now += 10_000; + expect(remote.currentTime).toBe(4); await remote.resume(); await tick(); - expect(context.resume).toHaveBeenCalledTimes(1); - stop(); + expect(remote.state).toBe("running"); + expect(remote.currentTime).toBe(4); + now += 1000; + expect(remote.currentTime).toBe(5); + context.currentTime = 4.25; + remote.createGain(); + await tick(); + expect(remote.currentTime).toBe(4.25); + context.state = "suspended"; + context.dispatchEvent(new Event("statechange")); + await tick(); + now += 10_000; + expect(remote.currentTime).toBe(4.25); + expect(remote.state).toBe("suspended"); + expect(remote.sampleRate).toBe(48_000); }); }); diff --git a/packages/core/__tests__/worker.spec.ts b/packages/core/__tests__/worker.spec.ts index d0e760acb3..9bd24046e6 100644 --- a/packages/core/__tests__/worker.spec.ts +++ b/packages/core/__tests__/worker.spec.ts @@ -39,8 +39,22 @@ function page() { style: {} as Record, transferControlToOffscreen: () => offscreen, }); - const container = { append: vi.fn() }; + const children: unknown[] = []; + const container = { append: vi.fn((child: unknown) => children.push(child)), children }; + Object.assign(canvas, { + remove: () => { + const index = children.indexOf(canvas); + if (index >= 0) children.splice(index, 1); + }, + }); + let nextFrame = 0; + const frames = new Map(); const view = Object.assign(new EventTarget(), { + requestAnimationFrame: (callback: FrameRequestCallback) => { + frames.set(++nextFrame, callback); + return nextFrame; + }, + cancelAnimationFrame: (id: number) => frames.delete(id), devicePixelRatio: 2, location: { href: "http://127.0.0.1:5173/game/index.html?tnWorker=1" }, }) as unknown as EventTarget & Record; @@ -57,7 +71,19 @@ function page() { disconnect() {} }, ); - return { canvas, container, offscreen, resize: () => resize?.(), view }; + return { + canvas, + container, + offscreen, + resize: () => resize?.(), + view, + frame: () => { + const callbacks = [...frames.values()]; + frames.clear(); + for (const callback of callbacks) callback(0); + }, + frames, + }; } function keyEvent(type: string, code: string): Event { @@ -69,6 +95,401 @@ afterEach(() => { }); describe("worker game host (PRD-752)", () => { + it("forwards gamepad snapshots each connected frame and stops polling on disconnect or stop", async () => { + const { scope, worker, toWorker } = link(); + const { container, view, frame, frames } = page(); + const pad = { + id: "controller", + index: 1, + connected: true, + mapping: "standard", + timestamp: 10, + buttons: [{ pressed: true, value: 1, touched: true }], + axes: [0.5, -0.25], + }; + let pads: (typeof pad | null)[] = [null, pad]; + const getGamepads = vi.fn(() => pads); + view.navigator = { getGamepads }; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const platform = scope.__tnGamePlatform as IGamePlatformSource; + try { + getGamepads.mockClear(); + frame(); + await tick(); + expect(platform.input()).toEqual([ + null, + { + id: "controller", + index: 1, + connected: true, + mapping: "standard", + timestamp: 10, + buttons: [{ pressed: true, value: 1, touched: true }], + axes: [0.5, -0.25], + }, + ]); + expect(getGamepads).toHaveBeenCalledTimes(1); + const button = pad.buttons[0]; + if (button !== undefined) button.pressed = false; + pad.axes[0] = -1; + expect(platform.input()[1]?.buttons[0]?.pressed).toBe(true); + frame(); + await tick(); + expect(platform.input()[1]?.buttons[0]?.pressed).toBe(false); + pads = [null, null]; + view.dispatchEvent(new Event("gamepaddisconnected")); + await tick(); + expect(platform.input()).toEqual([null, null]); + expect(frames.size).toBe(0); + pads = [null, pad]; + view.dispatchEvent(new Event("gamepadconnected")); + await tick(); + expect(frames.size).toBe(1); + handle.stop(); + const sent = toWorker.length; + frame(); + view.dispatchEvent(new Event("gamepadconnected")); + expect(frames.size).toBe(0); + expect(toWorker).toHaveLength(sent); + } finally { + handle.stop(); + } + }); + + it("bounds HUD snapshots while the page is stalled and preserves non-state frames", async () => { + const { scope, worker } = link(); + const { container, view } = page(); + const received = vi.fn(); + const gameReceive = vi.fn(); + view.__tnUiReceive = received; + scope.__tnUiGameReceive = gameReceive; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const posts = vi.spyOn(MessagePort.prototype, "postMessage"); + try { + const post = scope.__tnUiPost as (frame: string) => void; + for (let score = 0; score < 100; score++) + post(JSON.stringify({ type: "tn:state", state: { score } })); + post('{"type":"tn:notice","value":1}'); + post('{"type":"tn:notice","value":2}'); + post("opaque legacy frame"); + const sentStates = posts.mock.calls.filter( + ([frame]) => typeof frame === "string" && frame.includes('"tn:state"'), + ).length; + await tick(30); + expect(sentStates).toBe(1); + expect( + received.mock.calls + .map(([frame]) => frame) + .filter((frame: string) => frame.includes('"tn:state"')), + ).toEqual([ + '{"type":"tn:state","state":{"score":0}}', + '{"type":"tn:state","state":{"score":99}}', + ]); + expect( + received.mock.calls + .map(([frame]) => frame) + .filter((frame: string) => !frame.includes('"tn:state"')), + ).toEqual([ + '{"type":"tn:notice","value":1}', + '{"type":"tn:notice","value":2}', + "opaque legacy frame", + ]); + expect(gameReceive).not.toHaveBeenCalled(); + } finally { + posts.mockRestore(); + handle.stop(); + } + }); + + it.each(["pointermove", "mousemove"])( + "coalesces 100 %s events per frame and flushes motion before edges", + async (type) => { + const { scope, worker, toWorker } = link(); + const { canvas, container, view, frame, frames } = page(); + const bounds = vi.spyOn(canvas, "getBoundingClientRect"); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const moves = () => { + for (let i = 0; i < 100; i++) + (type === "pointermove" ? canvas : view).dispatchEvent( + Object.assign(new Event(type), { + pointerId: 1, + movementX: 1, + movementY: -2, + clientX: 10 + i, + clientY: 20 + i, + }), + ); + }; + const sent: unknown[] = []; + try { + moves(); + const beforeFrame = toWorker.filter((message) => message.type === "tn-worker-event").length; + frame(); + sent.push(...toWorker.filter((message) => message.type === "tn-worker-event")); + expect(beforeFrame).toBe(0); + expect(sent).toHaveLength(1); + expect(sent[0]).toMatchObject({ + event: { + type, + movementX: 100, + movementY: -200, + clientX: type === "pointermove" ? 99 : 109, + clientY: type === "pointermove" ? 99 : 119, + }, + }); + expect(bounds.mock.calls.length).toBe(type === "pointermove" ? 1 : 0); + moves(); + canvas.dispatchEvent(Object.assign(new Event("pointerdown"), { pointerId: 1 })); + expect( + toWorker.filter((message) => message.type === "tn-worker-event").slice(-2), + ).toMatchObject([ + { event: { type, movementX: 100, movementY: -200 } }, + { event: { type: "pointerdown" } }, + ]); + moves(); + handle.stop(); + frame(); + expect(frames.size).toBe(0); + expect(toWorker.filter((message) => message.type === "tn-worker-event")).toHaveLength(3); + } finally { + handle.stop(); + } + }, + ); + + it("keeps HUD wheel events out of the game, matching InputMap's target rule", async () => { + const { scope, worker } = link(); + const { canvas, container, view } = page(); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const platform = scope.__tnGamePlatform as IGamePlatformSource; + const received: number[] = []; + platform.inputTarget?.addEventListener("wheel", (event) => + received.push((event as WheelEvent).deltaY), + ); + for (const [target, deltaY] of [ + [{}, 10], + [canvas, 20], + [view, 30], + ] as const) { + const event = Object.assign(new Event("wheel"), { deltaY }); + Object.defineProperty(event, "target", { value: target }); + view.dispatchEvent(event); + } + await tick(); + handle.stop(); + expect(received).toEqual([20, 30]); + }); + + it("clamps a hidden canvas to one pixel at init and resize", async () => { + const { scope, worker } = link(); + const { canvas, container, offscreen, resize } = page(); + canvas.clientWidth = 0; + canvas.clientHeight = 0; + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const platform = scope.__tnGamePlatform as IGamePlatformSource; + const initial = platform.viewport.readSize(offscreen as HTMLCanvasElement); + canvas.clientWidth = 100; + resize(); + await tick(); + handle.stop(); + expect(initial).toEqual({ width: 1, height: 1, aspect: 1 }); + expect(platform.renderer.readSize(offscreen as HTMLCanvasElement)).toEqual([100, 1]); + expect(platform.viewport.readSize(offscreen as HTMLCanvasElement).aspect).toBe(100); + }); + + it("reports a pointerlockerror-only refusal to the worker and detaches on stop", async () => { + const { scope, worker } = link(); + const { canvas, container, offscreen } = page(); + Object.assign(canvas, { requestPointerLock: () => undefined }); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const changes: boolean[] = []; + offscreen.addEventListener("pointerlockchange", (event) => + changes.push((event as Event & { captured: boolean }).captured), + ); + (offscreen as unknown as HTMLCanvasElement).requestPointerLock(); + await tick(); + globalThis.document.dispatchEvent(new Event("pointerlockerror")); + await tick(); + handle.stop(); + globalThis.document.dispatchEvent(new Event("pointerlockerror")); + await tick(); + expect(changes).toEqual([false]); + }); + + it.each(["pointerup", "pointercancel", "stop"])( + "captures drags outside the canvas and releases on %s", + async (end) => { + const { scope, worker } = link(); + const { canvas, container, offscreen } = page(); + const captured = new Set(); + Object.assign(canvas, { + setPointerCapture: (id: number) => captured.add(id), + releasePointerCapture: (id: number) => captured.delete(id), + }); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + const received = vi.fn(); + offscreen.addEventListener(end, received); + canvas.dispatchEvent(Object.assign(new Event("pointerdown"), { pointerId: 7 })); + const wasCaptured = captured.has(7); + if (end === "stop") handle.stop(); + else { + // The browser retargets an outside release to the element holding pointer capture. + (captured.has(7) ? canvas : new EventTarget()).dispatchEvent( + Object.assign(new Event(end), { pointerId: 7 }), + ); + await tick(); + handle.stop(); + } + expect(wasCaptured).toBe(true); + expect(captured.size).toBe(0); + if (end !== "stop") expect(received).toHaveBeenCalledTimes(1); + }, + ); + + it.each([true, false])( + "removes the canvas on stop and exits only its own pointer lock (%s)", + async (ownsLock) => { + const { scope, worker } = link(); + const { canvas, container } = page(); + const documentHost = globalThis.document; + const exitPointerLock = vi.fn(); + Object.assign(documentHost, { pointerLockElement: ownsLock ? canvas : {}, exitPointerLock }); + serveWorkerGame({ start: async () => undefined }, scope); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + handle.stop(); + expect(container.children).toEqual([]); + expect(exitPointerLock).toHaveBeenCalledTimes(ownsLock ? 1 : 0); + }, + ); + + it("resolves late bridge methods and live fields, including the runner's clock epoch", async () => { + const { scope, worker } = link(); + const { container, view } = page(); + const remote: Record = { describe: () => ({}) }; + serveWorkerGame( + { + start: async () => { + scope.__THREENATIVE_PLAYTEST_BRIDGE__ = remote; + }, + }, + scope, + ); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + await handle.ready; + let epoch = 0; + Object.defineProperty(remote, "_clockEpoch", { get: () => epoch }); + remote._setClockMode = () => { + epoch += 1; + return { epoch }; + }; + remote.sample = () => ({ clock: { mode: "wall-clock" } }); + const bridge = view.__THREENATIVE_PLAYTEST_BRIDGE__ as { + _clockEpoch: number; + _setClockMode(mode: string): Promise; + sample(): Promise; + $get(field: string): Promise; + }; + try { + expect(typeof bridge._setClockMode).toBe("function"); + await expect(bridge._setClockMode("wall-clock")).resolves.toEqual({ epoch: 1 }); + expect(bridge._clockEpoch).toBe(1); + await expect(bridge.$get("_clockEpoch")).resolves.toBe(1); + const before = bridge._clockEpoch; + epoch += 1; + await bridge.sample(); + expect(before === bridge._clockEpoch).toBe(false); + } finally { + handle.stop(); + } + }); + + it.each(["error", "messageerror", "stop"])( + "rejects ready and pending RPCs on %s and detaches listeners", + async (failure) => { + const { scope, worker, toWorker } = link(); + const { container, view } = page(); + serveWorkerGame( + { + start: () => { + scope.__THREENATIVE_PLAYTEST_BRIDGE__ = { wait: () => new Promise(() => {}) }; + return new Promise(() => {}); + }, + }, + scope, + ); + const handle = startWorkerGame( + worker as unknown as Worker, + container as unknown as HTMLElement, + ); + const errors: string[] = []; + void handle.ready.catch((error: Error) => errors.push(error.message)); + await tick(20); + const bridge = view.__THREENATIVE_PLAYTEST_BRIDGE__ as { wait(): Promise }; + void bridge.wait().catch((error: Error) => errors.push(error.message)); + await tick(); + if (failure === "stop") handle.stop(); + else + worker.dispatchEvent(Object.assign(new Event(failure), { message: "module load failed" })); + await tick(); + const sent = toWorker.length; + worker.dispatchEvent(new MessageEvent("message", { data: { type: "tn-worker-ready" } })); + await tick(); + handle.stop(); + expect(errors).toEqual( + failure === "stop" + ? [ + "TN_WORKER_STOPPED: the worker game stopped.", + "TN_WORKER_STOPPED: the worker game stopped.", + ] + : ["module load failed", "module load failed"], + ); + expect(worker.terminate).toHaveBeenCalledTimes(1); + expect(toWorker).toHaveLength(sent); + }, + ); + it("sends the canvas only after the worker reports ready", async () => { // A message sent while the worker module is still in its top-level await reaches no handler // and is lost, so the page waits for the worker's own ready message. @@ -461,7 +882,7 @@ describe("worker game host (PRD-752)", () => { const { scope, worker } = link(); const { container, view } = page(); const created: unknown[] = []; - view.AudioContext = class { + view.AudioContext = class extends EventTarget { currentTime = 0; destination = {}; listener = {}; diff --git a/packages/core/src/worker-audio.ts b/packages/core/src/worker-audio.ts index 336a7fdd1d..bf51452479 100644 --- a/packages/core/src/worker-audio.ts +++ b/packages/core/src/worker-audio.ts @@ -9,6 +9,12 @@ interface IRemoteAudioInit { readonly currentTime: number; } +interface IClockReply { + readonly op: "clock"; + readonly currentTime: number; + readonly state: AudioContextState; +} + interface IGetOp { readonly op: "get"; readonly id: number; @@ -41,9 +47,23 @@ interface IDecodeOp { interface IFireOp { readonly op: "fire"; readonly fn: number; + readonly once?: boolean; } -type PageOp = IGetOp | ICallOp | ISetOp | IDecodeOp; +interface IControlOp { + readonly op: "control"; + readonly method: "resume" | "suspend" | "close"; + readonly req: number; +} +interface IControlReply { + readonly op: "controlled"; + readonly req: number; + readonly currentTime: number; + readonly state: AudioContextState; + readonly message?: string; +} + +type PageOp = IGetOp | ICallOp | ISetOp | IDecodeOp | IControlOp; interface IDecodedReply { readonly op: "decoded"; @@ -72,42 +92,111 @@ interface IDecodeSlot { * Keys whose page value is a node or an `AudioParam`. Reading one yields a child handle instead of * a call, because `gain.gain` and `source.buffer` are objects to drive, not functions to run. */ +const PARAM_DEFAULTS: Readonly> = { + gain: 1, + playbackRate: 1, + detune: 0, + positionX: 0, + positionY: 0, + positionZ: 0, + orientationX: 1, + orientationY: 0, + orientationZ: 0, + forwardX: 0, + forwardY: 0, + forwardZ: -1, + upX: 0, + upY: 1, + upZ: 0, + threshold: -24, + knee: 30, + ratio: 12, + attack: 0.003, + release: 0.25, + frequency: 350, +}; const OBJECT_KEYS: ReadonlySet = new Set([ "destination", "listener", - "gain", - "playbackRate", - "detune", - "positionX", - "positionY", - "positionZ", - "orientationX", - "orientationY", - "orientationZ", - "forwardX", - "forwardY", - "forwardZ", - "upX", - "upY", - "upZ", - "threshold", - "knee", - "ratio", - "attack", - "release", - "frequency", "Q", + ...Object.keys(PARAM_DEFAULTS), +]); + +const PARAM_METHODS: ReadonlySet = new Set([ + "setValueAtTime", + "linearRampToValueAtTime", + "exponentialRampToValueAtTime", + "setTargetAtTime", + "setValueCurveAtTime", + "cancelScheduledValues", + "cancelAndHoldAtTime", +]); + +const METHODS: ReadonlySet = new Set([ + ...PARAM_METHODS, + "connect", + "disconnect", + "addEventListener", + "removeEventListener", + "start", + "stop", + "setPosition", + "setOrientation", + "setVelocity", + "setPeriodicWave", + "getChannelData", + "copyFromChannel", + "copyToChannel", + "getFloatFrequencyData", + "getByteFrequencyData", + "getFloatTimeDomainData", + "getByteTimeDomainData", + "getFrequencyResponse", + "createGain", + "createBufferSource", + "createPanner", + "createStereoPanner", + "createBiquadFilter", + "createDynamicsCompressor", + "createOscillator", + "createConstantSource", + "createAnalyser", + "createBuffer", + "createChannelMerger", + "createChannelSplitter", + "createConvolver", + "createDelay", + "createIIRFilter", + "createMediaElementSource", + "createMediaStreamSource", + "createMediaStreamDestination", + "createPeriodicWave", + "createScriptProcessor", + "createWaveShaper", ]); +interface IParamReply { + readonly op: "param"; + readonly id: number; + readonly values: Record; +} +interface IReleasedReply { + readonly op: "released"; + readonly id: number; +} + /** Worker side: a page-side `AudioContext`, with every node replaced by a proxy handle. */ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioInit): AudioContext { - const createdAt = performance.now(); + let anchoredAt = performance.now(); + let currentTime = init.currentTime; + let state: AudioContextState = "suspended"; const functions = new Map unknown>(); const handleIds = new WeakMap(); const slots = new Map(); + const controls = new Map void; reject: (error: Error) => void }>(); + const params = new Map; written: Set }>(); let nextId = 1; let nextFnId = 1; - let running = false; // One message per task: AudioListener alone moves nine AudioParams a frame. let queue: PageOp[] = []; @@ -137,36 +226,65 @@ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioIn return value; } - function readKey(id: number, cache: Map, key: string): unknown { + function readKey( + id: number, + cache: Map, + key: string, + proxy: Record, + ): unknown { const override = id === 0 ? contextValue(key) : undefined; if (override !== undefined) return override.value; if (cache.has(key)) return cache.get(key); if (OBJECT_KEYS.has(key)) { const out = nextId++; - const child = makeHandle(out); + const value = key === "Q" ? 1 : PARAM_DEFAULTS[key]; + // Standard defaults are available in this task; the page sends the actual param bounds. + const child = makeHandle( + out, + value === undefined ? undefined : { value, defaultValue: value }, + ); cache.set(key, child); post({ key, op: "get", id, out }); return child; } - // ponytail: a property the worker never wrote reads as a function, not the page's value. + if (!METHODS.has(key)) + throw new Error(`Remote AudioContext: unknown property "${key}" on handle ${id}`); return (...args: unknown[]) => { - const out = nextId++; + const fluent = PARAM_METHODS.has(key); + if (fluent && !key.startsWith("cancel") && typeof args[0] === "number") { + cache.set("value", args[0]); + params.get(id)?.written.add("value"); + } + const destination = key === "connect" ? args[0] : undefined; + const existing = + destination !== null && typeof destination === "object" + ? handleIds.get(destination) + : undefined; + const out = fluent ? id : (existing ?? nextId++); post({ args: args.map(encode), method: key, op: "call", id, out }); - return makeHandle(out); + return fluent ? proxy : existing === undefined ? makeHandle(out) : destination; }; } function makeHandle(id: number, seed?: Record): Record { const cache = new Map(Object.entries(seed ?? {})); + const written = new Set(); + let endedFn: number | undefined; + if (typeof seed?.value === "number") params.set(id, { cache, written }); const proxy = new Proxy({} as Record, { get: (_target, key) => { if (typeof key === "symbol" || key === "then") return undefined; - return readKey(id, cache, key); + return readKey(id, cache, key, proxy); }, has: (_target, key) => typeof key === "string" && cache.has(key), set: (_target, key, value) => { if (typeof key === "symbol") return true; + if (key === "onended") { + if (endedFn !== undefined) functions.delete(endedFn); + endedFn = typeof value === "function" ? nextFnId : undefined; + } cache.set(key, value); + written.add(key); post({ key, op: "set", id, value: encode(value) }); return true; }, @@ -175,9 +293,12 @@ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioIn return proxy; } - function control(method: string): Promise { - post({ args: [], method, op: "call", id: 0, out: nextId++ }); - return Promise.resolve(); + function control(method: IControlOp["method"]): Promise { + const req = nextId++; + return new Promise((resolve, reject) => { + controls.set(req, { resolve, reject }); + post({ op: "control", method, req }); + }); } function decodeAudioData( @@ -197,32 +318,17 @@ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioIn function contextValue(key: string): { readonly value: unknown } | undefined { switch (key) { case "currentTime": - return { value: init.currentTime + (performance.now() - createdAt) / 1000 }; + return { + value: currentTime + (state === "running" ? (performance.now() - anchoredAt) / 1000 : 0), + }; case "sampleRate": return { value: init.sampleRate }; case "state": - return { value: running ? "running" : "suspended" }; + return { value: state }; case "resume": - return { - value: () => { - running = true; - return control("resume"); - }, - }; case "suspend": - return { - value: () => { - running = false; - return control("suspend"); - }, - }; case "close": - return { - value: () => { - running = false; - return control("close"); - }, - }; + return { value: () => control(key) }; case "decodeAudioData": return { value: decodeAudioData }; default: @@ -231,7 +337,39 @@ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioIn } port.onmessage = (event: MessageEvent) => { - const reply = event.data as IDecodedReply | IDecodeFailedReply | IFireOp | undefined; + const reply = event.data as + | IDecodedReply + | IDecodeFailedReply + | IFireOp + | IClockReply + | IParamReply + | IReleasedReply + | IControlReply + | undefined; + if (reply?.op === "param") { + const param = params.get(reply.id); + if (param) + for (const [key, value] of Object.entries(reply.values)) { + if (!param.written.has(key)) param.cache.set(key, value); + } + return; + } + if (reply?.op === "released") { + params.delete(reply.id); + return; + } + if (reply?.op === "clock" || reply?.op === "controlled") { + currentTime = reply.currentTime; + state = reply.state; + anchoredAt = performance.now(); + if (reply.op === "controlled") { + const pending = controls.get(reply.req); + controls.delete(reply.req); + if (reply.message === undefined) pending?.resolve(); + else pending?.reject(new Error(reply.message)); + } + return; + } if (reply?.op === "decoded") { const slot = slots.get(reply.req); if (slot === undefined) return; @@ -257,7 +395,9 @@ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioIn } if (reply !== undefined && "fn" in reply) { const fired = reply as IFireOp; - functions.get(fired.fn)?.(); + const fn = functions.get(fired.fn); + if (fired.once) functions.delete(fired.fn); + fn?.(); } }; @@ -268,15 +408,22 @@ export function createRemoteAudioContext(port: MessagePort, init: IRemoteAudioIn export function serveRemoteAudio(port: MessagePort, context: BaseAudioContext): () => void { const objects = new Map([[0, context]]); const warned = new Set(); + const parents = new Map(); let stopped = false; + const syncClock = (): void => { + if (!stopped) + port.postMessage({ op: "clock", currentTime: context.currentTime, state: context.state }); + }; + context.addEventListener("statechange", syncClock); + syncClock(); - function decodeValue(value: unknown): unknown { - if (Array.isArray(value)) return value.map(decodeValue); + function decodeValue(value: unknown, once = false): unknown { + if (Array.isArray(value)) return value.map((entry) => decodeValue(entry)); if (value !== null && typeof value === "object") { const handle = (value as { $h?: number }).$h; if (typeof handle === "number") return objects.get(handle); const fn = (value as { $fn?: number }).$fn; - if (typeof fn === "number") return () => port.postMessage({ fn, op: "fire" }); + if (typeof fn === "number") return () => port.postMessage({ fn, op: "fire", once }); } return value; } @@ -311,29 +458,85 @@ export function serveRemoteAudio(port: MessagePort, context: BaseAudioContext): } } + async function serveControl(op: IControlOp): Promise { + let message: string | undefined; + try { + await (context as AudioContext)[op.method](); + } catch (error) { + message = error instanceof Error ? error.message : String(error); + } + if (!stopped) + port.postMessage({ + op: "controlled", + req: op.req, + currentTime: context.currentTime, + state: context.state, + ...(message === undefined ? {} : { message }), + }); + } + port.onmessage = (event: MessageEvent) => { if (stopped) return; for (const op of (event.data as { ops: PageOp[] }).ops) run(op); + syncClock(); }; + function release(id: number): void { + objects.delete(id); + port.postMessage({ op: "released", id }); + parents.delete(id); + for (const [child, parent] of parents) if (parent === id) release(child); + } + function run(op: PageOp): void { try { switch (op?.op) { case "get": { const source = objects.get(op.id) as Record | undefined; objects.set(op.out, source?.[op.key]); + parents.set(op.out, op.id); + const param = source?.[op.key] as Record | undefined; + if (typeof param?.value === "number") { + const values = Object.fromEntries( + ["value", "defaultValue", "minValue", "maxValue"].flatMap((key) => + typeof param[key] === "number" ? [[key, param[key]]] : [], + ), + ); + port.postMessage({ op: "param", id: op.out, values }); + } break; } case "call": { const target = objects.get(op.id) as Record; const method = target[op.method] as (...args: unknown[]) => unknown; - const result = method.apply(target, op.args.map(decodeValue)); + const result = method.apply( + target, + op.args.map((arg) => decodeValue(arg)), + ); if (result !== null && typeof result === "object") objects.set(op.out, result); + if ( + ["createBufferSource", "createOscillator", "createConstantSource"].includes(op.method) + ) { + const source = result as AudioScheduledSourceNode; + source.addEventListener( + "ended", + () => { + // A one-shot cannot restart; disconnect its graph edge and release its params too. + source.disconnect(); + release(op.out); + }, + { once: true }, + ); + } break; } case "set": { const target = objects.get(op.id) as Record; - target[op.key] = decodeValue(op.value); + target[op.key] = decodeValue(op.value, op.key === "onended"); + break; + } + case "control": { + void serveControl(op); break; } case "decode": { @@ -350,6 +553,9 @@ export function serveRemoteAudio(port: MessagePort, context: BaseAudioContext): return () => { stopped = true; + context.removeEventListener("statechange", syncClock); + objects.clear(); + parents.clear(); port.onmessage = null; port.close(); }; diff --git a/packages/core/src/worker.ts b/packages/core/src/worker.ts index ff74791d3a..1e821b9f36 100644 --- a/packages/core/src/worker.ts +++ b/packages/core/src/worker.ts @@ -9,6 +9,7 @@ import type { IGamePlatformSource } from "./game.js"; import { hostDocument, hostResizeObserver, hostWindow } from "./platform.js"; +import { GAME_STATE_MESSAGE } from "./ui-bridge.js"; import { createRemoteAudioContext, serveRemoteAudio } from "./worker-audio.js"; /** The realm `serveWorkerGame` runs in. Narrowed, because a test cannot build a real worker scope. */ @@ -49,22 +50,36 @@ interface IWorkerRpcMessage { name: string; method: string; args: unknown[]; + get?: boolean; } +type WorkerGamepad = Pick & { + axes: number[]; + buttons: { pressed: boolean; value: number; touched: boolean }[]; +}; + type WorkerInbound = | IWorkerInitMessage | IWorkerResizeMessage | IWorkerEventMessage - | IWorkerRpcMessage; + | IWorkerRpcMessage + | { type: "tn-worker-gamepads"; gamepads: (WorkerGamepad | null)[] }; type WorkerOutbound = | { type: "tn-worker-ready" } - | { type: "tn-worker-mirror"; name: string; methods: string[] } + | { type: "tn-worker-mirror"; name: string; epoch?: number } | { type: "tn-worker-started" } | { type: "tn-worker-failed"; message: string } | { type: "tn-worker-pointer-lock"; lock: boolean } | { type: "tn-worker-attribute"; name: string; value: string } - | { type: "tn-worker-rpc-result"; id: number; result?: unknown; error?: string }; + | { + type: "tn-worker-rpc-result"; + id: number; + name?: string; + epoch?: number; + result?: unknown; + error?: string; + }; type PageOutbound = WorkerInbound; @@ -111,6 +126,14 @@ const CANVAS_EVENTS = [ "click", ] as const; +// `serializeUiState` writes `{ type, state }` in that order, so a state frame starts with this. A +// prefix test, not a parse: the page parses each frame once already, and that thread is the one to keep free. +const STATE_FRAME_PREFIX = `{"type":${JSON.stringify(GAME_STATE_MESSAGE)},`; + +function isStateFrame(frame: unknown): frame is string { + return typeof frame === "string" && frame.startsWith(STATE_FRAME_PREFIX); +} + /** * The game end of the UI bridge, live before the page's port arrives. * @@ -124,6 +147,14 @@ function holdUiBridge(scope: Record): { attach(port: MessagePor let port: MessagePort | undefined; const outbound: unknown[] = []; const inbound: unknown[] = []; + let snapshot: string | undefined; + let snapshotInFlight = false; + const flushSnapshot = (): void => { + if (port === undefined || snapshotInFlight || snapshot === undefined) return; + snapshotInFlight = true; + port.postMessage(snapshot); + snapshot = undefined; + }; let receive = scope.__tnUiGameReceive as ((frame: unknown) => void) | undefined; Object.defineProperty(scope, "__tnUiGameReceive", { configurable: true, @@ -134,7 +165,10 @@ function holdUiBridge(scope: Record): { attach(port: MessagePor }, }); scope.__tnUiPost = (frame: string) => { - if (port === undefined) outbound.push(frame); + if (isStateFrame(frame)) { + snapshot = frame; + flushSnapshot(); + } else if (port === undefined) outbound.push(frame); else port.postMessage(frame); }; scope.__tnUiOverlayAttached = () => true; @@ -142,10 +176,16 @@ function holdUiBridge(scope: Record): { attach(port: MessagePor attach(next: MessagePort): void { port = next; next.onmessage = (event: MessageEvent) => { + if (event.data?.type === "tn-worker-ui-ack") { + snapshotInFlight = false; + flushSnapshot(); + return; + } if (typeof receive === "function") receive(event.data); else inbound.push(event.data); }; for (const frame of outbound.splice(0)) next.postMessage(frame); + flushSnapshot(); }, }; scope.__tnWorkerUi = held; @@ -173,6 +213,7 @@ export function serveWorkerGame( const observers = new Set<() => void>(); const size = { dpr: 1, height: 1, width: 1 }; let canvas: OffscreenCanvas | undefined; + let gamepads: (WorkerGamepad | null)[] = []; const post = (message: WorkerOutbound): void => scope.postMessage(message); const ui = holdUiBridge(scope); @@ -216,7 +257,7 @@ export function serveWorkerGame( }; }; const platform: IGamePlatformSource = { - input: () => [], + input: () => gamepads, inputTarget, mountCanvas: () => undefined, renderer: { @@ -253,10 +294,8 @@ export function serveWorkerGame( // holds start until the runner calls the bridge, so a mirror sent after start never arrives. const mirror = (name: string, value: unknown): void => { if (typeof value !== "object" || value === null) return; - const methods = Object.keys(value).filter( - (key) => typeof (value as Record)[key] === "function", - ); - post({ methods, name, type: "tn-worker-mirror" }); + const epoch = (value as { _clockEpoch?: number })._clockEpoch; + post({ epoch, name, type: "tn-worker-mirror" }); }; for (const name of MIRRORED_GLOBALS) { let value = scope[name]; @@ -300,17 +339,23 @@ export function serveWorkerGame( const handleRpc = async (message: IWorkerRpcMessage): Promise => { const id = message.id; try { - const receiver = scope[message.name] as - | Record unknown> - | undefined; + const receiver = scope[message.name] as Record | undefined; const method = receiver?.[message.method]; - if (typeof method !== "function") { + if (!message.get && typeof method !== "function") { throw new Error( `TN_WORKER_RPC_UNKNOWN: '${message.name}.${message.method}' is not a function.`, ); } - const result = await method.apply(receiver, message.args); - post({ id, result, type: "tn-worker-rpc-result" }); + const result = message.get + ? method + : await (method as (...args: unknown[]) => unknown).apply(receiver, message.args); + post({ + id, + result, + name: message.name, + epoch: receiver?._clockEpoch as number | undefined, + type: "tn-worker-rpc-result", + }); } catch (error) { post({ error: error instanceof Error ? error.message : String(error), @@ -339,6 +384,9 @@ export function serveWorkerGame( case "tn-worker-rpc": void handleRpc(message); break; + case "tn-worker-gamepads": + gamepads = message.gamepads; + break; } }; @@ -374,6 +422,7 @@ export function startWorkerGame( let nextRpcId = 0; const pending = new Map(); const installed = new Set(); + const capturedPointers = new Set(); const listeners: Array<[EventTarget, string, EventListener]> = []; const send = (message: PageOutbound, transfer?: Transferable[]): void => { @@ -385,9 +434,9 @@ export function startWorkerGame( const postResize = (): void => { send({ dpr: windowHost.devicePixelRatio, - height: canvas.clientHeight, + height: Math.max(1, canvas.clientHeight), type: "tn-worker-resize", - width: canvas.clientWidth, + width: Math.max(1, canvas.clientWidth), }); }; @@ -396,6 +445,35 @@ export function startWorkerGame( listeners.push([target, type, listener]); }; + let gamepadFrame: number | undefined; + const pollGamepads = (): void => { + if (gamepadFrame !== undefined) windowHost.cancelAnimationFrame(gamepadFrame); + gamepadFrame = undefined; + if (stopped) return; + const gamepads = Array.from(windowHost.navigator?.getGamepads?.() ?? [], (pad) => + pad === null + ? null + : { + id: pad.id, + index: pad.index, + connected: pad.connected, + mapping: pad.mapping, + timestamp: pad.timestamp, + axes: Array.from(pad.axes), + buttons: Array.from(pad.buttons, ({ pressed, value, touched }) => ({ + pressed, + value, + touched, + })), + }, + ); + send({ type: "tn-worker-gamepads", gamepads }); + if (gamepads.some((pad) => pad?.connected)) + gamepadFrame = windowHost.requestAnimationFrame(pollGamepads); + }; + listen(windowHost, "gamepadconnected", pollGamepads); + listen(windowHost, "gamepaddisconnected", pollGamepads); + const eventFields = (event: Event): Record => { const source = event as unknown as Record; const fields: Record = {}; @@ -405,21 +483,77 @@ export function startWorkerGame( return fields; }; + let motionFrame: number | undefined; + const motion = new Map(); + const postEvent = (message: IWorkerEventMessage, rect?: DOMRect): void => { + const fields = message.event; + // The worker draws on the canvas, so its pointer coordinates must be canvas-relative. + if (message.target === "canvas" || fields.type === "wheel") { + const bounds = rect ?? canvas.getBoundingClientRect(); + const { clientX, clientY } = fields; + if (typeof clientX === "number") fields.clientX = clientX - bounds.left; + if (typeof clientY === "number") fields.clientY = clientY - bounds.top; + } + send(message); + }; + const flushMotion = (): void => { + if (motionFrame !== undefined) windowHost.cancelAnimationFrame(motionFrame); + motionFrame = undefined; + if (stopped) motion.clear(); + if (motion.size === 0) return; + const rect = [...motion.values()].some((message) => message.target === "canvas") + ? canvas.getBoundingClientRect() + : undefined; + for (const message of motion.values()) postEvent(message, rect); + motion.clear(); + }; const forward = (event: Event, target: "window" | "canvas"): void => { if (stopped) return; + // Match InputMap: wheel events on DOM overlays belong to the HUD. + if ( + event.type === "wheel" && + event.target != null && + event.target !== windowHost && + event.target !== canvas + ) + return; const fields = eventFields(event); - // The worker draws on the canvas, so its pointer coordinates must be canvas-relative. - if (target === "canvas" || event.type === "wheel") { - const rect = canvas.getBoundingClientRect(); - const { clientX, clientY } = fields; - if (typeof clientX === "number") fields.clientX = clientX - rect.left; - if (typeof clientY === "number") fields.clientY = clientY - rect.top; + if (event.type === "pointermove" || event.type === "mousemove") { + const key = `${target}:${event.type}:${fields.pointerId ?? ""}`; + const previous = motion.get(key)?.event; + fields.movementX = Number(previous?.movementX ?? 0) + Number(fields.movementX ?? 0); + fields.movementY = Number(previous?.movementY ?? 0) + Number(fields.movementY ?? 0); + motion.set(key, { event: fields, target, type: "tn-worker-event" }); + motionFrame ??= windowHost.requestAnimationFrame(flushMotion); + return; } - send({ event: fields, target, type: "tn-worker-event" }); + flushMotion(); + postEvent({ event: fields, target, type: "tn-worker-event" }); }; for (const type of WINDOW_EVENTS) listen(windowHost, type, (event) => forward(event, "window")); - for (const type of CANVAS_EVENTS) listen(canvas, type, (event) => forward(event, "canvas")); + const releasePointer = (id: number): void => { + if (!capturedPointers.delete(id)) return; + try { + canvas.releasePointerCapture?.(id); + } catch { + // The browser may already have released a cancelled pointer. + } + }; + for (const type of CANVAS_EVENTS) + listen(canvas, type, (event) => { + const id = (event as PointerEvent).pointerId; + if (type === "pointerdown" && canvas.setPointerCapture !== undefined) { + try { + canvas.setPointerCapture(id); + capturedPointers.add(id); + } catch { + // Synthetic events have no active browser pointer to capture. + } + } + forward(event, "canvas"); + if (type === "pointerup" || type === "pointercancel") releasePointer(id); + }); listen(canvas, "contextmenu", (event) => event.preventDefault()); const pageLock = documentHost as Document & { pointerLockElement?: Element | null }; const sendLock = (captured: boolean): void => @@ -429,6 +563,7 @@ export function startWorkerGame( type: "tn-worker-event", }); listen(pageLock, "pointerlockchange", () => sendLock(pageLock.pointerLockElement === canvas)); + listen(pageLock, "pointerlockerror", () => sendLock(false)); const pointerLock = (lock: boolean): void => { if (!lock) { pageLock.exitPointerLock?.(); @@ -480,7 +615,11 @@ export function startWorkerGame( installed.add("__tnUiOverlayAttached"); uiPort.onmessage = (event: MessageEvent) => { const receive = page.__tnUiReceive; - if (typeof receive === "function") (receive as (frame: unknown) => void)(event.data); + try { + if (typeof receive === "function") (receive as (frame: unknown) => void)(event.data); + } finally { + if (isStateFrame(event.data)) uiPort.postMessage({ type: "tn-worker-ui-ack" }); + } }; let resolveReady: () => void = () => undefined; @@ -490,21 +629,32 @@ export function startWorkerGame( rejectReady = reject; }); - const installMirror = (name: string, methods: string[]): void => { - const host: Record Promise> = {}; - for (const method of methods) { - host[method] = (...args: unknown[]) => { - if (stopped) { - return Promise.reject(new Error("TN_WORKER_STOPPED: the worker game stopped.")); - } - const id = nextRpcId++; - return new Promise((resolve, reject) => { - pending.set(id, { reject, resolve }); - send({ args, id, method, name, type: "tn-worker-rpc" }); - }); - }; - } - page[name] = host; + const mirrorEpochs = new Map(); + const installMirror = (name: string, epoch = 0): void => { + mirrorEpochs.set(name, epoch); + const call = (method: string, args: unknown[], get = false): Promise => { + if (stopped) { + return Promise.reject(new Error("TN_WORKER_STOPPED: the worker game stopped.")); + } + const id = nextRpcId++; + return new Promise((resolve, reject) => { + pending.set(id, { reject, resolve }); + send({ args, get, id, method, name, type: "tn-worker-rpc" }); + }); + }; + page[name] = new Proxy( + {}, + { + get: (_target, method) => { + // Promise resolution probes "then"; the mirror itself is not a thenable. + if (typeof method !== "string" || method === "then") return undefined; + // The runner reads the epoch synchronously around sample(); each RPC refreshes it. + if (method === "_clockEpoch") return mirrorEpochs.get(name) ?? 0; + if (method === "$get") return (field: string) => call(field, [], true); + return (...args: unknown[]) => call(method, args); + }, + }, + ); installed.add(name); }; @@ -513,11 +663,11 @@ export function startWorkerGame( canvas: offscreen, dpr: windowHost.devicePixelRatio ?? 1, globals: pageGlobals(), - height: canvas.clientHeight, + height: Math.max(1, canvas.clientHeight), pageUrl: windowHost.location.href, type: "tn-worker-init", ui: gamePort, - width: canvas.clientWidth, + width: Math.max(1, canvas.clientWidth), }; const transfer: Transferable[] = [offscreen, gamePort]; if (audioContext !== undefined && audioChannel !== undefined) { @@ -529,6 +679,7 @@ export function startWorkerGame( transfer.push(audioChannel.port2); } send(init, transfer); + pollGamepads(); }; const onWorkerMessage = (event: Event): void => { @@ -545,18 +696,20 @@ export function startWorkerGame( canvas.setAttribute(message.name, message.value); break; case "tn-worker-mirror": - installMirror(message.name, message.methods); + installMirror(message.name, message.epoch); break; case "tn-worker-started": resolveReady(); break; case "tn-worker-failed": - rejectReady(new Error(message.message)); + stop(new Error(message.message)); break; case "tn-worker-rpc-result": { const call = pending.get(message.id); if (call === undefined) break; pending.delete(message.id); + if (message.name !== undefined && message.epoch !== undefined) + mirrorEpochs.set(message.name, message.epoch); if (message.error === undefined) call.resolve(message.result); else call.reject(new Error(message.error)); break; @@ -564,16 +717,19 @@ export function startWorkerGame( } }; - const stop = (): void => { + const stop = (error = new Error("TN_WORKER_STOPPED: the worker game stopped.")): void => { if (stopped) return; stopped = true; + rejectReady(error); + if (gamepadFrame !== undefined) windowHost.cancelAnimationFrame(gamepadFrame); + flushMotion(); for (const [target, type, listener] of listeners) target.removeEventListener(type, listener); listeners.length = 0; observer?.disconnect(); - worker.removeEventListener("message", onWorkerMessage); - for (const call of pending.values()) { - call.reject(new Error("TN_WORKER_STOPPED: the worker game stopped.")); - } + if (pageLock.pointerLockElement === canvas) pointerLock(false); + for (const id of capturedPointers) releasePointer(id); + canvas.remove(); + for (const call of pending.values()) call.reject(error); pending.clear(); uiPort.onmessage = null; uiPort.close(); @@ -585,6 +741,12 @@ export function startWorkerGame( worker.terminate(); }; - worker.addEventListener("message", onWorkerMessage); + listen(worker, "message", onWorkerMessage); + const onWorkerError = (event: Event): void => { + const message = (event as ErrorEvent).message; + stop(new Error(message || `TN_WORKER_FAILED: worker ${event.type}.`)); + }; + listen(worker, "error", onWorkerError); + listen(worker, "messageerror", onWorkerError); return { ready, stop }; } diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 66c33249a2..349b6c25e2 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -47,6 +47,8 @@ vi.mock("@playwright/test", () => ({ evaluate: async (fn: (() => unknown) | string) => { if (typeof fn === "string") return state.presented.shift() ?? 0; if (fn.toString().includes("requestAdapter")) return "hardware"; + if (fn.toString().includes("__tnCallCounts")) return { perFrame: 0, top: [] }; + if (fn.toString().includes("__tnGpuCounts")) return []; if (fn.toString().includes("getCurrentTexture")) return Array.from({ length: 300 }, () => 10); if (fn.toString().includes("requestAnimationFrame")) @@ -130,6 +132,59 @@ describe("profile:wasm-page scorecard", () => { const run = await measure("http://test", options(["--url", "http://test"])); expect(run.presented.p50).toBe(16); }); + it("reads engine census and memory from the game worker", async () => { + state.memory = { wasmMemoryBytes: 1, jsHeapUsedBytes: 2 }; + state.workers = [ + { + evaluate: async (fn) => { + if (typeof fn === "string") return "object"; + if (fn.toString().includes("__tnCallCounts")) + return { perFrame: 12, top: [{ name: "Thing.draw", perFrame: 12 }] }; + if (fn.toString().includes("__tnWasmMemoryBytes")) + return { wasmMemoryBytes: 65536, jsHeapUsedBytes: 1234 }; + return Array.from({ length: 600 }, () => 16); + }, + }, + ]; + const run = await measure( + "http://test", + options(["--url", "http://test", "--calls", "--memory"]), + ); + expect(run.calls).toEqual({ perFrame: 12, top: [{ name: "Thing.draw", perFrame: 12 }] }); + expect(run.memory).toEqual({ wasmMemoryBytes: 65536, jsHeapUsedBytes: 1234 }); + }); + + it("reads GPU calls from the game worker's presentations", async () => { + state.workers = [ + { + evaluate: async (fn) => { + if (typeof fn === "string") return "object"; + if (fn.toString().includes("__tnGpuCounts")) + return [{ name: "GPUCanvasContext.getCurrentTexture", perFrame: 1 }]; + return Array.from({ length: 600 }, () => 16); + }, + }, + ]; + const run = await measure("http://test", options(["--url", "http://test", "--gpu-calls"])); + expect(run.gpu).toEqual([{ name: "GPUCanvasContext.getCurrentTexture", perFrame: 1 }]); + }); + + it("divides page CPU work by game worker presentations", async () => { + const presentations = [100, 120]; + state.workers = [ + { + evaluate: async (fn) => { + if (typeof fn === "string") + return fn === "globalThis.__tnPresented" ? presentations.shift() : "object"; + return Array.from({ length: 600 }, () => 16); + }, + }, + ]; + const run = await measure("http://test", options(["--url", "http://test", "--cpu-work"])); + expect(run.profile.work).toMatchObject({ frames: 20, busyMsPerFrame: 0.05, wallMs: 4 }); + expect(run.profile.work?.threadMsPerFrame).toBeCloseTo(0.1, 6); + }); + it.each(["wasmMemoryBytes", "jsHeapUsedBytes"])( "fails closed when %s is unavailable", async (field) => { diff --git a/scripts/profile-wasm-page.ts b/scripts/profile-wasm-page.ts index ed2b4c68bf..8d37dc0b5e 100644 --- a/scripts/profile-wasm-page.ts +++ b/scripts/profile-wasm-page.ts @@ -257,8 +257,14 @@ async function threadSeconds(cdp: CDPSession): Promise { return found.value; } -async function cpu(page: Page, seconds: number, withWork: boolean, saveAs?: string) { - const framesBefore = withWork ? await presentedFrames(page) : 0; +async function cpu( + page: Page, + realm: Pick, + seconds: number, + withWork: boolean, + saveAs?: string, +) { + const framesBefore = withWork ? await presentedFrames(realm) : 0; const cdp = await page.context().newCDPSession(page); if (withWork) await cdp.send("Performance.enable"); const threadBefore = withWork ? await threadSeconds(cdp) : 0; @@ -268,7 +274,7 @@ async function cpu(page: Page, seconds: number, withWork: boolean, saveAs?: stri await page.waitForTimeout(seconds * 1000); const { profile } = (await cdp.send("Profiler.stop")) as { profile: ICpuProfile }; if (saveAs) writeFileSync(saveAs, JSON.stringify(profile)); - const frames = withWork ? (await presentedFrames(page)) - framesBefore : 0; + const frames = withWork ? (await presentedFrames(realm)) - framesBefore : 0; const work = withWork ? { ...cpuWork(profile, frames), @@ -323,17 +329,25 @@ async function cpu(page: Page, seconds: number, withWork: boolean, saveAs?: stri }; } -/** Counts every presented frame from page load; installed by --cpu-work (CPU_WORK_COUNTER). */ -const CPU_WORK_COUNTER = `(() => { globalThis.__tnPresented = 0; - const count = () => { globalThis.__tnPresented++; requestAnimationFrame(count); }; - requestAnimationFrame(count); })();`; +/** Counts game presentations, including capped loops and worker games (--cpu-work). */ +const CPU_WORK_COUNTER = `(() => { + globalThis.__tnPresented = 0; + const proto = globalThis.GPUCanvasContext?.prototype; + if (!proto) throw new Error("TN_PROFILE_NO_WEBGPU_CANVAS: no presentation counter available"); + const original = proto.getCurrentTexture; + proto.getCurrentTexture = function (...args) { + const texture = original.apply(this, args); + globalThis.__tnPresented++; + return texture; + }; +})();`; -async function presentedFrames(page: Page) { - return Number(await page.evaluate("globalThis.__tnPresented")); +async function presentedFrames(realm: Pick) { + return Number(await realm.evaluate("globalThis.__tnPresented")); } -async function calls(page: Page) { - return page.evaluate( +async function calls(realm: Pick) { + return realm.evaluate( () => new Promise<{ perFrame: number; top: { name: string; perFrame: number }[] }>( (resolve, reject) => { @@ -402,8 +416,8 @@ const GPU_CENSUS = `(() => { * would credit the arm with the faster callback rate. The divisor is `getCurrentTexture`, which both * back ends call once per frame they present; the window runs until 120 such frames passed. */ -async function gpuCalls(page: Page) { - return page.evaluate( +async function gpuCalls(realm: Pick) { + return realm.evaluate( () => new Promise<{ name: string; perFrame: number }[]>((resolve) => { const g = globalThis as { __tnGpuCounts?: Map }; @@ -469,6 +483,10 @@ export async function measure(url: string, o: IOptions) { const reading = readHostLoad(); const hostLoad = { ...reading, absolutes: absolutesVerdict(reading) }; const thread = await gameThread(page); + if (thread.kind === "worker") { + if (o.gpuCalls) await thread.realm.evaluate(GPU_CENSUS); + if (o.cpuWork) await thread.realm.evaluate(CPU_WORK_COUNTER); + } const frame = await frames(thread.realm); const presentedFrame = await presented(thread.realm); if (o.saveProfile !== undefined) mkdirSync(o.saveProfile, { recursive: true }); @@ -476,11 +494,11 @@ export async function measure(url: string, o: IOptions) { o.saveProfile === undefined ? undefined : join(o.saveProfile, `${new URL(url).port}.cpuprofile`); - const profile = await cpu(page, o.seconds, o.cpuWork, saveAs); - const census = o.calls ? await calls(page) : undefined; - const gpu = o.gpuCalls ? await gpuCalls(page) : undefined; + const profile = await cpu(page, thread.realm, o.seconds, o.cpuWork, saveAs); + const census = o.calls ? await calls(thread.realm) : undefined; + const gpu = o.gpuCalls ? await gpuCalls(thread.realm) : undefined; const memory = o.memory - ? await page.evaluate(() => ({ + ? await thread.realm.evaluate(() => ({ jsHeapUsedBytes: (performance as { memory?: { usedJSHeapSize?: number } }).memory ?.usedJSHeapSize, wasmMemoryBytes: ( From 32a62eb9452de541a54a95901db5e52eb759a166 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 00:49:08 -0700 Subject: [PATCH 19/29] profile:wasm-page --vsync keeps the browser's frame pacing for worker A/Bs (PRD-752) The default flags unthrottle the page's requestAnimationFrame, not a worker's OffscreenCanvas frames, so a worker arm read 37 ms frames against 10 ms on the page. With vsync on, the shooter worker ran 16.7 ms frames against 33.3 ms on the page thread. Also types the memory probe's realm (Page | Worker union). --- scripts/__tests__/profile-wasm-page.spec.ts | 15 ++++++++++++++- scripts/profile-wasm-page.ts | 17 +++++++++++++---- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 349b6c25e2..0f431467ef 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -6,6 +6,7 @@ const state = vi.hoisted(() => ({ presented: [0, 0] as number[], thread: [1, 1.002] as number[], clicks: [] as string[], + launchArgs: [] as string[], workers: [] as { evaluate(fn: (() => unknown) | string): Promise }[], })); vi.mock("node:os", () => ({ @@ -13,7 +14,8 @@ vi.mock("node:os", () => ({ })); vi.mock("@playwright/test", () => ({ chromium: { - launch: async () => ({ + launch: async ({ args }: { args: string[] }) => ({ + ...((state.launchArgs = args), {}), newPage: async () => ({ on: vi.fn(), addInitScript: vi.fn(), @@ -185,6 +187,17 @@ describe("profile:wasm-page scorecard", () => { expect(run.profile.work?.threadMsPerFrame).toBeCloseTo(0.1, 6); }); + it("keeps the browser's frame pacing with --vsync and records it (a worker's frames do not unthrottle)", async () => { + state.workers = []; + const paced = await measure("http://test", options(["--url", "http://test", "--vsync"])); + expect(state.launchArgs).not.toContain("--disable-gpu-vsync"); + expect(state.launchArgs).not.toContain("--disable-frame-rate-limit"); + expect(paced.vsync).toBe(true); + const unpaced = await measure("http://test", options(["--url", "http://test"])); + expect(state.launchArgs).toContain("--disable-gpu-vsync"); + expect(unpaced.vsync).toBe(false); + }); + it.each(["wasmMemoryBytes", "jsHeapUsedBytes"])( "fails closed when %s is unavailable", async (field) => { diff --git a/scripts/profile-wasm-page.ts b/scripts/profile-wasm-page.ts index 8d37dc0b5e..8796b5bf52 100644 --- a/scripts/profile-wasm-page.ts +++ b/scripts/profile-wasm-page.ts @@ -1,5 +1,5 @@ /** - * `pnpm profile:wasm-page -- --url [--control ] [--seconds 5] [--load 0] [--calls] [--memory] [--json]` + * `pnpm profile:wasm-page -- --url [--control ] [--seconds 5] [--load 0] [--vsync] [--calls] [--memory] [--json]` * * Profiles a running game page the way a Wasm-engine performance question needs: frame time from * requestAnimationFrame, a Chrome CPU profile grouped into engine Wasm, JavaScript and WebGPU calls @@ -8,6 +8,7 @@ * frame on both pages (render passes, draws, bind groups, buffer writes). `--control` runs a second page, the same game on * three.js, through the same measurement, so the verdict is a ratio on one lane, never an absolute. * `--load ` keeps `n` busy-loop processes alive for the whole page run: the stated CPU load. + * `--vsync` keeps the browser's frame pacing; a worker against page-thread verdict needs it. * * It refuses a software WebGPU adapter (`--allow-software` overrides) and needs a display: run it as * `sh scripts/xvfb.sh pnpm profile:wasm-page -- ...`. Wasm function names need a module linked with @@ -44,6 +45,11 @@ interface IOptions { saveProfile?: string; /** Busy-loop processes kept running for the whole page run: the stated CPU load. */ load: number; + /** + * Keep the browser's frame pacing. Without it the page's requestAnimationFrame runs unthrottled, + * but a worker's OffscreenCanvas frames do not, so a worker against page-thread verdict needs it. + */ + vsync: boolean; } export function options(argv: readonly string[]): IOptions { @@ -54,7 +60,7 @@ export function options(argv: readonly string[]): IOptions { const url = value("url"); if (url === undefined) throw new Error( - "usage: profile:wasm-page -- --url [--control ] [--seconds 5] [--load 0] [--calls] [--memory] [--json]", + "usage: profile:wasm-page -- --url [--control ] [--seconds 5] [--load 0] [--vsync] [--calls] [--memory] [--json]", ); const seconds = Number(value("seconds") ?? 5); const warmupMs = Number(value("warmup-ms") ?? 12000); @@ -75,6 +81,7 @@ export function options(argv: readonly string[]): IOptions { memory: argv.includes("--memory"), allowSoftware: argv.includes("--allow-software"), cpuWork: argv.includes("--cpu-work"), + vsync: argv.includes("--vsync"), ...(value("save-profile") === undefined ? {} : { saveProfile: value("save-profile") as string }), @@ -444,7 +451,7 @@ async function gpuCalls(realm: Pick) { export async function measure(url: string, o: IOptions) { const browser = await chromium.launch({ headless: false, - args: [...WEBGPU_BROWSER_ARGS, ...PERFORMANCE_BROWSER_ARGS], + args: [...WEBGPU_BROWSER_ARGS, ...(o.vsync ? [] : PERFORMANCE_BROWSER_ARGS)], }); // Started after the launch, so a launch failure leaves no busy process behind. const cpuLoad = startCpuLoad(o.load); @@ -483,6 +490,7 @@ export async function measure(url: string, o: IOptions) { const reading = readHostLoad(); const hostLoad = { ...reading, absolutes: absolutesVerdict(reading) }; const thread = await gameThread(page); + const realm: Pick = thread.realm; if (thread.kind === "worker") { if (o.gpuCalls) await thread.realm.evaluate(GPU_CENSUS); if (o.cpuWork) await thread.realm.evaluate(CPU_WORK_COUNTER); @@ -498,7 +506,7 @@ export async function measure(url: string, o: IOptions) { const census = o.calls ? await calls(thread.realm) : undefined; const gpu = o.gpuCalls ? await gpuCalls(thread.realm) : undefined; const memory = o.memory - ? await thread.realm.evaluate(() => ({ + ? await realm.evaluate(() => ({ jsHeapUsedBytes: (performance as { memory?: { usedJSHeapSize?: number } }).memory ?.usedJSHeapSize, wasmMemoryBytes: ( @@ -522,6 +530,7 @@ export async function measure(url: string, o: IOptions) { adapter, startupMs, gameThread: thread.kind, + vsync: o.vsync, frame, presented: presentedFrame, profile, From ca2bbf0285899b011fc1445349a5c698c903377e Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 01:47:42 -0700 Subject: [PATCH 20/29] Playtest: a worker game relaunches without the unlimited frame rate (PRD-752) Under a private Xvfb the runner adds --disable-frame-rate-limit so a page game's frames escape the compositor's cadence. The same flag throttles a worker's OffscreenCanvas frames (shooter: 36 ms presented with it, 16.7 ms without). The runner now sees a worker game once the page opens and relaunches once with that thread's flags; the report says gameThread: worker. --- .../playtest/__tests__/browser-input.spec.ts | 5 +++ packages/playtest/src/runner/browser.ts | 14 ++++++-- packages/playtest/src/runner/runner.ts | 32 ++++++++++++++++++- packages/playtest/src/runner/shared.ts | 5 +++ 4 files changed, 53 insertions(+), 3 deletions(-) diff --git a/packages/playtest/__tests__/browser-input.spec.ts b/packages/playtest/__tests__/browser-input.spec.ts index 84ec028a87..4ee7f960ed 100644 --- a/packages/playtest/__tests__/browser-input.spec.ts +++ b/packages/playtest/__tests__/browser-input.spec.ts @@ -63,6 +63,11 @@ test("a frame-time assertion unthrottles only the virtual display, so a real dis // The completion window measures unthrottled completion on the session display by design. expect(measurementBrowserArgs(false, true, "real")).toEqual(PERFORMANCE_BROWSER_ARGS); expect(measurementBrowserArgs(false, false, "virtual")).toEqual([]); + // PRD-752: Xvfb does not pace a worker's OffscreenCanvas frames, and an unlimited frame rate + // throttled them (shooter: 36 ms presented against 16.7 ms), so a worker game keeps the limit. + expect(measurementBrowserArgs(true, false, "virtual", "worker")).toEqual(["--disable-gpu-vsync"]); + expect(measurementBrowserArgs(false, true, "real", "worker")).toEqual(["--disable-gpu-vsync"]); + expect(measurementBrowserArgs(true, false, "virtual", "page")).toEqual(PERFORMANCE_BROWSER_ARGS); }); test("browser module loads without Node process and keeps software adapter detection", () => { diff --git a/packages/playtest/src/runner/browser.ts b/packages/playtest/src/runner/browser.ts index 1789a803ba..d2561735fb 100644 --- a/packages/playtest/src/runner/browser.ts +++ b/packages/playtest/src/runner/browser.ts @@ -23,9 +23,19 @@ export const PERFORMANCE_BROWSER_ARGS = [ * the display's cadence: unthrottled, a game faster than 60 fps read the swapchain's back-pressure * (one ~17 ms wait every few callbacks) as a 17.5 ms interval p95. The completion window measures * unthrottled completion on the session display by design. + * + * A game drawing from a worker (PRD-752) keeps the frame-rate limit: Xvfb's compositor does not + * pace a worker's OffscreenCanvas frames, and `--disable-frame-rate-limit` throttled them (the + * shooter presented 36 ms frames with it, 16.7 ms without), the opposite of its effect on a page. */ -export function measurementBrowserArgs(performance: boolean, completion: boolean, display: "real" | "virtual"): readonly string[] { - return completion || (performance && display === "virtual") ? PERFORMANCE_BROWSER_ARGS : []; +export function measurementBrowserArgs( + performance: boolean, + completion: boolean, + display: "real" | "virtual", + gameThread: "page" | "worker" = "page", +): readonly string[] { + const args = completion || (performance && display === "virtual") ? PERFORMANCE_BROWSER_ARGS : []; + return gameThread === "worker" ? args.filter((argument) => argument !== "--disable-frame-rate-limit") : args; } export function resolveBrowserArguments(browserArgs: readonly string[] | undefined): string[] { diff --git a/packages/playtest/src/runner/runner.ts b/packages/playtest/src/runner/runner.ts index 4d60d71935..2d22c7067a 100644 --- a/packages/playtest/src/runner/runner.ts +++ b/packages/playtest/src/runner/runner.ts @@ -192,6 +192,19 @@ export interface IStandalonePlaytestRunOptions { interface IStandalonePlaytestInternalOptions extends IStandalonePlaytestRunOptions { remoteBrowser?: IRemoteBrowserSession; + /** Set on the relaunch after the page showed a game drawing from a worker (PRD-752). */ + gameThread?: "worker"; +} + +/** The page runs its game in a worker, so the run relaunches with that thread's measurement flags. */ +class WorkerGameRelaunch extends Error {} + +/** A worker host installs its platform source in the worker before the game starts (PRD-752). */ +async function runsGameInWorker(page: Page): Promise { + for (const worker of page.workers()) { + if ((await worker.evaluate("typeof globalThis.__tnGamePlatform").catch(() => undefined)) === "object") return true; + } + return false; } export async function runStandalonePlaytest( @@ -207,6 +220,18 @@ export async function runStandalonePlaytest( async function runStandalonePlaytestInternal( config: IStandalonePlaytestConfig, options: IStandalonePlaytestInternalOptions = {}, +): Promise { + try { + return await runStandalonePlaytestOnce(config, options); + } catch (error) { + if (!(error instanceof WorkerGameRelaunch)) throw error; + return runStandalonePlaytestOnce(config, { ...options, gameThread: "worker" }); + } +} + +async function runStandalonePlaytestOnce( + config: IStandalonePlaytestConfig, + options: IStandalonePlaytestInternalOptions, ): Promise { const usesFreePort = config.server !== undefined && config.port === 0; const resolvedConfig = usesFreePort ? await resolveManagedServerConfig(config) : config; @@ -230,7 +255,7 @@ async function runStandalonePlaytestInternal( let completionWindow: ICompletionWindow | undefined; const completionDisplay = completionLabel === undefined ? undefined : completionWindowEnvironment("browser", activeConfig.headless); if (completionLabel !== undefined && options.remoteBrowser !== undefined) throw new Error("TN_PLAYTEST_COMPLETION_WINDOW_FAILED: requires a fresh local browser process"); - const measurementArgs = measurementBrowserArgs(scenario.assert?.performance !== undefined, completionLabel !== undefined, frameDisplay()); + const measurementArgs = measurementBrowserArgs(scenario.assert?.performance !== undefined, completionLabel !== undefined, frameDisplay(), options.gameThread); const browserConfig = measurementArgs.length === 0 ? activeConfig : { @@ -463,6 +488,9 @@ async function runStandalonePlaytestInternal( // Started before navigation: the frames a player sees first are the ones a cover must own. const coverRecorder = scenario.assert?.startup?.steadyCover === true ? await recordStartupCover(page) : undefined; const bridge = await openRunnerPage(page, browserConfig, scenario, options.remoteBrowser); + // Browser flags are per process, so the thread the game draws on can only pick them on a relaunch. + if (options.gameThread === undefined && measurementArgs.includes("--disable-frame-rate-limit") + && await runsGameInWorker(page)) throw new WorkerGameRelaunch(); // From here on the page is expected to stay put; anything that moves it is evidence. pageLifecycle.settled = true; const setupApplication = bridge?.setupApplication; @@ -923,6 +951,7 @@ async function runStandalonePlaytestInternal( completionWindow.provenance = { requestedDisplay: completionDisplay!, qualified: false, browserArgs: resolveBrowserArguments(browserConfig.browserArgs) }; } + if (options.gameThread !== undefined) report.gameThread = options.gameThread; report.acquisition = { requestUrls: [...requestUrls], ...(cpuLabels.length === 0 ? {} : { cpuWindows }), ...(completionWindow === undefined ? {} : { completionWindow }) }; await writeObservationArtifacts(activeConfig.artifactDirectory, scenario.artifacts, { console: consoleEntries, @@ -935,6 +964,7 @@ async function runStandalonePlaytestInternal( else await page.close(); return addPreflightDiagnostic(report, preflight); } catch (error) { + if (error instanceof WorkerGameRelaunch) throw error; // Retain already-collected observations under the existing artifact policy. // A failed bridge is never queried here; artifact I/O cannot replace the primary error. try { diff --git a/packages/playtest/src/runner/shared.ts b/packages/playtest/src/runner/shared.ts index a9dc765740..a57b3412a8 100644 --- a/packages/playtest/src/runner/shared.ts +++ b/packages/playtest/src/runner/shared.ts @@ -59,6 +59,11 @@ export interface IStandalonePlaytestReport extends IPlaytestReport { * which one this was — the same field the native production profile carries. */ clock: "fixed-step" | "wall-clock"; + /** + * `worker` when the page drew its game from a worker and the run relaunched with that thread's + * measurement flags (`measurementBrowserArgs`); absent for a page-thread game. + */ + gameThread?: "worker"; /** * What the run waited for before it observed anything, and what it settled on. * From d600e7b18caee4d62eaa9aba0c428b23b5932b8a Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 01:49:01 -0700 Subject: [PATCH 21/29] PRD-752: the worker KPI box, measured with vsync on both kits --- ...eb-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 9c8653ae91..517abf5ff0 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -102,8 +102,9 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo - [ ] Behind a flag, without COOP/COEP, the starter boots in a worker on an `OffscreenCanvas`; input, pointer lock and audio forward from the main thread, and the HUD takes one state snapshot per frame and sends intents back. proof: `pnpm test:templates` with the flag, starter scenario green - [ ] The playtest harness captures the worker canvas (screenshot not blank, frame counter advances). proof: `pnpm test:templates` with the flag; `node packages/playtest/dist/runner/cli.js doctor --url --text` -- [ ] Main-thread busy falls below the target set in phase 1 (first target: below 0.5x of the main-thread arm) and p50 is no worse, under the same stated load. proof: `profile:wasm-page` worker URL against the main-thread `--control` URL, three runs per arm - Open: busy is measured, p50 is not. Starter, desktop RTX 2080, `--load 24`, `load1` 28 to 52: main-thread busy 1.1 / 1.1 / 1.9% on `worker.html` against 57.5 / 52.8 / 45.7% on `/` (target at or below 28%). A second set at `load1` 35 to 74: 0.9 / 1.4% against 57.6 / 86.9%. Presented p50 was 80 to 101 ms in both arms at that load, and one worker run presented 268 frames in 60 s, so frame time waits for the quiet-window rerun in `slot.sh`. +- [x] Main-thread busy falls below the target set in phase 1 (first target: below 0.5x of the main-thread arm) and p50 is no worse, under the same stated load. proof: `profile:wasm-page` worker URL against the main-thread `--control` URL, three runs per arm. Result: `--url / --control /main.html --cpu-work --vsync --load 0`, RTX 2080, host `load1` 7 to 24, three pairs per kit. Shooter: busy 4.5 / 4.5 / 4.5% against 32.0 / 34.8 / 34.5%; presented p50 42.8 / 45.7 / 34.2 ms against 85.7 / 71.6 / 78.5 ms; p95 69 / 57 / 63 ms against 146 / 130 / 154 ms. Starter: busy 1.4 / 1.3 / 1.5% against 24.8 / 23.0 / 23.7%; presented p50 33.4 / 33.8 / 33.3 ms against 33.6 / 33.4 / 44.7 ms. At `--load 24` (`load1` 38 to 71, unmatched arms) the shooter's worker busy stayed at 0.5 to 1.4% against 44 to 47%, and p50 was lower in 2 of 3 pairs: frame time there is load, per the AFK rule + Without `--vsync` the profiler's default flags unthrottle only the page's requestAnimationFrame, so a worker arm read 37 ms frames against 10 ms on the page; the option exists for this verdict. + Earlier, before the raycast tree: Starter, desktop RTX 2080, `--load 24`, `load1` 28 to 52: main-thread busy 1.1 / 1.1 / 1.9% on `worker.html` against 57.5 / 52.8 / 45.7% on `/` (target at or below 28%). A second set at `load1` 35 to 74: 0.9 / 1.4% against 57.6 / 86.9%. Presented p50 was 80 to 101 ms in both arms at that load, and one worker run presented 268 frames in 60 s, so frame time waits for the quiet-window rerun in `slot.sh`. Shooter (current #574 look, before the raycast tree), private Xvfb, three pairs at host load (`load1` 9 to 28) and three at `--load 24` (`load1` 16 to 45): page-thread busy 3.3 / 4.2 / 3.6% and 2.2 / 1.7 / 1.1% on the worker against 65.9 / 88.0 / 86.7% and 95.6 / 99.1 / 92.2% on the page. Presented p50 45 to 69 ms (worker) against 38 to 50 ms (page) at host load, and p95 worse in 4 of 6 pairs. The game's own 90 ms update spikes and a concurrent C++ build dominated; rerun on the raycast-tree build. - [ ] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR - [ ] The shooter kit plays through the worker with the same checks. proof: `pnpm test:templates` with the flag on the shooter scenario From 3a07599340303081587df0f7c1e09d81d4fded88 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 01:50:20 -0700 Subject: [PATCH 22/29] profile:wasm-page spec: record the launch args without a comma expression (PRD-752) --- scripts/__tests__/profile-wasm-page.spec.ts | 108 ++++++++++---------- 1 file changed, 56 insertions(+), 52 deletions(-) diff --git a/scripts/__tests__/profile-wasm-page.spec.ts b/scripts/__tests__/profile-wasm-page.spec.ts index 0f431467ef..77a87c5232 100644 --- a/scripts/__tests__/profile-wasm-page.spec.ts +++ b/scripts/__tests__/profile-wasm-page.spec.ts @@ -9,66 +9,70 @@ const state = vi.hoisted(() => ({ launchArgs: [] as string[], workers: [] as { evaluate(fn: (() => unknown) | string): Promise }[], })); +const recordLaunch = vi.hoisted(() => (args: string[], browser: T): T => { + state.launchArgs = args; + return browser; +}); vi.mock("node:os", () => ({ default: { availableParallelism: () => 8, loadavg: () => [state.load, 0, 0] }, })); vi.mock("@playwright/test", () => ({ chromium: { - launch: async ({ args }: { args: string[] }) => ({ - ...((state.launchArgs = args), {}), - newPage: async () => ({ - on: vi.fn(), - addInitScript: vi.fn(), - goto: vi.fn(), - waitForTimeout: vi.fn(), - click: async (selector: string) => { - state.clicks.push(selector); - }, - workers: () => state.workers, - context: () => ({ - newCDPSession: async () => ({ - send: async (method: string) => - method === "Performance.getMetrics" - ? // the thread's CPU seconds: 2 ms more at the second read - { metrics: [{ name: "ThreadTime", value: state.thread.shift() ?? 0 }] } - : method === "Profiler.stop" - ? { - // 1 ms busy in `work`, 3 ms in `(idle)`: a quarter of the window is CPU work. - profile: { - nodes: [ - { id: 1, callFrame: { functionName: "(idle)", url: "" } }, - { id: 2, callFrame: { functionName: "work", url: "" } }, - ], - samples: [1, 2, 1, 1], - timeDeltas: [1000, 1000, 1000, 1000], - }, - } - : {}, + launch: async ({ args }: { args: string[] }) => + recordLaunch(args, { + newPage: async () => ({ + on: vi.fn(), + addInitScript: vi.fn(), + goto: vi.fn(), + waitForTimeout: vi.fn(), + click: async (selector: string) => { + state.clicks.push(selector); + }, + workers: () => state.workers, + context: () => ({ + newCDPSession: async () => ({ + send: async (method: string) => + method === "Performance.getMetrics" + ? // the thread's CPU seconds: 2 ms more at the second read + { metrics: [{ name: "ThreadTime", value: state.thread.shift() ?? 0 }] } + : method === "Profiler.stop" + ? { + // 1 ms busy in `work`, 3 ms in `(idle)`: a quarter of the window is CPU work. + profile: { + nodes: [ + { id: 1, callFrame: { functionName: "(idle)", url: "" } }, + { id: 2, callFrame: { functionName: "work", url: "" } }, + ], + samples: [1, 2, 1, 1], + timeDeltas: [1000, 1000, 1000, 1000], + }, + } + : {}, + }), }), + evaluate: async (fn: (() => unknown) | string) => { + if (typeof fn === "string") return state.presented.shift() ?? 0; + if (fn.toString().includes("requestAdapter")) return "hardware"; + if (fn.toString().includes("__tnCallCounts")) return { perFrame: 0, top: [] }; + if (fn.toString().includes("__tnGpuCounts")) return []; + if (fn.toString().includes("getCurrentTexture")) + return Array.from({ length: 300 }, () => 10); + if (fn.toString().includes("requestAnimationFrame")) + return Array.from({ length: 600 }, (_, i) => (i === 599 ? 40 : 10)); + vi.stubGlobal("performance", { + memory: { usedJSHeapSize: state.memory.jsHeapUsedBytes }, + }); + vi.stubGlobal( + "__tnWasmMemoryBytes", + state.memory.wasmMemoryBytes === undefined + ? undefined + : () => state.memory.wasmMemoryBytes, + ); + return fn(); + }, }), - evaluate: async (fn: (() => unknown) | string) => { - if (typeof fn === "string") return state.presented.shift() ?? 0; - if (fn.toString().includes("requestAdapter")) return "hardware"; - if (fn.toString().includes("__tnCallCounts")) return { perFrame: 0, top: [] }; - if (fn.toString().includes("__tnGpuCounts")) return []; - if (fn.toString().includes("getCurrentTexture")) - return Array.from({ length: 300 }, () => 10); - if (fn.toString().includes("requestAnimationFrame")) - return Array.from({ length: 600 }, (_, i) => (i === 599 ? 40 : 10)); - vi.stubGlobal("performance", { - memory: { usedJSHeapSize: state.memory.jsHeapUsedBytes }, - }); - vi.stubGlobal( - "__tnWasmMemoryBytes", - state.memory.wasmMemoryBytes === undefined - ? undefined - : () => state.memory.wasmMemoryBytes, - ); - return fn(); - }, + close: vi.fn(), }), - close: vi.fn(), - }), }, })); From 7fe753ba5b427db59f776e13a77b5f6baee0fc9e Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 01:55:34 -0700 Subject: [PATCH 23/29] Playtest: a live-clock hold counts the ticks the game ran, not the requests (PRD-752) A wall-clock advance answers a one-tick request with the ticks the host's pump ran: two per frame at 30 fps on a 60 Hz step. advanceFixedStep counted requests, so a 169-tick key hold ran about 338 ticks and the starter's player overshot into the gap on the worker page. The bridge client now returns the reported count and the loop counts it; a fixed-step bridge still runs exactly one tick per request. Starter play passes on / (worker) and /main.html. --- packages/core/capabilities.json | 2 +- .../agent-docs/references/capability-reference.md | 2 +- packages/create-threenative/capabilities.json | 2 +- packages/playtest/__tests__/runner.spec.ts | 15 +++++++++++++++ packages/playtest/src/runner/androidRunner.ts | 6 +++--- packages/playtest/src/runner/bridgeClient.ts | 6 ++++-- packages/playtest/src/runner/steps.ts | 13 ++++++++----- scripts/api-surface.json | 2 +- 8 files changed, 34 insertions(+), 14 deletions(-) diff --git a/packages/core/capabilities.json b/packages/core/capabilities.json index d4ef6a0d5b..268d95179a 100644 --- a/packages/core/capabilities.json +++ b/packages/core/capabilities.json @@ -5689,7 +5689,7 @@ "kind": "function", "overrides": [], "package": "@threenative/playtest", - "signature": "export async function advanceFixedStep( page: Page, bridge: Pick, ticks: number, render = false, ): Promise { … }", + "signature": "export async function advanceFixedStep( page: Page, bridge: { … }", "situations": [ "run a complete browser or device playtest", "capture diagnostics and screenshots from a managed server" diff --git a/packages/create-threenative/agent-docs/references/capability-reference.md b/packages/create-threenative/agent-docs/references/capability-reference.md index 5b0c597433..9621ee6e5c 100644 --- a/packages/create-threenative/agent-docs/references/capability-reference.md +++ b/packages/create-threenative/agent-docs/references/capability-reference.md @@ -4518,7 +4518,7 @@ const adb = discoverAdb(process.env); `function` — Execute scenario steps, assertions, and evidence capture. ```ts -export async function advanceFixedStep( page: Page, bridge: Pick, ticks: number, render = false, ): Promise { … } +export async function advanceFixedStep( page: Page, bridge: { … } ``` - **Use when:** run a complete browser or device playtest · capture diagnostics and screenshots from a managed server diff --git a/packages/create-threenative/capabilities.json b/packages/create-threenative/capabilities.json index d4ef6a0d5b..268d95179a 100644 --- a/packages/create-threenative/capabilities.json +++ b/packages/create-threenative/capabilities.json @@ -5689,7 +5689,7 @@ "kind": "function", "overrides": [], "package": "@threenative/playtest", - "signature": "export async function advanceFixedStep( page: Page, bridge: Pick, ticks: number, render = false, ): Promise { … }", + "signature": "export async function advanceFixedStep( page: Page, bridge: { … }", "situations": [ "run a complete browser or device playtest", "capture diagnostics and screenshots from a managed server" diff --git a/packages/playtest/__tests__/runner.spec.ts b/packages/playtest/__tests__/runner.spec.ts index 18485366e3..b333c69d1e 100644 --- a/packages/playtest/__tests__/runner.spec.ts +++ b/packages/playtest/__tests__/runner.spec.ts @@ -357,6 +357,21 @@ test("a rendered fixed-step advance draws one world frame per tick and lets the expect(page.evaluate).toHaveBeenCalledTimes(9); }); +test("a live-clock hold counts the ticks the game ran, not the requests (PRD-752)", async () => { + // A 30 fps host on a 60 Hz step runs two ticks per frame, so each one-tick wall-clock request + // returns two. Counting requests held a 169-tick key for about 338 ticks. + let ran = 0; + const bridge = { + advance: vi.fn(async (_ticks: number) => { + ran += 2; + return 2; + }), + }; + await advanceFixedStep({} as Page, bridge, 10); + expect(bridge.advance).toHaveBeenCalledTimes(5); + expect(ran).toBe(10); +}); + test.each([0, -1, 1.5, Number.NaN])("fixed-step driving rejects invalid tick count %s", async (ticks) => { const bridge = { advance: vi.fn() }; await expect(advanceFixedStep({} as Page, bridge, ticks)).rejects.toThrow("positive integer"); diff --git a/packages/playtest/src/runner/androidRunner.ts b/packages/playtest/src/runner/androidRunner.ts index 60e94445d5..e039b00b22 100644 --- a/packages/playtest/src/runner/androidRunner.ts +++ b/packages/playtest/src/runner/androidRunner.ts @@ -270,7 +270,7 @@ async function runDevicePlaytestInternal( // this the wait reads that as a slow loading gate and burns its whole deadline; with it // the report names the exit and carries the console tail that says why. hostAlive: () => target.driver.isAlive().catch(() => undefined), - pump: () => attached.advance(1), + pump: async () => { await attached.advance(1); }, }); await throwIfAborted(target); // Device requests are polled on animation frames; observe warmup draws, as the browser does, @@ -395,7 +395,7 @@ async function runDevicePlaytestInternal( } const attachedBridge = bridge; afterInput = await waitForResource({ - advance: () => attachedBridge.advance(1), + advance: async () => { await attachedBridge.advance(1); }, id: wait.id, path: wait.path, predicate: wait, @@ -494,7 +494,7 @@ async function runDevicePlaytestInternal( const pid = target.driver.processId; if (pid === undefined) throw new Error("TN_PLAYTEST_CPU_WINDOW_FAILED: native child PID unavailable"); return { rootPid: pid, pid, tid: pid }; - }, measuredBridge, () => measuredBridge.advance(1000))); + }, measuredBridge, async () => { await measuredBridge.advance(1000); })); } finally { if (holdRender) await transport.call("_holdAutomaticRender", false); } diff --git a/packages/playtest/src/runner/bridgeClient.ts b/packages/playtest/src/runner/bridgeClient.ts index c63008bdd9..fae860f164 100644 --- a/packages/playtest/src/runner/bridgeClient.ts +++ b/packages/playtest/src/runner/bridgeClient.ts @@ -85,7 +85,8 @@ export class PlaytestBridgeError extends Error { export interface IPlaytestBridgeClient { /** `render: true` draws a world frame per tick; the default advances without drawing. Browser bridges only. */ - advance(ticks: number, options?: { render?: boolean }): Promise; + /** Resolves with the ticks the game reported running, which a live clock can put above `ticks`. */ + advance(ticks: number, options?: { render?: boolean }): Promise; applySetup(request: IPlaytestSetupRequest): Promise; close(): Promise; drainEvents(limit?: number): Promise; @@ -233,7 +234,8 @@ export async function connectPlaytestBridgeTransport( return { advance: async (ticks, options) => { // Scaled by the work requested, not fixed. See PLAYTEST_ADVANCE_TICK_BUDGET_MS. - await transport.call("advance", ticks, advanceTimeoutMs(ticks), options); + const result = await transport.call<{ ticks?: unknown } | undefined>("advance", ticks, advanceTimeoutMs(ticks), options); + return typeof result?.ticks === "number" ? result.ticks : undefined; }, applySetup: async (request) => { assertBoundedPayload(request); diff --git a/packages/playtest/src/runner/steps.ts b/packages/playtest/src/runner/steps.ts index a7f16c8e9a..d65f8a1c7e 100644 --- a/packages/playtest/src/runner/steps.ts +++ b/packages/playtest/src/runner/steps.ts @@ -602,7 +602,7 @@ async function executeAimAtStep( */ export async function advanceFixedStep( page: Page, - bridge: Pick, + bridge: { advance(ticks: number, options?: { render?: boolean }): Promise }, ticks: number, render = false, ): Promise { @@ -612,11 +612,14 @@ export async function advanceFixedStep( // work during one tick to exceed a request's bounded budget. A measurement that couples updates // to world draws also asks for a drawn frame per tick and lets the HUD paint between ticks. // Every other step advances without drawing. - for (let tick = 0; tick < ticks; tick += 1) { + // A live clock answers a one-tick request with the ticks its own pump ran: two per frame at + // 30 fps on a 60 Hz step. Count those, or a held key stays down for twice its ticks (PRD-752). + // A stalled pump reports zero and still counts as one, so the loop always ends. + for (let tick = 0; tick < ticks; ) { for (let attempt = 0; ; attempt += 1) { try { - if (render) await bridge.advance(1, { render: true }); - else await bridge.advance(1); + const advanced = render ? await bridge.advance(1, { render: true }) : await bridge.advance(1); + tick += typeof advanced === "number" && advanced > 1 ? advanced : 1; break; } catch (error) { if ( @@ -628,7 +631,7 @@ export async function advanceFixedStep( await waitFrames(page, 1); } } - if (render && tick + 1 < ticks) await waitFrames(page, 1); + if (render && tick < ticks) await waitFrames(page, 1); } } diff --git a/scripts/api-surface.json b/scripts/api-surface.json index f5b90a2128..c34c2ce8f1 100644 --- a/scripts/api-surface.json +++ b/scripts/api-surface.json @@ -300,7 +300,7 @@ "@threenative/playtest/protocol#jsonByteLength": "export function jsonByteLength(value: JsonValue): number { … }", "@threenative/playtest/protocol#requestedPlaytestClockMode": "export function requestedPlaytestClockMode(): PlaytestClockMode | undefined { … }", "@threenative/playtest/runner#AdbAndroidDriver": "export class AdbAndroidDriver implements IAndroidDriver { … }", - "@threenative/playtest/runner#advanceFixedStep": "export async function advanceFixedStep( page: Page, bridge: Pick, ticks: number, render = false, ): Promise { … }", + "@threenative/playtest/runner#advanceFixedStep": "export async function advanceFixedStep( page: Page, bridge: { … }", "@threenative/playtest/runner#advanceTimeoutMs": "export function advanceTimeoutMs( ticks: number, perTickMs: number = PLAYTEST_ADVANCE_TICK_BUDGET_MS, ): number { … }", "@threenative/playtest/runner#androidMailboxPaths": "export function androidMailboxPaths( packageName: string, root = `/sdcard/Android/data/$ { … }", "@threenative/playtest/runner#androidTouchBatches": "export function androidTouchBatches( identity: readonly string[], positions: readonly string[], ): string[][] { … }", From e4d908136b6d1543feed7f80197ce8401ba933a9 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 02:00:37 -0700 Subject: [PATCH 24/29] PRD-752: starter and harness boxes ticked; worker-in-Xvfb and live-clock hold decisions --- ...-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 517abf5ff0..4ab4718432 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -81,6 +81,8 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me - 2026-10-27 (lead): **a skinned mesh keeps the brute-force scan.** `SkinnedMesh::getVertexPosition` poses vertices through bones, so a tree of stored positions would cull a posed triangle (the red test: 1 hit brute, 0 through the tree). Out of scope and logged: the engine `SkinnedMesh` raycast still rejects rays against the bind-pose bounding sphere, which three does not, so a limb posed far from its rest pose can miss. - 2026-10-27 (lead): **a fresh review (Codex sol 6.1, max effort) found 15 items**; each confirmed one got a red-green spec. Worker: startup errors reject `ready`, stop removes the canvas and releases lock and capture, pointer capture, `pointerlockerror`, a 0x0 container, HUD wheel, motion coalesced per page frame, one HUD snapshot in flight, gamepads forwarded, a lazy playtest mirror with `_clockEpoch`. Audio and profiler items in the next entry. - 2026-10-27 (lead): **review, audio and profiler.** The worker clock freezes while the page context is suspended and re-anchors on each page reply. A fluent `AudioParam` call returns its own handle, so 10,000 calls keep the table constant, and an ended one-shot releases its handles. Unwritten `AudioParam` scalars read the standard default (`AudioBus` fades start from a number), and an unknown property throws instead of reading as a function. `resume`/`suspend`/`close` settle from the page's reply. `profile:wasm-page` runs census and GPU probes in the game's realm. The lead added `addEventListener`, `removeEventListener` and `createStereoPanner` to the proxy's method list, with a red-green spec on an `ended` listener. +- 2026-10-27 (lead): **a worker game in the private Xvfb.** The runner adds `--disable-frame-rate-limit` so a page game's frames escape Xvfb's compositor cadence; the same flag throttles a worker's `OffscreenCanvas` frames (shooter, 1280x720: presented 36 ms with it, 16.7 ms without; the page thread 9.9 ms with it, 33.3 ms without). The runner now relaunches a worker game once without it and reports `gameThread: "worker"`. At 1920x1080 the worker still waits on Xvfb's present path (22 to 24 fps with any flag set, 46 to 49 ms idle per frame) while the real GPU path runs it at 56 to 60 fps: a worker's frames wait for the compositor's acknowledgement, which no Chrome switch lifts. Open: if the quiet-window rerun repeats, measure a worker game's frame rate on a virtual display from its render work (frame time minus host gap), the reading the unthrottle flag gives a page game. +- 2026-10-27 (lead): **a live-clock hold counted requests, not ticks.** A wall-clock `advance(1)` returns the ticks the host's pump ran: two per frame at 30 fps. The runner counted each request as one tick, so the starter's 169-tick hold ran about 338 and the player fell into the gap on the worker page. Harness layer: any game under 60 fps on a live clock drifted the same way. The bridge client now returns the count and the hold loop counts it (red-green spec in `runner.spec.ts`). ### Phase 1: Loaded-host baseline **Status:** DONE @@ -100,14 +102,15 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo ### Phase 3: Worker mode and v5 KPI **Status:** PARTIAL -- [ ] Behind a flag, without COOP/COEP, the starter boots in a worker on an `OffscreenCanvas`; input, pointer lock and audio forward from the main thread, and the HUD takes one state snapshot per frame and sends intents back. proof: `pnpm test:templates` with the flag, starter scenario green -- [ ] The playtest harness captures the worker canvas (screenshot not blank, frame counter advances). proof: `pnpm test:templates` with the flag; `node packages/playtest/dist/runner/cli.js doctor --url --text` +- [x] Behind a flag, without COOP/COEP, the starter boots in a worker on an `OffscreenCanvas`; input, pointer lock and audio forward from the main thread, and the HUD takes one state snapshot per frame and sends intents back. proof: `pnpm test:templates` with the flag, starter scenario green. Result: the flag became the default page (owner, Decisions); a scaffolded starter at `/` (worker), `--browser-recipe webgpu --headed`, RTX 2080 ("turing"), the worktree's runner at 7fe753ba5: `movement`, `survives`, `play`, `production-readiness` all exit 0; `play` reports `gameThread: "worker"` +- [x] The playtest harness captures the worker canvas (screenshot not blank, frame counter advances). proof: `pnpm test:templates` with the flag; `node packages/playtest/dist/runner/cli.js doctor --url --text`. Result: the same runs capture non-blank frames with `data-engine` provenance from the page canvas (forwarded attribute) and pass their visual and tone assertions; 9 of the 10 starter and shooter scenarios exit 0 on `/` - [x] Main-thread busy falls below the target set in phase 1 (first target: below 0.5x of the main-thread arm) and p50 is no worse, under the same stated load. proof: `profile:wasm-page` worker URL against the main-thread `--control` URL, three runs per arm. Result: `--url / --control /main.html --cpu-work --vsync --load 0`, RTX 2080, host `load1` 7 to 24, three pairs per kit. Shooter: busy 4.5 / 4.5 / 4.5% against 32.0 / 34.8 / 34.5%; presented p50 42.8 / 45.7 / 34.2 ms against 85.7 / 71.6 / 78.5 ms; p95 69 / 57 / 63 ms against 146 / 130 / 154 ms. Starter: busy 1.4 / 1.3 / 1.5% against 24.8 / 23.0 / 23.7%; presented p50 33.4 / 33.8 / 33.3 ms against 33.6 / 33.4 / 44.7 ms. At `--load 24` (`load1` 38 to 71, unmatched arms) the shooter's worker busy stayed at 0.5 to 1.4% against 44 to 47%, and p50 was lower in 2 of 3 pairs: frame time there is load, per the AFK rule Without `--vsync` the profiler's default flags unthrottle only the page's requestAnimationFrame, so a worker arm read 37 ms frames against 10 ms on the page; the option exists for this verdict. Earlier, before the raycast tree: Starter, desktop RTX 2080, `--load 24`, `load1` 28 to 52: main-thread busy 1.1 / 1.1 / 1.9% on `worker.html` against 57.5 / 52.8 / 45.7% on `/` (target at or below 28%). A second set at `load1` 35 to 74: 0.9 / 1.4% against 57.6 / 86.9%. Presented p50 was 80 to 101 ms in both arms at that load, and one worker run presented 268 frames in 60 s, so frame time waits for the quiet-window rerun in `slot.sh`. Shooter (current #574 look, before the raycast tree), private Xvfb, three pairs at host load (`load1` 9 to 28) and three at `--load 24` (`load1` 16 to 45): page-thread busy 3.3 / 4.2 / 3.6% and 2.2 / 1.7 / 1.1% on the worker against 65.9 / 88.0 / 86.7% and 95.6 / 99.1 / 92.2% on the page. Presented p50 45 to 69 ms (worker) against 38 to 50 ms (page) at host load, and p95 worse in 4 of 6 pairs. The game's own 90 ms update spikes and a concurrent C++ build dominated; rerun on the raycast-tree build. - [ ] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR - [ ] The shooter kit plays through the worker with the same checks. proof: `pnpm test:templates` with the flag on the shooter scenario + Open: `survives`, `input-control`, `five-soldiers`, `hold-to-fire`, `shot-tracks-crosshair` exit 0 on `/`; `performance` fails in the private Xvfb (minFps 29.94 / 20.04 / 15.01 across three runs, host `load1` up to 30) while `/main.html` passes. The same 1920x1080 build on the real GPU path (Chrome DevTools) runs the worker at 56 to 60 fps, presented p95 16.8 ms. Rerun in a quiet window; if it repeats, the runner needs a render-work frame rate for a worker game on a virtual display (Decisions). - [x] The shooter's frame on the worker page has no update spikes from scene ray queries: the engine raycast reads a cached triangle tree with hits identical to the brute-force scan. proof: `tn-native-engine-raycast-test` and `tn-native-engine-animation-skeleton-test raycast_posed_skin` in `packages/runtime-native/build/tn-engine-core` (`-DTN_ENGINE_CORE_ONLY=ON`); the shooter's `TN_FRAME_BUDGET` in Chrome on the RTX 2080. Result: 2,048 oracle rays 0 differ; 1,000 rays on a 20k-triangle mesh 8,471 ms -> 6.0 ms; a focused ray tests 4 of 20,000 triangles; posed skin hits 1/1 on both paths. Shooter on `/` (worker), 8 windows of 5 s: update p95 about 90 ms -> 5 to 10 ms, presented p95 100 ms -> 16.8 ms, 28 to 37 fps -> 56 to 60 fps ### Phase 4: Wasm threads and gate From 4bc3055dc077b5bfe38f895c5957afdcb4006948 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 02:11:35 -0700 Subject: [PATCH 25/29] PRD-752: visual judge NEUTRAL ticked; shooter performance on the real display matches the page thread --- ...web-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 4ab4718432..60dcb40086 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -108,9 +108,9 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo Without `--vsync` the profiler's default flags unthrottle only the page's requestAnimationFrame, so a worker arm read 37 ms frames against 10 ms on the page; the option exists for this verdict. Earlier, before the raycast tree: Starter, desktop RTX 2080, `--load 24`, `load1` 28 to 52: main-thread busy 1.1 / 1.1 / 1.9% on `worker.html` against 57.5 / 52.8 / 45.7% on `/` (target at or below 28%). A second set at `load1` 35 to 74: 0.9 / 1.4% against 57.6 / 86.9%. Presented p50 was 80 to 101 ms in both arms at that load, and one worker run presented 268 frames in 60 s, so frame time waits for the quiet-window rerun in `slot.sh`. Shooter (current #574 look, before the raycast tree), private Xvfb, three pairs at host load (`load1` 9 to 28) and three at `--load 24` (`load1` 16 to 45): page-thread busy 3.3 / 4.2 / 3.6% and 2.2 / 1.7 / 1.1% on the worker against 65.9 / 88.0 / 86.7% and 95.6 / 99.1 / 92.2% on the page. Presented p50 45 to 69 ms (worker) against 38 to 50 ms (page) at host load, and p95 worse in 4 of 6 pairs. The game's own 90 ms update spikes and a concurrent C++ build dominated; rerun on the raycast-tree build. -- [ ] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR +- [x] A fresh visual judge subagent returns IMPROVEMENT or NEUTRAL on worker against main-thread frames, at least two runs per arm, `--browser-recipe webgpu`, `adapter.info` checked. proof: the verdict in this box and the images on the PR. Result: OVERALL NEUTRAL (fresh Opus judge, 6 sheets: starter `after` and `tone-3`, shooter `after`, runs 1 and 2 per arm, `/` vs `/main.html`, `--headed`, all 8 runs `architecture: "turing"`). Exposure, tone, shadows, AA, resolution and HUD match; the only differences are gameplay state (starter position 57 vs 59, 60 vs 61). Images: branch `prd-752-screenshots` at dcf23602b - [ ] The shooter kit plays through the worker with the same checks. proof: `pnpm test:templates` with the flag on the shooter scenario - Open: `survives`, `input-control`, `five-soldiers`, `hold-to-fire`, `shot-tracks-crosshair` exit 0 on `/`; `performance` fails in the private Xvfb (minFps 29.94 / 20.04 / 15.01 across three runs, host `load1` up to 30) while `/main.html` passes. The same 1920x1080 build on the real GPU path (Chrome DevTools) runs the worker at 56 to 60 fps, presented p95 16.8 ms. Rerun in a quiet window; if it repeats, the runner needs a render-work frame rate for a worker game on a virtual display (Decisions). + Open: `survives`, `input-control`, `five-soldiers`, `hold-to-fire`, `shot-tracks-crosshair` exit 0 on `/`; `performance` fails in the private Xvfb (minFps 29.94 / 20.04 / 15.01 across three runs, host `load1` up to 30) while `/main.html` passes. The same 1920x1080 build on the real GPU path (Chrome DevTools) runs the worker at 56 to 60 fps, presented p95 16.8 ms. The quiet-window rerun (load 5.5) repeated it: minFps 20.04, cost p95 17.7 ms (CPU), because at 20 presented fps each frame carries about 3 catch-up ticks (update p50 7.5 ms vs 0.6 ms on the page thread). On the real KDE display (`TN_PLAYTEST_HOST_DISPLAY=1`) both arms match (update p95 7.3 page vs 8.0 worker, render 10.4 vs 10.8), minFps passes on both, and both fail the kit's own budgets the same way (796 draw calls > 320; frame p95 16.8 / 17.2 ms against 16.7). Left: CI's template lane runs this scenario under Xvfb against `/`, so the runner needs an honest worker measurement on a virtual display before #576 joins #574. - [x] The shooter's frame on the worker page has no update spikes from scene ray queries: the engine raycast reads a cached triangle tree with hits identical to the brute-force scan. proof: `tn-native-engine-raycast-test` and `tn-native-engine-animation-skeleton-test raycast_posed_skin` in `packages/runtime-native/build/tn-engine-core` (`-DTN_ENGINE_CORE_ONLY=ON`); the shooter's `TN_FRAME_BUDGET` in Chrome on the RTX 2080. Result: 2,048 oracle rays 0 differ; 1,000 rays on a 20k-triangle mesh 8,471 ms -> 6.0 ms; a focused ray tests 4 of 20,000 triangles; posed skin hits 1/1 on both paths. Shooter on `/` (worker), 8 windows of 5 s: update p95 about 90 ms -> 5 to 10 ms, presented p95 100 ms -> 16.8 ms, 28 to 37 fps -> 56 to 60 fps ### Phase 4: Wasm threads and gate From 298a8068753b80fe6bfa6317042b3fe128c1c2cf Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 02:19:15 -0700 Subject: [PATCH 26/29] PRD-752: no worker-only frame rate under Xvfb (scheduler probe, draw-call evidence) --- ...52-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 60dcb40086..8d41998bec 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -82,6 +82,7 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me - 2026-10-27 (lead): **a fresh review (Codex sol 6.1, max effort) found 15 items**; each confirmed one got a red-green spec. Worker: startup errors reject `ready`, stop removes the canvas and releases lock and capture, pointer capture, `pointerlockerror`, a 0x0 container, HUD wheel, motion coalesced per page frame, one HUD snapshot in flight, gamepads forwarded, a lazy playtest mirror with `_clockEpoch`. Audio and profiler items in the next entry. - 2026-10-27 (lead): **review, audio and profiler.** The worker clock freezes while the page context is suspended and re-anchors on each page reply. A fluent `AudioParam` call returns its own handle, so 10,000 calls keep the table constant, and an ended one-shot releases its handles. Unwritten `AudioParam` scalars read the standard default (`AudioBus` fades start from a number), and an unknown property throws instead of reading as a function. `resume`/`suspend`/`close` settle from the page's reply. `profile:wasm-page` runs census and GPU probes in the game's realm. The lead added `addEventListener`, `removeEventListener` and `createStereoPanner` to the proxy's method list, with a red-green spec on an `ended` listener. - 2026-10-27 (lead): **a worker game in the private Xvfb.** The runner adds `--disable-frame-rate-limit` so a page game's frames escape Xvfb's compositor cadence; the same flag throttles a worker's `OffscreenCanvas` frames (shooter, 1280x720: presented 36 ms with it, 16.7 ms without; the page thread 9.9 ms with it, 33.3 ms without). The runner now relaunches a worker game once without it and reports `gameThread: "worker"`. At 1920x1080 the worker still waits on Xvfb's present path (22 to 24 fps with any flag set, 46 to 49 ms idle per frame) while the real GPU path runs it at 56 to 60 fps: a worker's frames wait for the compositor's acknowledgement, which no Chrome switch lifts. Open: if the quiet-window rerun repeats, measure a worker game's frame rate on a virtual display from its render work (frame time minus host gap), the reading the unthrottle flag gives a page game. +- 2026-10-27 (lead): **no worker-only frame rate under Xvfb; the evaluator stays as it is.** A probe swapped the worker's `requestAnimationFrame` for an uncapped MessageChannel scheduler (shooter, Xvfb, 1920x1080): callbacks rose from 23-29/s to 630-950/s, but skipped draws rose from about 65 to about 3,200 per window. The surface waits for Xvfb's acknowledgement, so no scheduler lifts it, and Chrome has no OffscreenCanvas equivalent of `--disable-frame-rate-limit`. Under Xvfb both arms skip most draws (median draw calls per sample 0; real display about 160, peak 796 on both), so the page thread's Xvfb green also rests mostly on skipped draws. Replacing the worker's reading with a cost-derived rate would be a branch that softens a fail-closed check (`packages/playtest/AGENTS.md`), so it is not done. Consequence: a kit scenario that asserts `minFps` under Xvfb reads the worker's real Xvfb present rate. For #574's CI that means the shooter's `performance` scenario on `/`; its owner decides between a real-display lane, a `main.html` URL for that scenario, or a harness-wide virtual-display rule like `perf`'s `TN_PERF_VIRTUAL_DISPLAY`. - 2026-10-27 (lead): **a live-clock hold counted requests, not ticks.** A wall-clock `advance(1)` returns the ticks the host's pump ran: two per frame at 30 fps. The runner counted each request as one tick, so the starter's 169-tick hold ran about 338 and the player fell into the gap on the worker page. Harness layer: any game under 60 fps on a live clock drifted the same way. The bridge client now returns the count and the hold loop counts it (red-green spec in `runner.spec.ts`). ### Phase 1: Loaded-host baseline From 4bc3150a4522994e283f8881c8a68d0077897d4b Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 02:30:16 -0700 Subject: [PATCH 27/29] PRD-752: phase 4 ceiling measured (shooter about 6%, Midway about 25% splittable engine work) --- ...52-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 8d41998bec..4b1d59be41 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -82,6 +82,7 @@ Clean CPU profile of the shooter kit (NVIDIA, no DevTools, 2026-10-26): frame me - 2026-10-27 (lead): **a fresh review (Codex sol 6.1, max effort) found 15 items**; each confirmed one got a red-green spec. Worker: startup errors reject `ready`, stop removes the canvas and releases lock and capture, pointer capture, `pointerlockerror`, a 0x0 container, HUD wheel, motion coalesced per page frame, one HUD snapshot in flight, gamepads forwarded, a lazy playtest mirror with `_clockEpoch`. Audio and profiler items in the next entry. - 2026-10-27 (lead): **review, audio and profiler.** The worker clock freezes while the page context is suspended and re-anchors on each page reply. A fluent `AudioParam` call returns its own handle, so 10,000 calls keep the table constant, and an ended one-shot releases its handles. Unwritten `AudioParam` scalars read the standard default (`AudioBus` fades start from a number), and an unknown property throws instead of reading as a function. `resume`/`suspend`/`close` settle from the page's reply. `profile:wasm-page` runs census and GPU probes in the game's realm. The lead added `addEventListener`, `removeEventListener` and `createStereoPanner` to the proxy's method list, with a red-green spec on an `ended` listener. - 2026-10-27 (lead): **a worker game in the private Xvfb.** The runner adds `--disable-frame-rate-limit` so a page game's frames escape Xvfb's compositor cadence; the same flag throttles a worker's `OffscreenCanvas` frames (shooter, 1280x720: presented 36 ms with it, 16.7 ms without; the page thread 9.9 ms with it, 33.3 ms without). The runner now relaunches a worker game once without it and reports `gameThread: "worker"`. At 1920x1080 the worker still waits on Xvfb's present path (22 to 24 fps with any flag set, 46 to 49 ms idle per frame) while the real GPU path runs it at 56 to 60 fps: a worker's frames wait for the compositor's acknowledgement, which no Chrome switch lifts. Open: if the quiet-window rerun repeats, measure a worker game's frame rate on a virtual display from its render work (frame time minus host gap), the reading the unthrottle flag gives a page game. +- 2026-10-27 (lead): **phase 4 ceiling, measured before any job pool.** `--profiling-funcs` engine build, Chrome trace on the RTX 2080, 1920x1080, self time on the game's thread. Shooter (worker, 12.4 s sampled): JS 47.8%, engine Wasm 22.2%, physics Wasm 16.8%, idle 10.3%; the work a job pool can split (matrix walk 4.7%, cull/project 1.2%) is about 6%. Midway (page thread, on the deck, 6.5 s, 0% idle): engine Wasm 56.5%; splittable work is about 25% (projection `decide` + `contentClassOf` 10.8%, matrix walk about 9%, animation 3.2%, `RenderDatabase::project` 2.1%), so 4 threads return up to about 19% of the frame, inside the 15-20% expectation. Draw recording stays on the WebGPU thread, and the JS-to-Wasm binding (Midway 16.9%) is single-threaded. Two single-thread items are as large as the threaded win and belong to the owner's earlier waste order (engine hot-call guard), not to this PRD: `tn_context::share(std::string, ...)` 8.7% of Midway's main thread, and `projection::contentClassOf` 5.1%, which reads as a pure function of the geometry. A DevTools trace inflates JS self time, so the Wasm shares are lower bounds. - 2026-10-27 (lead): **no worker-only frame rate under Xvfb; the evaluator stays as it is.** A probe swapped the worker's `requestAnimationFrame` for an uncapped MessageChannel scheduler (shooter, Xvfb, 1920x1080): callbacks rose from 23-29/s to 630-950/s, but skipped draws rose from about 65 to about 3,200 per window. The surface waits for Xvfb's acknowledgement, so no scheduler lifts it, and Chrome has no OffscreenCanvas equivalent of `--disable-frame-rate-limit`. Under Xvfb both arms skip most draws (median draw calls per sample 0; real display about 160, peak 796 on both), so the page thread's Xvfb green also rests mostly on skipped draws. Replacing the worker's reading with a cost-derived rate would be a branch that softens a fail-closed check (`packages/playtest/AGENTS.md`), so it is not done. Consequence: a kit scenario that asserts `minFps` under Xvfb reads the worker's real Xvfb present rate. For #574's CI that means the shooter's `performance` scenario on `/`; its owner decides between a real-display lane, a `main.html` URL for that scenario, or a harness-wide virtual-display rule like `perf`'s `TN_PERF_VIRTUAL_DISPLAY`. - 2026-10-27 (lead): **a live-clock hold counted requests, not ticks.** A wall-clock `advance(1)` returns the ticks the host's pump ran: two per frame at 30 fps. The runner counted each request as one tick, so the starter's 169-tick hold ran about 338 and the player fell into the gap on the worker page. Harness layer: any game under 60 fps on a live clock drifted the same way. The bridge client now returns the count and the hold loop counts it (red-green spec in `runner.spec.ts`). From 6639da58379d5f517909c93a450b4bbda6de473a Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 03:07:51 -0700 Subject: [PATCH 28/29] PRD-752: phase 4 threaded-build probe (1-5% single-thread cost); job pool waits on PRD-573 v3.1 --- ...-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index 4b1d59be41..b1467be587 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -116,13 +116,16 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo - [x] The shooter's frame on the worker page has no update spikes from scene ray queries: the engine raycast reads a cached triangle tree with hits identical to the brute-force scan. proof: `tn-native-engine-raycast-test` and `tn-native-engine-animation-skeleton-test raycast_posed_skin` in `packages/runtime-native/build/tn-engine-core` (`-DTN_ENGINE_CORE_ONLY=ON`); the shooter's `TN_FRAME_BUDGET` in Chrome on the RTX 2080. Result: 2,048 oracle rays 0 differ; 1,000 rays on a 20k-triangle mesh 8,471 ms -> 6.0 ms; a focused ray tests 4 of 20,000 triangles; posed skin hits 1/1 on both paths. Shooter on `/` (worker), 8 windows of 5 s: update p95 about 90 ms -> 5 to 10 ms, presented p95 100 ms -> 16.8 ms, 28 to 37 fps -> 56 to 60 fps ### Phase 4: Wasm threads and gate -**Status:** NOT STARTED +**Status:** PARTIAL — ceiling measured, threaded build probed; the job pool waits on PRD-573 v3.1 (Blocked on) - [ ] A `TN_WEB_THREADS` build costs at most 2% on one thread against the non-threaded build. proof: `profile:wasm-page` A/B, three runs per arm + Open: a `-pthread` build (no pool threads) boots in the worker under COOP/COEP with no errors, but costs about 1 to 5% on one thread. Engine `TN_FRAME_BUDGET` render p50, alternating arms, real display, vsync, RTX 2080: shooter 5 runs per arm, median 6.84 ms shipped vs 7.14 ms (+4.4%), pairs spread -17% to +27%; Midway on the deck, 5 runs per arm, median 20.02 vs 20.87 ms (+4.2%), +1.2% in the slow mode and +5.4% in the fast mode. `profile:wasm-page` cannot profile a worker (Playwright opens no CDP session on one), so the engine's own phases are the reading. `-sGROWABLE_ARRAYBUFFERS=2` (the glue-only fix Emscripten suggests) draws nothing (render phase 0 ms, no GPU time) and fails the Node 20 post-build check, so the facade's heap views need work before it can separate glue cost from Wasm atomics. - [ ] Without cross-origin isolation the engine reports `threads: off` and runs unchanged; with isolation (COOP/COEP) it reports the pool size. proof: `native_engine_wasm_browser_backend` for both isolation states - [ ] The job pool cuts engine frame CPU on the threaded build. proof: `profile:wasm-page` A/B on the shooter, threaded against not - [ ] Stability gate S is green on web with and without isolation headers. proof: the S commands in [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md) ## Blocked on -- None. The platform proof rule applies: a helper admitted for being unportable ships native proof in the same commit (`packages/runtime-native/AGENTS.md`). +- The job pool itself: [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md) Phase v3.1 "Jobs on every core" (mixers and transform subtrees as jobs, bit-equal to one thread) on epic PR #493. A second pool here would duplicate it. Phase 4's pool box, its isolation box and gate S resume when v3.1 lands; the threaded build then has to pay for its measured 1 to 5% single-thread cost out of the pool's win (Midway ceiling about 19% at 4 threads). +- The shooter `performance` scenario under Xvfb on `/`: PR #574's owner picks the CI lane for it (Decisions, 2026-10-27). +- The platform proof rule applies: a helper admitted for being unportable ships native proof in the same commit (`packages/runtime-native/AGENTS.md`). From c4fe5efcc0cd42d8200cae8ba2a33a2c84f849b0 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Tue, 27 Oct 2026 03:25:27 -0700 Subject: [PATCH 29/29] PRD-752: threaded-build cost re-measured on one source tree (about 1.5-3%); earlier A/B void --- ...web-games-run-in-a-worker-so-the-page-cannot-stall-them.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md index b1467be587..f414e66f7f 100644 --- a/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md +++ b/docs/PRDs/performance/PRD-752-web-games-run-in-a-worker-so-the-page-cannot-stall-them.md @@ -119,13 +119,13 @@ Sets the KPI baseline. Shooter kit and starter kit, same machine, same stated lo **Status:** PARTIAL — ceiling measured, threaded build probed; the job pool waits on PRD-573 v3.1 (Blocked on) - [ ] A `TN_WEB_THREADS` build costs at most 2% on one thread against the non-threaded build. proof: `profile:wasm-page` A/B, three runs per arm - Open: a `-pthread` build (no pool threads) boots in the worker under COOP/COEP with no errors, but costs about 1 to 5% on one thread. Engine `TN_FRAME_BUDGET` render p50, alternating arms, real display, vsync, RTX 2080: shooter 5 runs per arm, median 6.84 ms shipped vs 7.14 ms (+4.4%), pairs spread -17% to +27%; Midway on the deck, 5 runs per arm, median 20.02 vs 20.87 ms (+4.2%), +1.2% in the slow mode and +5.4% in the fast mode. `profile:wasm-page` cannot profile a worker (Playwright opens no CDP session on one), so the engine's own phases are the reading. `-sGROWABLE_ARRAYBUFFERS=2` (the glue-only fix Emscripten suggests) draws nothing (render phase 0 ms, no GPU time) and fails the Node 20 post-build check, so the facade's heap views need work before it can separate glue cost from Wasm atomics. + Open: a `-pthread` build (no pool threads) boots in the worker under COOP/COEP with no errors, and costs about 1.5 to 3% on one thread, which 5 runs cannot separate from a 2% budget. Engine `TN_FRAME_BUDGET` render p50, alternating arms, real display, vsync, RTX 2080, both arms built from this PR's sources (stamp checked): Midway on the deck, 5 runs per arm, median 16.65 ms vs 17.11 ms (+2.8%), runs 1-4 mean 16.41 vs 16.66 ms (+1.5%), pairs -5.8% to +7.8%, plus one `-pthread` outlier at 24.65 ms under a load spike (`load1` 10.6). An earlier A/B (Midway +4.2%, shooter +4.4%) is void: its single-thread arm came from a stale `build/web` pair built from another tree. `profile:wasm-page` cannot profile a worker (Playwright opens no CDP session on one), so the engine's own phases are the reading. `-sGROWABLE_ARRAYBUFFERS=2` (the glue-only fix Emscripten suggests) draws nothing (render phase 0 ms, no GPU time) and fails the Node 20 post-build check, so the facade's heap views need work before it can separate glue cost from Wasm atomics. - [ ] Without cross-origin isolation the engine reports `threads: off` and runs unchanged; with isolation (COOP/COEP) it reports the pool size. proof: `native_engine_wasm_browser_backend` for both isolation states - [ ] The job pool cuts engine frame CPU on the threaded build. proof: `profile:wasm-page` A/B on the shooter, threaded against not - [ ] Stability gate S is green on web with and without isolation headers. proof: the S commands in [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md) ## Blocked on -- The job pool itself: [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md) Phase v3.1 "Jobs on every core" (mixers and transform subtrees as jobs, bit-equal to one thread) on epic PR #493. A second pool here would duplicate it. Phase 4's pool box, its isolation box and gate S resume when v3.1 lands; the threaded build then has to pay for its measured 1 to 5% single-thread cost out of the pool's win (Midway ceiling about 19% at 4 threads). +- The job pool itself: [PRD-573](./PRD-573-unreal-class-performance-in-release-cycles.md) Phase v3.1 "Jobs on every core" (mixers and transform subtrees as jobs, bit-equal to one thread) on epic PR #493. A second pool here would duplicate it. Phase 4's pool box, its isolation box and gate S resume when v3.1 lands; the threaded build then has to pay for its measured 1.5 to 3% single-thread cost out of the pool's win (Midway ceiling about 19% at 4 threads). - The shooter `performance` scenario under Xvfb on `/`: PR #574's owner picks the CI lane for it (Decisions, 2026-10-27). - The platform proof rule applies: a helper admitted for being unportable ships native proof in the same commit (`packages/runtime-native/AGENTS.md`).