From db9cc85f8c003363cfc57f5cbeb41e0dc54b6522 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Thu, 5 Nov 2026 00:09:13 -0700 Subject: [PATCH 001/253] docs(PRD-824): the mystral desktop host confines untrusted game JS Store audit (threenative-store docs/security/engine-capabilities-audit.md): playtest.respond/receive write, read and delete any path; file reads, module loads and screenshots take any path on desktop; curl has no scheme limit. Fixes keep dev, CLI, benchmark and playtest behavior unchanged. --- .../PRD-824-mystral-host-confines-game-js.md | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 docs/PRDs/native/PRD-824-mystral-host-confines-game-js.md diff --git a/docs/PRDs/native/PRD-824-mystral-host-confines-game-js.md b/docs/PRDs/native/PRD-824-mystral-host-confines-game-js.md new file mode 100644 index 0000000000..ea548abfef --- /dev/null +++ b/docs/PRDs/native/PRD-824-mystral-host-confines-game-js.md @@ -0,0 +1,57 @@ +# PRD-824 — The `mystral` desktop host confines untrusted game JS + +**Status:** NOT STARTED +**Priority:** P1 — Phases 1-3 are open: game JS in the `mystral` host can write, read and delete any file the player can reach, so a store game on this host could install malware. +**Complexity:** 4 (HIGH) — 1-5 files (+1), crosses the engine/store boundary (+2), external API: curl options (+1); risk override: code execution on player machines. +**Owner:** João +**Depends on:** None +**Consumer:** `threenative-store` PRD-815 (`docs/PRDs/PRD-815-native-builds-run-only-the-engine.md`); until this PRD is done the store ships native builds only on `tn-native-engine-player-v8`. + +## Context + +Audit (read-only, 2026-11-05, `develop` at `e9707dd6c`, cross-checked by DeepSeek V4.1 Flash): `threenative-store/docs/security/engine-capabilities-audit.md`. Findings in `packages/runtime-native`: + +| # | Capability | Code | Today | +|---|---|---|---| +| 1 | Write any file, ≤ 1 MB | `__THREENATIVE_NATIVE__.playtest.respond(path, payload)`, `src/runtime.cpp:3817-3830` | Always installed (`:3875`), no path check | +| 2 | Read and delete any file | `__THREENATIVE_NATIVE__.playtest.receive(path)`, `src/runtime.cpp:3797-3815` | Always installed, no path check | +| 3 | Read any file | `__readFileSync`/`__readFileAsync` (`fetch("file://…")`), `src/runtime.cpp:2328-2430`; `resolveFetchFilePath`, `:169-185` | Desktop passes the raw path to `std::ifstream` | +| 4 | Load JS from outside the game | `require`/`import`, `src/js/module_resolver.cpp:154-167` | Absolute and `..` paths pass when no `game.bundle` is present | +| 5 | Write a PNG anywhere | `__THREENATIVE_NATIVE__.captureScreenshot(path)`, `src/runtime.cpp:3640-3645` | No path check | +| 6 | curl schemes | `src/http/http_client.cpp:138-196` | No `CURLOPT_PROTOCOLS` (`file://`, `ftp://` UNVERIFIED) | + +Row 1 is code execution: a `.desktop` file in `~/.config/autostart/` or a `.bat` in the Windows Startup folder runs at the next login. + +Not affected: `tn-native-engine-player-v8` (`src/engine/player/v8_main.cpp:1211-1318` exposes no file, network or module access; its mailbox uses fixed names under `TN_PLAYTEST_MAILBOX_ROOT`, `src/engine/player/mailbox.cpp:49-59`). No process spawn, `dlopen` or FFI binding exists in either host. + +## Solution + +Owner constraint (2026-11-05): no regressions and no nerfing. Block only what a legitimate game never does; turn on stricter limits only where a store needs them. + +- **A. Playtest gate (always).** Install `__THREENATIVE_NATIVE__.playtest` only when `TN_PLAYTEST_MAILBOX_ROOT` is set, and confine `receive`/`respond` to that root with `canonicalize` + prefix check, the same check `native/ui-overlay/src/desktop.rs:167-171` uses. The playtest runner always sets the variable (`packages/playtest/src/runner/desktop.ts:123`). +- **B. Store mode (`--store`).** A new CLI flag. Store launchers pass it with fixed arguments; dev, CLI tools, benchmarks and playtests do not, so they keep today's behavior exactly. Game JS cannot start a process, so it cannot drop the flag. +- **C. In store mode,** confine file reads (row 3) and module resolution (row 4) to the game folder, the way Android maps to the bundle (`runtime.cpp:174-178`). A refused path throws `TN_STORE_PATH_REFUSED` naming the path. +- **D. In store mode,** confine `captureScreenshot` (row 5) to the game's save folder. +- **E. Always,** set `CURLOPT_PROTOCOLS` and `CURLOPT_REDIR_PROTOCOLS` to HTTP and HTTPS (row 6). Network stays open for games; in store mode the host logs each contacted host once per run, so a store can show what a game talks to. + +## Acceptance Criteria + +#### Phase 1: Always-on fixes (A, E) +- [ ] Without `TN_PLAYTEST_MAILBOX_ROOT`, `__THREENATIVE_NATIVE__.playtest` is `undefined`; with it, `respond` to a path outside the root returns false and writes nothing. proof: new runtime-native test (red on `develop` first) — Evidence: pending. +- [ ] `fetch("ftp://…")` and a redirect from HTTPS to `file://` both fail. proof: same test file — Evidence: pending. +- [ ] No regression: the runtime-native conformance run and the desktop playtest specs (`packages/playtest/__tests__/desktop-playtest.spec.ts`) pass unchanged. proof: those commands — Evidence: pending. + +#### Phase 2: Store mode (B, C, D) +- [ ] With `--store`, `fetch("file:///etc/hostname")`, `require("/tmp/x.js")` and `captureScreenshot("/tmp/x.png")` each fail with `TN_STORE_PATH_REFUSED`; the same game's own assets, modules and save-folder screenshot work. proof: new runtime-native test — Evidence: pending. +- [ ] Without `--store`, the same calls behave as on `develop` today (dev mode untouched). proof: same test — Evidence: pending. +- [ ] A `mystral`-host starter and a React UI starter run 300 frames with `--store` and log zero refused paths. proof: the desktop starter verification with `--store` — Evidence: pending. + +#### Phase 3: Store consumer +- [ ] The store launcher passes `--store` for `mystral` builds, and the store builder accepts `mystral` native builds again (reverting the store's engine-player-only rule). proof: `threenative-store` PRD-815 follow-up, `cargo test -p tn-store-builder` and a real `build-game.yml` run — Evidence: pending. + +## Integration Ledger + +| Capability | Reachable consumer/trigger | Replaces / disposition | Evidence | +|---|---|---|---| +| Playtest gate | host start → native host setup in `runtime.cpp` | Replaces the always-on playtest object | Phase 1 | +| Store mode | `mystral --store` from the store launcher | New flag; default behavior unchanged | Phase 2, 3 | From 3d65a735c515d9397bd666bc775b288d4373ffe8 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Fri, 6 Nov 2026 14:24:16 -0700 Subject: [PATCH 002/253] docs(PRD-828): JS keeps only game logic, engine work moves into C++, measured on the stress lab against three.js --- .../PRD-828-js-keeps-only-game-logic.md | 147 ++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 docs/PRDs/performance/PRD-828-js-keeps-only-game-logic.md diff --git a/docs/PRDs/performance/PRD-828-js-keeps-only-game-logic.md b/docs/PRDs/performance/PRD-828-js-keeps-only-game-logic.md new file mode 100644 index 0000000000..06dae34463 --- /dev/null +++ b/docs/PRDs/performance/PRD-828-js-keeps-only-game-logic.md @@ -0,0 +1,147 @@ +# PRD-828 — JS keeps only game logic: engine work moves into C++ + +**Status:** NOT STARTED +**Priority:** P1 — on the stress lab about 30% of the busy main thread is JS (2026-11-02 profile). The animation tick (about 2.2 ms a frame) and the per-pass uniform sync have no owning PRD. Every box below is open. +**Complexity:** 8 (HIGH) — 11+ implementation files (3), crosses the C++/Wasm ABI and the JS package release boundary (+2), animation listener and wrapper lifetime state (+2), new ABI exports on web and native (+1 compatibility). Risk override: none. +**Owner:** João +**Depends on:** PRD-770 (the fair window, `pnpm stress:loop fair`) and PRD-771 (the stage meter). Both tools ship. Coordinates with PRD-772 (C++ skeleton update and GPU joints), PRD-765 (bulk physics interpolation), PRD-768 (uniform scan scope), PRD-773 and PRD-774 (C++ plan). None of their scope moves here. + +## Context + +Owner, 2026-11-03: "whatever you can move to C++/gpu you move, reduce js<>wasm bridge crossings". +Owner, 2026-11-06: move code execution out of JS into C++ so that as little as possible runs in JS, for significant performance gains. Measure it on the stress benchmark against three.js, and use the benchmark artifacts to pick what moves. + +### What one crossing costs + +- A **facade crossing** (`browser-backend.ts` `invoke`: alloc, `toEngine`, `wrap`, `countCall`) costs about 0.8 µs. `Color.lerp` measured 796-939 ms per 1 M calls, against 4 ms in three.js (PRD-742). +- A **direct Emscripten export** (`module._tnw_web_frame(n)`) costs tens of nanoseconds. +- Rule of this PRD: **one crossing does the work of many objects.** That is a typed-array span, a whole loop in C++, or a GPU pass. Tiny per-call math stays in JS (PRD-742). + +### Evidence 1: the stress-lab CPU profile (measured) + +Artifact: `~/.cache/tmp-tn/stress-loop/hunt/js1102.cpuprofile` and `js1102.json`, 2026-11-02, `?stress`, 15 s, 1,013 frames at 67.5 fps. Main thread busy 14.6 s, so 1% of samples is about 0.144 ms a frame. Report: frame p50 14.09 ms, update p50 2.16 ms, 734 crossings p50, 2,304 KB GPU upload p50. Counts: 4,000 trees, 6,000 rocks, 120 houses, 600 props, 120 crowd, 300 bodies, 18,000 particles, 16 lights. Self time by owner: Wasm 52.8%, facade `web-engine-runtime.js` 14.9%, `core` 9.2%, Emscripten glue 2.8%, three math about 2%. + +Clues ranked by JS time that a move removes: + +| # | Clue (inclusive share, ms a frame) | Call chain in the profile | Where it lives | Owner | +| --- | --- | --- | --- | --- | +| P1 | Animation tick 8.1% (1.17 ms) | `Stress.ts:384` → anim `update` → `#evaluate` 3.5%, `#syncStride` 3.1%, both mostly facade `value`→`invoke` | `packages/core/src/animation.ts:1180-1267,922-988,1498` | **this PRD** | +| P2 | Animation rate bands 7.0% (1.01 ms) | `onRender` → `prepareAnimation` → `#visitAnimation` → `#observeAnimation` (3.3% self JS) | `packages/core/src/render-camera-cull.ts:408-448,622-720` | **this PRD** | +| P3 | Facade value-class crossings 12.4% (1.79 ms), across P1, P5 and P6 | `value` → `invoke` 9.4% | `packages/three-native/src/browser-backend.ts` | PRD-742 for the classes; the loops that call them move here | +| P4 | WebGPU calls from Wasm into JS 10.2% (1.47 ms) | `wasm-to-js` → `writeBuffer` 4.2%, `beginRenderPass` 1.9%, `createCommandEncoder`, `submit`, `executeBundles` | the C++ renderer's per-call `wgpu*` use | **this PRD** (coalesce) | +| P5 | Uniform sync 4.6% (0.66 ms), once per nested render and per compute | `Renderer` → `sync` (3.0%), `compute` → `sync` (1.4%), under core `updateBefore` pass renders (6.0%) | `packages/three-native/src/tsl-uniforms.ts:223-296,430-460` | **this PRD** (write once a frame); PRD-768 keeps the scan scope | +| P6 | Compute-driven systems 5.9% (0.85 ms) | `processRender` → `process` → `compute` 3.1%, `Box3.applyMatrix4` 0.8%, `multiplyMatrices` 0.4% | `packages/core/src/fluid-particles.ts:1181-1215`, `particles.ts:477-582` | **this PRD** | +| P7 | Physics interpolation apply 3.3% (0.48 ms) | `onRender` → `apply` → `value` | `packages/core/src/game-frame-loop.ts` interpolation | PRD-765 | + +PRD-771's stage split agrees (2026-10-30, 14.7 ms frame): jsBefore 2.65, update 2.26, engineJs 0.81 (735 host calls), residual 0.41. That is about 6 ms of JS. + +### Evidence 2: code survey and scouts (counted from code, not measured) + +Three Sonnet surveys and four DeepSeek v4.1 flash max scouts read the code on 2026-11-06. Their reports are in `~/.cache/tmp-tn/prd828/out/{A,B,C,D}.md`. Findings that match a profile clue join the phases below. The scout briefs are DeepSeek output: each one needs a full code review before it is implemented. + +- Scout B confirms P1. A starter rig with 6 clips pays about 52 ABI calls a frame, and a crowd rig pays 2 (`animation.ts:1180-1267`). The C++ mixer already owns actions, weights and time scales (`packages/runtime-native/src/engine/animation/mixer.cpp:317,352,850`). +- Scout D confirms P6. A fluid step issues 25 `renderer.compute()` calls, and each opens its own compute pass (`compute.cpp:126-135`). +- Down-ranked: the 55 `_tnw_web_frame` telemetry reads (`browser-renderer.ts:650-760`) are direct exports, about 3 µs a frame. They are not a lever. + +## Benchmark: the stress lab against three.js + +Every lever is kept or rejected on the stress benchmark that exists. This PRD adds no new meter. + +- **Scene:** `sandbox/stress-lab` (`main.html?stress`). `?load=N` scales density. +- **Arms:** `threenative-wasm` (the default owned engine) against `three-vanilla` (the same scene in plain three.js). `threenative-threejs` (the `engine: "threejs"` override) is an optional third arm. +- **Tool:** `pnpm stress:loop fair threenative-wasm=?stress&arm=