diff --git a/docs/PRDs/BLOCKED/README.md b/docs/PRDs/BLOCKED/README.md index cbbee9a95b..ab4e0dc5a7 100644 --- a/docs/PRDs/BLOCKED/README.md +++ b/docs/PRDs/BLOCKED/README.md @@ -23,7 +23,7 @@ dependency is not ready. | [`requires-release-credentials/`](requires-release-credentials/) | [PRD-445](requires-release-credentials/PRD-445-public-release-hygiene.md) | An upstream `threenative-sculpt-mcp` release that moves its `sharp` pin off `0.35.3`, and three owner calls on tracked repository junk. Every box in it is ticked; only the `## Blocked on` list is open. Filed here 2026-09-25 under R6. The `CLOUDFLARE_API_TOKEN` item left this list the same day: the owner set both `site-production` secrets and the `site` run on `main` (36063649413) went green | | [`requires-owner-provider-checkpoint/`](requires-owner-provider-checkpoint/) | [PRD-P2-5](requires-owner-provider-checkpoint/PRD-P2-5-evidence-storage-boundary.md) | An owner checkpoint naming the bulk-evidence provider, credentials source, retention policy, cost bound, and restore owner. Phase 1 (immutable evidence manifests, Git-only) is delivered and green; only provider selection is an owner decision no agent can make | | [`requires-examples-checkout/`](requires-examples-checkout/) | [PRD-443](requires-examples-checkout/PRD-443-lossless-gltf-flatten-join-instance.md) | A commit in the `ThreeNativeHQ/examples` checkout replacing the five AC-5-verified compacted GLBs, and Midway's own `tools/run-handoff.sh` gates (`check-carrier-cycle`, `check-repair`, `maxPassTriangles`) going green there. No code change is left in this repository: the engine passes, proof and 0.3.4 release all landed. Every box is ticked; only the `## Blocked on` list is open. Filed here 2026-09-25 under R6 | -| [`requires-owner-decision/`](requires-owner-decision/) | [PRD-299](requires-owner-decision/PRD-299-request-decomposition-index.md) | An owner ruling in §0 of the PRD. | +| [`requires-owner-decision/`](requires-owner-decision/) | [PRD-299](requires-owner-decision/PRD-299-request-decomposition-index.md), [PRD-866](requires-owner-decision/PRD-866-a-generated-forest-stands-under-mountains-with-real-shapes.md) | An owner ruling in §0 of the PRD; for PRD-866, its `## Blocked on` line (AC-3 frame cost). | | [`requires-sky-light-occlusion/`](requires-sky-light-occlusion/) | [PRD-635](requires-sky-light-occlusion/PRD-635-ambient-occlusion-darkens-only-the-light-that-comes-from-around.md) | PRD-662 (sky light occlusion when dynamic GI is off) lands; then a rerun of `pnpm unreal-look:compare --layer L6` for AC-1 and a fresh visual judge on the L9 interior frames | **A PRD whose only remaining work is `## Blocked on` items lives here** (owner decision, diff --git a/docs/PRDs/BLOCKED/requires-owner-decision/PRD-866-a-generated-forest-stands-under-mountains-with-real-shapes.md b/docs/PRDs/BLOCKED/requires-owner-decision/PRD-866-a-generated-forest-stands-under-mountains-with-real-shapes.md new file mode 100644 index 0000000000..b766670045 --- /dev/null +++ b/docs/PRDs/BLOCKED/requires-owner-decision/PRD-866-a-generated-forest-stands-under-mountains-with-real-shapes.md @@ -0,0 +1,144 @@ +# PRD-866: A generated forest stands under mountains with real shapes + +**Status:** BLOCKED (owner decision) — phases 1-3 landed (7/7 boxes); AC-1 and AC-2 passed; AC-3 fails (the far band costs about 23 ms GPU, 20 -> 15 FPS) and needs the owner's call +**Priority:** P1 — the owner named the mountains as one of the two defects that keep the forest from "good to go" (2026-11-09); AC-1 is that defect. +**Complexity:** 6 (MEDIUM) — 6 to 10 files (2), a new placement module (2), state across bake and cook (2); risk override: none +**Owner:** João +**Depends on:** [PRD-854](../../done/PRD-854-one-terrain-request-decides-the-landform-water-size-and-survey.md) (the survey fetch this PRD reuses, landed), [PRD-864](../../strata-terrain-hardening/PRD-864-prop-and-collar-mechanisms-leave-the-strata-preview.md) (only for a kit game to draw the far ring; no box here waits on it) +**Epic:** [Strata terrain hardening](../../strata-terrain-hardening/README.md). Continues the shape work of [PRD-856](../../strata-terrain-hardening/PRD-856-the-mountains-carry-real-silhouettes-erosion-and-rock.md), whose open boxes stay there. + +## Problem + +Owner, 2026-11-09: "these crappy mountains that doesn thave proper textures or AAA mountain look", then "think about heightmaps etc. real shapes. how to make it procedural. and most importantly, making sure we can create variations of this forest with the MCP from strata". + +The forest's near massif is two smooth stamps (`examples/strata-terrain-preview/scripts/recipe.mjs`, `nearMassif`: a `ridge` and a `mountain` stamp, `roughness: 0.1`) with ridged noise and droplet erosion on top (`alpineCrests`). It reads as smooth clay lobes with vertical flutes. The far ranges come from the same generator and read as mountains, because they are small on screen. + +Three measured causes (2026-11-09, worktree `strata-forest-aaa`, RTX 2080, 1600 x 900): + +| Cause | Measurement | +|---|---| +| The shape is not a mountain | Stamp profiles: median slope 50 degrees from 200 m to 700 m, 63 degrees above 700 m. A real mountain averages 30 to 40 degrees with cliff bands. | +| The mesh cannot carry rock at this range | The massif stands 1.1 to 2.4 km from the vista eye. One pixel covers 1.4 to 3.1 m there (`distance x 0.00128`). The far ring has 9.8 m cells, drawn on a 14 to 18 m ring grid, so one cell is 3 to 13 pixels of flat facet. | +| The lower slopes are bare | The far forest plants up to 36 to 46 degrees (`packages/terrain/starter/forest/biome.data.json`, `foliage[0].far.slope`). Only 29 to 38% of the massif between 200 m and 700 m is under 40 degrees, so most of it is bare rock from foot to top. | + +Also true today: + +- No sun shadow reaches the massif. The shadow window ends at 256 m (`src/render/sky.ts`, `RIG.shadowExtents: [24, 320]`), so a gully is lit like the rib beside it. +- `terrain_generate` writes no far ring. A cooked world has no mountains around its square at all (`packages/terrain/src/mcp/cook.ts`; PRD-856 records the same gap). +- The cooked forest kit is not this forest. The kit draws CC0 art with its own `world.ts` and `sky.ts`. The example draws the owner's licensed FAB assets with `src/render/`. The cook also places props from seeded rows (`stream: "seeded"`, `cook.ts:684`), not the example's ordered stream. + +## What was tried and failed (do not repeat) + +All four were captured and reverted on 2026-11-09. Frames are local only: `~/Downloads/strata-forest-aaa/`. + +| Attempt | Result | +|---|---| +| Ridged `crags` noise (amplitude 80, scale 360 m, 4 octaves) plus 160 000 droplets | Marginal. Curvature kurtosis fell from 41.5 to 25.0, under the floor of 40 in `forest-backdrop.spec.ts`. With crags at 40 000 droplets it fell to 14.7, the value of smooth blobs. Frame `10-crags`. | +| Rock normal map on the faces, unwarped, 220 m and 60 m tiles | A faint crack net. No rock form. Frame `11-relief-clear`. | +| Dark gullies from the collar's `hollow` field | Blotches at the field's own 9.8 m scale. Frame `12-gully-shade`. | +| Longs Peak summit block (1.2 km each way, 9.8 m cells) pasted with `blend: "max"`, an offset and `falloff: 0.45` | Worse: one huge smooth bare cone. The crop was a summit dome, not a whole mountain, and the feather made its flank a ramp. Snow area rose from under 15% to 22%. Frame `13-survey-massif`. | + +What does work, same renderer: the alpine world's surveyed headwall at 2 m cells reads as a real mountain (`ref-alpine-world/forest-overview.png`). Real shape at enough resolution is the lever. Noise and material tweaks are not. + +## Solution + +```mermaid +flowchart LR + R["terrain_generate request + seed"] --> C["cook: ground, water, scatter"] + C --> M["placeMassif: seeded survey tile, foot to foot"] + M --> H["far ring: heights + relief texture"] + H --> K["cooked world (kit look)"] + H --> B["example bake from the cooked world"] + B --> P["preview: this forest's look"] +``` + +1. **Real shapes from surveys.** A massif is a height tile cut from a public survey, whole, foot to foot. The example cuts tiles with `scripts/dem/crop.mjs` from USGS 3DEP (public domain, 1 m source). The generator fetches them with the installed `surveyHeights` (`packages/terrain/src/mcp/survey.ts:228`) at terrarium zoom 14 or 15 (about 7.3 m and 3.7 m per pixel at 40 degrees north). Rows live beside the landform surveys in `packages/terrain/src/core/surveys.ts`, each with its attribution. +2. **Foot to foot, base relative.** A tile covers the whole mountain (3 km or more), so its edge is valley floor. The placement subtracts the tile's own base (the 10th percentile of its perimeter), clamps at zero and adds the rest to the host ground. No offset, no `max` plateau, no feather ramp. +3. **Procedural by seed.** The seed picks the row, the bearing sector, the distance, a rotation that turns the tile's steepest face to the play area within 20 degrees, a mirror, and a vertical scale of 0.85 to 1.15. One seed gives one mountain. Another seed gives another. +4. **Rock where the mesh ends.** The collar already bakes a relief texture from its 9.8 m cells (`src/render/terrain.ts`, `ICollarRelief`: normal, hollow, scree). Over the massif footprint the same bake reads the tile at its source spacing, 4 m or finer, so the faces take real ribs, couloirs and ledges in their light. No new shader path. +5. **Forest on the lower slopes.** A real tile has real slopes, so the installed far-forest rule plants it. A gate measures the planted share. +6. **This forest, varied through the MCP.** The cook writes the far ring. The example's bake takes a cooked world as its ground, so the preview draws this forest's look on generated ground. + +Auto by default: the cook adds the ring and a massif when the landform is `alpine` or `temperateMountains`. Named override on the same request: `backdrop: "none"`. The result reports the choice in `decided.backdrop`. + +## Execution Phases + +### Phase 1 — one real mountain in the example + +**Files:** `examples/strata-terrain-preview/scripts/dem/` (new tiles and metadata), `scripts/recipe.mjs` (`nearMassif`), `src/render/terrain.ts` (`ICollarRelief` source), `__tests__/`. + +- [x] Three whole mountains are cut foot to foot at 4 m cells or finer, each 3 km square or more, with the same metadata and hash file the other surveys carry. Start with Longs Peak (the 1 m source is already named in `scripts/dem/alpine.json`). proof: a new `__tests__/massif-tiles.spec.ts` asserts, per tile, the hash, relief of 700 m or more, and a perimeter whose 90th percentile is under 80 m above its base + - result: `scripts/dem/massif.mjs` cut Longs Peak (4.4 km, 1101²), Grand Teton (3.6 km, 901²) and Maroon Bells (3.6 km, 901²) at 4 m from the 3DEP 1 m lidar service, with metadata, lidar tile list and sha256. `__tests__/massif-tiles.spec.ts` 6/6: hash, raw relief 1191/1900/1243 m, lift above the perimeter membrane 649/888/696 m (bar 600, see Decisions), summit in the central half, border exactly 0. +- [x] `nearMassif` places one tile base relative (Solution 2) where the stamps stood. proof: `pnpm exec vitest run --config vitest.config.ts __tests__/forest-backdrop.spec.ts` keeps its 7 cases green (relief 800 to 1200 m, kurtosis over 40, the vista skyline, snow under 15%), plus one new case: 60% or more of the massif cells between the valley floor plus 50 m and the treeline band are under 40 degrees + - result: seed 6 (Maroon Bells, 594 m above its foot, face south, summit at [-2050, -150]); `forest-backdrop.spec.ts` 8/8 green; the new case measures 65.4% of 330 773 lower-slope cells under 40 degrees. Snow 11% of the collar, kurtosis 65, relief 1116 m. `node scripts/stamp.mjs` (through tsx) exits 0. +- [x] The collar relief texture reads the tile at its source spacing over the massif footprint. proof: a spec asserts texels of 4 m or less inside the footprint and unchanged bytes outside it + - result: one atlas texture (the ground already samples 16): collar at 9.8 m on the left, the massif's 4 m cells on the right. `__tests__/massif-relief.spec.ts` asserts spacing 4 m, every collar texel byte-equal to a build without the massif, and over 90% of massif texels written; red when the massif write is removed (0 written). + +### Phase 2 — the generator owns the mountain + +**Files:** `packages/terrain/src/core/surveys.ts` (massif rows), a new `packages/terrain/src/core/massif.ts` (`placeMassif`), `packages/terrain/src/mcp/cook.ts`, `packages/terrain/__tests__/`. + +- [x] `placeMassif(terrain, { seed, tile })` ships in `@threenative/terrain` and the example's `nearMassif` calls it; no mountain stamp code stays in the example. proof: a spec in `packages/terrain/__tests__/` shows the same seed gives the same height hash, and 8 seeds each pass the Phase 1 gates (relief, kurtosis, slope share); `rg "massif-ridge|massif-summit" examples` returns nothing + - result: `packages/terrain/__tests__/massif.spec.ts` 10/10 on a checked-in 16 m Maroon fixture: same seed gives the same hash, seed 3 differs from seed 4, and seeds 0-7 each give relief 594-758 m, kurtosis 48-178 and a lower-slope share of 0.715-0.837 (bar 0.6). `rg "massif-ridge|massif-summit" examples` exits 1 (no match). +- [x] `terrain_generate` writes the far ring with a seeded massif for `alpine` and `temperateMountains`, reports `decided.backdrop`, and writes none under `backdrop: "none"`. Without network it fails closed and names the tile, as the ground fetch does. proof: red-green cases in `packages/terrain/__tests__/terrain-mcp.spec.ts` on a checked-in tile fixture + - result: the cases live in `packages/terrain/__tests__/terrain-survey.spec.ts`, "the far ring" (the survey fixtures sit there), on checked-in Terrarium tiles `12/847/{1545,1546}` and `13/{1692,1693}/{3093,3094}`: ring.json with the survey continued and a surveyed massif, `decided.backdrop` by default and off by argument, no ring.json under `backdrop: "none"`, fail closed naming `13/1692/3093.png`, and a z13 massif sharp enough on a flat host (relief > 550 m, kurtosis > 40, seeds 1-3). Red before `farRing`, green after; `vitest run terrain-survey terrain-mcp massif` 172/172 (2026-11-09). + +### Phase 3 — variations of this forest through the MCP + +**Files:** `examples/strata-terrain-preview/scripts/bake.mjs`, `scripts/recipe.mjs`, one new verify script. + +- [x] The example bakes its forest from a cooked world: `STRATA_WORLD= pnpm bake` takes that world's ground, water, scatter seed and far ring, and keeps the example's biome row and look. Unset, the bake is the pinned one. proof: a spec bakes from a fixture cook and asserts the baked heights equal the cook's; `node scripts/stamp.mjs` exits 0 on the pinned bake + - result: `examples/strata-terrain-preview/__tests__/cooked-world.spec.ts` bakes from a fixture cook (seed-9 eroded ground, 5120 m ring) and asserts the baked heights equal the cook's and the horizon is the ring; `isFresh` is false under the cooked bake and true after the pinned re-bake (`stamp.mjs` is a module `bake.mjs` imports, so `isFresh` is its proof, not `node scripts/stamp.mjs`). 24/24 with forest-backdrop, bake-golden, biome-values and the massif specs (2026-11-09). +- [x] Three seeds give three forests. proof: a new `scripts/verify-variations.mjs` cooks "a spruce forest under mountains" with seeds 1, 2 and 3, bakes each, and asserts three different height hashes, three different `decided.backdrop` choices or poses, and the Phase 1 gates on each + - result: `node scripts/verify-variations.mjs` exit 0 (2026-11-09), five cooks through the MCP server over stdio. Seeds 1-3 at 512 m: floor relief 86/32/59 m, slope p90 24/16/7 degrees, seam p90 5.6/0.9/1.6 m, Maroon Bells at three poses, massif relief 1012-1170 m, lower slopes 93-97% under 40 degrees. "a small ..." seed 4: 256 m square, Grand Teton, relief 1089 m. "a huge ..." seed 5: 1024 m square, floor 145 m over 1 km at 16 degrees, seam p90 2.9 m. Five distinct ground hashes and poses. Kurtosis reported only (9-30 in a real ring; the flat-host gate is in the spec). + +## Acceptance Criteria + +- [x] AC-1 [local]: A fresh read-only visual judge returns IMPROVEMENT on the forest `overview` and `vista` frames against the frames of the commit before this PRD, with the owner's reference frame as the target (spruce wood at low sun; mountains with forested lower slopes, rock, snow and low cloud). Headed WebGPU, real adapter checked, two runs per arm, BEFORE and AFTER burned in. proof: the judge verdict and the sheets on the PR + - result (2026-11-10, judge PASS, HIGH): overview IMPROVEMENT (the three bare hills between the lake and the massif are forested ridges in haze; spruce climbs the massif's lower slopes), vista IMPROVEMENT (the stamped spire is a broad surveyed massif with a snowfield and forested lower slopes; the bare right hill is gone). BEFORE `~/Downloads/strata-forest-aaa/prd866/BEFORE-run{1,2}` (1fbcf58bc's example files), AFTER the shipped code (94b8219fb, `prd866/march/BEFORE-run{1,2}`, the march test's control arm), sheets `prd866/march/sheets/ac1-final-*.png`. A first judge on an impostor-march arm later reverted also returned PASS. Largest gap left: one flat lilac-grey rock tone (no warm sunlit against cool shaded faces), patchy forest on the lower slopes, no low cloud. +- [x] AC-2 [local]: The same judge finds one consistent look across the three seeded forests of Phase 3 and no REGRESSION against seed 0. proof: the three-seed sheet and verdict on the PR + - progress (2026-11-10, open): first judge FAIL (seeds 1 and 3 put the massif behind the camera; seed 1 a bare plain). The cook now draws the massif's bearing from the kit's spawn camera (+-25 degrees) and retries up to six seeded placements for a wooded foot (`massifBearing`, `woodedFoot`, terrain-survey spec). Re-judge (sheets `~/Downloads/strata-forest-aaa/prd866/seeds2/sheets`): seeds 2 and 3 CONSISTENT, NO REGRESSION, each with a snow-capped massif behind the forest; seed 1 REGRESSION: a kilometre-scale far-field meadow (`far-field.ts`, macro noise on ground under `meadowSlope`) sits at the square's edge and reads as a bare level lawn from the overview. A far-field meadow fade did not move it (reverted). Cause: the bake's grass rule faded the meadow out at 66 m absolute, and a tree needs the meadow (`grassThreshold` 0.3), so no tree stood above 57.6 m; seed 1's knoll reaches 73 m. The forest row's grass `height` is now `null` (the pinned world tops out at 38.4 m, so it never acted there); red-green `biome-values.spec.ts`. Judge (sheets `prd866/seeds4/sheets`): seed 1 overview and vista IMPROVEMENT, meadow-close NEUTRAL; seed 0 NEUTRAL on all three. Final judge (2026-11-10, 995af266e, sheets `~/Downloads/strata-forest-aaa/prd866/seeds4/sheets/all-*.png`, two runs per seed): seeds 1, 2 and 3 CONSISTENT, NO REGRESSION; OVERALL PASS. Worst remaining: a dark ravine crease on seed 1's massif (read as terrain, not a bug). +- [ ] AC-3 [local]: Steady-state frame rate at `overview` and `vista` does not fall. proof: `measure-steady-state-fps`, paired arms on one code state, the numbers on the PR + - reading (2026-11-10, shipped code 94b8219fb, RTX 2080 Turing, 1600x900, headed, 20 s settle then 30 s rAF window, control page 16.7 ms, two passes, no errors; `?farForest=off` -> default `near`): overview GPU 35.4-35.7 -> 57.3-57.7 ms, presented p50 49.9 -> 66.7 ms (20 -> 15 FPS), p95 50.1 -> 66.8; vista GPU 39.8-41.0 -> 64.0-64.5 ms, p50 50.0 -> 66.7 ms (20 -> 15 FPS), p95 50.1 -> 83.4. FAILS. Levers tried on the band's remaining cost (impostor fragments of about 22 k far trees): a coarser impostor march at the fine mips cut the band to 9-12 ms but darkened the overview canopy 2-5% (judge REGRESSION); the full march through mip 2 kept the look and saved nothing; impostor gate 30-400 m and band reach 300-700 m moved 0-2.6 ms; shadow receive is 0.9 ms; drawing the far impostors after the terrain and the square's trees moved -0.9 to +0.2 ms; keeping the band's trees as coarse meshes longer costs more (gate 400 m: overview 63.0, vista 67.4 ms; 1000 m: 73.9, 86.2 ms); one extra atlas read per march step costs only 0.7-1.5 ms, so reads are not the cost; skipping the march when the ray's path crosses no coverage saved nothing (the path spans a coarse mip that is almost always covered). All reverted. The band's cost is the impostor fragment itself at mips 0-2; a cheaper far canopy needs a different renderer (its own PRD). + +## Blocked on + +- AC-3, owner decision (requires-owner-decision): the wooded band past the square that AC-1 judges (`?farForest=near`, the default) costs about 23 ms GPU and drops the overview and vista from 20 to 15 FPS on the RTX 2080. Pick one: accept the cost and its threshold, set `?farForest=off` as the default (AC-3 holds, the hills past the square go bare again, which the owner flagged as a bug on 2026-11-09), or fund a lossless far-canopy renderer (a canopy-card or merged-cluster impostor) as its own PRD. + +## Integration Ledger + +| Capability | Reachable consumer | Replaces | Evidence | +|---|---|---|---| +| Seeded real-shape massif | `scripts/recipe.mjs` `nearMassif` → `pnpm bake` → `src/world/horizon.json` → the collar mesh | The two stamps; they are deleted | Phase 1 box 2, Phase 2 box 1 | +| Far ring from the cook | `terrain_generate` → `cookKit` (`cook.ts:155`) → the cooked world's ring file | Nothing; the cook wrote no ring | Phase 2 box 2 | +| Example bake from a cooked world | `STRATA_WORLD` → `scripts/bake.mjs` → the preview | Nothing; the pinned bake stays the default | Phase 3 | + +## Decisions + +- 2026-11-09 (author, from the four failed attempts): the mountain shape comes from survey tiles, not from more noise or erosion on stamps. Noise lowered the sharpness metric and did not change the read. +- 2026-11-09 (author): a tile is a whole mountain, placed base relative. A summit crop lifted by an offset made a cone. +- 2026-11-09 (author): the look half of "this forest through the MCP" (the kit's own art and render files reaching this forest's look, and the ordered scatter stream in the cook) is not in this PRD. Phase 3 reaches this forest's look by baking the example from a cooked world. A kit that looks like this forest without the owner's licensed assets is separate work and needs its own PRD. +- 2026-11-09 (executor): the tile bar "perimeter 90th percentile under 80 m above its base" is replaced. No probed whole mountain meets it (valleys climb to cols on every side: Teton 1148, Moran 1099, Shuksan 990, Maroon 801, Longs 690 m of relief above a perimeter membrane). `liftMassif` removes a Coons-patch membrane through the low-passed border, so the border is exactly 0 by construction; the bar is now 600 m or more above that membrane, with the summit in the central half. +- 2026-11-09 (executor): tiles come from the 3DEP dynamic ImageServer (2 m bilinear pixels, a 2x2 box to 4 m), not the 1 m GeoTIFF downloads: one request per 2 km chunk and no 2 GB intermediates. The TNM products API names the 1 m lidar tiles behind each cut. +- 2026-11-09 (executor): the summit stands at [-2050, -150], about 2.4 km from the vista eye, facing south (`face: 90`). At [-1450, -50] the mountain was a wall beside the lake with the camera under ground; facing the vista, the wall was backlit by the west-south-west sun and showed no rock. +- 2026-11-09 (executor): `top: [1100, 1160]` bounds every cell inside the world, not only the summit; the lift is signed with a 40 m soft floor at the lowest host border height, so a valley inside the tile fills to its outlet instead of an island on a flat plane. +- 2026-11-09 (executor): the forest look's treeline band moves to [11, 14] and its snow cap to [17, 21] walkable ranges (the other four looks keep [6.5, 10] and [14, 17]). Spruce then climbs to about 450-575 m, so the massif's lower slopes can carry forest, and snow holds above 700-860 m. +- 2026-11-09 (executor, from the first judge): the massif keeps 650 m clear of the centre and fades in over 600 m. At 420 m on a 160 m ramp the near flank rose beside the overview camera as a pale wall inside the haze. +- 2026-11-09 (executor): bake time is unchanged by the tiles: 13.5 s before, 9.9-10.1 s after on the same machine (both inside machine-load noise). + +- 2026-11-09 (executor): the far ring needs a survey ground; it is the ground's survey row continued to ten times the world's size at zoom - 2, and the massif is fetched one zoom finer (z13, 257 cells) with its own detail grid in ring.json, so the example packs it at its own spacing. +- 2026-11-09 (executor): "a spruce forest under mountains" reads "mountains" after a backdrop word (under, below, beneath, behind, among, beside) as the far ring's range, not the ground's landform. The ground keeps the kit's own temperate survey. Before, the alpine landform squeezed a 6 km alpine crop into the 512 m square as a tan miniature range (owner screenshot, seed 1). +- 2026-11-09 (executor): a forest under a backdrop cooks a true-scale floor: a 512 m crop at z15, scale 1, at a seeded spot within 0.4 of the row's extent; the first spot with relief under 90 m per 512 m of side and a slope p90 under 28 degrees wins, the row's own centre last. The preset's fluvial erosion is replaced by hydraulic plus thermal erosion, because it lowered the square's border 10-40 m under its ring. +- 2026-11-09 (executor): curvature kurtosis gates the massif only on a flat host (spec); inside a real ring the smooth survey around it dilutes the number to 9-30 with the same mountain. +- 2026-11-09 (executor): the forest look's treeline and snow are metres above the floor (`bandSpan: 41`), not shares of the square's own range, so a hillier cooked square keeps the pinned forest's bands. +- 2026-11-09 (executor): the far forest's default is the band 700 m past the square (`?farForest=near`); `on` plants the whole ring, `off` none. A far spruce draws its coarsest cooked level until the impostor (vista props 28.6 M -> 16.3 M triangles). +- 2026-11-10 (executor): the far forest's remaining cost is its impostors' main-pass fragments, not shadow work. The `gpuShadow` span was mislabelled: three@0.187.0 `WebGPUBackend.js:3268` (`copyFramebufferToTexture`) reuses the shared `_renderPassTimestampWrites`, so the main pass's time lands on the shadow pass's slot; the GPU total is right. Each impostor quad was its bounding sphere's square, about four times a spruce's silhouette; the quad now fits the baked box (`packages/core`, e92ff9c54, judge NEUTRAL). GPU ms at 1600x900 on the RTX 2080, four windows: overview off 37.6 -> 34.6, near 66.5 -> 57.7; vista off 46.0 -> 40.7, near 72.8 -> 64.0. The band still costs about 23 ms, so AC-3 stays open; `near` stays the default because AC-1's look asks for a wooded ring, and the threshold is the owner's call. + +## Notes for the executor + +- Read first: `scripts/recipe.mjs` (backdrop, `nearMassif`, `alpineCrests`), `packages/terrain/src/core/operations.ts` (the `stamp` and `paste` case, about lines 474 to 540), `packages/terrain/src/core/presets.ts:269` (`landformBackdrop`), `__tests__/forest-backdrop.spec.ts`, `src/render/benchmark.ts` (the `vista` pose). +- `alpineCrests` runs ridged noise and erosion over everything above 320 m. Place the massif after it, or mask it out, so the survey's own rock is not carved again. +- Capture with the `capture-strata-worlds` skill. Any edit to example source or a bake reloads the dev server and spoils a running capture. `KeyV` steps the views to `vista`. +- The GPU on this machine is shared with other sessions. A single FPS reading is not evidence; use paired arms. +- Erosion is about 80% of a landform bake (PR-CHAIN spike). A survey tile needs none, so the bake should not slow down. Record the bake time before and after. +- Follow-up, not in this PRD: a baked sun-horizon term so the massif takes shadow past the 256 m window (PRD-856 removed a first try because the world was then inside the window). diff --git a/docs/PRDs/Unreal/research/unreal-visuals-and-performance-feature-map.md b/docs/PRDs/Unreal/research/unreal-visuals-and-performance-feature-map.md index 1f90a98b35..5f9a8ee0eb 100644 --- a/docs/PRDs/Unreal/research/unreal-visuals-and-performance-feature-map.md +++ b/docs/PRDs/Unreal/research/unreal-visuals-and-performance-feature-map.md @@ -262,7 +262,7 @@ flowchart LR | Niagara runtime (+ shader, vertex factories) | 255,512 | in C++ | **255,512** | 9 | High | **Partial.** `GPUParticles3D`; PRD-569, 609 done; PRD-610, 611 (`vfx/`) open; graph editor out of scope (the repo rules close editors) | | Landscape (incl. editor code in runtime) | 86,937 | 2,567 | 89,504 | 8 | High | **Partial.** PRD-566, 567 done; PRD-493 open | | World Partition and HLOD (incl. editor code) | 70,618 | 0 | 70,618 | 9 | High | **Partial.** `WorldCells`, `streaming.ts`; PRD-456, 459, 460, 606, 607 open | -| Texture and mesh streaming | 20,414 | 0 | 20,414 | 7 | Medium | **Open.** PRD-VQ-10 | +| Texture and mesh streaming | 20,414 | 0 | 20,414 | 7 | Medium | **Partial.** [PRD-860](../../done/assets/PRD-860-the-first-scene-plays-before-the-game-finishes-downloading.md) done on #643, not yet in `develop` (was PRD-VQ-10) | | Anim budget, sharing, AnimGraphRuntime | 29,104 | 0 | 29,104 | 6 | Medium | **Open.** PRD-570 (prd:75), PRD-486 | | GPU skinning | 6,782 | 1,207 | 7,989 | 5 | Low | Shipped through three.js and the owned engine | diff --git a/docs/PRDs/done/PRD-806-epic-review-medium-and-low-findings.md b/docs/PRDs/done/PRD-806-epic-review-medium-and-low-findings.md new file mode 100644 index 0000000000..ae74d5fdcd --- /dev/null +++ b/docs/PRDs/done/PRD-806-epic-review-medium-and-low-findings.md @@ -0,0 +1,83 @@ +# PRD-806 — Fix the medium and low findings of the PR #483 epic review + +**Status:** DONE 2026-11-09 on PR #635 — every medium and low finding is fixed or rejected with its reason. Scope: the 14 medium findings (13 after one duplicate) and 45 low findings verified on HEAD `21c713b7d` on 2026-11-05. The high findings are in [PRD-805](PRD-805-epic-review-findings.md). +**Priority:** P2 — each open finding below is a real defect, and none makes the board red at HEAD +**Complexity:** 5 (MEDIUM); many small fixes in independent files +**Owner:** ThreeNative maintainers + +## Problem + +The per-finding facts, the cited `file:line` and a one-line fix sketch are in [the evidence file](../refactoring/strata-refactoring/PRD-805-epic-review-findings.evidence.md). The verifier was a DeepSeek arm, so read each cited line before you fix it. Each fix gets a red-green spec. A fix that changes what reaches the screen (E46, E56, E73) also needs the visual judge. + +## Execution Phases + +#### Phase 1: Engine and asset pipeline + +- [x] E09 and E11 `packages/assets/src/compile.ts`: the pre-scan fails odd GLBs even with `models: "none"` (:339), and the terrain-world cache key ignores `PIPELINE_VERSION` (:584). proof: red-green specs in `packages/assets/__tests__/` — FIXED 2026-11-05: E09 in `32bcc9244`, E11 in `f6bd216f6` (DeepSeek arm, reviewed; red at `b1e89ac2a`, green after). `packages/assets/__tests__` 55 files, 868 tests pass. +- [x] E46 `packages/assets/src/lod/generate.ts:982`: scaled far cards clamp at the quantized POSITION bounds. proof: a red-green spec in `packages/assets/__tests__/` — FIXED 2026-11-07 on PR #635 (`5ba984939`, from `c2a06bdfc`): `card-copies-quantized.spec.ts` 1 failed without the fix, passes with it. The forest golden `world.glb` hash moves with it (`9000247e1`); the 25 MiB budget holds. Visual judge (headed WebGPU, RTX 2080, upstream three build, 2 runs per arm, 5 worlds): NEUTRAL, no visible change at the capture poses; the alpine shadow difference is run-to-run flake (see E73). +- [x] E49 `packages/runtime-native/src/engine/scene/texture.h:138`: a Texture copy copies its listeners. proof: a red-green case in the engine texture test — FIXED (`7d9510f88`, carried into PR #635). 2026-11-09, engine-only V8 player build (`-DTN_ENGINE_ONLY=ON -DTN_ENGINE_PLAYER=ON -DMYSTRAL_USE_V8=ON`): `tn-native-engine-v8-textures-test clone_listeners` FAILS with the copy constructor defaulted (`v8_textures_test.cpp:135` `clone->languageListeners == nullptr`, `:137` `fired == 1`) and PASSES with the fix; `half_float` passes. +- [x] E73 `packages/runtime-native/src/engine/renderer/renderer.cpp:4999`: the custom MRT slot of a `depthWrite=false` draw is not written at 1x. proof: a red-green case in the engine renderer test — FIXED 2026-11-08 on PR #635 (`8c1c9695d`): native `attachment-mask` 3/3, `pnpm wasm:build` 0; owned-engine judge (`TN_ENGINE=threenative`, headed WebGPU, RTX 2080, 2 runs per arm, 5 worlds × 4 poses) NEUTRAL, no lost shadow, halo or colour shift. + History: the fix is on PR #635 (`460f9e5a4`; reverted in `d55471289` and restored in `8c1c9695d`). The Wasm build compiles it (`pnpm wasm:build` exit 0), and the native `attachment-mask` test passes 3/3. The first judge called the alpine boulder shadows lost and blamed E73. That was wrong: the preview's `build` ships upstream three unless `TN_ENGINE=threenative` is set (`examples/strata-terrain-preview/vite.config.ts:129`), so those frames never ran the C++ engine. Two more runs of one unchanged build showed the long boulder shadows in one run and not in the other: the shadow presence at the capture pose is run-to-run flake. + +#### Phase 2: Game, editor and scripts + +- [x] E17 and E29 (Strata example): a prop restream cancels the in-flight terrain update (`src/render/editorView.ts:251`), and the status line lost `role=status` and `aria-live` (`src/render/selection.ts:57`). proof: red-green specs in `examples/strata-terrain-preview/__tests__/` — E17 FIXED 2026-11-05 in `32829237d` (DeepSeek arm, reviewed; red at `b1e89ac2a`, green after): the drag restream and the terrain update now supersede only their own lane. E29 NOT-REAL: the epic moved every announcement into the single `#editor-announcements` live region on purpose, so the status line needs no own live region. +- [x] E39, E40 and E41 (terrain editor): the ortho extent subtracts `target[0]` from every axis (`cameras.ts:185`), the picker shows `assets[0]` but scatter paints `"pine"` (`tools.ts:112`), and the CSS ids do not match the camera rows (`cameras.ts:158`). proof: red-green specs in `packages/terrain/__tests__/` — FIXED 2026-11-05: E39 in `5b2d7cfdb`, E40 in `1695544a6`, E41 in `5ab0ba340` (DeepSeek arm, reviewed; red at `b1e89ac2a`, green after). +- [x] E52 and E59 (one finding) `scripts/ci-runner-admission.mjs:82`: rain goes to the GPU runner but is admitted as a CPU slot. proof: a red-green spec in `scripts/__tests__/` — FIXED 2026-11-05 in `6c526a522` (DeepSeek arm, reviewed; red at `b1e89ac2a`, green after). The `ci-*` script specs pass: 24 files, 838 tests. +- [x] E56 and E62 (terrain package): `bendCrownNormals` is not idempotent (`starter/forest/world.ts:154`), and the matrix pins 35 empty worlds as success (`__tests__/terrain-mcp.spec.ts:1173`). proof: `pnpm exec vitest run packages/terrain/__tests__/` + Open: E56 FIXED 2026-11-07 on PR #635 (`b8f6c9db0`): `forest-crown-normals.spec.ts` 1 failed without the fix, passes with it; the five preview worlds judge NEUTRAL (upstream three build). E62 is on PR #635 (`b41bfaa61`) and `packages/terrain/__tests__/` passes 531/531, but it changes what a landform cook draws (firs on forest and coastal presets), so it needed a judge on an MCP-cooked world. + E62 judge, 2026-11-09: `scripts/verify-forest-playtest.ts --target web` with `terrain_generate {kit: "forest", landform: "rollingHills"}`, spawn pose, two runs per arm, headed WebGPU nvidia/turing. First verdict REGRESSION (`prd-806/e62-spawn.jpg` on `pr635-screenshots`): firs grew through the spawn pad and inside the arena walls, because a preset's foliage had no keep-out (the kit worlds stay clear only through their own terrain). Fixed in `20fd01f7c`: each landform foliage rule clears a 23 m circle at the world centre, the starter arena's half-diagonal plus a fir crown; `terrain-mcp.spec.ts` `cooks 'rollingHills' under 'forest'|'coastal'` fail without it (nearest tree 3.3 m) and pass with it, `packages/terrain` 605/605. Rejudge IMPROVEMENT (`prd-806/e62-spawn-fixed.jpg`): a fir stand around the arena, none on the pad, floor or walls; the play area meters about 10 % brighter under the canopy, still readable. Judge notes, not regressions: sparkle on near fir branches, uniform trunk lean, a bare floor under the stand. + +#### Phase 3: The 45 low findings + +- [x] Fix or reject each REAL low finding in the evidence file, with the reason beside each rejection. proof: one line per finding in this box + 2026-11-09: 44 fixed, 1 rejected. + - E01 FIXED in the EPIC squash `972d1e371`: fix(core): assert the concurrent spawn waiters clear their deadlines. + - E02 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): a failed horizon fetch no longer poisons every later world load. + - E03 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): the kit proof empties a reused artifact directory again. + - E05 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): the swimmer enters the sea the world draws. + - E06 FIXED in the EPIC squash `972d1e371`: fix(strata-example): name the wildwood stage arm in the comments that list the arms. + - E07 FIXED in the EPIC squash `972d1e371`: fix(wasm-engine-boot): the normal-node scenario fails a run that fell back to the geometric normal. + - E13 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): run the shared water regressions from the test lane. + - E14 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): drop the forest skip flag whose false arms its own throw made unreachable. + - E15 FIXED in the EPIC squash `972d1e371`: fix(core): restore the frame-buffer texture name only once, after overlapping compiles. + - E16 FIXED in the EPIC squash `972d1e371`: fix(core): the dither-partition test reads the mask instead of plain JS numbers. + - E18 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): name ?farForest=on as the override the flag actually reads. + - E19 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): delete the ?massif=before knob and the survey it swapped in. + - E22 FIXED in the EPIC squash `972d1e371`: fix(create-threenative): stop the terrain-kit symlink walk at the first existing folder. + - E23 FIXED in the EPIC squash `972d1e371`: fix(core): make the bundle row-cap test really cross 1,024 rows. + - E24 FIXED in the EPIC squash `972d1e371`: fix(core): exercise the depth-buffer width check without a resize event. + - E25 FIXED in the EPIC squash `972d1e371`: fix(core): endFrame's doc says a hitch still returns its split. + - E26 FIXED in the EPIC squash `972d1e371`: fix(core): scale a zero-area kept card about its own centre. + - E27 FIXED in the EPIC squash `972d1e371`: fix(playtest): a worker relaunch no longer reports the discarded attempt's cpu profile. + - E28 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): sway()'s doc says the shadow casts the rest pose, and why. + - E30 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): delete the ?peaks and ?meadowBase A/B knobs. + - E31 FIXED in the EPIC squash `972d1e371`: fix(engine-mcp): drop a stop word's inflections too, not just the word. + - E32 FIXED in the EPIC squash `972d1e371`: fix(physics): a forced trunk gets a positive capsule radius. + - E33 FIXED in the EPIC squash `972d1e371`: fix(strata): the spruce terminal minimum is taken by the loader, and the gate test drives it. + - E34 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): bound each invalidated shadow region by the instance's own span. + - E35 FIXED in the EPIC squash `972d1e371`: fix(strata-terrain-preview): river rocks must sit on the river's bank, not just be counted. + - E37 FIXED in the EPIC squash `972d1e371`: fix(core): size the chunk shadow proxy index pool from the shadow ladder. + - E38 FIXED in the EPIC squash `972d1e371`: fix(core): keep a file URL's triple slash when resolving terrain splat layer maps. + - E42 FIXED in the EPIC squash `972d1e371`: fix(runtime-native): a missing MRT slot refuses one read instead of killing the web host. + - E43 FIXED in the EPIC squash `972d1e371`: fix(native): state the parity bound the first-frame shadow row enforces. + - E45 FIXED in the EPIC squash `972d1e371`: fix(terrain): a 409 rebase the external writer already landed settles instead of rejecting. + - E47 FIXED in the EPIC squash `972d1e371`: fix(assets): report a GLB with no JSON chunk where the container shows it. + - E50 FIXED in the EPIC squash `972d1e371`: fix(terrain): wait for the export cook to start instead of a fixed sleep. + - E53 FIXED in the EPIC squash `972d1e371`: fix(three-native): release an object-valued uniform with the value it reads. + - E54 FIXED in the EPIC squash `972d1e371`: fix(terrain): let the example guard see path-string reads, not only imports. + - E55 FIXED in the EPIC squash `972d1e371`: fix(terrain): describe the placement record as position, quaternion and scale. + - E58 FIXED in the EPIC squash `972d1e371`: fix(native-renderer): a mirror's pass is reused only while its reflector lives. + - E60 FIXED in the EPIC squash `972d1e371`: fix(scripts): describe terrain peers as shipped or external. + - E61 FIXED in the EPIC squash `972d1e371`: fix(scripts): resolve the terrain reply only on a newline-terminated line. + - E63 FIXED in the EPIC squash `972d1e371`: fix(terrain): cook both AC-3 arms as tool calls in a fresh process. + - E65 FIXED in the EPIC squash `972d1e371`: fix(scripts): the startup gate compares Three patch digests and records the arm's own patch. + - E66 FIXED in the EPIC squash `972d1e371`: fix(terrain): the cook reports the water it floods, not the rows it was asked for. + - E74 FIXED in the EPIC squash `972d1e371`: fix(native-renderer): accept a legal eight-attachment aux pass. + - E75 FIXED in the EPIC squash `972d1e371`: fix(native-renderer): a volume texture carries the usage a texture copy reads. + - E57 FIXED 2026-11-09 on PR #635 (`feaf012c6`): alpine, desert and tundra set `lighting.lods: true`, so the `boulder-mid.glb` each bakes at 35 m is sky-dressed; coastal bakes no mid level and keeps `false`. `kit-runtime.spec.ts` "dresses the mid level its own assets.json bakes": 3 failed without the fix, 5 passed with it; `packages/terrain/__tests__/` 605/605. Visual judge, 2026-11-09: `KIT=alpine pnpm test:kit` at `feaf012c6^` and `2c9262966`, two runs per arm, headed WebGPU nvidia/turing, sheets `prd-806/e57-{boulders,ground,edge}.jpg` on `pr635-screenshots`: a fresh judge returns NEUTRAL on all three frames, no regression (arm RMSE 0.0010 against run noise 0.0026-0.0035 on `boulders`). It also says the frames do not show the gain: the one far boulder in view draws the same in both arms, so these poses prove no regression, not the sky on the mid level. + - E68 REJECTED: an explicit `kit` over a `biome` argument is the pinned contract (`terrain-mcp.spec.ts:1060` "keeps kit as the explicit override over a biome name"), and a request word that loses to an argument is reported as `decided.biome.source: "argument"` (`tools.ts:474`). A guard that threw on the conflict broke that pinned spec, so it was reverted. + +## Acceptance Criteria + +- [x] AC-1: `pnpm typecheck` and `pnpm lint` exit 0 after the fixes. proof: command output in the box — 2026-11-09: local `pnpm typecheck` exit 0 at `3817a9529` and `pnpm lint` exit 0 at `ecebc2fe5`; CI run 42609379659 on `2c9262966` (after every fix, E57 included): `typecheck` success, `lint` success, `ci-required` success. diff --git a/docs/PRDs/done/PRD-854-one-terrain-request-decides-the-landform-water-size-and-survey.md b/docs/PRDs/done/PRD-854-one-terrain-request-decides-the-landform-water-size-and-survey.md new file mode 100644 index 0000000000..346d0363ab --- /dev/null +++ b/docs/PRDs/done/PRD-854-one-terrain-request-decides-the-landform-water-size-and-survey.md @@ -0,0 +1,47 @@ +# PRD-854: One terrain request decides the landform, water, size and a real survey + +**Status:** PARTIAL — Phases 1 to 3 landed (2026-11-07). The two acceptance boxes are open. +**Priority:** P1 — `terrain_generate` drops every request word except the biome word, so "a forest with mountains and a river" cooks a flat default forest (Phase 1). +**Complexity:** 5 (MEDIUM) +**Depends on:** [PRD-762](../strata-terrain-hardening/PRD-762-an-mcp-terrain-runs-on-the-default-engine.md) (same MCP tool, path must start first) +**Epic:** [Strata terrain hardening](../strata-terrain-hardening/README.md), chain step PR-5 in [PR-CHAIN.md](../strata-terrain-hardening/PR-CHAIN.md) + +## Problem + +Owner direction (2026-11-07): the terrain MCP receives a request such as "create a forest with mountains and a river" and decides everything itself. Real elevation is one of its internal choices, not an input the agent prepares. + +Today `parseGenerateInput` (`packages/terrain/src/mcp/tools.ts:192`) reads only the biome words (`BIOME_WORD_ROWS`). "Mountains", "river", "island", "huge" and every other word are dropped silently. `landform`, `water` and `sizeMeters` come only from explicit arguments. + +The spike in [PR-CHAIN.md](../strata-terrain-hardening/PR-CHAIN.md) measured the parts: AWS terrarium tiles need no key (Mount Fuji z12 decodes to 1 317–3 754 m at 31.2 m/px), and `decodeHeightPNG`, `decodeRAW16` and the `heightmap` layer already ship. + +All work is bake time. No network call at runtime. + +## Execution Phases + +### Phase 1 — request words pick the landform, water and size + +- [x] Request words pick the landform ("mountains" → `alpine` or `temperateMountains`), the water rows ("river", "lake", "island" → `coastalCliffs` plus an ocean row) and the size ("huge", "small"). An explicit argument wins over a word. proof: a red-green case per word row in `packages/terrain/__tests__/terrain-mcp.spec.ts` — 2026-11-07: `-t "request words"` 30 failed before the change, 43 passed after; whole file 144/144; word tables in `packages/terrain/src/core/request-words.ts`. +- [x] The result carries `decided` (each choice with the word that made it, or `"argument"`) and `unused` (request words that matched nothing). proof: the same spec asserts both fields — 2026-11-07: unit cases plus the built server's reply (`-t "savanna in a game project"` asserts `decided.water.source` and `unused: ["zeppelin"]`), 1/1 pass. + +### Phase 2 — the tool picks the ground source + +- [x] A curated survey table: one or more real locations per landform, each with lat, lon, zoom and attribution. The cook picks a row from the landform. proof: a spec asserts every landform has a row and every row an attribution — 2026-11-07: `packages/terrain/src/core/surveys.ts`, one real place per landform (Matterhorn, Rocky Mountain NP, Driftless WI, Monument Valley, Étretat, Trail Ridge, Mount Fuji), each with its joerd source attribution and a measured range; `terrain-survey.spec.ts` asserts every row and attribution. +- [x] The cook fetches AWS terrarium tiles, caches them on disk, decodes `height = R·256 + G + B/256 − 32768` and feeds the `heightmap` layer. The kit README gets the attribution. proof: a spec on a checked-in tile fixture, plus one real cook from "alpine mountains" — 2026-11-07: `packages/terrain/src/mcp/survey.ts` (Node zlib decoder, no new dependency), cache in `TN_TERRAIN_TILE_CACHE`; fixture Mount Fuji z12 tile; a real `{"kit":"forest","request":"alpine mountains"}` cook fetched 4 Matterhorn tiles, 6.1 s, and wrote the EU-DEM attribution into the kit README. Look not judged yet (PRD-762 Phase 3 frames). +- [x] Without network the cook fails closed and names the missing tile, unless `ground: "procedural"`. No silent switch to procedural. proof: a spec with the fetch stubbed to fail — 2026-11-07: the spec stubs `fetch` to fail; the cook throws naming the tile URL and `ground: "procedural"`. + +### Phase 3 — named overrides + +- [x] `ground: "procedural" | { heightmap: <16-bit PNG or .r16 path> } | { location: { lat, lon } }` overrides the automatic choice, and the result names the override. proof: a spec per variant — 2026-11-07: `terrain-survey.spec.ts` covers each variant and `decided.ground`. +- [x] A PNG over 4 097 px or an 8-bit file throws and names the field. proof: a spec per refusal — 2026-11-07: both refusals name `ground.heightmap`; a `.r16` without `min`/`max` and a PNG without an embedded range also throw. Terrain specs 591/591, `tsc` 0. + +## Acceptance Criteria + +Open gap (2026-11-07): "create a forest with mountains and a river" now fails loudly with `cook: water row 'river' has no river layer in the recipe to take it`. A landform preset authors no river or lake layer, and `applyWaterRows` only re-shapes an existing layer. The AC needs the presets to grow the body a water word asks for. That changes the cooked look, so it needs a visual judge. +Closed for rivers (2026-11-08): a river row with no layer now runs down the finished ground's own drainage (`routeRivers` in `cook.ts`, on the installed `fillDepressions`/`routeFlow`): the main stem of the largest basin that leaves the world, from where its catchment is a tenth of the outlet's to the edge, every 16 m, with the forest starter's bed, depth and shoulder; a second row takes the next basin at least an eighth of the world away. A lake row with no layer still fails closed: a lake needs a basin, and a fluvially eroded preset has none. `terrain-mcp.spec.ts` cases red on HEAD (`has no river layer`, 2 failed) then green; whole MCP and survey specs 157/157. Visual judge of the cooked river in a game: open, with the PRD-762 Phase 3 frames. + +- [x] "create a forest with mountains and a river" cooks a forest kit on a mountain landform with a river, and the result lists every choice and the word behind it. proof: one real `terrain_generate` cook, result JSON in the PR — 2026-11-08: the server's own `handleLine` `tools/call`, survey ground (Matterhorn, EU-DEM attribution in the README), 6.0 s, 20 draws, 3 200 firs, `water: [{kind: "river", width: 5.32}]`; `water.json` holds one river of 164 points falling 8.6 to 5.6 m, monotone, ending at the world's edge along the lowland drainage line (hillshade check). `decided`: biome "forest", landform "alpine" from "mountains", water from "river", size default, ground survey default; `unused: []`. Result JSON on the PR. +- [x] The MCP surface snapshot names the new fields. proof: the terrain MCP surface snapshot spec — 2026-11-09: `terrain-mcp-tools.json` names `decided`, `unused` and `ground`; `scripts/__tests__/mcp-tool-schemas.spec.ts` plus `packages/terrain/__tests__/terrain-mcp.spec.ts` 2 files, 153/153 pass. The capability-manifest half was dropped (see Decisions). + +## Decisions + +- 2026-11-09 (agent): the AC no longer asks the capability manifest to name the new MCP fields. `capabilities.json` indexes code exports only and holds no MCP tool (`terrain_patch` and `terrain_biome_new` are absent too). The MCP `tools/list` reply and its snapshot spec already carry every tool field. Reversible: index MCP tools in the manifest in its own PRD. diff --git a/docs/PRDs/done/PRD-863-world-building-mechanisms-leave-the-strata-preview.md b/docs/PRDs/done/PRD-863-world-building-mechanisms-leave-the-strata-preview.md new file mode 100644 index 0000000000..0739646102 --- /dev/null +++ b/docs/PRDs/done/PRD-863-world-building-mechanisms-leave-the-strata-preview.md @@ -0,0 +1,53 @@ +# PRD-863: World-building mechanisms leave the Strata preview + +**Status:** IN PROGRESS — Phase 1 started 2026-11-10 (PR #635). +**Priority:** P1 — a fix in the preview leaves every kit and every game with the bug. +**Complexity:** 6 (MEDIUM-HIGH): water, streaming and terrain fields; props, models and the collar are [PRD-864](../strata-terrain-hardening/PRD-864-prop-and-collar-mechanisms-leave-the-strata-preview.md). +**Epic:** [Strata terrain hardening](../strata-terrain-hardening/README.md); rides PR #635 (owner, this session: no extra PRs). + +## Problem + +Owner, 2026-11-10: "are these structures being hardcoded on examples/strata-terrain-preview/? shouldnt these abstractions to create rivers, etc, line on the terrain package? it should be reusable. also the performance related ones (core?)", then "extract all world building abstractions you can" and "hook with engine_capabilities". + +The proof: the river surface stopped in a straight edge at its first station while core's `WaterBodies` floods a round cap past it, so the player swam in water nobody drew. The preview, the forest kit and the tundra kit each drew their own ribbon, and none matched the query. A read-only inventory (DeepSeek arm, 2026-11-10) lists 17 mechanisms that live only in `examples/strata-terrain-preview/src/`, about 3 000 lines, with 342 lines of lake geometry copied into two kits. + +## Rules + +- Runtime mechanism goes to `@threenative/core` (`/world` for terrain and water); `packages/terrain` stays authoring and bake only, because a shipped game plays without it (`test:consumer`). +- Look stays game source in `src/render/`: every colour, ripple, opacity and timing comes from the caller. +- Each moved export carries `@situation`, `@constraint` and `@example` JSDoc, so `engine_search_capabilities` returns it; the pre-commit `capabilities:sync` regenerates the manifest. +- Each move is red-green: a core spec, the preview and kit callers switched in the same commit, the old copy deleted. +- A move that changes a frame gets before/after captures and a fresh visual judge. + +## Execution Phases + +### Phase 1 — water + +- [x] River surface geometry: core `riverRibbonGeometry` beside `WaterBodies`, a reach past each end; the preview and both kits call it. proof: `world-river-ribbon.spec.ts`, `river-ribbon.spec.ts` (forest bake), `test:water`, the river-head judge — 2026-11-10: `river-ribbon.spec.ts` red on the old ribbon (the swim query floods half a reach past the head, nothing drawn) then green; core spec 4/4; preview 274/274 and `test:water` PASS; the kits pass a half-width reach and 3 columns, so only their end caps change. Fresh judge on the river-head sheets (two runs, NVIDIA Turing, `~/Downloads/strata-river-head/`): IMPROVEMENT both runs — the straight edge is gone, the carved head fills, no new artefact. +- [x] Lake surface geometry: core `lakeSurfaceGeometry` draws the disc `WaterBodies` floods (out to `lakeReach`, wet cells only, `metres` depth); the preview calls it. proof: `world-river-ribbon.spec.ts` lake cases, the forest lake mesh before and after — 2026-11-10: core 7/7 (every point the lake query floods over ground under the level is drawn); the forest lake is 6 992 triangles before and after, positions and `metres` exact, `waveDepth` within 6e-8, so no frame changes and no judge is due. The port also sizes the vertex buffer to the rings it fills (the preview's held one unused ring at the origin). +- [x] Shore distance and open-sea exposure field: core `shoreField` for the preview and the coastal kit. proof: `world-shore-field.spec.ts`, the coastal bake's field before and after — 2026-11-10: core 3/3; on the coastal bake both arrays are identical to the old inline code across 66 049 nodes (5 733 exposed), so no frame changes; the kit ran the same 57 lines, now one call. + +### Phase 2 — streaming and startup + +- [x] Streamed ground: the per-frame budget is core's existing `AdmissionBudget` (now exported and tagged), the settled block is `TerrainTiles.settledHalf`, and the warm-before-reveal is core `warmUnculled`; `streamedGround.ts` keeps the game's tile size, colliders and material (170 -> 100 lines). proof: `world-terrain-tiles.spec.ts`, `world-warm.spec.ts`, `stream.playtest.json` `pipelinesSinceReveal` 0 — 2026-11-10: core 82/82 (settled block red without the method); `stream.playtest.json` (`?stream=1&tnTerrainValidate=1`, load 17) `pipelinesSinceReveal` 0 and every stream counter passes. The run exits 1 on 3 console errors and 2 `TN_SCENE_WARNING`s, and HEAD's code shows the same 3 errors on the same host (A/B below), so they are not this change. +- [x] Lake warm behind the curtain: the preview's hand-written lake hook is now `warmUnculled(ctx, lake.mesh)`, the same helper the tiles use. proof: `world-warm.spec.ts`, `loop.playtest.json` 0 late pipelines — 2026-11-10: `loop.playtest.json` `pipelinesSinceReveal` 0 with HEAD's `game.ts` and with this one (load 17); both runs log the same 3 `copyFramebufferToTexture` console errors inside the held startup compile, a pre-existing core bug: on the owned engine `warmUpWorldPass` is undefined, so `renderWarmPasses` draws every object unculled straight to the `rgba8unorm` canvas, and a refracting water material copies that into its `rgba16float` scene copy. Tracked in PRD-790's streaming box. + +### Phase 3 — terrain fields + +- [x] `measureBiome` uses the terrain package's `measureBands`. proof: preview tests, the bands before and after — 2026-11-10: `measureBiome` is now a 15-line adapter from the preview's fraction record to `measureBands` layers; the measured biome is deep-equal to the old code on all five worlds; preview 274/274. +- [x] Drawn-surface sampler: core `surfaceSampler(field, matrixWorld)` replaces the preview's `createGroundSampler` (same logic; the unused geometry argument is gone). proof: `world-surface-sampler.spec.ts`, the preview's `groundSampler.spec.ts` against a raycaster and the collider — 2026-11-10: core 2/2; the preview's 8 cases pass on the core function; preview 274/274. + +## Decisions + +- 2026-11-10 (agent, AFK): the spawn gate stays in the preview. Its generic part, a deadline that holds the loading curtain, is already core's `ctx.startup.hold(label, promise, budgetMs, progress)`; what is left is this game's `TN_STRATA_SPAWN_FAILURE` receipt, its witness fields and its `worldReady`/`loadingError` state. + +- 2026-11-10 (agent, AFK): the curvature texture stays in `src/render/terrain.ts`. Its Laplacian gain, 8 m neighbourhood, wet margin, river reach and flow normalisation decide how the ground shades, and look stays game source. +- 2026-11-10 (agent, AFK): `skyVisibility` moves to `@threenative/terrain` with the next rebake, not now. `scripts/stamp.mjs` hashes its source into every bake, so a move alone rebakes all five worlds and commits about 50 MB of new JSON for no runtime change. + +- 2026-11-10 (agent): the forest and tundra kits keep their own lake surfaces; the box that switched them to `lakeSurfaceGeometry` is deleted. `lakeSurfaceGeometry` exists so the drawn lake matches `WaterBodies`' swim query, and the kits run no swim query, so there is no mismatch to fix. The tundra kit also clips each frozen kettle at the ground's contour, which a disc of wet cells would coarsen. A kit that adds swimming switches then. + +## Acceptance Criteria + +- [x] AC-1: No water, streaming or terrain-field mechanism in the inventory is left only in the preview, or each one left has a written reason here. proof: the inventory table in this PRD's PR comment — 2026-11-10: [PR #635 comment](https://github.com/ThreeNativeHQ/threenative-engine/pull/635#issuecomment-6750056714), 12 mechanisms: 9 moved, 3 kept with the reasons under Decisions. +- [x] AC-2: `engine_search_capabilities` returns each moved export for its situation. proof: `searchCapabilities` (the MCP tool's function) on the worktree manifest — 2026-11-10: 7/7 rank first: `riverRibbonGeometry`, `lakeSurfaceGeometry`, `shoreField`, `AdmissionBudget`, `TerrainTiles` (`settledHalf`), `warmUnculled`, `surfaceSampler`. Note: `call.mjs` from this worktree reaches the main checkout's `.mcp.json` and its older manifest; an installed game reads its own. +- [x] AC-3: The five worlds and the kits look the same or better (fresh judge per visual phase). proof: judge verdicts on the PR — 2026-11-09: forest river head IMPROVEMENT (above). Kits, forest and tundra, 5 shots, 2 runs per arm, headed WebGPU on NVIDIA Turing: NEUTRAL, no regression; no close river end in their shots. Preview tundra, 3 river-end poses pinned by script in headed Chrome on the same GPU: the first judge found a REGRESSION at the meltwater head, where the cap past the head hung 2-3 m over snow that falls away upstream. Fix: core `riverRibbonGeometry` drops a cap whose ground sits below the bed at its end station (`world-river-ribbon.spec.ts` red, then 8/8; core world specs 973/973; forest ribbon index arrays identical before and after, tundra 27 600 -> 27 522 triangles). Fresh judge on the fixed arm: NEUTRAL at all three poses, 2 runs per arm. Coastal, alpine and desert draw no river; the other moves are equality-proven. The kit judge ran before the fix; the fix only drops caps, toward BEFORE. Open beside it: the swimmer query (`WaterBodies`) still floods past the meltwater head, as before this PRD. diff --git a/docs/PRDs/done/assets/PRD-860-the-first-scene-plays-before-the-game-finishes-downloading.md b/docs/PRDs/done/assets/PRD-860-the-first-scene-plays-before-the-game-finishes-downloading.md new file mode 100644 index 0000000000..2dec1a13ce --- /dev/null +++ b/docs/PRDs/done/assets/PRD-860-the-first-scene-plays-before-the-game-finishes-downloading.md @@ -0,0 +1,318 @@ +--- +prd_contract: v1 +--- + +# PRD-860 — The first scene plays before the game finishes downloading + +**Status:** DONE (2026-11-10) — Midway reaches first play after 22.4 MB (from 139.5 MB) in 10.4 s (from 39.0 s) at 40 Mbps, with texture and mesh streaming on, zero late assets, and a NEUTRAL visual judge. +**Complexity:** 8 (HIGH) — 6–10 implementation files (+2): the loader, a mip-streaming module, the KTX2 path in the owned engine, the model cook and the Midway game; a new streaming module (+2); request queues and residency are concurrent state (+2); the change crosses the TypeScript packages and the owned C++ engine build (+2). +**Owner:** João +**Depends on:** None. Reuses [PRD-729](PRD-729-assets-ship-smallest-and-preload-fastest.md) (cooked bytes, throttled preload probe) and [PRD-496](../../performance/PRD-496-the-loading-screen-is-bytes-not-files.md) (loader fan-out). + +## Context + +The owner asked for web asset streaming (2026-11-08): "instead of downloading a 2GB game, it only +gets 50mb for the first scene, then downloads the rest on the go in background". Midway is the +subject (owner, 2026-11-08: "use midway as a sample game for this, its large enough"). + +What ships today: + +- `ctx.assets` (`packages/core/src/assets.ts`) downloads what a scene awaits, then + `backgroundFill` (on by default, `assets.ts:65`) fetches the rest of the manifest at idle priority. + `prefetch(paths)` (`assets.ts:215`) warms named files early. Priority `now` goes before `soon`. +- `TN_STARTUP_OFFSCREEN_BYTES` (`packages/core/src/startup-offscreen-bytes.ts:40`) reports the bytes + the reveal waited for that the spawn camera does not show. +- `WorldCells`, `TerrainTiles`, `ScatterColliders` and `propPackageFiles` stream open worlds by + position. +- Playtest scenarios throttle the network (`artifacts.network.downloadKbps`, `latencyMs`, + `packages/playtest/src/scenario/schema-validate.ts:327`). + +What is missing: + +- **Midway does not use any of it.** `src/scenes/Midway.ts` `load()` awaits every model and all + audio in one `Promise.all`. The game ships 197 MB (`dist/`, measured 2026-11-08): 29 GLBs and + 146 OGG files. Its GLBs are not cooked: `akagi.glb` is 26.3 MB with 33 PNG/JPEG images and no + KTX2 or meshopt data. +- **No texture streaming.** A texture downloads and uploads all its mips at once. The feature map + row "Texture and mesh streaming" is open (`docs/PRDs/Unreal/research/unreal-visuals-and-performance-feature-map.md:265`). +- **No mesh LOD streaming.** A cooked LOD chain (`lodChainOf`, `TN_discrete_lod`) shares the vertex + buffers of its base level, so the coarse level cannot download first. + +## Solution + +Three slices, each proven on Midway with a throttled playtest: + +1. **Midway boots on its spawn bytes.** Cook Midway through `compileAssets` (KTX2 textures, + meshopt geometry). Change its `load()` to await only what the spawn camera shows. Every other + file goes to `prefetch` or `backgroundFill`. The engine adds one log line, `TN_ASSET_LATE`, for + a file that a scene requests after the reveal and that is not yet resident. That line makes a + pop-in visible to a playtest. +2. **Textures stream by mip.** A KTX2 file keeps its level index in its header, so the loader can + read each mip level with one HTTP `Range` request. The loader first fetches the header and the + small mip tail. It then fetches the larger levels that the screen size asks for. +3. **Meshes stream by LOD.** The model cook writes each streamed LOD level with its own vertex and + index ranges, coarsest first. The loader draws the coarsest level, then fetches finer levels by + projected error. + +```mermaid +sequenceDiagram + participant Scene as Midway scene + participant Loader as ctx.assets + participant Net as HTTP (Range) + participant GPU as Engine upload + Scene->>Loader: await spawn models (now) + Loader->>Net: header + mip tail + coarsest LOD + Net-->>Loader: small bytes + Loader->>GPU: upload tail mips + coarse LOD + Loader-->>Scene: reveal (first play) + loop each update + Loader->>Loader: wanted mips / LOD from screen size + Loader->>Net: Range fetch of the next level (soon) + Net-->>GPU: finer mip or LOD + end + Loader->>Net: backgroundFill (idle) for the rest of the manifest +``` + +### Unreal reference (UE 5.8.3, read 2026-11-08 at `~/projects/UnrealEngine`) + +Borrow the algorithm and defaults. Do not copy code. + +- **Wanted texture mips.** `ceil(1 + log2(max(1, screenSize)))`, clamped to the allowed range. + UE 5.8.3: Engine/Source/Runtime/Engine/Private/Streaming/StreamingTexture.cpp:309. +- **Wanted mesh LODs.** The smallest LOD count whose normalized screen size covers the wanted size. + UE 5.8.3: Engine/Source/Runtime/Engine/Private/Streaming/StreamingTexture.cpp:320. +- **Always-resident tail.** The mips the cook inlines stay resident, and at least one mip always + stays resident. UE 5.8.3: Engine/Source/Runtime/Engine/Private/TextureDerivedData.cpp:2658. +- **Visible first.** A request loads the visible-wanted mips before the mips for hidden primitives. + A request is important when it misses more than two mips (one for meshes). + UE 5.8.3: Engine/Source/Runtime/Engine/Private/Streaming/StreamingTexture.cpp:503. +- **Budgets.** `r.Streaming.PoolSize` (-1: platform default, :120), `r.Streaming.PoolSizeForMeshes` + (below 0: meshes share the texture pool, :126) and `r.Streaming.MaxTempMemoryAllowed` (50 MB of + in-flight mip memory, :139). UE 5.8.3: Engine/Source/Runtime/Engine/Private/Streaming/TextureStreamingHelpers.cpp. +- **Mesh streaming is off by default in Unreal** (`r.MeshStreaming` 0). + UE 5.8.3: Engine/Source/Runtime/Engine/Private/ContentStreaming.cpp:38. Here it is on by + default ("Auto by default"), with a named override. + +### What execution measured (2026-11-09, local) + +Four findings change the slices below. Each is measured on this machine, not assumed; the numbers +come from Midway's own cooked manifest (`public/assets.manifest.json`, 228 entries). + +- **The default engine refuses the compression this PRD's Phase 1 is built on.** `webRuntimeDecoders` + returns `{ ktx2: false, meshopt: false }` unless the game sets `engine: "threejs"` + (`packages/create-threenative/src/web-engine.ts:11`), because the owned Wasm engine has no decoder + for either. The bake says so out loud: `TN_ASSETS_COMPRESSION_SKIPPED model: 29 file(s), 63.2 MB` + and `texture: 52 file(s), 13.2 MB`. Midway's cook is therefore quantize + compact + LOD + WebP + only: **166.2 MB shipped** (90.1 MB of manifest entries plus 76.1 MB of shared images) from 188 MB + of sources, and a 165 MB `dist` against 197 MB before. Phase 1's "KTX2 textures, meshopt geometry" + is unavailable until the owned engine decodes them — filed as + [PRD-861](PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md): the meshopt decoder is already + vendored in `third_party/meshopt/` and never compiled, and KTX2 needs a Basis transcoder that is + not in the tree at all. +- **The load split, not the cook, is what reaches AC-2's 50 MB — and the runtime now says how much.** + On a cooked build the engine's own line reports `awaitedBytes` **139.5 MB**, `visibleBytes` + **42.2 MB**, `unattributedBytes` **21.9 MB** (textures and audio: no scene object ties them to a + view) and `offscreenBytes` **75.4 MB**, naming `akagi.glb` 17.2 MB hidden, `carrier.yorktown.glb` + 10.8 MB not-placed, `midway-atoll.glb` 10.7 MB not-placed, `destroyer.samidare.glb` 10.4 MB + not-placed and `structures.radar-station.glb` 2.8 MB outside-view. So what the spawn camera shows + is 42.2 MB, the off-screen wait is 75.4 MB, and the gap to 50 MB closes by deferring the + off-screen files and moving the 13 MB of cockpit dials and most of the 8.7 MB of audio out of the + launch. ⚠️ Two of the four named files are `not-placed`, and the engine's own fix line warns why: + a model whose geometry the game merged into a new buffer loses its provenance, so Yorktown and the + atoll may be on screen and merely unattributable — check before deferring them. +- **The cooked game boots and draws.** A cooked Midway briefing run on the owned engine loads in + 11.3 s, enters at 15.6 s, draws 120 frames, passes its tone and diagnostics assertions, and loads + every one of the 40 cockpit textures through `ctx.assets`; its only failure is + `TN_PLAYTEST_SOFTWARE_ADAPTER` — Chromium on the runner's private Xvfb falls back to SwiftShader. +- **The compact `instance` pass and the engine's `clone()` disagree.** Cooking turned repeated hull + parts into `EXT_mesh_gpu_instancing`, and the shipped engine then threw + `TN_ABI_6: clone(): TN_NATIVE_UNSUPPORTED TN_NATIVE_CLONE_UNSUPPORTED: InstancedMesh` inside + `createCarrier` before the briefing drew — every ship in this game is built by cloning its model. + `assets.models.compact.instance: false` in the game's config clears it. Either the clone path or + the cook has to learn about the other; that is a follow-up, not this PRD's subject. +- **A cooked game exposes two game-side defects an uncompiled one hides**, both fixed here because + without them the cooked build cannot boot: + - 106 logical paths were written `/assets/x.glb`. The manifest's keys are **source-relative** + (`x.glb`), and `resolveCandidates` throws `Asset '…' is not listed in the asset manifest` for a + path it does not list, so the whole game had to move to the templates' convention. The + source-relative spelling also makes the delete-test fallback work: the loader tries `x.glb` + first, then `assets/x.glb`. + - `cockpit-detail.ts` loaded its 40 dial and label textures through its own `TextureLoader` on a + literal `cockpit/dial-rpm.png`, which a cook renames to `cockpit/dial-rpm..png` — so the + launch died on `TN_STARTUP_FAILED: The source image could not be decoded`. It now goes through + `ctx.assets.texture`, which resolves the manifest, keeps the disk cache warm, and already picks + `createImageBitmap` on a host with no ``; ~25 lines of duplicated decode logic went with it. +- **The playtest lane on this machine cannot settle Midway.** The runner takes a private Xvfb and + Chromium falls back to SwiftShader (`TN_GPU_CLASS: software`), where Midway's startup warm-up runs + past the runner's 195 s `describe` window; a 30-minute run at the 40 Mbps profile was still + loading. Every byte figure above is therefore read from the manifest, and the playtest boxes stay + open until a run on a real adapter settles (`--allow-software` is not evidence for a time claim). + +### Conventions + +- **On by default, named override.** `ctx.assets` options take `streamTextures: false` and + `streamMeshes: false`. An override is reported in the startup line, not silent. +- **Measured, not configured.** The wanted level comes from the render camera and the viewport. + The browser exposes no GPU memory size, so the default pool has no cap below what fits. It + reports resident bytes in `TN_STREAMING_POOL`. `texturePoolBytes` is the named override, and it + is a hard cap that reports pressure. +- **Native.** A native host reads installed files, so a `Range` read becomes a file offset read. + The native proof shows the same settled frame (Rule 5: web-only is unfinished). + +### Risks + +- ⚠️ The owned C++ engine is the default on web. It needs a path that uploads one mip level into an + existing texture (`packages/runtime-native/src/webgpu/bindings_resources.cpp`). If that path + does not exist, Phase 2 adds it. +- ⚠️ A static file host without `Range` support returns 200 with the full body. The loader must + detect this and keep the full file, not fail. +- ⚠️ Lower mips first changes the image for a short time. The visual judge rule applies to the + settled frame and to the pop-in. + +## Acceptance Criteria + +Thresholds: the playtest network profile is 40 Mbps down and 20 ms latency (the PRD-729 profile). +"First play" is `ctx.startup.whenReady()` settling. + +- [x] End to end [local]: a cold browser profile plays cooked Midway at 40 Mbps from the spawn through the flight with texture and mesh streaming on, after the AC-7 byte count, with zero `TN_ASSET_LATE` lines. proof: the AC-3 flight playtest after Phase 3 — Evidence (2026-11-10): `fleet.playtest.json` on Midway `c2ab782` with packs from `911c7cb8b` (KTX2 cook, both streams on), loopback network namespace at 40 Mbps / 20 ms, headed WebGPU, RTX 2080 nvidia/turing: exit 0, **0** `TN_ASSET_LATE`, 0 `TN_ASSETS_STREAM_FAILED`, 0 console errors, first play 9.99 s, `enemyFleetM` 16,708 → 673 m. + +#### Phase 1 — Midway boots on its spawn bytes + +- [x] AC-1 [local]: The Midway launch playtest at 40 Mbps records the baseline: bytes before first play, time to first play and total shipped bytes. proof: `TN_PLAYTEST_HOST_DISPLAY=1 DISPLAY=:0 npx threenative-playtest playtests/launch.playtest.json --browser-recipe webgpu --headed` against `vite preview` of Midway `00719cf`+`a52e839`, RTX 2080 (`TN_GPU_CLASS` nvidia/turing) — Evidence (2026-11-09): `TN_STARTUP_OFFSCREEN_BYTES.awaitedBytes` **139,518,197** (visible 42.2 MB, unattributed 21.9, off-screen 75.4, warn); first play (`TN_LOAD_STEP ready`) **38.97 s** in the playtest, 36.8 s in a Playwright probe of the same build at the same link (147.6 MB on the wire before it); **161.4 MB** shipped (manifest outputs plus shared images). The run completed every step; its one red is `maxPassTriangles.main` 1,360,253 > 1,300,000, which the baseline carries. +- [x] AC-2 [local]: Cooked Midway reaches first play after 50 MB or less, and `TN_STARTUP_OFFSCREEN_BYTES` reports no warn. proof: the same playtest on Midway `f8591d4` — Evidence (2026-11-09): `awaitedBytes` **49,950,371**, `warn: false` (visible 42.2 MB, unattributed 3.2, off-screen 4.5); first play **17.8 s** in the playtest (19.5 s in the probe), from 38.97 s; zero `TN_ASSET_LATE` lines through the run, and the streamed fleet built at 33.0 s. 22.9 MB of the cut were three dead loads (`destroyer.samidare.glb`, `carrier.yorktown.glb`, `weapon.torpedo.glb`) that nothing drew; the rest moved behind `prefetch` after the reveal (Japanese fleet, Akagi, atoll, escorts, cockpit textures, speech, combat cues). Frame time is held open under Phase 1 "Open" below, not under this box. +- [x] AC-3 [local]: The flight playtest reaches the enemy fleet with zero `TN_ASSET_LATE` lines, and the engine test proves that a late request emits the line. proof: `/playtests/fleet.playtest.json` at 40 Mbps and `pnpm exec vitest run packages/core/__tests__/assets*.spec.ts` — Evidence (2026-11-09): engine half `assets-late.spec.ts` green (assets suites 259/259 with this PR's loader change). Flight half: `fleet.playtest.json` (Midway `c2ab782`; `flight.playtest.json` never left the launch climb) starts airborne, flies 6,622 fixed steps toward Kaga and closes `enemyFleetM` from 16,708 m to **673 m** (bound 2,000 m); **0** `TN_ASSET_LATE`, 0 console and network errors, 0 `TN_BRIDGE_WARNING`, first play 16.5 s, exit 0, RTX 2080 (`TN_GPU_CLASS` nvidia/turing, discrete), loopback-only network namespace at 40 Mbps / 20 ms, engine `9dbe79811`. The first run failed only on 11 `TN_BRIDGE_WARNING` lines (`ModelLod.triangles` re-read `geometry.index` 528 times a frame); fixed in `9dbe79811`, which caches a level's triangle count. ⚠️ The launch and flight scenarios' deck click at (183,542) misses the TAKE THE DECK button (x 90–298, y 577–630): the PRD's recorded deck-click misses are that, not hitches. + +#### Phase 2 — Textures stream by mip + +- [x] AC-4 [local]: A KTX2 texture uploads its resident tail first, then fetches the wanted levels with `Range` requests. A host without `Range` keeps the full file. proof: `pnpm exec vitest run packages/core/__tests__/texture-streaming.spec.ts` — Evidence (2026-11-09): 21/21 green. `openKtx2Stream` (`packages/core/src/texture-streaming.ts`) reads a 4096 px, 13-level file with one 64 KiB `Range` read (under 2% of the file; it holds the 64 px tail and two more levels), hands back a whole KTX2 of those levels for the upload, then one more `Range` read per want (300 px wants 10 levels, from byte 65536 on), coarse first; the full chain matches the source level for level. A host that answers 200 is read once and kept whole. BasisLZ files keep the kept levels' image descriptors. three's own `ktx-parse` reads every rebuilt file with the source's descriptor, key/value data, level bytes and BasisLZ descriptors. A host that answers from the wrong offset fails with `TN_KTX2_FETCH`. The upload into the engine and Midway's bytes are AC-5. +- [x] AC-5 [local]: On cooked Midway at 40 Mbps, texture bytes before first play drop below the AC-2 result, and the owned engine (the default) draws the streamed mips. proof: the AC-2 playtest, with `TN_STREAMING_POOL` read from the console — Evidence (2026-11-10): packs from `911c7cb8b`, KTX2 cook (68 `.ktx2`, 24 coarse companions), owned engine, RTX 2080 nvidia/turing, launch playtest at 40 Mbps / 20 ms, 2 runs per arm. `awaitedBytes` **22,376,134** in both runs (AC-2: 49,950,371), first play 10.44 / 10.46 s (AC-2: 17.8 s). On the wire (cold context, 40 Mbps / 20 ms, resource buffer raised to 100,000, 252–261 entries, so nothing truncated): `.ktx2` bytes ending before ready are 3,523,064 with `streamTextures: false` in both runs, and 1,265,962 and 4,115,588 streamed. The full-model upgrade and its whole KTX2 files start as soon as the coarse parse ends, so in some runs they finish before the reveal (follow-up below). The bar holds on `awaitedBytes`, the AC-2 metric, and on the pool's 2,870,667 B held back. `TN_STREAMING_POOL` before ready: 23 files, 23 ranged, 338,575 resident of 3,209,242 B; `upgraded: 109` after. 0 `TN_ASSET_LATE`, 0 `TN_ASSETS_STREAM_FAILED`, 0 console errors. `awaitedBytes` counts manifest bytes, so it is equal with and without texture streaming. Every run of every arm exits 1 on the same two frame asserts: `maxPassTriangles.main` 1.30–1.31 M (carried since AC-1) and `maxFrameMsP95` 33.35 ms on Midway's pinned 0.3.12 build too (2 runs, 33.35 ms, engine frame 6.95 ms against 4.65 ms on this branch): the display cadence locked at two vsyncs that day, not this change. +- [x] AC-6 [local]: A fresh visual judge returns NEUTRAL or IMPROVEMENT for the settled spawn frame and one flight pose, streamed against `streamTextures: false`, two runs per arm. proof: headed `--browser-recipe webgpu` captures, `adapter.info` checked, images on the PR — Evidence (2026-11-10): packs from `f0b6911a9`, headed WebGPU, `adapter.info` nvidia/turing, spawn at ready + 20 s and flight at the airborne start + 6 s, 2 runs per arm. The first judge returned **REGRESSION**: the SBD's wing star smeared, because the swapped-in whole texture lost the `anisotropy` Midway set (fixed in `f0b6911a9`, red-green in `model-lod-runtime.spec.ts`; frames kept in `~/Downloads/prd-860-ac6/v1-regression/`). On the fix, a second fresh judge returned **NEUTRAL** for both poses: wing-star Laplacian 1,367 / 1,366 streamed against 1,364 / 1,329 at spawn and 818 / 815 against 812 / 820 in flight, no wrong colour, no new artifact. The SBD texture reads 2048² BC7, all 12 levels written, sampler `maxAnisotropy` 16. Frames: branch `prd-860-screenshots`, folder `ac6/` (`055ece4`). + +#### Phase 3 — Meshes stream by LOD + +- [x] AC-7 [local]: The model cook writes each streamed LOD with its own vertex and index ranges, and Midway's ships draw the coarsest level first. Bytes before first play drop below the AC-5 result. proof: `pnpm exec vitest run packages/assets/__tests__/model-pass*.spec.ts` and the AC-2 playtest — Evidence (2026-11-09, cook): the cook writes 24 coarse companions for Midway (10.0 MB coarse against 60.6 MB full; `lod-generation.spec.ts` coarse tests, and `ba3ba62be` keys the compile cache on the companion after a warm cache served all 228 models without one). With the WebP cook, `streamMeshes` alone took `awaitedBytes` from 49,950,371 B to 32,794,675 B (−34.3%). Evidence (2026-11-10, KTX2 cook, the AC-5 runs): `streamMeshes: false` awaits **28,787,830 B** in both runs (first play 12.41 / 12.03 s); streamed meshes await **22,376,134 B** (−22.3%), first play 10.44 / 10.46 s. Texture streaming rides on the coarse parse, so the bar reads as meshes-whole against both streams on. +- [x] AC-8 [local]: A fresh visual judge returns NEUTRAL or IMPROVEMENT for the settled frames, streamed against `streamMeshes: false`, and the native desktop host draws the same settled frame. proof: the AC-6 capture recipe and `pnpm native:verify:desktop` on cooked Midway — Evidence (2026-11-09, web half): a fresh read-only judge returned **NEUTRAL** for the spawn pose (briefing over Enterprise's deck, 20 s after the reveal; `streamMeshes: false` r2, r3 against streamed r1–r4) and the flight pose (airborne start + 6 s; r2, r3 against r1, r3, r4), headed WebGPU, `adapter.info` nvidia/turing, frames in `~/Downloads/prd-860-ac8/`. Excluded: `before-r1-*` (the first cold load of that port never drew, which matches the parked first-load crash) and one black streamed flight frame (`after-r2-flight`), which did not recur in r3 and r4 with a console error hook (0 warnings). Native half: cooked Midway built with `threenative build --target desktop` on `tn-native-engine-player-v8` from this branch, `native-playtests/briefing.playtest.json` passes on the RTX 2080 (exit 0, 0 `TN_ASSETS_STREAM_FAILED`), and a second fresh judge returned **SAME**, no regression, for `native-desktop-spawn.png` against the web r3 frames: island, SBD, deck and crew at full detail, no material defect. The native proof is that Midway scenario, not `pnpm native:verify:desktop`, which runs the smoke example and not Midway. + +## Integration Ledger + +| Capability | Reachable consumer/trigger | Replaces / disposition | Evidence | +|---|---|---|---| +| Spawn-only boot | Midway `src/scenes/Midway.ts` `load()` → `ctx.assets` | One `Promise.all` of every asset becomes spawn awaits plus `prefetch` — **open**: `WorldView` cannot build a hull it has not loaded | 136.4 MB awaited today, 64.3 MB spawn-only | +| Late-asset report | `ctx.assets` request after the reveal → `TN_ASSET_LATE` | New line, no old path | `packages/core/src/assets.ts:84`, `packages/core/__tests__/assets-late.spec.ts` | +| Cooked Midway | `pnpm build` in the game → `public/assets.manifest.json` | 188 MB of uncompiled sources become a 228-entry manifest | 166.2 MB shipped, 165 MB `dist` | +| Texture mip streaming | `ctx.assets` KTX2 load → owned-engine mip upload | Whole-file KTX2 load stays only for hosts without `Range` | `openKtx2Stream` (`packages/core/src/texture-streaming.ts`), unit-proven; not yet called by `ctx.assets` (no KTX2 output on the default engine, [PRD-861](PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md)) | +| Mesh LOD streaming | `compileAssets` model pass → `ctx.assets` model load | Shared-buffer LOD chains stay for non-streamed models | cook: `coarseDocument` (`packages/assets/src/lod/coarse.ts`), the model pass writes `.coarse..glb` under `streamedLods`; runtime: `ctx.assets.model` draws it first and `upgradeStreamedGeometry` (`packages/core/src/model-lod.ts`) gives every coarse geometry the full model and chain in place; `assets.streamMeshes: false` reports `TN_ASSETS_STREAM_MESHES_OFF` | + +Fill the `file:line` entry points when each phase lands. + +## Execution Phases + +#### Phase 1: Midway boots on its spawn bytes + +**Status:** IN PROGRESS — the engine line and the cooked build landed; the load split and every +playtest gate are open, for the reasons this section records. +**Estimate:** about half a day. +**Files:** `/src/scenes/Midway.ts` and its asset build config (game layer: what the spawn +shows is the game's choice); `packages/core/src/assets.ts` (engine layer: `TN_ASSET_LATE`, which +every game needs); `/playtests/launch.playtest.json` and `flight.playtest.json` (network +profile). +**Implementation:** Measure AC-1 before any change. Cook the GLBs and audio with `compileAssets`. +Split `load()`: await the spawn carrier, the player aircraft, the sky and the spawn audio. Send the +enemy fleet, distant hulls and later audio to `prefetch` in the order the flight reaches them. +**Verification:** the AC-1 to AC-3 commands. Work on Midway at `~/projects/threenative/sandbox/midway-open-pacific` +(`` above). Engine edits go into `packages/` and are reinstalled with +`scripts/midway-bench/install.sh`, never patched into the game copy. + +**Landed:** + +- `TN_ASSET_LATE` (`packages/core/src/assets.ts:84`; the reveal is marked in `startBackgroundFill` + at `:1558`, the `prefetch` warm set is at `:1142`): one line per file a scene asks for after the + reveal that the game did not `prefetch`, carrying `{ bytes, kind, ms, path }`. Three unit tests in + `packages/core/__tests__/assets-late.spec.ts`; + `pnpm exec vitest run packages/core/__tests__/assets*.spec.ts` is green (94 tests) and + `packages/create-threenative/capabilities.json` carries the constraint. +- Midway's sources moved to `assets/` (the pipeline's source directory; source and output must be + disjoint or the bake refuses with `TN_ASSETS_OVERLAP`), so `pnpm build` now cooks 228 files and + `public/assets.manifest.json` is Midway's first real manifest. +- The 106 logical paths, the cockpit texture loader and the instancing pass, as recorded under + "What execution measured" above. +- Both playtests carry the 40 Mbps / 20 ms profile with console and network artifacts + (`/playtests/launch.playtest.json`, `flight.playtest.json`). + +**Measured:** 166.2 MB shipped cooked from 188 MB of sources; the runtime reports 139.5 MB awaited +before first play (visible 42.2, unattributed 21.9, off-screen 75.4); the bake skips 63.2 MB of model +and 13.2 MB of texture compression for want of an engine decoder ([PRD-861](PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md)). + +**Landed 2026-11-09:** + +- Midway `f8591d4` (game layer): three dead loads deleted; the hulls the spawn camera does not + show load after the reveal behind one `prefetch` and the world builds each ship when its model + lands (`ModelPendingError`, `WorldView.buildStreamed`, in `addInSlices` slices); the cockpit + interior attaches late (`attachCockpitInterior`); speech, combat and enemy-engine cues load late. +- `TN_ASSET_UNUSED` (`packages/core/src/startup-offscreen-bytes.ts`, `measureUnusedModels`, + `reportUnusedModelsLater`): a minute after the reveal, every awaited model no scene object came + from, with its bytes. It is the line that would have named Midway's 22.9 MB of dead loads; the + offscreen line could only call them not-placed. Tests in `startup-offscreen-bytes.spec.ts`. +- The playtest handshake gets the startup window (`packages/playtest/src/runner/bridgeClient.ts`): + core answers `describe` only after the start scene's `load()`, so a 30 s handshake failed every + Midway run at 40 Mbps 370 ms short. A failed `load()` now rejects `describe` at once + (`game-startup.ts`), instead of leaving it to time out. +- The web engine parses a late model off the main thread (`web_host.cpp` + `tnw_web_load_gltf_begin/done/finish`, one loader pthread; `gltf-loader-web.ts` awaits it). A + DevTools trace of Midway's stream-in had 2.36 s of `loadGltf` long tasks on the main thread; after + it, none. The loader thread builds engine objects beside the main thread, so the object, material, + geometry and attribute id counters are atomic (`concurrent_ids`: red 3/3 with plain counters, + green 3/3). Per-thread CPU: the shared eight workers decode a landing model at 649% for ~2 s, and + idle workers read 1-2% CPU each, so the pool parks rather than spins. A pool of two for the loader + alone (`JobPool::useOnThisThread`) was tried and reverted: on a quiet host it passed the frame + bound 2/5 against the shared pool's 1/3, no measured win, and it left a third of the decode lanes. +- `packages/core/src/texture-streaming.ts`: KTX2 level index, the UE wanted-mip rule and + coarse-first mip ranges, pure and tested — Phase 2 groundwork, not yet exported. + +**Open, in the order it blocks this phase:** + +1. Frame time while the fleet streams in. Paired launch runs in a private network namespace (the + host's docker churn fails fetches with `ERR_NETWORK_CHANGED`), frame p95 bound passed: baseline + 2/3, streaming with the synchronous parse 0/3, with the loader thread 1/3. On a quiet host (load + 9-18, 5 runs per arm) the baseline passed 4/5 (the fifth had no GPU reading). The streamed + arms' misses are one 33 ms window at first play with one long task. What still runs on the + main thread then: Midway's `attachCockpits` (0.9-2.3 s of bridge calls, game layer), pipeline + compiles, and Akagi's ~220 ms merged stand-in. Hitches there also cost the launch scenario a + one-tick `KeyC` press and the deck click (HUD `btn-camera`, `center-tip`), which the baseline + never meets because nothing streams after its reveal. +2. AC-3's scenario: one that flies to the enemy fleet. +3. Engine follow-up (owner question 2026-11-09, "can this be pushed to the engine?"): a + `ctx.assets.model(path, { defer: true })` handle that returns a placeholder group at once and + fills it when the bytes land, so a game defers a file with one flag instead of Midway's + restructure. Fully automatic deferral was rejected: games that read geometry at build time + (Midway measures wingspans and bakes stand-ins) would break silently. +4. The compression question stays with [PRD-861](PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md). +5. Follow-up (2026-11-10, AC-5 wire bytes): a streamed model's full file and its whole KTX2 files + start downloading when the coarse parse ends, at `soon` priority, so on some runs they share the + link with the first scene before the reveal. Holding them for `startBackgroundFill()` in the + game's own loader would keep the first-play wire bytes flat; it also keeps low mips on screen + longer after the reveal, so it needs its own visual judge. Not a box of this PRD. + +#### Phase 2: Textures stream by mip + +**Status:** DONE — AC-4, AC-5 and AC-6 met (2026-11-10). +**Estimate:** 2 to 3 days. +**Files:** a new `packages/core/src/texture-streaming.ts` (wanted mips, request queue, pool +report); `packages/core/src/assets.ts` (KTX2 `Range` path); `packages/runtime-native/src/webgpu/bindings_resources.cpp` +(single-level upload, if missing); `packages/core/__tests__/texture-streaming.spec.ts`. +**Implementation:** Port the wanted-mip formula and the visible-first order from the Unreal +reference above. Read the KTX2 level index from the header. Keep the in-flight bytes under a cap +in the manner of `MaxTempMemoryAllowed`. Grade from the render camera, as `WorldCells` does. +Update the feature map row to cite this PRD, then run `pnpm unreal:status`. +**Verification:** the AC-4 to AC-6 commands, plus `pnpm typecheck && pnpm lint`. + +#### Phase 3: Meshes stream by LOD + +**Status:** DONE — AC-7 and AC-8 met (2026-11-10). +**Estimate:** 2 to 3 days. +**Files:** the model pass in `packages/assets/src/` (per-level vertex and index ranges, coarsest +first); `packages/core/src/assets.ts` (LOD fetch by projected error, the same `autoLod` error +measure that `WorldCells` uses); `packages/assets/__tests__/`. +**Implementation:** Port the mesh LOD rule from the Unreal reference. Reuse the cooked chain and +its projected-error switch distances. A skinned mesh keeps its current path. +**Verification:** the AC-7 and AC-8 commands, plus `pnpm native:build` before the native frame. diff --git a/docs/PRDs/done/assets/PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md b/docs/PRDs/done/assets/PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md new file mode 100644 index 0000000000..a78b51a8cf --- /dev/null +++ b/docs/PRDs/done/assets/PRD-861-the-owned-engine-decodes-meshopt-and-ktx2.md @@ -0,0 +1,117 @@ +--- +prd_contract: v1 +--- + +# PRD-861 — The owned engine decodes meshopt and KTX2 + +**Status:** DONE — the owned web engine decodes meshopt and KTX2 in its own glTF loader, and the web cook ships both for it. +**Priority:** P1 — every cook for the default engine ships its geometry and its textures uncompressed, which is why cooked Midway is 166.2 MB instead of tens of MB (PRD-860 (the first scene plays before the game finishes downloading, not yet filed in this tree) records the measurement). +**Complexity:** 6 (MEDIUM-HIGH) — 5 implementation files (+2): the bake gate, the engine's glTF loader, its CMake wiring, a TypeScript unit test and a conformance case, across the TypeScript packages and the C++ engine build (+2); one new vendored transcoder (+2). +**Owner:** João (owner); an agent executes +**Depends on:** PRD-860 (the first scene plays before the game finishes downloading, not yet filed in this tree) is the consumer whose byte ceiling this unblocks. `EXT_texture_webp` in the same loader is the pattern to follow (`packages/runtime-native/cmake/NativeEngineCore.cmake:239`). + +## Context + +The owner asked (2026-11-09) why a cook for the owned engine cannot ship compressed assets: "we should make this work with wasm engine, no?" It should. Today it cannot, and the reason is a missing decoder in the engine's glTF loader, not a missing format. + +What ships today: + +- `webRuntimeDecoders` (`packages/create-threenative/src/web-engine.ts:11`) answers `{ ktx2: false, meshopt: false }` for every game that does not set `engine: "threejs"`. `engine: "threenative"` is the default on web and desktop, so the default cook ships uncompressed geometry and textures and prints the reason itself: `TN_ASSETS_COMPRESSION_SKIPPED model: 29 file(s), 63.2 MB ... texture: 52 file(s), 13.2 MB` on Midway's bake. +- Models on the web build do not go through three's `GLTFLoader`: `packages/create-threenative/src/web-engine.ts:266` rewrites `three/addons/loaders/GLTFLoader.js` to `packages/three-native/src/addons/gltf-loader-web.ts`, which hands the bytes to the engine's own loader in Wasm (`tnw_web_load_gltf`). That loader is `packages/runtime-native/src/engine/assets/gltf/loader.cpp` over cgltf, and it refuses the two extensions by name: `kSupported` (`:107`) omits `KHR_texture_basisu` and `EXT_meshopt_compression`, and `loader.h:11` says why — "needs a decoder this loader does not have (draco, meshopt, basisu, avif)". +- **The meshopt decoder is already in this repository and is never compiled.** `packages/runtime-native/third_party/meshopt/` holds `vertexcodec.cpp`, `indexcodec.cpp`, `vertexfilter.cpp` and `meshoptimizer.h` — the decode-only subset. No CMake file mentions it: `packages/runtime-native/cmake/NativeEngineCore.cmake:230` lists `loader.cpp`, `cgltf_impl.cpp` and `image_decode.cpp` and nothing else. +- cgltf already parses the extension and documents the seam this PRD fills: `cgltf_buffer_view` carries `has_meshopt_compression` and `meshopt_compression`, and its `data` member is "overrides buffer->data if present, filled by extensions" (`packages/runtime-native/third_party/cgltf/cgltf.h:321`, `:322`). +- KTX2 has no transcoder in the tree: `third_party/` has `webp`, `stb`, `cgltf`, `meshopt` and no basisu. Three's `KTX2Loader` and `basis_transcoder.wasm` are bundled for the *JS* texture path (`packages/core/src/assets.ts` `loadCompiledKtx2`), and the browser engine's compressed-upload lane is gated by the conformance case `compressed-texture-preparation` (`packages/runtime-native/conformance/registry.json`), so a `CompressedTexture` reaches the GPU today. The native player's lane refuses it by design: `packages/runtime-native/src/engine/player/core-compressed-texture-preparation.mjs:11` rejects with "WebGPU renderer lacks bounded compressed texture preparation". +- The native cooked-package path is a separate question and stays closed: `targetDecoders()` answers 0 on every target (`packages/runtime-native/src/engine/assets/package.cpp:108`) and `verifyPackage` fails a decoder-backed entry with `TN_NATIVE_KTX2_UNSUPPORTED` / `TN_NATIVE_MESH_COMPRESSION_UNSUPPORTED` (`:114`). + +## Solution + +Two slices, smallest first. Each flips one half of `webRuntimeDecoders` and proves it on a cooked web game running the owned engine. + +```mermaid +flowchart LR + cook[compileAssets] -->|meshopt bufferViews| glb[GLB] + glb -->|tnw_web_load_gltf| loader[loader.cpp + cgltf] + loader -->|has_meshopt_compression| decoder["meshopt (vendored)"] + decoder -->|view->data| scene[scene] + cook -->|KTX2 images| glb2[GLB] + glb2 --> loader + loader -->|KHR_texture_basisu| transcoder["basisu transcoder -> CompressedTexture"] + transcoder --> upload[compressed upload lane] +``` + +1. **Meshopt, because the decoder is already here.** Compile the vendored decoder into `tn_engine_gltf`, decode every bufferView that carries the extension into a retained buffer, and point `cgltf_buffer_view::data` at it before anything reads the view. Then `webRuntimeDecoders` answers `{ ktx2: false, meshopt: true }` for a web build — the two formats resolve independently, so half the win does not wait for the transcoder. +2. **KTX2, which needs a transcoder.** Vendor the Basis Universal transcoder (decode only), transcode `KHR_texture_basisu` images to the format the target device supports, and hand the result to the compressed-upload lane the conformance case already gates. `webRuntimeDecoders` then answers both true for a web build. + +### Conventions + +- **The gate is per format and per target, not per renderer.** `engine: "threejs"` was standing in for "this build can decode", which stopped being true when the owned engine became the default. A build says what its loader can decode; `engine` says which renderer draws. +- **A format the loader cannot decode stays refused by name**, never silently loaded uncompressed: `TN_NATIVE_GLTF_EXTENSION_UNSUPPORTED` already does this, and the cook's `TN_ASSETS_COMPRESSION_SKIPPED` names the bytes it withheld. +- **Auto by default.** Once a target decodes the format, the bake emits it with no game-side switch, and a game that wants the uncompiled path keeps `assets.textures: "none"` / the model pass switches. +- **Native stays honest.** The player's package path answers `targetDecoders() == 0` until a decoder is qualified there; a desktop or Android build keeps the uncompressed cook and says so. + +### Risks + +- ⚠️ Meshopt decoding allocates per bufferView. The decoded bytes must outlive the whole load and be freed with it; a per-view `new` would leak on every model. +- ⚠️ The vendored decoder is the decode-only subset and carries no version pin in a CMake comment today. Pin it the way `webp-source` is pinned, or a future bump silently changes the bitstream. +- ⚠️ KTX2 is a licence decision as much as an engineering one: Basis Universal's transcoder has its own terms, and `third_party/` is reconstructed by `download-deps` rather than patched. +- ⚠️ A cooked model that fails to decode must fail loudly at load, not draw an empty mesh: the LOD plugin's `failClosed` is the precedent. + +Thresholds: a cooked web build of a template, run on the owned engine, is the subject; "settles" means `ctx.startup.whenReady()`. The box for each criterion sits with the phase that proves it. + +## Integration Ledger + +| Capability | Reachable consumer/trigger | Replaces / disposition | Evidence | +|---|---|---|---| +| Meshopt geometry | `compileAssets` model pass → `webRuntimeDecoders` (`packages/create-threenative/src/web-engine.ts:22`) → `tn_engine_meshopt` (`packages/runtime-native/cmake/NativeEngineCore.cmake:148`) in the loader (`loader.cpp:119`); the web engine refuses to link without it (`NativeEngineCore.cmake:1169`) | `webRuntimeDecoders`'s blanket `false` for the default engine | AC-1, AC-4 | +| KTX2 textures | an authored `KHR_texture_basisu` model → `tn_engine_basisu` (`NativeEngineCore.cmake:164`, pinned at `scripts/download-deps.mjs:321`) → `decodeImage` RGBA8 transcode (`image_decode.cpp:34`) via `loader.cpp:179`; the cook does not emit it for this engine (`web-engine.ts:22`) | the JS-only `KTX2Loader` path stays for a standalone `.ktx2` file | AC-5 | +| Skip line | `skippedCompressionRows` names only the decoders a target lacks (`packages/assets/src/compile-rows.ts:479`) | the model row named meshopt and KTX2 even when meshopt ran | AC-1 | +| Native package decoders | `verifyPackage` → `targetDecoders()` | unchanged: still 0 until a target qualifies | AC-6 | + +## Execution Phases + +#### Phase 1: meshopt geometry in the engine's loader + +- [x] AC-1 [local]: the bake for a web game with `engine: "threenative"` emits meshopt geometry and reports no `TN_ASSETS_COMPRESSION_SKIPPED model` line. proof: `pnpm --filter create-threenative exec threenative build` in a cooked template, then read its bake log — Evidence: half. 2026-11-09, starter scaffolded from this branch's packs, `pnpm exec threenative build` exit 0: `model native-proof.glb (EXT_meshopt_compression, KHR_mesh_quantization): 624 -> 1260 bytes`, manifest extensions the same. + Closed 2026-11-09 with KTX2 on (PRD-867 AC-4): a starter scaffolded from the branch's packs, `pnpm exec threenative build` exit 0, prints no `TN_ASSETS_COMPRESSION_SKIPPED` line (before: `model: 2 file(s), 0.3 MB kept its textures out of KTX2`); `model mannequin.glb (EXT_meshopt_compression, KHR_mesh_quantization): 664944 -> 263628 bytes`. +- [x] AC-2 [local]: the engine's glTF loader decodes `EXT_meshopt_compression` and the scene it builds equals the decoded-from-uncompressed one. proof: `pnpm exec vitest run packages/three-native/__tests__/gltf-loader-web.spec.ts` and the corpus test in `packages/runtime-native/tests/native-engine/` — Evidence: 2026-11-09, `gltf-loader-web.spec.ts` with `native-ktx2.spec.ts` 32/32 passed; `tn-native-engine-gltf-meshopt-test` on the Wasm toolchain under node PASS `decoded_matches_source`, `khr_decodes_alike`, `malformed_is_refused` (3/3); red on an engine linked without meshopt: `TN_NATIVE_GLTF_EXTENSION_UNSUPPORTED EXT_meshopt_compression`; `tests/web-host-decoders.test.mjs` loads the fixture in headless Chromium through the real Wasm engine, 2/2. +- [x] AC-3 [local]: a cooked web template runs on the owned engine with meshopt geometry, and `TN_GPU_CLASS`/`adapter.info` name a real adapter. proof: `node packages/playtest/dist/runner/cli.js