Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# PRD-755 — The default frame stops copying textures it already holds

**Status:** IN PROGRESS
**Priority:** P1 — on Iris Xe, 14.0 ms of the 19 ms Hop Garden frame does not scale with the render resolution ([PRD-741](../../done/PRD-741-the-owned-engine-shades-fewer-pixels-and-its-temporal-resolve-writes-the-display-frame.md), final box); the copies and blits below are part of it, on every game with the default look.
**Complexity:** 4 (MEDIUM) — the temporal resolve and the post-target render path in the native renderer (+2), bit-identical output proven by readback (+1), a GPU A/B on Iris Xe (+1).
**Owner:** João
**Parent:** [PRD-400](../../performance/PRD-400-the-frame-gets-cheaper-one-measured-cost-at-a-time.md) (it owns the remaining fixed cost). Sibling: [PRD-756](PRD-756-the-temporal-resolve-feeds-one-shared-post-chain.md). Lane of the PR #574 follow-up.
**Depends on:** none.

**Update rule:** this PRD has no PR of its own. Its commits land on `lane/pr574-gpu` and merge into the PR #574 follow-up. Tick each box, with its result, in the commit that proves it.

## Problem

Found by reading the code and the PRD-741 per-pass table (2026-10-27):

- The resolve copies three textures into history every frame instead of swapping them: the display-size `RGBA16Float` colour, the input-size depth and the `RG8` motion mask (`packages/runtime-native/src/engine/renderer/post/traa.cpp:653-654`). That is about 50 MB of memory traffic a frame at 1080p (estimate: bytes read plus bytes written). The pass timer does not see copies, so no dashboard shows this cost.
- Every post-target render (bloom chain, local exposure, meters) draws its scene pass, then an `output` pass into the target (`Renderer::outputPass`, `packages/runtime-native/src/engine/renderer/renderer.cpp:4467`). The PRD-741 table holds 28 `output / 2 / N` occurrences, about 0.45 ms summed on Iris Xe (measured per pass).

## What Unreal does (UE 5.8.3)

- TSR keeps its history as render-graph textures and extracts the new ones for the next frame; it copies nothing. `UE 5.8.3: Engine/Source/Runtime/Renderer/Private/PostProcess/TemporalSuperResolution.cpp:3505-3518` (`QueueTextureExtraction`).

## Goal

The resolve and the post targets do the same work with no history copy and no extra output pass, the frame is bit-identical, and a test fails when either lever is undone.

### Phase 1: The resolve swaps its history

- [x] The colour history and the motion mask are swapped each frame instead of copied, and the resolve writes the next history directly. A counter reports the resolve's texture copies per frame. proof: a new ctest, red at 3 copies a frame, green at 0 (or 1 if the scene depth stays a copy, with the reason here); every `native_engine_traa_*` ctest passes with unchanged metric values. Result (`aecafb9ba`): `native_engine_traa_history_swap` red `steady=3 (colour 1, depth 1, mask 1)`, green `steady=1 (colour 0, depth 1, mask 0)`, also into a caller's target. The depth stays one copy: it is the caller's scene depth, which the resolve only samples. With a caller target, the next history is a third colour attachment (one 8 B/px write in place of an 8 B/px read plus write). `ctest -R "native_engine_traa|native_engine_renderer|velocity_needs_traa"` 95/95 (93/93 before, metric lines unchanged), rerun by the lead.
- [x] The resolved frame is bit-identical to the copy path over 60 frames of a moving scene. proof: a ctest that reads back both paths. Result: `native_engine_traa_history_swap_is_bit_identical` `120 frames compared, 0 differing`; with the swap undone, 118 differ, so the check is not vacuous.

### Phase 2: A post target draws without the extra output pass

- [x] A render into a post target whose output stage is an identity draws its scene pass straight into the target. proof: a ctest, red at one `output` pass per target render, green at 0. Result: already landed after the trace this PRD quotes (source `618c0670e`, 2026-10-25): `a0ecc6f0d` (PRD-748) gives a render target's renderer `skipOutputPass()` (`render_target_pass.cpp:34-35`), and `226ec257a` (PRD-741) guards it with `!sceneOnly_` (`renderer.cpp:4440`). Its red-green is that commit's: `native_engine_render_target` red at 1 output pass, green flat (57 to 33 passes a frame); rerun here, passed, and `ctest -L native-engine` 484/484 (gpu2 lane, no code change).

### Phase 3: Prove it on games

- [ ] Paired GPU A/B on Iris Xe, Hop Garden medium and the shooter, three pairs from one code state, GPU p50 and presented p95, load and temperature recorded. proof: the table here.
Hop Garden done, shooter open. 2026-10-28, Iris Xe at 0.72 medium, BEFORE PR #574 head `916682671`, AFTER `aecafb9ba`, Wasm hashes checked different. A first series (B1 A1 B2 A2 B3 A3) was rejected: every AFTER run started 5-10 C hotter (88-93 C). Balanced rerun, each run admitted below 78 C, all at 1200 MHz, presented p95 16.70 ms in every run:

| pair | order | BEFORE GPU p50 | AFTER GPU p50 | start C |
| --- | --- | --- | --- | --- |
| 1 | A then B | 18.87 | 18.68 | 74 / 76 |
| 2 | B then A | 18.91 | 19.11 | 77 / 77 |
| 3 | A then B | 18.97 | 19.07 | 75 / 80 |
| median | | **18.91** | **19.07** | |

No measurable gain on Iris Xe: +0.16 ms, inside the AFTER spread (0.43 ms). With `?tnGpuPasses=1`, `traaResolve` is 4.546/4.532 ms BEFORE and 4.556/4.566 ms AFTER: the third colour attachment costs what the two copies cost, and the copies never showed as passes of their own (inference: the copy engine overlaps them).
- [ ] The kit scenarios pass on both kits. proof: `pnpm test:templates` on Hop Garden and the shooter.

## Done when

Every box above is ticked.

## Decisions

- 2026-10-27, lead: the Phase 2 pixel box is moot and is removed. The skipped output pass drew into the renderer's own `RGBA8Unorm` `colorView_` (`renderer.cpp:4529, 4557`), never into the target, whose pixels are its own linear scene texture (`render_target_pass.cpp:90, 103`), so skipping it cannot change the target. Of the 0.45 ms, 23 `output / 2 / N` rows are 0.321 ms; the 5 `localExposure.*.outputBlit` rows (0.115 ms) have another producer, unverified.
- 2026-10-27, lead: follow-up, not a box here: a reflection pass still draws an unread output pass with the main tone-mapped program (`render_database.cpp:1768-1805`), about 0.3 ms at 0.5 scale per reflector per frame (estimate). Only the sailing kit places water reflectors, so it cannot move the Hop Garden number.

- 2026-10-27, lead: with no caller target the resolve still writes a third attachment beside `resolve_`. Resolving straight into the next history slot would remove that write too, at the cost of a second pipeline. Left for Phase 3's measurement to price.

## Blocked on
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# PRD-756 — The temporal resolve feeds one shared post chain and keeps a half-size history

**Status:** PARTIAL — Phase 1 Deferred (0.44 ms removable, under its 1 ms bar)
**Priority:** P1 — on Iris Xe at the 0.72 tier start, the TRAA resolve (4.36 ms), bloom at display size (1.55 + 1.21 ms) and local exposure (2.9-3.2 ms) are most of the 14.0 ms that does not scale with resolution ([PRD-741](../../done/PRD-741-the-owned-engine-shades-fewer-pixels-and-its-temporal-resolve-writes-the-display-frame.md), measured per pass).
**Complexity:** 6 (MEDIUM) — the resolve outputs and the native post passes (+2), core's bloom and local exposure nodes (+1), a history format with an adapter fallback (+1), visual judge rounds (+2).
**Owner:** João
**Parent:** [PRD-400](../../performance/PRD-400-the-frame-gets-cheaper-one-measured-cost-at-a-time.md). Sibling: [PRD-755](PRD-755-the-default-frame-stops-copying-textures-it-already-holds.md). Lane of the PR #574 follow-up.
**Depends on:** PRD-755 Phase 1 (the resolve writes its history in place).

**Update rule:** this PRD has no PR of its own. Its commits land on `lane/pr574-gpu` and merge into the PR #574 follow-up. Tick each box, with its result, in the commit that proves it.

## Problem

- Bloom and local exposure each downsample the resolved display-size frame on their own (PRD-741 table: a 960x540 to 30x17 chain per effect, plus meters).
- The history is `RGBA16Float` with 32-bit shader math (`packages/runtime-native/src/engine/renderer/post/traa.cpp:503`). The resolve still costs 3.3 ms with one colour load instead of nine (PRD-741 deletion probe), so memory traffic, not arithmetic, sets its cost (inference).
- The history keeps the game's colour alpha (`traa.cpp:97`, `:346`), so a format without alpha is not a drop-in change.

## What Unreal does (UE 5.8.3)

- TSR writes half, quarter and eighth-resolution scene colour while it resolves, and the post chain starts from them. `UE 5.8.3: Engine/Source/Runtime/Renderer/Private/PostProcess/TemporalAA.h:216-235`.
- Bloom, eye adaptation and local exposure share one scene downsample chain. `UE 5.8.3: Engine/Source/Runtime/Renderer/Private/PostProcess/PostProcessing.cpp:214` (`r.PostProcessing.DownsampleChainQuality`).
- TSR history is `R11G11B10` and its math is 16-bit by default. `UE 5.8.3: Engine/Source/Runtime/Renderer/Private/PostProcess/TemporalSuperResolution.cpp:64` (`r.TSR.History.R11G11B10`), `:99` (`r.TSR.16BitVALU`).

## Goal

The default frame on Iris Xe loses at least 1.5 ms of GPU time with a NEUTRAL or IMPROVEMENT visual verdict.

### Phase 1: One shared post chain

- [x] Size it: the bloom chain, the display-size bloom passes and the local exposure passes, per pass, on the current build, and what a shared chain removes. If that is under 1 ms, this phase moves to Deferred with the number. proof: `?tnGpuPasses=1` on Iris Xe, two runs, table here.
Result (2026-10-28, Iris Xe, Hop Garden medium at 0.72, PR #574 head `916682671`, runs PB1 and PB2, 3614 and 3514 steady frames, frame-total p50 19.56 and 19.25 ms):

| group | PB1 ms | PB2 ms |
| --- | --- | --- |
| scene (1382x778) | 4.239 | 4.234 |
| traaResolve (1920x1080) | 4.546 | 4.532 |
| bloom chain (960x540 to 30x17 and back) | 1.623 | 1.621 |
| display-size bloom | 0.596 | 0.596 |
| local exposure and small chain | 2.892 | 2.888 |
| auto-exposure meters | 0.180 | 0.180 |
| output (1920x1080) | 1.475 | 1.475 |
| other (aux, velocity, input-size post, shadow) | 3.248 | 3.223 |

A shared chain removes at most the bloom chain's own downsample from the display frame: 960x540 0.312, 480x270 0.068, 240x135 0.029, 120x68 0.014, 60x34 0.010, 30x17 0.008 ms, **0.44 ms** in all (PB2). The local exposure passes are a bilateral grid at input size, not a mip chain, so they would not read the shared levels (inference from their extents: 240x576, 240x540, 480x9). Under 1 ms: Phase 1 is **Deferred**.
- [ ] The resolve writes a half-resolution colour, and bloom and local exposure start from one shared chain. proof: ctests for the chain levels; the core look specs pass.
- [ ] Visual judge NEUTRAL or IMPROVEMENT on Hop Garden, Lantern Vale and the shooter, two runs per arm. proof: the verdict here and the sheets on the follow-up PR.

### Phase 2: Half-size history

- [ ] Audit who reads the history alpha and whether the adapters in the supported matrix expose `rg11b10ufloat-renderable` and `shader-f16`. proof: the findings here, with file:line.
- [ ] The history moves to `RG11B10Ufloat` (alpha kept elsewhere if it is read) with 16-bit math where the adapter allows it, and the current path where it does not. proof: ctests on both paths; the resolve ms on Iris Xe, two runs per arm.
- [ ] Visual judge NEUTRAL or IMPROVEMENT on the same kits, two runs per arm. proof: the verdict here and the sheets on the follow-up PR.

### Phase 3: Prove it on games

- [ ] Paired GPU A/B on Iris Xe, Hop Garden medium and the shooter, three pairs from one code state, GPU p50 and presented p95, load and temperature recorded. proof: the table here.

## Done when

Every box is ticked, or a phase is Deferred with its measured number.

## Decisions

- 2026-10-28, lead: Phase 1 is Deferred with its number, as its sizing box set: a shared chain removes at most 0.44 ms on Iris Xe at 0.72. Revisit when a `?tnGpuPasses=1` profile shows the display-frame downsample levels at 1 ms or more, for example at a 4K display. The larger cost is the resolve itself (4.5 ms), which is Phase 2's target.

## Blocked on
73 changes: 73 additions & 0 deletions packages/core/__tests__/render-camera-cull.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,64 @@ import { describe, expect, it, vi } from "vitest";
import { formatProjectionWindow } from "../src/projection-marker.js";
import {
DEFAULT_MINIMUM_PROJECTED_PIXELS,
type IAnimationRateState,
RenderCameraCull,
alwaysRender,
inFrustum,
rayMeetsSphere,
setFrustumPlanes,
trackAnimationRate,
} from "../src/render-camera-cull.js";

// The camera pass tests its frustum and forward ray in plain numbers: on the web engine three's Frustum,
// Ray and Sphere are engine objects, one boundary call per test, 70 a frame on the shooter.
describe("the cull's plain frustum and ray math", () => {
// The win is the class, not the answer: the equivalence test below passes just as well if the math is
// right while an engine Ray, Frustum or Sphere comes back to hold it, which is the cost the lever
// removed (66e637840: cull 0.78 -> 0.66 ms on the laptop, about 70 boundary calls a shooter frame).
// So count what a real cull frame constructs, not what the source spells: on the web engine each of
// these is an object across the boundary, whichever helper or alias reaches it.
it("constructs no three Frustum, Ray or Sphere in a real cull frame", () => {
// One frame of a small scene, one object per plain-math path: a rig part on the camera's forward
// ray (near band), a rig part off it (the band test), an off-camera shadow caster (the frustum
// test) and a sub-pixel prop to cull.
const onRay = new Group();
onRay.add(ball(1, -60));
const lateral = ball(1, -60);
lateral.position.x = 8;
const offRay = new Group();
offRay.add(lateral);
const onRayState = rateState();
const offRayState = rateState();
trackAnimationRate(onRay, onRayState);
trackAnimationRate(offRay, offRayState);
const caster = ball(0.01, -1_000);
caster.position.set(100_000, 0, -1_000);
caster.castShadow = true;
const scene = sceneWith(onRay, offRay, caster, ball(0.1, -1_000));
const view = camera();

// The scene above is scaffolding. Everything from here is the cull: its own fields and its frame,
// since the engine objects the lever removed were built once per cull and reused every frame.
engineObjects.Frustum = 0;
engineObjects.Ray = 0;
engineObjects.Sphere = 0;
const cull = new RenderCameraCull();
cull.prepareAnimation(scene, view, 720);
cull.apply(scene, view, 720);

// The frame reached every path, so the zero below is not vacuous: the on-ray rig took the near
// band, the lateral one the band test (4, not the unseen 8), and the caster the frustum test.
expect([onRayState.rate, offRayState.rate]).toEqual([1, 4]);
expect(cull.report.exemptShadowCasters).toBe(1);
expect(cull.report.culled).toBe(1);
expect({
Frustum: engineObjects.Frustum,
Ray: engineObjects.Ray,
Sphere: engineObjects.Sphere,
}).toEqual({ Frustum: 0, Ray: 0, Sphere: 0 });
});

it("answers as three's Frustum and Ray for cameras and spheres all around", () => {
let seed = 7;
const random = () => {
Expand Down Expand Up @@ -85,6 +133,8 @@ describe("the cull's plain frustum and ray math", () => {
*/

const vector3Allocations = vi.hoisted(() => ({ count: 0 }));
/** Constructions of the three classes that are engine objects on the web engine, counted, not spied. */
const engineObjects = vi.hoisted(() => ({ Frustum: 0, Ray: 0, Sphere: 0 }));

vi.mock("three", async (importOriginal) => {
const actual = await importOriginal<typeof import("three")>();
Expand All @@ -96,9 +146,32 @@ vi.mock("three", async (importOriginal) => {
vector3Allocations.count += 1;
}
},
Frustum: class CountingFrustum extends actual.Frustum {
constructor(...planes: ConstructorParameters<typeof actual.Frustum>) {
super(...planes);
engineObjects.Frustum += 1;
}
},
Ray: class CountingRay extends actual.Ray {
constructor(...args: ConstructorParameters<typeof actual.Ray>) {
super(...args);
engineObjects.Ray += 1;
}
},
Sphere: class CountingSphere extends actual.Sphere {
constructor(...args: ConstructorParameters<typeof actual.Sphere>) {
super(...args);
engineObjects.Sphere += 1;
}
},
};
});

/** A rig's rate state, as `AnimationPlayer` registers one (the cull only writes `rate` and `frame`). */
function rateState(): IAnimationRateState {
return { rate: 1, frame: 0, source: undefined, evaluated: false, evaluation: 0 };
}

/** A perspective camera at the origin looking down -Z, matching the default Three camera. */
function camera(fov = 60): PerspectiveCamera {
const result = new PerspectiveCamera(fov, 1, 0.1, 1_000_000);
Expand Down
Loading
Loading