Skip to content

Present snapshots with the encode thread, on by default on iOS; wait ledger (DM_NVN_WAITS) - #157

Closed
v0lumehi wants to merge 5 commits into
seedds:mainfrom
v0lumehi:snap-enc
Closed

v0lumehi wants to merge 5 commits into
seedds:mainfrom
v0lumehi:snap-enc

Conversation

@v0lumehi

@v0lumehi v0lumehi commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Part of #142: Breath of the Wild performance on iPhone and Mac (60 → 120 fps, then 4K). Start there to see how the PRs fit together and the merge order.

Builds on #87 (present snapshots): this PR contains its commits; only the last commit is new here (compare).

With this PR and #158, Breath of the Wild runs at a steady 60 fps at docked resolution on an iPhone 17 Pro Max.

Problem

On the iPhone, Metal encoding runs on its own thread by default (DM_NVN_ENC_THREAD). #87 turned present snapshots off for any present that comes from another thread than the submitting one while that thread is on, because the frame's work could still sit in the encode thread's record queue: a copy committed at once would run before it, and a GPU wait for the frame's fence would hold the queue in front of that very commit. Breath of the Wild presents from its own Presentation Thread, so on the iPhone it never got a snapshot. It took its two swapchain textures back only on display ticks, and a frame that ran long was shown late.

At 60 fps that matters more than it looks. iOS lowers the GPU clock until the GPU is about 85–92 % busy, so the frame has little headroom once the clock has dropped (after about 45 s here). Late frames make the frame step step to 30 fps. At 30 fps the governor sees even more idle time and lowers the clock to about a third, and the game never gets back to 60. That was the 30 fps in #142.

Two runs on the same iPhone, both without Metal's capture layer:

The two runs also differ in the frame step and in #158, so they show the combination, not this PR alone. My earlier iPhone CPU numbers in #142 were taken with Metal's capture layer on, which was enabled in my local Xcode project and not in the repo.

Change

  1. Snapshots with the encode thread. With DM_NVN_ENC_THREAD on, the copy into the ring slot is a block of the encode thread's own queue (nvn_rec.h rec_own_begin / rec_own_push). It goes behind every block published so far, and a submission publishes its records before it returns, so the copy runs after the frame's work. The block holds the copy's commit, the ring event, and a note that the present waits for before its blit (snap_commit_wait): the encode thread's progress, not the GPU.
    • The presenting thread neither waits for the encode thread nor commits anything itself, so a present from another thread gets a snapshot too.
    • The copy has its own barriers: after the frame's work and the slot's last blit, and before the game's next writes. The CPU waits for the slot's previous copy only when that copy has not finished.
    • DM_PRESENT_SNAP=<n> is the ring's length, the number of frames the game may run ahead of the screen (2 to 4; 1 means 2).
  2. On by default on iOS. DM_PRESENT_SNAP=0 turns snapshots off there. On the Mac they stay off unless DM_PRESENT_SNAP is set, as in Mac: present snapshots (DM_PRESENT_SNAP=1): a game with two swapchain textures runs a frame ahead, and the frame step judges lateness by the screen #87.
  3. Wait ledger (DM_NVN_WAITS=1, diagnostic, off by default). It counts every place a thread waits on the CPU for the GPU or the encode thread: the game's SyncWait (split into newest fence, older fence, acquire sync and polls), QueueFinish, the allocator, the occlusion ring, multi-draw counts, readbacks, the snapshot ring, upload batches and slots, the encode thread drain, the uniform copy worker, and old-queue command buffers. Every 300 frames it logs calls, blocked calls and blocked ms per frame ([waits] up to frame N, per frame: …), and how long after the GPU end a blocking SyncWait wakes. DM_NVN_TSV rows gain @sync_ms, the frame's blocked SyncWait time. An acquire sync now has its own ledger line (hle_nvn.c). This is how the numbers below were taken, so I kept it in this PR.

It merges with #141 (upload batches flushed outside a submission are handed to the encode thread at once): the finish path still hands over the encode thread's current block before it waits for upload batches, and now records the wait in the ledger.

Measurements

iPhone 17 Pro Max, iOS 27.0.1, Metal's capture layer off, a route on the Great Plateau, frame step active. The build was these commits plus #158 on the previous base. Frames are counted from frame 1500.

300 s of steady play 120 s
Frames counted 17,392 7,324
On-screen interval p50 / p90 / p99 / max 16.67 / 16.67 / 16.67 / 16.7 ms the same
Frames over 25 ms 0 0
Hold 1 throughout, no step to 30 the same
Thermal state 0 → 1 after 198 s 0 → 1 after 83 s
Submit thread CPU per frame 3.30 ms –
Encode thread per frame 4.51 ms –
GPU busy per frame 13.2 ms 12.4 ms
Blocked SyncWait per frame 0.83 ms 0.05 ms

The GPU clock (the median of a tiny probe pass) is full, 6–7 µs per pass, until about frame 2,700. It then drops to about half, 11–14 µs, and stays there, and the game still holds 60 fps.

The cost is latency. From present to screen takes about 74 ms with snapshots (ring depth 2), against about 46–60 ms without. A ring of depth 1, or a present without the copy, is a planned follow-up.

On the Mac (Breath of the Wild, encode thread and snapshots on, DM_FPS_TARGET=120), the picture is right (DM_NVN_SHOT of the snapshot) and there is no red line. In the ledger, one blocking SyncWait per frame remains, 4.9–7.0 ms on the newest fence by the main thread, woken 0.29 ms after the GPU end. The encode thread drain at the end of each submission blocks 0.25–0.39 ms, and the uniform copy worker 0.1–0.16 ms. Nothing else blocks in steady play. These counts come from a loaded machine (load 6–8), so they are counts only, not timings.

Reproduce

  • iPhone: the defaults (encode thread and snapshots on). DM_PRESENT_SNAP=0 for the comparison; DM_NVN_WAITS=1 for the ledger; DM_NVN_TSV for per-frame rows with @sync_ms.
  • Mac: DM_NVN_ENC_THREAD=1 DM_PRESENT_SNAP=1, plus DM_NVN_WAITS=1. The log says present snapshots (DM_PRESENT_SNAP=…) at the first present, and [waits] … every 300 frames.

Risk

This changes the iPhone's default present path for every NVN game. A game that reads back its swapchain texture after presenting is unaffected, because the snapshot is a copy. With snapshots off (DM_PRESENT_SNAP=0), or on the Mac without the variable, nothing changes. Ring depths 3 and 4 add a frame of latency each and were not measured.

Tested

On this branch (current main plus #87):

Not tested on this branch: the iPhone (the numbers above are from the previous base), and other games.

…ain textures can run a frame ahead of the screen

Breath of the Wild has two swapchain textures and gets one back only when the display link has taken its previous present, so each frame's CPU work starts on a tick and the GPU idles from the end of one frame to the first work of the next. macOS reads that idle time as room and holds the GPU at its lowest clock (444 MHz on an M2 Max) even while frames miss the screen. With DM_PRESENT_SNAP=1 each present is copied on the game's queue, right after the frame's work, into one of two host textures (nvn_mtl.m snap_take), the display link shows the copy, and the game's texture is free at once: the game runs up to a frame ahead of the screen and a frame whose GPU work runs long is absorbed instead of shown late. The presenting thread's wait for a free ring slot paces the game; AcquireTexture then does not wait for a display link tick, and the frame step counts the slot wait as waiting, not as the frame's cost (counted as cost, it stepped Breath of the Wild down to 30 fps). Presents from another thread while Metal encoding runs on its own thread (DM_NVN_ENC_THREAD) are not copied. Off by default.

Checked on the same change on the previous base (the Breath of the Wild Mac shader library here is of that chain): Zora's Domain from a Cemu save, M2 Max, 144 Hz screen at 120 Hz, about 145 s each, frames on screen longer than 1/60 s by a quarter: 64 of 8,684 with neither, 53 with snapshots alone (at 444 MHz the GPU work does not fit for stretches, which no buffer absorbs), 41 with the GPU filler of PR seedds#81 alone, 4 of 9,000 with both. On this branch the Mac host builds and t_present_crop, t_pace and t_fstep pass. docs/reference/frame-rate.md (Present snapshots) and the note of Breath of the Wild say so
… not the frame's cost

With snapshots the game runs up to a frame ahead of the screen, so a frame whose work runs long is absorbed instead of shown late, and its cost is no evidence against the hold. Judged by cost, Breath of the Wild in Zora's Domain stepped to 30 fps five times in 145 s, three of them at a GPU time of 0.75-0.89 vsyncs: the frame's CPU time held the game's waits on its own GPU work, which runs a frame later now. fstep_tick now hands the controller the screen's verdict (nvn_fstep.screen): the on-screen interval of the frame 8 presents back against its hold (pres_expect_ms), late by a quarter or dropped. Without snapshots, or without presented times, nothing changes. t_fstep: costly frames shown on time keep hold 1, three frames late on screen step up whatever their cost, the verdict holds for one call (50 checks pass).

Checked on the same change on the previous base: Zora's Domain from a Cemu save, M2 Max, 144 Hz screen at 120 Hz, snapshots without the GPU filler, 180 s with a shader build running beside it: 4 of 10,800 frames on screen longer than 1/60 s by a quarter (53 of 8,700 judged by cost, 64 of 8,684 without snapshots), no step to 30, the GPU at 444 MHz throughout. docs/reference/frame-rate.md (Present snapshots) and the note of Breath of the Wild say so
…after frame is late too, and frames the buffer absorbed stay out of the probes' reference

Judged by the screen alone, a game steadily slower than its hold never stepped up: with 18.9 ms frames at hold 1 (DM_TEST_PRESENT_LAG=1,12 on Breath of the Wild) the screen missed one refresh in 7, never three in a row, and the game ran at a jittering 53 fps with a payback every few frames instead of a steady 30. fstep_tick now also counts a frame late when 12 of the last 16 started later than their hold (the bits are cleared when the hold changes, so a step is judged against the hold in force). The controller's probe reference (crun, ccur) takes only frames whose cost fit their hold while the screen judges: frames the buffer absorbed made the scene look as cheap as the one that held hold 1, and the probes flapped between 60 and 30 about once a second. t_fstep: a steadily too slow game shown on time, then late, stays at hold 2 with no probes (51 checks pass).

Checked on the same change on the previous base: the same test now holds hold 2 (31.7 ms frames, 10-12 of 300 late on screen) where it ran at 53 fps with 40 of 300 late
…d by time and the frame step does not judge by the screen

With snapshots a frame starts as soon as a ring slot is free, and a slot frees when the display link has taken its present. While the link is silent (the window in the background or covered; mtl_present_wait's stall fallback, STALL_NS) frames are dropped as fast as they come, so nothing paced the game, and the dropped frames counted as late on screen and stepped the frame step to 3 vsyncs: Breath of the Wild ran at 2.5 to 4.8 times real time behind another window. AcquireTexture now sleeps to the texture's slot time while the link is stalled (mtl_link_stalled), as without snapshots, and fstep_tick takes no screen verdict then (frames the fallback drops say nothing about the game; it judges by cost).

Checked on the same change on the previous base, Breath of the Wild with the display link paused for 40 s (DM_TEST_LINK_STALL=45,40): game time against real time during the pause 1.000 (2.48 and 4.83 before), after it 1.003 and 1.000 (2.29 before). On this branch the Mac host builds and t_fstep, t_pace and t_present_crop pass
…OS; wait ledger (DM_NVN_WAITS=1) and the frame's blocked SyncWait time (@sync_ms)

Snapshots: with Metal encoding on its own thread (DM_NVN_ENC_THREAD) the copy into the ring slot is now a block of the encode thread's own queue (nvn_rec.h rec_own_begin / rec_own_push: one piece behind every block published so far; a submission publishes its records before it returns, so the copy runs after the frame's work), with its commit, the ring event and a RO_NOTE the present waits for before its blit (snap_commit_wait: the encode thread's progress, not the GPU). The presenting thread neither waits for the encode thread nor commits itself, so a present from another thread than the submitting one (Breath of the Wild's Presentation Thread) gets a snapshot too (2b6d446 had turned them off there). The copy has its own barriers (after the frame's work and the slot's last blit; the game's next writes after the copy); the CPU wait for the slot's previous copy is taken only when it has not finished. DM_PRESENT_SNAP=<n> is the ring's length, the frames the game may run ahead of the screen (2 to 4; 1 = 2); on by default on iOS (DM_PRESENT_SNAP=0 turns them off), off on the Mac as before. Mac, Breath of the Wild, encode thread on, snapshots on, DM_FPS_TARGET=120: the picture is right (DM_NVN_SHOT of the snapshot), no red line.

Wait ledger: every place a thread waits on the CPU for the GPU or the encode thread (the game's SyncWait split into newest fence / older fence / acquire sync / polls, QueueFinish, allocator, occlusion ring, multi-draw count, readbacks, snapshot ring, upload batches and slots, the encode thread drain, the uniform copy worker, old-queue command buffers), calls / blocked calls / blocked ms per frame every 300 frames, with how long after the GPU end a blocking SyncWait wakes. Breath of the Wild on an M2 Max (load 6-8, counts only): one blocking SyncWait a frame, 4.9-7.0 ms, on the newest fence, by the main thread (a sync WindowAcquireTexture handed out, and its own fence of the last submission, which the game waits for again; with the acquire sync on an older fence the second wait blocked as long), woken 0.29 ms after the GPU end (the feedback handler runs 0.28 ms after it); the encode thread drain at the end of each submission 0.25-0.39 ms; the uniform copy worker 0.1-0.16 ms; nothing else blocks in steady play. hle_nvn.c: an acquire sync is fenced 2 (its own ledger line); @sync_ms in DM_NVN_TSV rows

t_present_crop takes the last queued frame when a present snapshot hands back no ticket (with the encode thread and DM_PRESENT_SNAP=1 its present is now copied). Merged with seedds#141 (b87a0cc): the finish path still hands the encode thread its current block before it waits for upload batches, now with its wait site. docs/reference/frame-rate.md (Present snapshots) describes the ring, the default and the encode thread path.
seedds added a commit that referenced this pull request Oct 25, 2026
Integrate the present snapshots and frame-step work from #87 together with
the encode-thread support and wait ledger from #157. Keep snapshots off
by default on both platforms pending wider device validation.

Offer Frame buffering in the iOS launcher for NVN builds. The existing
launcher remembers the choice and preserves env.txt overrides. Pass the
packer's detected graphics API to the settings generator.

Wait for snapshot capacity at acquire; a present reaching a busy slot
falls back to its original texture ticket. Drain snapshot commits and GPU
completion during suspend even when display tickets were discarded.
Regression tests cover both waits, fallback tickets and copied pixels.

Validated on M4 Pro: full gfx run list, 38 snapshot lifecycle checks,
219 checks with encoding and snapshots enabled, Mac host build, 133-file
iOS compile check, and eight launcher/docs/packing tests. The host unit
suite cannot link t_heap_size because dm_ue_heap_owns is missing, reproduced
on unchanged main cbbb600. No iPhone runtime performance claim.

Co-authored-by: v0lumehi <[email protected]>
@seedds

seedds commented Oct 25, 2026

Copy link
Copy Markdown
Owner

Incorporated #87 and #157 together into main in a188613, with contributor credit, two lifecycle fixes, and an NVN-only Frame buffering launcher toggle.

Snapshots default off on both iOS and Mac pending wider device performance/latency validation. The launcher remembers the choice; Documents/env.txt (DM_PRESENT_SNAP=1 / 0) overrides it.

The targeted reproductions on an M4 Pro found and fixed:

  • Suspend could return while a snapshot was still queued on the encoder. It now waits for the snapshot commit and GPU completion, including copies whose display tickets were discarded.
  • With a simulated 150 ms display stall, present calls blocked for about 155 ms. Ring pacing now happens at acquire; presents reaching a busy slot use the original texture ticket. Present calls in the same test stay below 0.5 ms. A busy copy allocator is not reused.

Validation on the integrated main revision: full graphics run list; 38 snapshot regression checks; 219 existing checks with encoder and snapshots enabled; Mac host build; iOS compile check (133 files); eight launcher, docs and packing tests. The host unit suite stops linking t_heap_size on missing dm_ue_heap_owns; the same failure was reproduced on unchanged main cbbb600.

The earlier first-use residency concern did not reproduce on this Mac and is not treated as a confirmed blocker. No iPhone runtime benchmark was performed here; the combined performance measurements from this PR remain the author's results. #158 is not included and will need to account for this integration.

Closing this PR as incorporated into main. Thank you for the snapshot, frame-step and encoder work.

@seedds seedds closed this Oct 25, 2026
@v0lumehi
v0lumehi deleted the snap-enc branch October 26, 2026 09:29
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
Integrate the present snapshots and frame-step work from seedds#87 together with
the encode-thread support and wait ledger from seedds#157. Keep snapshots off
by default on both platforms pending wider device validation.

Offer Frame buffering in the iOS launcher for NVN builds. The existing
launcher remembers the choice and preserves env.txt overrides. Pass the
packer's detected graphics API to the settings generator.

Wait for snapshot capacity at acquire; a present reaching a busy slot
falls back to its original texture ticket. Drain snapshot commits and GPU
completion during suspend even when display tickets were discarded.
Regression tests cover both waits, fallback tickets and copied pixels.

Validated on M4 Pro: full gfx run list, 38 snapshot lifecycle checks,
219 checks with encoding and snapshots enabled, Mac host build, 133-file
iOS compile check, and eight launcher/docs/packing tests. The host unit
suite cannot link t_heap_size because dm_ue_heap_owns is missing, reproduced
on unchanged main 57ebf66. No iPhone runtime performance claim.

Co-authored-by: v0lumehi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants