Skip to content

nvn/pso: task / mesh pipelines are bases compiled when the game makes the program - #2765

Closed
ericschaal wants to merge 4 commits into
seedds:mainfrom
ericschaal:mesh-pipeline-bases
Closed

ericschaal wants to merge 4 commits into
seedds:mainfrom
ericschaal:mesh-pipeline-bases

Conversation

@ericschaal

@ericschaal ericschaal commented Nov 15, 2026 •

Copy link
Copy Markdown

What and why

A native task / mesh pipeline was compiled whole at its first draw unless an M row in a prewarm list named it, and recording rows in play does not converge: Ocarina of Time has 500 mesh shaders and 1,284 mesh → pixel pairs, and each day of play found dozens of new ones (#2550 and the other row PRs: #2280, #2324, #2466, #2518, #2642).

With the Metal 4 compiler, a mesh pipeline is now its base specialized, as a vertex / pixel pair's is:

  • Base (mtk_base, mtk_build): the task, mesh and pixel functions compiled with every colour attachment left unspecialized. The specialization (formats, blend, write masks) compiles nothing, so one base serves every render state its program is drawn in. The base is kept in the pipeline archive.
  • At program creation (mtk_prefetch from mtl_prog_prefetch, nvnProgramSetShaders): a base depends on the program's functions alone, so the host compiles it in the background when the game makes the program. A draw that comes while its base still compiles waits for the rest of that compile. DM_NVN_MESH_PREFETCH=0 turns it off.
  • Per-primitive values stay whole. A pipeline specialized from a base gives the fragment 0 for every per-primitive value of the mesh function, its attributes as well as the primitive ID, whichever colour property the base left open (macOS 27.2, Apple M5 Pro; a standalone probe showed it, and the game hung the GPU on it). A platform limit. So a mesh shader that hands per-primitive values on (NvShader.mesh_prim) and a pixel shader that reads the primitive ID keep whole pipelines, built at the first draw or from an M row. Ocarina has four such mesh shaders.
  • The pipelines up to frame line counts the bases asked for, taken from the archive and compiled, and the first draws that found theirs ready or waited.

Measured

Ocarina of Time 0400486024DEC000 v131072, Apple M5 Pro, macOS 27.2, bench.py --route field-run (130 s; the route's inputs from a save in a forest area, not Hyrule Field as this first said; 60 mesh pipelines drawn for the first time), no M rows, an empty pipeline archive for each run:

Run Bases compiled in background First-draw stalls Stall time in all Worst frame GPU hangs
on, 1 215 (66.6 s of thread time) 1 756 ms 782 ms 0
on, 2 215 (68.6 s) 1 758 ms 784 ms 0
DM_NVN_MESH_PREFETCH=0, 1 0 54 4,577 ms 2,466 ms 0
DM_NVN_MESH_PREFETCH=0, 2 0 54 4,539 ms 2,491 ms 0

The one stall left is a program the game makes just before drawing it. On the next start with the archive of an "on" run: 235 bases from the archive, 59 first draws ready, none waited, worst frame 124 ms. GPU time per frame is the same with the compiles on or off (11.2-11.3 ms median).

Open, and what was not checked

  • These game runs were on chain c030a898168d (this change applied to d02bede), as I have no Ocarina library for the current chain. The unit tests ran on this branch.
  • scripts/regress.py check was not run. The reach is the native task / mesh path only (games with task or mesh shaders, on the Metal 4 compiler).
  • M1 / M2 and iOS are untested. Ocarina of Time's intermittent GPU hangs measured: 0.03-0.27 per 1,000 frames, against 5.00 in the run macOS stopped, lining up with mesh pipelines built while the game ran #1756 measured GPU hang bursts on an M1 Max that line up with mesh pipelines compiled while the game ran. This change compiles many bases in the background during play; it produced no hang here, but an M5 says little about an M1. A run with and without DM_NVN_MESH_PREFETCH=0 on an M1 or M2 would settle it; if hangs rise, the compile queue's width is the place to change.
  • The offline tools do not enumerate mesh bases, so a first start with no archive still waits once for a program drawn right after it is made. An IPA built with a Mac host carries the prewarm lists' pipelines made offline (R rows' bases and M rows' mesh pipelines), and Ocarina of Time's seed list gets an iPhone 17's own rows #2674 bundles the M rows' pipelines through the host's prebuild tool; enumerating every mesh → pixel pair there would close this.
  • The reflection does not say whether a mesh function has per-primitive values, so shader_load looks in the compiled library, where Metal's compiler names each per-primitive output in the function's metadata (air.mesh_primitive_data). The reflection should carry it at the next chain change.
  • Corrected after opening: the first commit read the shader's source for this, which an app's shader image does not carry, so in a packed app every mesh pipeline was still built whole. The second commit fixes it. With the Mac app's own shader image (Hyrule Field, empty archive, no M rows): 327 bases asked for, 322 compiled in the background, 68 first draws found theirs ready, none waited, no GPU hang.

Related: #1583 (its mesh_archive probe; the per-primitive limit would be a probe there), #1720 (reports dropped mesh draws; draws wait since ea0c98d), #2413 (touches the same files).

Checked

t_pso_archive (373 checks, mesh_pipelines updated for bases), t_mesh_first_use, t_pso_prewarm, t_pso_warm (both compiler paths), t_pso_bundle, t_pso_stall, t_pso_pending, t_geometry_mesh, test_dmsh_geometry.py, dm/tools/tests/test_docs.py.

🤖 Generated with Claude Code

seedds and others added 2 commits November 15, 2026 09:52
… the program

With the Metal 4 compiler a native mesh pipeline is now its base specialized:
the task, mesh and pixel functions compiled with the colour state left open,
kept in the pipeline archive. The host compiles the base in the background at
nvnProgramSetShaders (mtk_prefetch), so a first draw specializes it without a
compile; a draw that comes while the base still compiles waits for the rest.
DM_NVN_MESH_PREFETCH=0 turns the prefetch off.

A pipeline specialized from a base gives the fragment 0 for the mesh
function's per-primitive values (a platform limit, macOS 27.2), so a mesh
shader that hands such values on, and a pixel shader that reads the primitive
ID, keep whole pipelines. Open: the offline tools do not enumerate mesh bases.

Ocarina of Time, Hyrule Field run with no M rows (M5 Pro): 58 of 60 first
draws found their base ready with an empty archive, 59 on the next start with
none waiting.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400486024DEC000 A game labels Nov 15, 2026
seedds and others added 2 commits November 15, 2026 10:18
… not Hyrule Field

The field-run route's inputs ran from a save that loads in a forest area.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
…iled library

shader_load looked for dm_prim_t in the shader's source to decide whether a
mesh pipeline may be a specialized base. An app's shader image carries no
sources, so there every mesh shader counted as one with per-primitive values
and every mesh pipeline was built whole: the bases did nothing in a packed app.

It now looks in the compiled library, where Metal's compiler names each
per-primitive output in the function's metadata (air.mesh_primitive_data).

Ocarina of Time with the Mac app's shader image, an empty archive and no M
rows (M5 Pro, Hyrule Field): 327 bases asked for, 322 compiled in the
background, 68 first draws found theirs ready, none waited.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
@ericschaal ericschaal closed this Nov 15, 2026
@ericschaal

ericschaal commented Nov 15, 2026 •

Copy link
Copy Markdown
Author

This PR was closed before I could mark it as a draft; recording why it should not be reopened as it is. A first playthrough on a packed Mac app built from this branch (chain 8e41177b6eb3, M5 Pro) ended after about 13.5 minutes with GPU access revoked: 36 kIOGPUCommandBufferCallbackErrorHang in a row from frame 39086, during an area load. The first failed submission drew two mesh shaders for the first time (842d6cfcc9e9df67, 37b6b6175f323fed, neither with per-primitive outputs), so both went through a specialized base.

Not yet shown to be this change: the same build is also the first run on this chain, and the failed submission holds the task shader a5288d0a6c4971fd that earlier hang reports name. The hang needs to be reproduced and explained first.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants