Repository navigation
nvn/pso: task / mesh pipelines are bases compiled when the game makes the program - #2765
ericschaal wants to merge 4 commits into
Conversation
… the program With the Metal 4 compiler a native mesh pipeline is now its base specialized: the task, mesh and pixel functions compiled with the colour state left open, kept in the pipeline archive. The host compiles the base in the background at nvnProgramSetShaders (mtk_prefetch), so a first draw specializes it without a compile; a draw that comes while the base still compiles waits for the rest. DM_NVN_MESH_PREFETCH=0 turns the prefetch off. A pipeline specialized from a base gives the fragment 0 for the mesh function's per-primitive values (a platform limit, macOS 27.2), so a mesh shader that hands such values on, and a pixel shader that reads the primitive ID, keep whole pipelines. Open: the offline tools do not enumerate mesh bases. Ocarina of Time, Hyrule Field run with no M rows (M5 Pro): 58 of 60 first draws found their base ready with an empty archive, 59 on the next start with none waiting. Co-Authored-By: Claude Fable 5.2 <[email protected]>
… not Hyrule Field The field-run route's inputs ran from a save that loads in a forest area. Co-Authored-By: Claude Fable 5.2 <[email protected]>
…iled library shader_load looked for dm_prim_t in the shader's source to decide whether a mesh pipeline may be a specialized base. An app's shader image carries no sources, so there every mesh shader counted as one with per-primitive values and every mesh pipeline was built whole: the bases did nothing in a packed app. It now looks in the compiled library, where Metal's compiler names each per-primitive output in the function's metadata (air.mesh_primitive_data). Ocarina of Time with the Mac app's shader image, an empty archive and no M rows (M5 Pro, Hyrule Field): 327 bases asked for, 322 compiled in the background, 68 first draws found theirs ready, none waited. Co-Authored-By: Claude Fable 5.2 <[email protected]>
|
This PR was closed before I could mark it as a draft; recording why it should not be reopened as it is. A first playthrough on a packed Mac app built from this branch (chain Not yet shown to be this change: the same build is also the first run on this chain, and the failed submission holds the task shader 🤖 Generated with Claude Code |
What and why
A native task / mesh pipeline was compiled whole at its first draw unless an
Mrow in a prewarm list named it, and recording rows in play does not converge: Ocarina of Time has 500 mesh shaders and 1,284 mesh → pixel pairs, and each day of play found dozens of new ones (#2550 and the other row PRs: #2280, #2324, #2466, #2518, #2642).With the Metal 4 compiler, a mesh pipeline is now its base specialized, as a vertex / pixel pair's is:
mtk_base,mtk_build): the task, mesh and pixel functions compiled with every colour attachment left unspecialized. The specialization (formats, blend, write masks) compiles nothing, so one base serves every render state its program is drawn in. The base is kept in the pipeline archive.mtk_prefetchfrommtl_prog_prefetch,nvnProgramSetShaders): a base depends on the program's functions alone, so the host compiles it in the background when the game makes the program. A draw that comes while its base still compiles waits for the rest of that compile.DM_NVN_MESH_PREFETCH=0turns it off.NvShader.mesh_prim) and a pixel shader that reads the primitive ID keep whole pipelines, built at the first draw or from anMrow. Ocarina has four such mesh shaders.pipelines up to frameline counts the bases asked for, taken from the archive and compiled, and the first draws that found theirs ready or waited.Measured
Ocarina of Time
0400486024DEC000v131072, Apple M5 Pro, macOS 27.2,bench.py --route field-run(130 s; the route's inputs from a save in a forest area, not Hyrule Field as this first said; 60 mesh pipelines drawn for the first time), noMrows, an empty pipeline archive for each run:DM_NVN_MESH_PREFETCH=0, 1DM_NVN_MESH_PREFETCH=0, 2The one stall left is a program the game makes just before drawing it. On the next start with the archive of an "on" run: 235 bases from the archive, 59 first draws ready, none waited, worst frame 124 ms. GPU time per frame is the same with the compiles on or off (11.2-11.3 ms median).
Open, and what was not checked
c030a898168d(this change applied to d02bede), as I have no Ocarina library for the current chain. The unit tests ran on this branch.scripts/regress.py checkwas not run. The reach is the native task / mesh path only (games with task or mesh shaders, on the Metal 4 compiler).DM_NVN_MESH_PREFETCH=0on an M1 or M2 would settle it; if hangs rise, the compile queue's width is the place to change.Mrows' pipelines through the host's prebuild tool; enumerating every mesh → pixel pair there would close this.shader_loadlooks in the compiled library, where Metal's compiler names each per-primitive output in the function's metadata (air.mesh_primitive_data). The reflection should carry it at the next chain change.Mrows): 327 bases asked for, 322 compiled in the background, 68 first draws found theirs ready, none waited, no GPU hang.Related: #1583 (its
mesh_archiveprobe; the per-primitive limit would be a probe there), #1720 (reports dropped mesh draws; draws wait since ea0c98d), #2413 (touches the same files).Checked
t_pso_archive(373 checks,mesh_pipelinesupdated for bases),t_mesh_first_use,t_pso_prewarm,t_pso_warm(both compiler paths),t_pso_bundle,t_pso_stall,t_pso_pending,t_geometry_mesh,test_dmsh_geometry.py,dm/tools/tests/test_docs.py.🤖 Generated with Claude Code