Repository navigation
host/vi: HDR output for Switch 2 games on screens that can show EDR - #1176
maxdavidson wants to merge 1 commit into
Conversation
4f40096 to
79dc8ca
Compare
70395e9 to
f94b279
Compare
a0c3511 to
e863278
Compare
Port PR seedds#1176 at a360806f onto oot-hyper-main 64a163ba. Preserve SDR tagging, quality filtering, AOT hooks and absolute pacing. Keep HDR off by default and bypass the SDR MetalFX scaler for PQ input. Static Ocarina v131072 modules and resources contain the HDR status calls, paper-white controls, PQ constants and BT.2020 output matrix. Native present and quality checks, settings/docs tests and the host build pass. Gameplay, displays and HDR pipeline archive coverage remain unverified. Front-ends: new: Mac Switch 2 builds can offer default-off DM_HDR; HDR layer formats override SDR colorspace tagging. Co-authored-by: seedds <[email protected]> Co-authored-by: beecrepaldi-afk <[email protected]>
a360806 to
173e358
Compare
24e84ef to
69676b9
Compare
|
Tested this branch on a screen with EDR headroom, which the description lists as untested. Setup: MacBook Pro M2 Pro, built-in Liquid Retina XDR (potential headroom 16, about 1600 nits), macOS 27.2. Ocarina of Time (0400486024DEC000, v131072), the branch cherry-picked onto main 88c7e03 with #1331, #1447, #1462 and #1368. Result: it works. Status answers HDR, format 1. The game switches its window from 0x25 to 0x3d and declares 0..203 / 0..710 nits; the PQ layer gets the HDR10 metadata. The title holds 60 fps with no new red line, and HDR looks right by eye. How the game uses the two numbers (one relaunch each):
Ocarina declares min(peak, 3.5 × paper white). The paper white is what lifts its highlights; the default peak never limits it. Testing gotcha: a window behind other windows reads a current headroom of 1.2 whatever it declares. In front, the same window reads 16 at low brightness and 3.2 at full brightness (1600 / 500 nits). Calibration proposal, working locally on top of this branch (about 100 lines with tests and docs; happy to open a PR or share the patch):
Minor: the MetalFX present scaler running on PQ codes looked over-sharpened next to Standard (bilinear) with the 3D at full resolution. Subjective. |
d945160 to
65af3ec
Compare
0e44b7c to
5074a70
Compare
5074a70 to
a4bac21
Compare
…n the game's queue only draws it The present command buffer ran the MetalFX spatial scaler (1920x1080 → the fullscreen canvas) on the game's queue at each display-link tick, inside or right after the frame's segments: about 2.6 ms of GPU time a frame at the low clock macOS keeps for Breath of the Wild, which took frames from about 15 to 17–18 ms (48–56 fps). With present snapshots, snap_take now scales the presented rectangle into an upscaled texture of the slot, in the snapshot's command buffer right after the frame's last segment, and the present draws it 1:1 (0.3–0.5 ms). A canvas size or format change remakes the scaler and the slot's texture (the old one stays resident until the slot's next use), and any mismatch takes the old path for that frame. DM_PRESENT_FX_AT_SNAP=0 restores it, ab compares both; DM_PRESENT_UPSCALE takes ab too. @present_gpu_ms and @snap_gpu_ms give both command buffers' GPU time per frame. M5 Pro run, the two placements switched every 30 s: the present 2.58–2.66 → 0.34–0.52 ms; the user reports a cleaner picture. t_present_snap fxsnap (both encode modes, Metal validation): exact scaled pictures, a refused resized or reformatted canvas, a canvas change remaking the scaler, residency of the replaced textures. Hand merge (nvn_mtl.m): main and this fork have since changed the present: the fork's HDR output (PR seedds#1176) gives the pass a conversion for its target (present_mode, the PQ layer's paper white and roll-off) and main lets MetalFX write the drawable itself (fx_encode's direct write). present_draw_m4 takes the conversion from present_encode_m4 (mode, knee, target) and reads the lower-left origin itself; snap_take scales at the snapshot only a picture the present would draw as stored (present_mode 0 and no roll-off), and the present draws it so. A picture converted for its target (HDR on, or an HDR10 window into an SDR drawable) is scaled by the present, as before. (cherry picked from commit 3ee6ec4)
…s the screen's own colour space, and fullscreen on a 4K HDR TV over HDMI in HDR mode, with nothing over the layer, showed a magenta band that came and went Hand merge (front_mac.m): the sRGB tag goes on the SDR layer first; the fork's HDR output (PR seedds#1176, DM_HDR) then sets its RGB10A2 layer's own colour space, BT.2100 PQ, as before. (cherry picked from commit eed212e)
a4bac21 to
91ff4db
Compare
…s the screen's own colour space, and fullscreen on a 4K HDR TV over HDMI in HDR mode, with nothing over the layer, showed a magenta band that came and went Hand merge (front_mac.m): the sRGB tag goes on the SDR layer first; the fork's HDR output (PR seedds#1176, DM_HDR) then sets its RGB10A2 layer's own colour space, BT.2100 PQ, as before. (cherry picked from commit eed212e)
AMD FSR 1's EASU (MIT) in half precision, drawn by the present pass straight into the drawable: no input copy, no scaler output texture, no second pass. iPad Pro 13" M4, Breath of the Wild 1920x1440 to 2752x2064: present command buffer 1.62 ms against MetalFX's 2.21 ms and Standard's 0.31-0.42 ms. Hand merge (present.metal, nvn_mtl.m, env_switches.tsv): the present pass has since gained the fork's HDR conversion (PR seedds#1176: Crop.mode) and main's DM_PRESENT_UPSCALE=ab (sw_mode, where 2 and up mean ab), and the pass itself is present_draw_m4 (PR seedds#1045). easu keeps its own body and reads the fork's Crop (rect and edge; mode unused). fsr is a flag of its own (g_fsr, fsr_on) with MetalFX off, not mode 2; present_encode_m4 decides easu and hands it to present_draw_m4, and only for a picture drawn as stored (mode 0): with the HDR output on, or an HDR10 window into an SDR drawable, the bilinear pass converts it as before. The snapshot path's 1:1 draw never uses it. (cherry picked from commit 917dee7)
Two additions to the present pass, both of which work for any game because neither needs anything from the game. DM_PRESENT_UPSCALE_OUT=1.0..4.0 reconstructs above the viewport and averages back down. The present shader took one bilinear sample, which reads 4 texels whatever the reduction, so reconstructing larger and sampling down this way threw nearly all of it away. The tap count now follows the reduction ratio, capped at 4x4, which makes it real supersampling. At 1.0, the default, the scaler still targets the viewport and the shader still takes one tap, so nothing changes. In the launcher it is Upscaling detail, a sub-setting of Upscaling (needs=DM_PRESENT_UPSCALE=metalfx, seedds#418): its row is drawn only while MetalFX is chosen. DM_SHARPEN=0..1 is contrast adaptive sharpening of the finished picture: the 5 tap cross, clamped to the neighbourhood's own range so it cannot ring or halo. It is a second fragment function taken instead of the blit, not after it, so it adds no pass and costs nothing when off. The present without a Metal 4 queue passes one tap and no sharpening, so it draws as before. Measured with the same two changes in a fork, on Breath of the Wild 1.9.0 on an iPad Pro 13" M4, 1920x1080 content into a 2752x1548 viewport, all at 60.00 fps with thermal state nominal and a 16.67 ms budget: upscaling output scaler target GPU off - - 8.53 ms metalfx 1.0 2752x1548 9.09 ms metalfx 2.0 5504x3096 11.34 ms Before the box filter, output 2.0 looked identical to 1.0 while costing the same 11 ms, which is what led to the change. Compiles. Not yet run against this main: our devices are built from an older chain and rebuilding them is a long job, so the device numbers above come from the fork. Opening it now because both settings are off by default. Hand merge (present.metal, nvn_mtl.m, settings, docs): the present's constants (Crop) carry the fork's HDR conversion (PR seedds#1176, mode) in the slots this change used, so Crop has a fourth float4, box (taps, one texel, sharpening), and every buffer of it is 16 floats; fs and sharp both end in present_convert, the conversion fs did. The taps are more than 1 only for MetalFX's output reconstructed above the viewport (DM_PRESENT_UPSCALE_OUT above 1.0): at the default a picture shown smaller than rendered still takes one tap, as before (the change computed taps from any reduction). fx_encode's output size takes DM_PRESENT_UPSCALE_OUT ahead of main's scaler check (fx->sc fields), and a larger output is never the drawable written directly; snap_take (PR seedds#1045) scales at the snapshot only at 1.0. FSR 1 (easu) wins over sharpening when both are set. Upscaling detail, Sharpening and FSR 1 are all off by default. (cherry picked from commit 704ca34)
Symptom: Nintendo Switch Sports Resort's docked output (2560x1440) shown fullscreen on a 2560x1080 screen (a 1920x1080 picture) or in a 1280x720 window has stair-stepped, uneven edges that crawl as they move, more than the game's own frame shows. Cause: the present pass (present.metal) took one bilinear sample of the swapchain texture per drawable pixel. Shown smaller, a pixel covers more than a texel and one tap weighs texels by where its centre falls (at 2560 -> 1920 a texel counts fully in one pixel and a quarter in the next), and past 2:1 texels are skipped: one-texel stripes at 3:2 came out as columns of 64 and 191. Fix (every game; the host's own present shader, not the shader chain: no game shader or pipeline archive rebuild): a picture shown at its size or larger still takes one bilinear tap, so 1:1, upscales and MetalFX's 1:1 draw are unchanged. One shown smaller is filtered over each pixel's footprint with a Catmull-Rom kernel stretched by the scale (texels per pixel from the texture coordinate's derivatives, at most 3), two texels an axis per bilinear fetch at the pair's weighted centre, fetches clamped to the crop. Both presents, DM_NVN_SHOT_OUT and whole-window captures use it; a shot of a cropped window, drawn at the window texture's own size since 5582c54, is never smaller than its crop and keeps the single tap. Lanczos-2 with separate texel reads gave a like picture at 1.8 ms. Checked: t_present_crop (4 env combinations): the filter against a CPU model within 1 at 3:2 (stripes, ramp), 2:1, 5:1 (width capped at 3:1) and with the lower-left origin, 1:1 unchanged, 3:2 stripes' spread at most 50 (one tap 127); with the old shader the 5 filter checks fail. make -C dm/gfx/tests GAME=010015100B514000 run on this branch; test_docs, test_env_switches, test_pr_areas. GPU cost on an M2 Max, standalone: 2560x1440 -> 1920x1080 0.27 ms (one tap 0.03), -> 1280x720 0.20 ms. In the game (the host from this change on the fork's branch, pointer calibration screen, 1280x720 window, DM_UPSCALER=metalfx DM_PRESENT_UPSCALE=metalfx, one run each): present command buffer 0.37 ms p50 (0.06 before), game GPU p50 12.4 ms either way, frame time p95 20-24 ms (24-27 before); DM_NVN_SHOT_OUT=1920x1080 shots at the same frames show the court line and bench edges smoother at the same sharpness (Laplacian 23.7 vs 24.2). In an 11-minute session on 2026-11-12 with a build of the fork's branch that has this change and the fork's other fixes (not the keep-busy switch), a player played two-player skate (the practice zone, then leaving it) and two-player Avioneta with no crash. Not run: other games, regress.py (its shots of uncropped windows are the texture's own pixels and do not pass through the present; cropped ones are drawn at least 1:1), iOS. Docs: rendering.md (A picture shown smaller than it is drawn: stair-stepped edges from the present's single tap), the game note, symptom index regenerated. Off by default on this branch: DM_PRESENT_MINIFY=1 turns it on (new switch, docs/env_switches.tsv), since it changes the picture where a frame is shown smaller than drawn. The filter is a fragment function of its own, minify, taken in place of fs (legacy and Metal 4 present alike; not with FSR 1, sharpening or the supersampled box filter of DM_PRESENT_UPSCALE_OUT), and it ends in the fork's HDR conversion (present_convert, PR seedds#1176) as fs does. t_present_crop runs its minification cases in one more pass with DM_PRESENT_MINIFY=1; the reference entry and the Switch Sports note say it is off. (cherry picked from commit bcfd475)
00d3a3a to
1d7ca68
Compare
3c42f77 to
2f0fe2a
Compare
2f0fe2a to
8562086
Compare
A Switch 2 game asks vi::QueryDisplayHdrStatus for the display's HDR state. The host answered SDR, so the game never made its HDR10 window (RGB10A2, PQ in BT.2020) and the present had no decode for one. - hle_vi.c: with DM_HDR=1 (the HDR row, on by default) the status answers HDR on, format 1, once the window has the HDR layer on a screen that can show EDR, and SDR elsewhere, where the game draws its own SDR picture. The games ask every frame, so the answer follows the window to another screen, and the iOS in-game menu turns HDR on and off while the game runs. - The layer follows the game and the screen (dm_hdr_layer_tick, called by both front ends on their main thread four times a second, in every run-loop mode, and on screen changes): the SDR layer of before until a Switch 2 game runs with HDR on (dm_hdr_boot, once the loader knows it, so the layer is there for the game's first status query) on a screen that can show EDR; then the HDR layer, RGB10A2 in BT.2100 PQ, and HDR answered once it exists, not waiting for the headroom's climb (Donkey Kong Bananza decides its window at its first query while it boots and, with a cold pipeline archive, did not ask again for two minutes); back to the SDR layer once HDR is no longer wanted and the game has presented its SDR window twice, so no HDR frame reaches the SDR layer. A screen change counts after half a second. An SDR screen, a Switch 1 game and HDR off show what they showed before, with no conversion pass and no EDR power. - The present (present.metal, nvn_mtl.m present_params) writes the game's HDR10 window as it is, and encodes an SDR or float source with its white at 100 nits; the conversions write alpha 1. - The game's white always lands at the screen's SDR white, so HDR output is as bright as SDR and follows the brightness slider: Core Animation shows a PQ layer relative to the screen's white, 100 nits at EDR 1.0 (CAEDRMetadata.h for HDR10 metadata, WWDC21 10161 for an app converting PQ itself; measured for the PQ layer without metadata), and the game is told a paper white of 100, so its window goes to the layer as it is. In a prototype that told Donkey Kong Bananza BT.2408's 203 and scaled its window to match, it drew the same picture (median and mean light equal, highlights to 1.61 and 1.67 times its white), so the paper white stays 100, fixed, and the HDR paper white and HDR peak settings rows are gone: nothing a player would set improves the picture. The peak is automatic: the paper white times the screen's current headroom, in steps of about 6 %, at least 1.25 times the paper white, a new step taken once the readings have stayed a second within two neighboring steps (a headroom flickering between two steps held the answer back before). Donkey Kong Bananza, Ocarina of Time and Mario Kart World keep their pixels within the peak they are told. - No HDR10 metadata on the layer by default (DM_HDR_META=1 adds it). With metadata above 100 times the headroom, Core Animation compresses the whole picture, mid-tones included, and Mario Kart World declares about 1.8 times the peak it is told. In fullscreen, a layer with metadata of any range shows rectangles of older frames under GPU load, also in a standalone app with no game. What the game may draw beyond what the screen shows right now is rolled off in the present instead (dm_hdr_rolloff), the automatic peak included: a rational shoulder above max(paper white, 3/4 of the target), the game's peak landing on the target, pixels below the knee left bit for bit. - With metadata in fullscreen on the Mac, the drawable is handed over once its present pass has completed, without signalDrawable (DM_PRESENT_AFTER_GPU): that removes the rectangles but makes frames late when a pass ends close to the refresh, so without metadata the present keeps Apple's order. - An HDR source is scaled bilinearly, not by MetalFX's spatial scaler (present_hdr_source): its perceptual mode expects sRGB, and on PQ code values it over-sharpened highlights past the peak the game was told (a synthetic scene: 31 to 65 % worse than bilinear in ΔE_ITP, where SDR through MetalFX is 10 to 27 % better). The launchers hide the Upscaling row while HDR is on (settings.tsv needs=DM_HDR=0), and DM_PRESENT_UPSCALE=metalfx with an HDR picture logs a warning. - A screenshot of the HDR10 window goes through the present pass: converted to BT.709 first, then compressed into SDR white with the same shoulder, so highlights keep their hue (compressing before the conversion turned orange yellow); the code values go to <path>.raw, and the HDR picture to a 10-bit HEIC tagged BT.2100 PQ (.heic for .png), which Preview and Photos show in HDR, re-encoded with the paper white at 203 nits, the SDR white of Apple's image path (the window's own codes showed at half brightness there). The present pipeline is kept per target format, so a screenshot beside the HDR layer makes none again (one was made for each, while command buffers could still hold the old one). - The game's SDR bytes are decoded and encoded with a pure power curve of 2.2, not the piecewise sRGB curve (an SDR window in the HDR layer, a screenshot of an HDR frame): they are sRGB-encoded with BT.709 primaries (Horizon's color management unit takes the framebuffer as sRGB and passes it through), but the screens they are made for show them through a power curve (the Switch's LCD measures about 2.2, sRGB's reference display is 2.2, TVs BT.1886), and the piecewise curve gave code 16 2.3 times the light of 2.2. DM_SDR_COLORSPACE=g22|g24|srgb picks the curve and a matching color space on the SDR layer together, for comparisons. The SDR layer is tagged sRGB explicitly, when it is made and when it comes back from the HDR layer: its pixel format gives a fresh layer kCGColorSpaceSRGB on macOS 27.2, but coming back it kept the PQ color space, or none once set to NULL (not color-matched: oversaturated on a P3 screen). Checked on main e4b0d9b: t_present_hdr (71 checks, also under the Metal validation layer); make -C dm/host/tests run, every test but t_audio's device-stall case, which fails on main too on this Mac; make -C dm/gfx/tests run, every test but t_geometry, t_geometry_feedback, t_geometry_hazards and t_geometry_mesh, which do not compile on main (t_geometry.m still reads g_mdi_v, which ad1ee5b removed); test_docs and test_settings. regress.py check against main e4b0d9b (DM_HDR unset, the SDR path; no HDR line in any run), shaders rebuilt for chain a9bfaeb9fa8c (no MSL changed), two runs a side: Breath of the Wild, Ocarina of Time and Donkey Kong Bananza PASS, the same fps or better; Super Mario Odyssey the same fps, its 95 s screenshot catching the opening cutscene at another moment (both sides loaded); Mario Kart World crashed at frame 93 on both sides (-[AGXG13XFamilyRenderContext_mtlnext setRenderPipelineState:] from draw_), a crash of main's. Its 30 fps hold after the CPU-bound stretch at 152 s, open on the earlier base, happens on main too: in five alternating pairs on d6b5fc1, two main runs and one run here held it. Donkey Kong Bananza with HDR on: the HDR layer from boot, its first status query answered HDR, the peak following the brightness slider. Ocarina of Time and Mario Kart World with HDR on, and Mario Kart World on iOS with an earlier version. Five reference entries in docs/reference/rendering.md, and the three games' notes. Front-ends: new: HDR output for Switch 2 games (the DM_HDR row, on by default; the host's own default, DM_HDR unset, stays SDR): the front end switches its layer between the SDR one and RGB10A2 in kCGColorSpaceITUR_2100_PQ with wantsExtendedDynamicRangeContent as hle_vi.c's dm_hdr_layer_tick(can, headroom, layer_pq, now_ms) asks, calling it on its main thread four times a second and on screen changes (front_mac.m, front_ios.m) Front-ends: new: front_hdr_headroom() (the screen's current EDR headroom) is a weak hook in hle_vi.c; a front end calls dm_hdr_layer_meta_set() with whether it gave the layer HDR10 metadata, and gets the game's declared range from front_hdr_content_range() Front-ends: new: main.c calls dm_hdr_boot() once the modules are loaded; a front end with its own startup calls it there too Front-ends: new: the SDR layer is tagged kCGColorSpaceSRGB explicitly, when it is made and when it comes back from the HDR layer (front.h front_sdr_colorspace_create) Front-ends: interface: the HDR paper white and HDR peak rows are removed (DM_HDR_PAPER_WHITE with them; DM_HDR_PEAK stays a diagnostic switch), and the Upscaling row (DM_PRESENT_UPSCALE) carries needs=DM_HDR=0 Co-authored-by: beecrepaldi-afk <[email protected]> Co-authored-by: b1tdr3amer24 <[email protected]> Co-authored-by: Amost <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]>
8562086 to
3e12087
Compare
What and why
Switch 2 games can draw an HDR picture, with highlights brighter than normal white and richer color, but the host always told them the screen was SDR. With this change, a Switch 2 game on a Mac or iPhone screen that can show HDR is told so and draws its HDR picture. Its normal brightness stays the same as with HDR off, and highlights use the screen's spare brightness. On a screen without HDR, for Switch 1 games, or with the HDR setting off, nothing changes. Terms are explained under Terms below.
vi::QueryDisplayHdrStatusanswers HDR when the HDR setting is on (DM_HDR, on by default, Switch 2 games only) and the window is on an HDR screen with its HDR layer ready. Elsewhere it answers SDR, as a Switch 2 does on a TV without HDR. The game asks every frame, so moving the window switches it.dm_hdr_layer_tick). The HDR layer is ready before the game first asks (dm_hdr_boot). SDR screens, Switch 1 games and HDR off look and cost what they did before.DM_NVN_SHOTwrites the SDR PNG with highlights compressed and their hue kept, an HDR HEIC that Preview and Photos show in HDR, and the exact values in<path>.raw.How it works, and why, is in the reference entries under HDR output in
docs/reference/rendering.md:Each game's findings are in its note: Donkey Kong Bananza, Ocarina of Time, Mario Kart World.
Built on beecrepaldi-afk's PQ layer, b1tdr3amer24's present after the GPU, and Amost's paper white and peak handling; they are co-authors of the commit.
Terms
HDR terms used in this PR
Checked
Tests, on main
e4b0d9b7:t_present_hdrpasses all 71 checks, also under the Metal validation layer.make -C dm/host/tests runpasses exceptt_audio's device-stall case, which fails the same way on main on this Mac.make -C dm/gfx/tests runpasses exceptt_geometry,t_geometry_feedback,t_geometry_hazardsandt_geometry_mesh, which don't compile on main:t_geometry.mstill readsg_mdi_v, whichad1ee5bfremoved.test_docsandtest_settingspass.Donkey Kong Bananza
040074A01BF12000v458752:Ocarina of Time
0400486024DEC000v131072 and Mario Kart World0400C3F00006E000v0:regress.py check --recordagainst maine4b0d9b7, two runs a side, shaders rebuilt for chaina9bfaeb9fa8cwith no shader newly failing and no changed MSL (DM_HDRunset, so the SDR path; no HDR line in any run's log):-[AGXG13XFamilyRenderContext_mtlnext setRenderPipelineState:]fromdraw_), a crash of main'sMario Kart World's 30 fps hold after the CPU-bound stretch at 152 s, open on the earlier base, happens on main too: in five alternating pairs on
d6b5fc1b, two main runs and one PR run held it.Devices:
Not checked yet with the current code: a window moved between an HDR and an SDR screen; iOS; Ocarina of Time and Mario Kart World at full headroom.
Front ends
Front-ends: new: HDR output for Switch 2 games (the DM_HDR row, on by default; the host's own default, DM_HDR unset, stays SDR): the front end switches its layer between the SDR one and RGB10A2 in kCGColorSpaceITUR_2100_PQ with wantsExtendedDynamicRangeContent as hle_vi.c's dm_hdr_layer_tick(can, headroom, layer_pq, now_ms) asks, calling it on its main thread four times a second and on screen changes (front_mac.m, front_ios.m)
Front-ends: new: front_hdr_headroom() (the screen's current EDR headroom) is a weak hook in hle_vi.c; a front end calls dm_hdr_layer_meta_set() with whether it gave the layer HDR10 metadata, and gets the game's declared range from front_hdr_content_range()
Front-ends: new: main.c calls dm_hdr_boot() once the modules are loaded; a front end with its own startup calls it there too
Front-ends: new: the SDR layer is tagged kCGColorSpaceSRGB explicitly, when it is made and when it comes back from the HDR layer (front.h front_sdr_colorspace_create)
Front-ends: interface: the HDR paper white and HDR peak rows are removed (DM_HDR_PAPER_WHITE with them; DM_HDR_PEAK stays a diagnostic switch), and the Upscaling row (DM_PRESENT_UPSCALE) carries needs=DM_HDR=0
Docs
docs/reference/rendering.mdentries, andpython3 dm/tools/docs_symptoms.pywas run (the symptom index)🤖 Generated with Claude Code