Skip to content

nvn/present: supersampled upscaling output and sharpening - #345

Open
Xipzer wants to merge 1 commit into
seedds:mainfrom
Xipzer:present-quality
Open

Xipzer wants to merge 1 commit into
seedds:mainfrom
Xipzer:present-quality

Conversation

@Xipzer

@Xipzer Xipzer commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #315, rewritten on top of current main now that DM_PRESENT_UPSCALE has landed. Both settings are off by default and neither needs anything from the game.

DM_PRESENT_UPSCALE_OUT

fx_encode targets the viewport, and the present then draws the result 1:1. That draw takes one bilinear sample, which reads 4 texels whatever the reduction. So reconstructing above the viewport and sampling back down threw nearly all of it away: output 2.0 looked identical to 1.0 while costing the same GPU time.

The tap count now follows the reduction ratio, capped at 4x4, so the whole source footprint is averaged. Reconstructing large and sampling down becomes real supersampling.

At 1.0, the default, the scaler still targets the viewport and the shader still takes one tap. Nothing changes for anyone who does not set it. Sizes above Metal's 16384 texture limit fall back to the viewport.

DM_SHARPEN

Contrast adaptive sharpening of the finished picture: the 5 tap cross, low pass from the four neighbours, clamped to the neighbourhood's own min and max so it cannot ring or halo.

It is a second fragment function taken instead of the blit, not a pass after it, so it adds no encoder and costs nothing when off. It helps most where a game renders below the screen and no upscaler is on, which is the common case on the Pokémon titles that have no render scale stub.

Measured

Same two changes in a fork, Breath of the Wild 1.9.0, iPad Pro 13" M4, 1920x1080 content into a 2752x1548 viewport. All runs 60.00 fps, thermal state nominal, 16.67 ms budget.

Upscaling Output Scaler target GPU
off — — 8.53 ms
metalfx 1.0 2752x1548 9.09 ms
metalfx 2.0 5504x3096 11.34 ms

MetalFX at the viewport costs 0.55 ms; at 2.0 output, 2.8 ms. Worst case leaves 5 ms spare.

Changes

  • dm/gfx/nvn/nvn_mtl.m: fx_out(), the box filter and sharp in the present shader, a second pipeline, the present constants widened from 8 to 12 floats
  • dm/games/settings.tsv: two rows in Game
  • docs/settings.md, docs/env_switches.tsv

State of testing

Compiles clean. Not yet run against this main: our devices are built from an older translation chain and rebuilding them is a long job, so the device numbers above come from the fork's equivalent implementation rather than from this diff. I will post numbers from this branch once that rebuild is done.

test_docs, test_env_switches and test_launcher_settings pass.

Related: #335 (anisotropic and trilinear filtering), which is the sampler-side half of the original #315.

@Xipzer

Xipzer commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main. Upscaling detail now uses needs=DM_PRESENT_UPSCALE=metalfx in the #418 meta column, so it only shows with MetalFX on.

Xipzer pushed a commit to Xipzer/nxnative that referenced this pull request Oct 28, 2026
Game carries six unrelated rows: how fast the game runs, how large it renders,
which console mode it reports, how the finished frame is scaled, which language
it speaks, and a frame-rate read-out. The only thing they have in common is that
none of them belonged in Controls or Performance.

Picture takes the two rows that are about the finished frame rather than about
the game: Render scale and Upscaling. Game keeps Frame rate, Console mode and
Game language. The frame-rate counter moves to Performance next to the
Performance HUD, since both are read-outs rather than settings.

The split matters more as the section grows. With the anisotropic and trilinear
filtering of seedds#335 and the upscaling detail and sharpening of seedds#345, Game would
hold ten rows, six of which are about how the picture looks.

The heading order on screen is the order of SECTIONS, and rows keep their table
order inside a section, so settings.tsv still decides the layout on its own.
@Xipzer

Xipzer commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on current main. The present shader moved into dm/gfx/msl/present.metal (8907429), so the box filter and the sharp function live there now; present_pipeline loads sharp from the same host library. Checked on the Mac with BotW: DM_PRESENT_UPSCALE_OUT=2.0 gives a 1920x1080 → 5120x2880 MetalFX target averaged down, DM_SHARPEN=0.5 takes the sharpening pipeline, present GPU p50 4.54 ms; launcher and env switch tests pass.

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation labels Nov 3, 2026
@Xipzer Xipzer changed the title present: supersampled upscaling output (DM_PRESENT_UPSCALE_OUT) and sharpening (DM_SHARPEN) nvn/present: supersampled upscaling output and sharpening Nov 13, 2026
@Xipzer

Xipzer commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main (2e6edda) as one commit, retitled to the area/component form. One fix is new: the present on the old queue (present_encode) bound 32 bytes of the 48-byte Crop block this PR introduces, so the shader read its tap count past them and t_present_crop's letterboxed case failed on the branch (2 of 112). It binds the whole block now (one tap, no sharpening) and the test passes. iOS compile check: 263 files, 0 failed.

Checked on this commit (M4 Max): host build, make -C dm/gfx/tests run (every target passes), the settings / docs / switch-table tool tests. The host tests pass except audio-device checks (t_audio's device stall here, t_audio_device on clean main in the same session).

Xipzer added a commit to Xipzer/nxnative that referenced this pull request Nov 13, 2026
Game carries eleven unrelated rows: how fast the game runs, how large it
renders, which console mode it reports, how the finished frame is scaled,
fitted and shaped, which language it speaks, how the Mac window opens, and a
frame-rate read-out. The only thing they have in common is that none of them
belonged in Controls or Performance.

Picture takes the rows that are about the finished frame rather than about
the game: Render scale, Upscaling, Game's upscaler, Picture and Picture
shape. Game keeps Frame rate, Dynamic resolution, Console mode, Game language
and the Mac's Display mode. The frame-rate counter moves to Performance next
to the Performance HUD, since both are read-outs rather than settings. The
iOS in-game menu follows the sections, so the counter comes after the
controls there.

The split matters more as the section grows. With the anisotropic and
trilinear filtering of seedds#335 and the upscaling detail and sharpening of seedds#345,
Game would hold fifteen rows, nine of which are about how the picture looks.

The heading order on screen is the order of SECTIONS, and rows keep their
table order inside a section, so settings.tsv still decides the layout on its
own.

In docs/settings.md the Picture section takes the anchor #picture, and the
Picture setting inside it becomes #picture-1.

The thermal cap (seedds#220) holds the frame step at 30 fps while iOS reports a
serious or critical thermal state, and DM_THERMAL_CAP=0 turned it off only
from an env file. It is a trade a player should be able to make: with a phone
cooler, or for a player who would rather have an uneven 45 to 55 than an even
30, it costs more than it saves.

"Halve when hot" in Performance, on by default, so nothing changes for anyone
who does not touch it. Shown only for a game with a frame step, as Hold
frames when late is.

The note says what the thermal state does not catch. On an iPhone 17 Pro Max,
Breath of the Wild 1.9.0 dropped a third of its frames for ten minutes with
the state reading nominal throughout, and one scene's GPU time went from
8.40 ms to 21.83 ms while it still read nominal: iOS lowers the clock well
before it calls the device hot. The same phone, charged over USB while large
apps were installed onto it, started a game already at serious and was held
at 30 from the first frame, which is the case the setting is for.

The tests follow: Render scale is a basic row under Picture, and in the Mac
window's list for Breath of the Wild every Game row is basic, so its Advanced
settings start at Picture.
@Xipzer

Xipzer commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Now that this Mac has the chain lock's compiler (Xcode 27.2), scripts/regress.py check on Breath of the Wild 1.9.0, main (2e6edda) against this branch, replacing my earlier note that I could not run it. Two runs: the first was flagged on its screenshots only, the second (below) passes.

regress.py check (full): PASS

Baseline 2e6edda4d334feaf48326407c1718cb96c43e2de 2e6edda, candidate 704ca34ee5ca13a53963f3651e454d453bd77ba4 704ca34; Apple M4 Max, macOS 27.2.1, Xcode 27.2; 297.4 s

game status shaders (base → cand) newly failing newly building changed MSL interface (new / known) boot fps p50 frame p99 ms peak MB prepare s new RED
BOTW 01007EF00011E000 PASS 19592/19592 → 19592/19592 0 0 0 0 / 0 ok 59.8 → 59.8 17.83 → 17.29 3243 → 3244 0.05 → 0.05 0

BOTW 01007EF00011E000: PASS

  • screenshots (s: dHash bits / RGB diff / baseline noise): 60: 6/2.71/0, 80: 7/3.51/0
  • baseline: exit None, first frame 2.1 s, max frame 6270, reached True, prepare 0.05 s, prebuild 168.7 s (15767 built, 0 failed), fps 59.8, p50/p99 16.67/17.83 ms, peak 3243 MB
  • candidate: exit None, first frame 2.6 s, max frame 6222, reached True, prepare 0.05 s, prebuild 168.0 s (15767 built, 0 failed), fps 59.8, p50/p99 16.67/17.29 ms, peak 3244 MB
  • time: chain 18.6 s, smoke 278.8 s

First run's screenshots: screenshots (s: dHash bits / RGB diff / baseline noise): 60: 16/8.23/0, 80: 16/14.27/0

Screenshot noise of this route with my save: main (2e6edda) checked against itself three times gave PASS (80 s: RGB 2.88), PASS (3.17) and REGRESSION (60 s: RGB 11.6, 80 s: 34.28). Cloud shadows cross the field where my save stands, so a screenshot flag on this route does not separate a change from the weather. Every other column is the same in all runs.

The route's save is my own outdoors save, not the one smoke.tsv was written with. The route leaves DM_PRESENT_UPSCALE_OUT and DM_SHARPEN unset.

Two additions to the present pass, both of which work for any game because
neither needs anything from the game.

DM_PRESENT_UPSCALE_OUT=1.0..4.0 reconstructs above the viewport and averages
back down. The present shader took one bilinear sample, which reads 4 texels
whatever the reduction, so reconstructing larger and sampling down this way
threw nearly all of it away. The tap count now follows the reduction ratio,
capped at 4x4, which makes it real supersampling. At 1.0, the default, the
scaler still targets the viewport and the shader still takes one tap, so
nothing changes.

In the launcher it is Upscaling detail, a sub-setting of Upscaling
(needs=DM_PRESENT_UPSCALE=metalfx, seedds#418): its row is drawn only while MetalFX
is chosen.

DM_SHARPEN=0..1 is contrast adaptive sharpening of the finished picture: the
5 tap cross, clamped to the neighbourhood's own range so it cannot ring or
halo. It is a second fragment function taken instead of the blit, not after
it, so it adds no pass and costs nothing when off.

The present without a Metal 4 queue passes one tap and no sharpening, so it
draws as before.

Measured with the same two changes in a fork, on Breath of the Wild 1.9.0 on
an iPad Pro 13" M4, 1920x1080 content into a 2752x1548 viewport, all at
60.00 fps with thermal state nominal and a 16.67 ms budget:

  upscaling  output  scaler target  GPU
  off        -       -               8.53 ms
  metalfx    1.0     2752x1548       9.09 ms
  metalfx    2.0     5504x3096      11.34 ms

Before the box filter, output 2.0 looked identical to 1.0 while costing the
same 11 ms, which is what led to the change.

Compiles. Not yet run against this main: our devices are built from an older
chain and rebuilding them is a long job, so the device numbers above come
from the fork. Opening it now because both settings are off by default.
@Xipzer

Xipzer commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main, which now has MetalFX read the presented texture and write the drawable itself. Upscaling detail sizes the scaler before main's checks, so above 1.0 the scaler writes its own texture and the present pass averages it down. The sharpening pipeline is built through main's compile wrapper. t_present_crop (117) and t_present_alloc pass.

Xipzer added a commit to Xipzer/nxnative that referenced this pull request Nov 18, 2026
Game carries eleven unrelated rows: how fast the game runs, how large it
renders, which console mode it reports, how the finished frame is scaled,
fitted and shaped, which language it speaks, how the Mac window opens, and a
frame-rate read-out. The only thing they have in common is that none of them
belonged in Controls or Performance.

Picture takes the rows that are about the finished frame rather than about
the game: Render scale, Upscaling, Game's upscaler, Picture and Picture
shape. Game keeps Frame rate, Dynamic resolution, Console mode, Game language
and the Mac's Display mode. The frame-rate counter moves to Performance next
to the Performance HUD, since both are read-outs rather than settings. The
iOS in-game menu follows the sections, so the counter comes after the
controls there.

The split matters more as the section grows. With the anisotropic and
trilinear filtering of seedds#335 and the upscaling detail and sharpening of seedds#345,
Game would hold fifteen rows, nine of which are about how the picture looks.

The heading order on screen is the order of SECTIONS, and rows keep their
table order inside a section, so settings.tsv still decides the layout on its
own.

In docs/settings.md the Picture section takes the anchor #picture, and the
Picture setting inside it becomes #picture-1.

The thermal cap (seedds#220) holds the frame step at 30 fps while iOS reports a
serious or critical thermal state, and DM_THERMAL_CAP=0 turned it off only
from an env file. It is a trade a player should be able to make: with a phone
cooler, or for a player who would rather have an uneven 45 to 55 than an even
30, it costs more than it saves.

"Halve when hot" in Performance, on by default, so nothing changes for anyone
who does not touch it. Shown only for a game with a frame step, as Hold
frames when late is.

The note says what the thermal state does not catch. On an iPhone 17 Pro Max,
Breath of the Wild 1.9.0 dropped a third of its frames for ten minutes with
the state reading nominal throughout, and one scene's GPU time went from
8.40 ms to 21.83 ms while it still read nominal: iOS lowers the clock well
before it calls the device hot. The same phone, charged over USB while large
apps were installed onto it, started a game already at serious and was held
at 30 from the first frame, which is the case the setting is for.

The tests follow: Render scale is a basic row under Picture, and in the Mac
window's list for Breath of the Wild every Game row is basic, so its Advanced
settings start at Picture.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant