Repository navigation
Conversation
The game's image is stretched to the drawable by one bilinear sample in the present shader. On an iPad Pro 13" M4 that is 1920x1080 to 2752x1548, a 2.0x upscale per axis, and it is the largest source of aliasing on a panel that much larger than the game's output: larger than the game's own render scale. MTL4FXSpatialScaler reconstructs the same upscale instead. It needs no motion vectors and no depth, so it is per game free, and newSpatialScalerWithDevice: compiler: gives an MTL4 scaler that encodes on the command buffer the present already uses. Only the whole-image case is taken; a cropped present keeps the bilinear path, as does any failure to build the scaler or its output texture. DM_METALFX=1 turns it on, DM_METALFX_OUT=0.5..4.0 sets the output as a share of the present viewport (above 1.0 reconstructs larger and the present samples it back down). Both are in the launcher. Measured on Breath of the Wild 1.9.0, iPad Pro 13" M4, 60 fps held at 100 % either way, GPU time per frame unchanged within run to run noise. Metal HUD confirms Scaling Spatial, input 1920x1080, target 2752x1548. Two things worth knowing for anyone touching this file: MetalFX.h pulls in <complex.h>, whose I macro collides with the local static const MTLVertexFormat I[] in vfmt_for_ct, hence the #undef I. present_encode_m4's src parameter is objc_externally_retained, so ARC refuses to reassign it and the blit source has to be tracked in separate locals.
|
Thanks, and good to see the M4 iPad numbers: on an iPhone 17 Pro Max we measured 2–5 ms GPU for the same spatial pass, so the cost clearly varies by device. Present-level MetalFX is already in |
Three additions on top of the spatial scaler, all of which work for every game because none of them needs to know anything about the game. Box downsample. The present shader took one bilinear sample, which reads 4 texels whatever the reduction, so reconstructing above screen size and sampling back down threw most of it away: DM_METALFX_OUT=2.0 looked identical to 1.0. The tap count now follows the reduction ratio, capped at 4x4, so it is real supersampling. Anisotropic filtering (DM_ANISO=2..16). The host already builds every sampler the game uses and maxAnisotropy already flows through from nvnSamplerBuilderSetMaxAnisotropy. The games ask for almost none, being built for a handheld, so ground and wall textures blur at a glancing angle. This raises the floor: sampling only, no render target, no allocation, and a sampler that asks for more keeps what it asked for. Sharpening (DM_SHARPEN=0..1). Contrast adaptive, 5 tap cross, on the final picture. Clamped to the neighbourhood min and max so it cannot ring or halo. A second pipeline from the same library, selected per frame, so it costs nothing when off. Measured on Breath of the Wild 1.9.0, iPad Pro 13" M4, all 60.00 fps, thermal nominal: render scale MetalFX scaler target GPU 100% off - 8.53 ms 100% 100% 2752x1548 9.09 ms 100% 200% 5504x3096 11.34 ms 140% off - 8.38 ms 140% 100% 2752x1548 8.94 ms 140% 200% 5504x3096 11.77 ms MetalFX at screen size costs 0.55 ms, at 200% output 2.8 to 3.3 ms. Worst case 11.77 ms of 16.67 ms.
|
Thanks, that is the answer I needed. I was working from an older Splitting what is left into small PRs on top of current
I will close this branch once 2 and 3 are up rather than rebase it, since almost none of the original diff survives. On the 2 to 5 ms you measured for the spatial pass on an iPhone 17 Pro Max against 0.55 ms on the M4 iPad: that gap is larger than the raw GPU difference, so part of it is probably the output size. The M4 iPad number is a 1920x1080 to 2752x1548 scale; an iPhone at 2868x1320 from 1280x720 handheld is a bigger ratio. I have the same phone and will post a matched pair. On temporal: I dumped a full frame of Breath of the Wild with |
|
Here's the pre-HUD scaling experiment, on the
Possibly relevant to #335: d67e7f27 passes the game's LOD bias through to the NVN and Vulkan samplers. Worth checking that the two don't overlap. |
|
Closing in favour of two small PRs on current |
The game's image is stretched to the drawable by one bilinear sample in the present shader. On an iPad Pro 13" M4 that is 1920x1080 to 2752x1548, a 2.0x upscale per axis.
That single filter is the largest source of aliasing on a panel much larger than the game's output, ahead of the game's own render scale.
MTLFXSpatialScalerreconstructs the same upscale instead.Why spatial
It needs no motion vectors and no depth, so it costs nothing per game and works for every title through the shared
present_encode_m4path.newSpatialScalerWithDevice:compiler:(iOS 26.0+) returns anid<MTL4FXSpatialScaler>that encodes on theMTL4CommandBufferthe present already uses, so there is no Metal 3 bridging.Only the whole-image case is taken. A cropped present keeps the bilinear path, as does any failure to build the scaler or its output texture.
Settings
DM_METALFX=1turns it onDM_METALFX_OUT=0.5..4.0sets the output as a share of the present viewport. Above 1.0 reconstructs larger and the present samples it back down. Clamped to 8192 keeping aspect.Both are in the launcher.
Measurements
Breath of the Wild 1.9.0, iPad Pro 13" M4. 60 fps held at 100 % with and without, GPU time per frame unchanged within run to run noise. Metal HUD confirms
Scaling Spatial, input1920x1080, target2752x1548.At
DM_METALFX_OUT=2.0the scaler runs 1920x1080 to 5504x3096 and 60 fps still held, though thermal state reached 2. The extra reconstruction is mostly averaged away by the present's single bilinear tap on the way back down, so 1.0 is the useful setting.Two things worth knowing for this file
MetalFX.hpulls in<complex.h>, whoseImacro collides with the localstatic const MTLVertexFormat I[]invfmt_for_ct. Hence the#undef Iafter the import.present_encode_m4'ssrcparameter isobjc_externally_retained, so ARC refuses to reassign it. The blit source and its dimensions are tracked in separate locals.Not included
MTL4FXTemporalScalerandMTL4FXFrameInterpolatorare both in the iOS 27 SDK. Both need per pixel motion vectors and depth, which means identifying the NVN render targets that carry them per game. That is a separate investigation.