Skip to content

Mac app carries the launcher settings; shader_split.py reports its progress - #352

Merged
seedds merged 2 commits into
seedds:mainfrom
v0lumehi:mac-launch-settings
Oct 27, 2026
Merged

seedds merged 2 commits into
seedds:mainfrom
v0lumehi:mac-launch-settings

Conversation

@v0lumehi

@v0lumehi v0lumehi commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Update (2026-10-27): rebased on main 75065df. The Mac app now takes the RomFS as rfs.source(game) (the streamed images from e09ff95), and the split's progress sits beside the new unreadable-file handling (5623c9a). Launcher, Mac app, incremental and split-progress tests pass, the lint ratchet reports 0 new. NX2Apple's launch options already read this launcher.json (its front end README), and #387 documents it for front ends.

Two small pipeline changes that a Mac front end needs, kept apart from any UI so they hold for the command line too.

Mac app: the launcher settings go into the app, as in the IPA

The IPA carries dm/launcher.json (settings.py from settings.tsv), which the iOS launcher draws its screen from. The Mac app had no such list, so a Mac front end could not offer a game's settings.

  • The NSP script writes the Mac app's list (settings.py with SETTINGS_PLATFORM=mac), and pack_app.py --settings puts it at Contents/Resources/dm/launcher.json.
  • The two settings only the iOS front end reads, the frame-rate counter (front_ios.m) and touch feedback (DmGamepad.m), are marked with a new condition platform:ios in settings.tsv and left out of the Mac list. Every other row is read by the Mac host as well (checked: DM_FPS_TARGET, DM_RENDER_SCALE, DM_OPMODE, DM_PRESENT_UPSCALE, DM_LANG, DM_RUMBLE*, DM_PRESENT_SNAP, DM_PSO_*, DM_NVN_ENC_THREAD, DM_FPS_HOLD, DM_METAL_HUD). build_ipa.sh is unchanged: ios is the default, so the IPA's list stays as it is.
  • dm/games/README.md and docs/settings.md say where the Mac list is.

shader_split.py: report its progress, counting jobs as they finish

The shader split printed nothing until it ended, so its stage was a spinner for minutes, although shader_split.py knows how many files and archive ranges it hands to its pool.

  • It now prints progress: done/total at most twice a second, which split_shaders forwards as the stage's count (files).
  • The count is of the jobs finished in any order. Counted in pool.map's order it would stand still behind one slow archive range (Breath of the Wild's Bootup_Graphics.pack) while the other workers go on. The jobs are submitted as futures; the results are still collected in job order, so the tables do not change. Checked on Breath of the Wild's three shader packs and 200 event packs: the package and the four tables are byte-identical to the previous version's.

Tests

test_launcher_settings.py (the platform: condition), test_nsp_mac_app.py (--settings, launcher.json in the bundle, the NSP script's command line), test_nsp_incremental.py, and a new dm/gfx/tests/test_shader_split_progress.py (a slow first job does not hold the count; batches and an offset count every job once; the results keep the job order). test_docs.py passes.

Context

These come from the Mac launcher I worked on for the NX2Apple fork (a library of the built games with Play and per-game launch options from this list; LNPANDY/NX2Apple#27). Upstreaming them here means the fork does not have to carry engine changes, and any front end can use them.

v0lumehi added a commit to v0lumehi/NX2Apple that referenced this pull request Oct 26, 2026
…gress go to seedds

The launch-settings list for the Mac app (settings.py, settings.tsv,
pack_app.py --settings, its step in the NSP script) and the shader split's
progress lines are engine changes. The fork takes those files from upstream
on every sync (build_ipa_from_nsp.py and shader_split.py are patch carriers,
checked for drift), so carried here they would block the sync or be
reverted by it. They are proposed upstream instead (seedds/nxnative#352),
and come back with the sync once merged.

The launcher works without them: an app without dm/launcher.json shows no
launch options, and a step without a count shows its stripes and its clock.
@v0lumehi
v0lumehi force-pushed the mac-launch-settings branch from e221f2a to 5769477 Compare October 26, 2026 23:47
The IPA carries dm/launcher.json, the game's settings from settings.tsv,
which the iOS launcher draws its screen from. The Mac app had no such list,
so a Mac front end could not offer a game's settings. The NSP script now
writes the Mac app's list (settings.py with SETTINGS_PLATFORM=mac) and
pack_app.py --settings puts it at Contents/Resources/dm/launcher.json.

The two settings only the iOS front end reads (the frame-rate counter in
front_ios.m, touch feedback in DmGamepad.m) are marked platform:ios, a new
condition in settings.tsv, and left out of the Mac list; build_ipa.sh keeps
the iOS list as it was (ios is the default).
The shader split printed nothing until it ended, so the NSP script showed
the stage as a spinner although shader_split.py knows how many files and
archive ranges it hands to its pool. It now prints 'progress: done/total'
at most twice a second, which split_shaders forwards as the stage's count.

The count is of the jobs finished in any order: pool.map yields in job
order, so counted that way it would stand still behind one slow archive
range while the other workers go on. The jobs are submitted as futures and
the results are still collected in job order, so the tables do not change
(checked against the previous version on Breath of the Wild's packs).
@v0lumehi
v0lumehi force-pushed the mac-launch-settings branch from 5769477 to 28c60fe Compare October 27, 2026 09:10
LNPANDY pushed a commit to LNPANDY/NX2Apple that referenced this pull request Oct 27, 2026
#35 Display mode: Window or Fullscreen for a Mac game, under the render
scale. The host starts fullscreen when DM_FULLSCREEN is set at all
(front_mac.m), so Window leaves the variable out instead of setting it
empty. A list from settings.tsv will carry the row once it has a
platform condition (seedds/nxnative#352); until then the launcher adds it
to a list that has none.

#36 Render scale in pixels: with the game's base size, the row shows
0.75 as 1200x675 and says what it is a share of ("75% of 1600x900
docked"), for the console mode chosen (1280x720 handheld for Breath of
the Wild). The base size comes from the row's base (seedds/nxnative#399:
a profile's render_base, written by settings.py), else from the game's
profile in the repository, so apps packed before still show pixels. A
game without one keeps the percentage. Play goes through the same list,
so Display mode reaches the app.

LaunchOptionsTests: Display mode's place and its unset Window, the
pixels docked and handheld from the profile, and the percentage without
a base size.
@seedds

seedds commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Local review and validation of 28c60fef7e7e09d2539a46a3d5bb2259a8e1765e, against 75065df74762d8277c4a93277d560429c2d43a7e:

  • All five GitHub CI checks pass.
  • Complete local tool suite: 348 tests, OK (16 skipped), using /Volumes/S790/nsp/lint-venv/bin/python with the project's dependencies.
  • iOS tests: 31 tests, OK.
  • Focused launcher settings, Mac app packaging, incremental build, JSON progress, shader split progress, docs and chain tests: 104 tests, OK. The Mac packaging tests exercise the compiled launcher, settings resource, signature, relocation, environment precedence, exit codes and signal forwarding with synthetic inputs.
  • make -C dm/gfx/tests GAME=010015100B514000 -j4 run: passed on this Mac, after generating the profile header in the clean checkout.
  • Settings generation compared with the base revision for all 79 profiles and three graphics API values (237 comparisons), with synthetic language/capability inputs: iOS metadata is unchanged; Mac metadata omits exactly DM_SHOW_FPS and DM_TOUCH_HAPTICS.
  • Python lint/type ratchet: 405 existing findings, 0 new, 0 fixed findings left listed. git diff --check passes.
  • The translation-chain version remains 157e1b96c8da; shader_split.py is outside the chain inputs. The progress change keeps scan results in submission order.

scripts/regress.py impact origin/main..HEAD reports 0 directly matched games, 3 through broad packaging/profile rules (Hades, Breath of the Wild, Tears of the Kingdom), and 76 profiles without usage manifests. The host and translation-chain sources are unchanged.

Full local regression command:

python3 scripts/regress.py --root /Volumes/S790/nx-harness --games-dir /Volumes/S790/nsp check --baseline 75065df74762d8277c4a93277d560429c2d43a7e --candidate 28c60fef7e7e09d2539a46a3d5bb2259a8e1765e --jobs 4 --out /private/tmp/nx352-regress-report

Status: stopped before the first game comparison; no game comparison or gameplay pass claimed. The harness detected 29 Mac builds; none has a committed smoke route. Their NVN maps contain roughly 437,000 shader rows in total, and the run was rebuilding the first baseline despite the unchanged chain. Game-derived outputs remain in each game's external-drive work folder. The maintainer explicitly chose to merge after the focused checks and document this full-run limitation; the baseline rebuild was stopped on that basis. There is no completed full-game regression report.

@seedds
seedds merged commit 2a4a1e3 into seedds:main Oct 27, 2026
5 checks passed
seedds added a commit that referenced this pull request Oct 27, 2026
Mac launcher metadata inherited the iOS defaults for Metal 4 pipelines
and deferred encoding. A frontend applying those values could disable
the Mac pipeline archive and enable its normally disabled encode thread.
Resolve ios|mac defaults in the shared table before profile macros, and
check both native defaults across profiles with Metal 4 on and off.

Release the shader-progress test's blocked worker directly from the
observed progress line, with bounded failure and executor cleanup.
Document and test argparse's exit 2 and stderr-only error path before
JSON progress starts. Record the metadata finding in the packing
reference and Wonder's note.

Follow-up to #352 and #387.

Co-authored-by: v0lumehi <[email protected]>
@seedds

seedds commented Oct 27, 2026

Copy link
Copy Markdown
Owner

The review follow-up is on main in 6138a54, co-authored with v0lumehi.

Mac metadata now matches the native host: Metal 4 pipelines default on, and the encode thread defaults off. settings.tsv supports ios|mac defaults, selected before resolving a profile macro or build variable, so iOS retains its profile-dependent Metal 4 default and enabled encode thread. Invalid platforms and malformed pairs fail clearly. The metadata regression test fails against the previous implementation and passes with the fix.

The progress test now releases its blocked worker directly when it observes progress: 19/20, with bounded failure and cleanup through the executor. The polling daemon thread is gone. The finding is recorded in the packing reference, the symptom index and Wonder's note.

Validation on this follow-up:

  • Full tool suite: 352 tests, OK (16 skipped).
  • iOS suite: 31 tests, OK.
  • Focused settings, JSON progress, shader progress and documentation checks: 22 tests, OK.
  • 81 profiles × 3 API variants = 243 metadata comparisons: iOS unchanged; Mac changes only the two intended defaults.
  • Python lint/type ratchet: 405 existing findings, 0 new or stale findings. git diff --check passes.

The host and shader chain are unchanged. The maintainer's explicit full-game/shader-rebuild waiver remains in effect; no new gameplay pass or completed full-game regression is claimed. The impact tool's runtime manifests do not capture frontend consumption of metadata, so its zero-match result is not evidence that Mac apps are unaffected.

regress.py impact HEAD^..HEAD

regress.py impact HEAD^..HEAD

0 games use what the change touches, 0 only through a broad rule (every game of its API), 4 not affected, 77 unknown (no usage manifest); 10 files changed, 0 NVN / HLE entry points reached through the host's callers, 2 rules of dm/tools/impact_map.tsv

game status what it uses of the change (kind: name ×count; for broad, the broad rows)
Hades 0100535012974000 not affected
The Legend of Zelda: Breath of the Wild 01007EF00011E000 not affected
Pokémon Scarlet 0100A3D008C5C000 not affected
The Legend of Zelda: Tears of the Kingdom 0100F2C0115B6000 not affected

Unknown (no usage.tsv): 77 profiles (--all lists them).

Why

  • profile ← dm/games/settings.py: one game's profile
  • profile ← dm/games/settings.tsv: one game's profile

seedds pushed a commit to Xipzer/nxnative that referenced this pull request Oct 27, 2026
…ON Schema front ends read

Draft for the discussion in seedds#388. Two open PRs want the ninth column of
settings.tsv for different things (seedds#355: needs, seedds#399: level), and seedds#379
brings its own metadata. This makes the ninth column one optional meta
column of key=value pairs, so any number of optional fields fit:

- settings.py's META lists the keys with the pattern each value must
  match (level=basic, needs=<VAR>=<value>); an unknown key, a key given
  twice, a value that does not match, or needs naming a variable no row
  sets stops settings.py. The fields go into the row of launcher.json.
- launcher.json gets "schema": 1. Schema 1 grows only by optional
  fields: a reader ignores fields it does not know, and a field keeps its
  meaning. docs/launcher.schema.json describes the file as a JSON Schema,
  so a front end can check it and generate types from it.
- The five settings a front end shows first carry level=basic, and
  Rumble strength carries needs=DM_RUMBLE=1, as seedds#355 has it.

The platform split stays the shown column's platform:<ios|mac> (seedds#352).

test_launcher_settings.py: the fields and the version in a written list,
the strict meta parsing, META agreeing with docs/frontends.md and the
schema, and every list written for Breath of the Wild, Wonder and Hades
(iOS and Mac) conforming to the schema (a small validator for the
keywords the schema uses, as CI has no jsonschema package).
test_docs, test_lint_python, test_launcher_storage, test_nsp_mac_app and
test_pso_offline pass.
surajmandalcell added a commit to surajmandalcell/nxnative that referenced this pull request Oct 27, 2026
The Mac .app now shows the launcher settings from dm/games/settings.tsv before
it starts dmhost: a Basic view with an Advanced settings disclosure, resolution
in real pixels, and a Window / Fullscreen row. Skip it with the check box,
--no-launcher or DM_LAUNCHER=0; hold Option to show it again.

Builds on seedds#352.
Xipzer pushed a commit to Xipzer/nxnative that referenced this pull request Oct 27, 2026
Mac app carries the launcher settings; shader_split.py reports its progress
Xipzer pushed a commit to Xipzer/nxnative that referenced this pull request Oct 27, 2026
Mac launcher metadata inherited the iOS defaults for Metal 4 pipelines
and deferred encoding. A frontend applying those values could disable
the Mac pipeline archive and enable its normally disabled encode thread.
Resolve ios|mac defaults in the shared table before profile macros, and
check both native defaults across profiles with Metal 4 on and off.

Release the shader-progress test's blocked worker directly from the
observed progress line, with bounded failure and executor cleanup.
Document and test argparse's exit 2 and stderr-only error path before
JSON progress starts. Record the metadata finding in the packing
reference and Wonder's note.

Follow-up to seedds#352 and seedds#387.

Co-authored-by: v0lumehi <[email protected]>
Xipzer pushed a commit to Xipzer/nxnative that referenced this pull request Oct 27, 2026
…ON Schema front ends read

Draft for the discussion in seedds#388. Two open PRs want the ninth column of
settings.tsv for different things (seedds#355: needs, seedds#399: level), and seedds#379
brings its own metadata. This makes the ninth column one optional meta
column of key=value pairs, so any number of optional fields fit:

- settings.py's META lists the keys with the pattern each value must
  match (level=basic, needs=<VAR>=<value>); an unknown key, a key given
  twice, a value that does not match, or needs naming a variable no row
  sets stops settings.py. The fields go into the row of launcher.json.
- launcher.json gets "schema": 1. Schema 1 grows only by optional
  fields: a reader ignores fields it does not know, and a field keeps its
  meaning. docs/launcher.schema.json describes the file as a JSON Schema,
  so a front end can check it and generate types from it.
- The five settings a front end shows first carry level=basic, and
  Rumble strength carries needs=DM_RUMBLE=1, as seedds#355 has it.

The platform split stays the shown column's platform:<ios|mac> (seedds#352).

test_launcher_settings.py: the fields and the version in a written list,
the strict meta parsing, META agreeing with docs/frontends.md and the
schema, and every list written for Breath of the Wild, Wonder and Hades
(iOS and Mac) conforming to the schema (a small validator for the
keywords the schema uses, as CI has no jsonschema package).
test_docs, test_lint_python, test_launcher_storage, test_nsp_mac_app and
test_pso_offline pass.
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
Mac app carries the launcher settings; shader_split.py reports its progress
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
Mac launcher metadata inherited the iOS defaults for Metal 4 pipelines
and deferred encoding. A frontend applying those values could disable
the Mac pipeline archive and enable its normally disabled encode thread.
Resolve ios|mac defaults in the shared table before profile macros, and
check both native defaults across profiles with Metal 4 on and off.

Release the shader-progress test's blocked worker directly from the
observed progress line, with bounded failure and executor cleanup.
Document and test argparse's exit 2 and stderr-only error path before
JSON progress starts. Record the metadata finding in the packing
reference and Wonder's note.

Follow-up to seedds#352 and seedds#387.

Co-authored-by: v0lumehi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants