Skip to content

Serialize dirty-page protection with NVN pool retirement - #686

Closed
widmonstertony wants to merge 1 commit into
seedds:mainfrom
widmonstertony:fix/dirty-pool-retirement
Closed

widmonstertony wants to merge 1 commit into
seedds:mainfrom
widmonstertony:fix/dirty-pool-retirement

Conversation

@widmonstertony

Copy link
Copy Markdown
Contributor

What and why

Pool retirement can finish between nvn_dirty_protect finding a live range and changing its first page to BUSY. Retirement restores write access and unpublishes the range; protect then makes the retired storage read-only. A subsequent host read() into that storage fails with EFAULT, and the tracker no longer owns the page.

Hold the existing pool-slot mutex from protect's lookup through its final mprotect. Pool creation and retirement already use that mutex. The fault handler and host pre-write path remain independent of that mutex, so they can complete a BUSY transition while protection waits.

This extracts the shared tracker fix from fork commit 587d5206, discussed in #310. It is independent of the older renderer, deployment baseline, tvOS and game-specific changes. This is a CPU-side pool-lifetime race present on current main; it does not assert that an additional Metal 4 encoder race or any particular game's black screen is fixed.

Checked

  • Apple M4, macOS 27.2, Apple clang 21.0.0. Based on 85578fc9 from current main.
  • Negative control: current upstream dirty tracker plus the lookup test hook, without the mutex change, fails 3/3 runs: retirement completes, the range is untracked, and a 16-byte read() returns -1 with errno=EFAULT.
  • Candidate: the same regression reads all 16 bytes after retirement; existing t_dirty_race and t_dirty_slots pass. The new regression also passes ASan/UBSan and ThreadSanitizer. It is included in the CPU-only CI list and the normal graphics run list.
  • Full make -C dm/gfx/tests GAME=010015100B514000 -j2 run and make -C dm/host/tests GAME=010015100B514000 -j2 run pass. These were run on 710559cf; the final base increment 85578fc9 changes NSP extraction/tools/docs only, with no host, graphics or iOS source changes.
  • iOS source compile check: 183 files, 0 failures. iOS Python tests: 39 pass; documentation tests: 11 pass. Tool tests on 710559cf: 608 discovered, OK with 17 skipped. On the final 85578fc9 base with the existing .NET runtime on PATH: 613 discovered, 596 passed, 16 skipped and 1 environment failure: test_extractor_streams_populated_and_empty_filesystems cannot run dotnet run because this machine has no .NET SDK. The same test fails identically on pristine 85578fc9 without this PR (exit 155, “No .NET SDKs were found”). The new NSP scan tests pass 7/7, macOS bundle tests pass 44/44. This is not an all-green local tools result; .NET SDK 10 CI validation remains open. Python lint ratchet: 392 existing findings, 0 new findings.
  • The shader chain and profile inputs are unchanged by this PR. No game assets, keys, extracted shaders or platform fallback are included.

Gameplay validation remains open

This is a draft for focused review. No game was automatically launched: the device owner explicitly handles gameplay testing. The deterministic reproducer and synthetic GPU suite establish the tracker invariant, not a new five-game gameplay result or an iPhone runtime result.

regress.py impact upstream/main..HEAD reports 3 affected profiles with usage manifests, 0 broad-only, 1 unaffected and 110 unknown. The affected manifests are Breath of the Wild (01007EF00011E000), Pokémon Scarlet (0100A3D008C5C000) and Tears of the Kingdom (0100F2C0115B6000); all use the NVN pool/texture submission path. Fresh gameplay checks of those titles, and NFS: Hot Pursuit Remastered plus Animal Crossing to cover pool churn and CPU texture uploads, remain unrun. Reviewers' current-device tests would be useful before marking ready. Protection calls now serialize across pools through the existing global slot mutex; gameplay throughput and lock contention have not been benchmarked. The change claims correctness, not a speed improvement.

Full usage-impact report (static, not a gameplay result)

regress.py impact upstream/main..HEAD

3 games use what the change touches, 0 only through a broad rule (every game of its API), 1 not affected, 110 unknown (no usage manifest); 6 files changed, 11 NVN / HLE entry points reached through the host's callers, 1 rules of dm/tools/impact_map.tsv

game status what it uses of the change (kind: name ×count; for broad, the broad rows)
The Legend of Zelda: Breath of the Wild 01007EF00011E000 affected nvn: nvnTextureInitialize ×321706, nvn: nvnQueuePresentTexture ×3765, nvn: nvnQueueSubmitCommands ×3765, nvn: nvnWindowAcquireTexture ×3765, nvn: nvnDeviceGetProcAddress ×1068, hle: nvnBootstrapLoader ×2
Pokémon Scarlet 0100A3D008C5C000 affected nvn: nvnQueuePresentTexture ×5354, nvn: nvnQueueSubmitCommands ×5354, nvn: nvnWindowAcquireTexture ×5354, nvn: nvnTextureInitialize ×2177, nvn: nvnDeviceGetProcAddress ×2128, hle: nvnBootstrapLoader ×2 … (7 in all)
The Legend of Zelda: Tears of the Kingdom 0100F2C0115B6000 affected nvn: nvnTextureInitialize ×55959, nvn: nvnQueueSubmitCommands ×2705, nvn: nvnQueuePresentTexture ×2704, nvn: nvnWindowAcquireTexture ×2704, nvn: nvnDeviceGetProcAddress ×1068, nvn: nvnTextureReadTexels ×25 … (7 in all)
Hades 0100535012974000 not affected

Unknown (no usage.tsv): 110 profiles (--all lists them).

Why

  • api:nvn ← dm/gfx/nvn/nvn_dirty.c: NVN on Metal: every NVN game
  • hle:nvnBootstrapLoader ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnDeviceGetProcAddress ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnQueueBuilderSetQueuePriority ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnQueueFinish ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnQueuePresentTexture ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnQueueSubmitCommands ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnTextureInitialize ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnTextureReadTexels ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnTextureReadTexelsStrided ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvnWindowAcquireTexture ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat
  • nvn:nvn_lookup ← dm/gfx/nvn/nvn_dirty.c: calls reach it from nvn_dirty_protect, nvn_dirty_stat

Docs

  • General mechanism documented in docs/reference/nvn-objects.md under “A retired pool remains read-only”; symptom index regenerated.
  • No shader-chain tool changed; no chain header update is needed.
  • No game files, keys or extracted data in the change or attachments.

Keep the range descriptor live until the final mprotect so retirement cannot restore writable pages and unpublish the slot before protection starts. The signal handler and host pre-write path remain lock-free with respect to the slot mutex.

Extract the shared tracker fix from fork commit 587d520. A staged lookup/retirement regression fails with EFAULT on current main and succeeds with the fix; include it in the CPU-only CI list.
seedds added a commit that referenced this pull request Oct 30, 2026
…st mprotect (PR #686)

nvn_dirty_protect looked up a live range without the slot mutex. Pool retirement could finish between that lookup and the
first page-state change: it restored the protected pages to writable and marked the range dead, then protect set the retired
storage read-only with no live range left to release it (a host read() into it failed with EFAULT; a game write faulted with
no owner). In the same window a new pool could reuse the slot, and protect computed its pages from the new pool's base and
state array. Protect now holds g_slot_mu, which pool creation and retirement already hold, through its last mprotect. The
page-fault and host pre-write paths still take no lock, so a BUSY page they hold finishes while protect waits.

t_dirty_retire_race stages the retirement between protect's lookup and its first page-state exchange by wrapping
atomic_compare_exchange_strong, as t_dirty_race wraps atomic_store; it fails with EFAULT without the mutex and passes with it,
also under ASan/UBSan and TSan. It joins the CPU-only CI list.

Co-authored-by: Xinpei Tan <[email protected]>
@seedds

seedds commented Oct 30, 2026

Copy link
Copy Markdown
Owner

Landed on main as af616f5, with you as co-author. Thanks for the clear diagnosis and the staged reproducer. Two changes on landing: the test now sets up the race by redefining atomic_compare_exchange_strong inside the test, as t_dirty_race does, so nvn_dirty.c carries no test hook. The reference entry moved under "CPU-written textures", next to the other dirty-tracker entries, and also notes that a new pool could reuse the slot during protect. Checked with a negative control (fails 3/3 with EFAULT without the lock), TSan, ASan/UBSan, and the full graphics and host suites. No game run was done for this landing.

@seedds seedds closed this Oct 30, 2026
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…st mprotect (PR seedds#686)

nvn_dirty_protect looked up a live range without the slot mutex. Pool retirement could finish between that lookup and the
first page-state change: it restored the protected pages to writable and marked the range dead, then protect set the retired
storage read-only with no live range left to release it (a host read() into it failed with EFAULT; a game write faulted with
no owner). In the same window a new pool could reuse the slot, and protect computed its pages from the new pool's base and
state array. Protect now holds g_slot_mu, which pool creation and retirement already hold, through its last mprotect. The
page-fault and host pre-write paths still take no lock, so a BUSY page they hold finishes while protect waits.

t_dirty_retire_race stages the retirement between protect's lookup and its first page-state exchange by wrapping
atomic_compare_exchange_strong, as t_dirty_race wraps atomic_store; it fails with EFAULT without the mutex and passes with it,
also under ASan/UBSan and TSan. It joins the CPU-only CI list.

Co-authored-by: Xinpei Tan <[email protected]>
@widmonstertony
widmonstertony deleted the fix/dirty-pool-retirement branch November 13, 2026 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants