Skip to content

Save sync through a cloud folder for the Mac and iOS apps (DM_SAVE_SYNC) - #255

Closed
parsifal295 wants to merge 13 commits into
seedds:mainfrom
parsifal295:save-sync
Closed

parsifal295 wants to merge 13 commits into
seedds:mainfrom
parsifal295:save-sync

Conversation

@parsifal295

@parsifal295 parsifal295 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Saves can now move between the Mac app and the iOS app through a folder both devices share, normally in iCloud Drive (DM_SAVE_SYNC). The host does the sync itself, so the same code serves both platforms: dm/host/savesync_apple.m, hooked into the transactional save in hle_fs.c.

Format. Each save directory is one file in the folder, <folder>/<program ID>/save.nxsave (save_device.nxsave for a device save of its own). It is one file rather than a tree because iCloud Drive delivers a file whole but a tree file by file, so another device could read half a save. The file is a text header (format, SHA-256 of the body, device, time) followed by a raw-deflate body: every directory and file in path order, with its guest name (org.d2rmobile.guest-name, for case-collision names) and its bytes. It stores no times, so one save always has the same digest. A 10 MB Unicorn Overlord save packs to 244 KB.

Rules. Each device keeps the digest it last synced in <save>.sync/base. At the first mount of a save in a run (in mount_ex, outside its locks, before the work directory is cloned), the host compares this device's committed save, the folder's and that base:

  • Only the folder changed: pull. The save is unpacked beside the old one and swapped in with renamex_np(RENAME_SWAP), the same move commits use, so a failure leaves the save as it was.
  • Only this device changed: push.
  • Both changed: the front end asks which save to keep (front_save_sync_choose: an NSAlert on the Mac, a UIAlertController on iOS), or DM_SAVE_SYNC_RESOLVE=local|cloud decides. "Not now" syncs nothing that run.

After each commit, commit_mount_locked reads the committed save under its locks and queues a push on a serial queue; on iOS the push holds a background task. The push reads the folder again first. If another device saved during the run, it pushes nothing more and tells the player (front_save_sync_notice), so the next start asks. Whatever a sync replaces is kept first in <save>.sync/backup/ (the last 10):

  • this device's save, before a pull;
  • the folder's save, before a conflict is decided for this device;
  • versions iCloud set aside when two devices wrote at once (NSFileVersion).

dm/tools/nxsave.py lists or unpacks an archive. If the folder cannot be read within the time-out (30 s at mount, 15 s per push), or its file does not match its digest, nothing is synced that run.

Setup.

  • Mac: with iCloud Drive on, the launcher creates iCloud Drive/nxnative-saves and uses it, so nothing needs configuring. DM_SAVE_SYNC=<folder> in env.txt picks another folder, and an empty value turns sync off.
  • iOS: a new Sync folder row in the launcher picks that folder with the system folder picker. The folder is kept as a security-scoped bookmark (dm/ios/Sources/SaveSync.swift), so no entitlement or paid account is needed, and main.m resolves it at boot.
  • LiveContainer: the app's Fix File Picker setting must be on, or the picker's Open button stays disabled.

hle_fs.c carries do-nothing weak defaults of the hooks for the unit tests that build it alone. docs/playing.md (the player tip), docs/settings.md (Save sync, with first-time steps) and docs/development.md (Save sync, the Mac app, the tools table) describe it.

Related PRs

Checked

  • dm/host/tests/t_savesync.m (new, in make run) covers:

    • a first push;
    • a pull, keeping guest names and empty directories;
    • a push after a commit, and an unpushed save pushed at start;
    • a conflict left alone, decided for the folder, or decided for this device by DM_SAVE_SYNC_RESOLVE, each with its backup;
    • a push refused after another device saved;
    • a damaged folder file;
    • archive paths that leave the directory.

    It passes with its folders in /tmp and inside iCloud Drive. The whole host test run passes (GAME=010069401ADB8000).

  • dm/tools/tests/test_nxsave.py (new). The tool tests pass (171, 14 skipped), including the docs check.

  • The iOS smoke build (dm/ios/tools/build_smoke.py) succeeds, and the app links the strong hooks.

  • Unicorn Overlord's Mac app (010054B01AD92000, Asian 1.05), syncing through iCloud Drive:

    • The first start pushed its save. nxsave.py reads the archive with the same digest a separate Python walk of the save gives.
    • A staged conflict (the folder's archive with an extra file, a stale base) waited at the save mount. With DM_SAVE_SYNC_RESOLVE=local it kept the Mac's save and backed up the folder's.
    • A newer folder archive was pulled at start, with the Mac's save backed up first.
    • A change made only on the Mac was pushed at the next start.
    • With no env.txt, the launcher's default folder synced; DM_SAVE_SYNC= (empty) synced nothing.
  • A new user, simulated with a HOME holding an empty iCloud Drive (13 Sentinels' Mac app): the launcher created nxnative-saves, the game made its first save at boot, and the push after that commit put it in the folder.

  • iPhone through LiveContainer (tested by me on the device): picked the folder in the launcher, saved in game, then started the Mac app, which pulled the iPhone's progress. The picker needed LiveContainer's Fix File Picker.

  • Not checked:

    • the conflict alerts on screen, on either platform;
    • the bookmark across restarts on a device;
    • iCloud's own conflict versions.

…iOS app take turns on one save. Each transactional save directory is kept in the folder as one archive file, <folder>/<program ID>/save.nxsave (save_device.nxsave for a device save of its own): one file and not a tree, because iCloud Drive delivers a file whole but a tree file by file, and another device could read half a save. The archive is a text header (format, SHA-256 of the body, device, time) and a raw-deflate body of every directory and file in path order with its guest name (org.d2rmobile.guest-name) and bytes, no times, so one save always has one digest. Each device keeps the digest it last synced in <save>.sync/base. At the first mount of a save in a run (mount_ex, outside its locks, before the work directory is cloned) the host compares this device's committed save, the folder's and that base: only the folder changed → pull (unpacked beside the save and swapped in with renamex_np RENAME_SWAP, as commits are); only this device changed → push; both → the front end asks which to keep (front_save_sync_choose: an NSAlert on the Mac, a UIAlertController on iOS), or DM_SAVE_SYNC_RESOLVE=local|cloud decides; "Not now" syncs nothing that run. After each commit (commit_mount_locked) the committed save is read under the locks and pushed on a serial queue; the push re-reads the folder first and, if another device saved during the run, pushes nothing more and tells the player (front_save_sync_notice), so the next start asks. Whatever a sync replaces, this device's save before a pull or another device's before a push over it (and versions iCloud set aside when two devices wrote at once, from NSFileVersion), is kept in <save>.sync/backup/ (the last 10); dm/tools/nxsave.py lists or unpacks one. A folder that cannot be read within the time-out (30 s at mount, 15 s per push), or a file that does not match its digest, syncs nothing that run. iOS: the launcher's new Save sync row picks the folder with the system folder picker and keeps it as a security-scoped bookmark (dm/ios/Sources/SaveSync.swift; no entitlement or paid account), which main.m resolves at boot unless an env file sets DM_SAVE_SYNC; the push holds a background task. Mac: with iCloud Drive on, the launcher creates iCloud Drive/nxnative-saves and sets DM_SAVE_SYNC to it (dm/mac/launcher.c), so the folder is there to pick on the iPhone after one start on the Mac, even with no save yet; env.txt can name another (~ expanded) or turn it off with an empty value. hle_fs.c carries do-nothing weak defaults of the hooks for the unit tests that build it alone. README, docs/settings.md (Save sync, with the first-time steps) and docs/development.md (Save sync, the Mac app, the tools table) describe it.

- dm/host/tests/t_savesync.m (new, in `make run`): first push, pull with guest names and empty directories kept, push after a commit, push at start of an unpushed save, a conflict left alone / decided for the cloud / decided for this device by DM_SAVE_SYNC_RESOLVE (each with its backup), a push refused after another device saved, a damaged folder file, archive paths that leave the directory. Passes with its folders in /tmp and inside iCloud Drive (~/Library/Mobile Documents/com~apple~CloudDocs). The whole host test run passes (GAME=010069401ADB8000).
- dm/tools/tests/test_nxsave.py (new); the tool tests pass (123, 14 skipped), docs check included.
- iOS smoke build (dm/ios/tools/build_smoke.py) succeeds; the app links the strong hooks.
- Mac app of Unicorn Overlord (010054B01AD92000, profile from seedds#181) syncing through iCloud Drive (DM_SAVE_SYNC from env.txt, then from the launcher's default): a first start pushed its 10 MB save as a 244 KB archive, which nxsave.py reads with the same digest a separate Python walk of the save gives; a staged conflict (folder archive with an extra file, stale base) waited at the save mount for an answer, and with DM_SAVE_SYNC_RESOLVE=local kept the Mac's save and backed up the folder's; a newer folder archive was pulled at start with the Mac's save backed up first; a change made only on the Mac was pushed at the next start. The boot commit rewrote identical bytes, so no push followed it.
- A new user, simulated with a HOME holding an empty iCloud Drive folder (13 Sentinels' Mac app): the launcher created nxnative-saves, the game made its first save at boot, and the push after that commit put it in the folder (10 KB archive).
- iPhone, installed through LiveContainer (user): the folder picked in the launcher, a save made on the iPhone, then the Mac app started: it pulled the iPhone's progress. The folder picker's Open button stayed disabled until LiveContainer's per-app Fix File Picker setting was turned on; README and docs/settings.md say so.
- Not checked: the conflict alerts on screen (Mac or iPhone), the bookmark across restarts on a device, and iCloud's own conflict versions.
…n) and the save sync call before the locks; list DM_SAVE_SYNC and DM_SAVE_SYNC_RESOLVE in docs/env_switches.tsv
…, whose Python has no os.setxattr (unpacked backups lost case-collision names there); the test now checks it on macOS too. Fixes the lint ratchet's five new findings (os.setxattr / os.getxattr unknown, the module spec's Optional)
# Conflicts:
#	dm/mac/launcher.c
#	docs/development.md
#	docs/env_switches.tsv
# Conflicts:
#	dm/mac/launcher.c
#	docs/development.md
#	docs/settings.md
…save sync folder in boot_env; the Sync folder row is locked when the launch environment sets DM_SAVE_SYNC too, as the launcher's other rows are
…ith $(QUIET), as main's tests now do, so the sanitizer jobs cover it (passes under make asan)
…ave-on-iPhone-and-Mac tip moves to docs/playing.md beside saves and logs
@parsifal295

Copy link
Copy Markdown
Contributor Author

Superseded by #617 (123c7ff), which landed the same Mac/iPhone save sync on main. Closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant