Skip to content

Proposal: decide syncs by save versions instead of guessing from files #29

Open

Description

@ChakraFusion

Part of #21.

The problem

Today a sync decides what to do from the files alone: their timestamps, which ones exist on each side, and what both devices held at the last sync (the lineage). That works while every transfer completes. Once one doesn't, the guesses go wrong in ways that cost data. All of these happened on my four PCs:

  • A device that received part of a large save offered its partial copy back. The files it lacked were read as deleted over there, and 23,000 files were deleted on the device that still had them. Later, a device not yet updated did the same to a restored save: 6,927 files.
  • A device with unsynced work of its own (the watcher still waiting for the game to finish writing) looked merely "behind", and took the other device's save over its own.
  • Two devices that had never agreed on anything were merged file by file into a save no game ever wrote.
  • Conflict prompts appeared for changes nobody had made: timestamps re-written by a sync, exclusion rules changed, launcher files.

The proposal

Each device records, per game, which version of the save it holds, as a version vector: one counter per device, moved only when the save really changes on that device, never by a sync. The version travels in the manifest response. Two devices compare versions before anything else:

Comparison What happens
One is older That device is behind. It takes the newer save whole (every file and every deletion), only from a device that holds it completely, and only then adopts its version. Until then it is a source for nobody: nothing is taken from it, pushed by it, or deleted on its word. A pull that stops part-way is remembered across restarts, so the half-copy is never mistaken for a change.
Same Nothing to do.
Neither is newer Both changed independently: a real conflict, and the only case that asks. Resolution stays mutual: "Keep mine" on one device doesn't make the other simply older. The other device is asked in turn, and the answer travels with the version.

More details:

  • Targets. A device remembers the newest version it has heard of. Two devices that both know a newer one exists do nothing with each other until a device holding it is back. This is what stops a half-finished copy being passed on.
  • An empty folder holds no version. It fills itself and never raises a conflict.
  • Unreported changes count. Every sync, and every manifest served, first checks the save against the hash recorded with its version. A change the watcher hasn't reported yet is therefore already a version before anyone compares.
  • Excluded files and rule changes aren't versions. The recorded hash is tagged with the exclusion rules it was taken under and simply re-taken when they change.

Saves from before versions

  • Existing saves get a starting version named after their content, so devices that already agree agree immediately.
  • Between two such saves that differ, the order is:
    1. the one still holding the state both last agreed on is the older;
    2. otherwise, the one newer in every file that differs is the newer;
    3. otherwise, the save holding the most recent work wins (the latest-written differing file, then the one newer in more files);
    4. only an exact tie is asked about.
  • The other device takes the winning save whole, never file by file, and keeps a snapshot of its own first.
  • Two saves with no file in common (the same emulator save under different names) are asked about.
  • "Use this save everywhere" (game header and conflict dialog) settles states nobody can rank, such as half-arrived copies and mixtures.

Mixed versions during the transition

  • A peer without versions decides by the old file comparison, so whatever its copy lacks it reads as deleted. Nothing is therefore taken from such a peer, and deletions it requests are refused, on the LAN and over the relay. It can still take from devices with versions.
  • The device list marks it "needs update".
  • Peers announce versions with "versions": true in the manifest response (LAN protocol revision 3). The relay path now carries the version too.
  • Games with automatic sync off are not watched, so they keep the old file comparison.

Storage

One table, game_versions (migration 0039_game_versions.sql in the PR, after 0038_peer_links from the network-aware PR).

Code

  • The full implementation is on my fork: https://github.com/ChakraFusion/OpenSave/tree/pr/save-versions. It's about 2,450 changed lines across 7 commits, with unit tests (version_test.go has 17 scenario tests).
  • It has run on 4 PCs through emptied folders, interrupted restores, offline changes and real conflicts.
  • It's stacked on the "Keep mine" and own-writes PRs. It relies on internal/owntouch to tell OpenSave's writes apart from the game's.
  • I'd open it as a draft PR alongside this issue, so the design can be discussed against real code.

Does this direction work for you? I'm happy to change names, protocol fields, or how the transition is handled.


Draft PR with the implementation: #30 (stacked PRs #31–#38 build on it).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions