Skip to content

Harden backup/delete against data loss; add encrypted backups and snapshot restore - #7

Merged
dbpecka merged 14 commits into
mainfrom
claude/blissful-dijkstra-5r6y4o
Oct 26, 2026
Merged

Harden backup/delete against data loss; add encrypted backups and snapshot restore#7
dbpecka merged 14 commits into
mainfrom
claude/blissful-dijkstra-5r6y4o

Conversation

@dbpecka

@dbpecka dbpecka commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

Several review passes over the app found bugs that could lose user data. Examples: a "messages only" delete whose attachment records were deleted by chat.db's own triggers, an orphan cleaner that deleted Live Photo videos, and backups that silently skipped every voice memo after the first. This PR fixes those, adds end-to-end tests against Apple's real chat.db schema, tightens the build's security settings, and adds four features: encrypted backups, restore from snapshot, free-space checks, and folder-type attachments in JSON/PDF backups.

New features

  • Encrypted backups: an optional AES-256 encrypted APFS .sparsebundle made with hdiutil, opened in Finder with a password. The password goes over stdin and is redacted from debug output. The image is mounted only while writing and is unmounted on every exit path. It counts as a backup for the delete step.
  • Restore from snapshot: the delete step lists the snapshots taken before deletes, cleanups and restores, and restores one.
    • Only the app's own snapshot folders can be restored, and the snapshot must pass SQLite's integrity check.
    • Messages must be quit, and iCloud needs an acknowledgement.
    • The current database is snapshotted first, so a restore can itself be undone.
    • The restore copies the snapshot in with SQLite's backup API rather than swapping files.
  • Free-space checks: backups and snapshots refuse up front, with sizes in the message, instead of failing halfway.
  • Folder attachments: bundle-style attachments are copied whole in JSON/PDF backups, so those backups can satisfy the delete step's backup check.

Delete safety

  • Backup check enforced by the backend: it compares a fingerprint of exactly what the backup contained (messages, attachments, and which files were actually copied) with what the delete would remove. New messages, or files downloaded after the backup, break the match. PDF exports don't count, because they can't store non-Latin-1 text.
  • What the user confirmed is what gets deleted: the delete must match the counts shown, the user types DELETE, and Messages must be quit.
  • iCloud: an acknowledgement is required, and the warning says deletes are expected to sync (chat.db's triggers queue them for iCloud).
  • Unrecoverable attachments: attachments that are only in iCloud need their own acknowledgement.
  • Removed "messages only": Apple's triggers delete an attachment's record with its last message, so the option couldn't keep what it promised.
  • Shared data: attachments and files still used by kept messages are excluded, and tapbacks on deleted messages are removed and saved with the backup.
  • Snapshots: taken before every delete in the app's own folder, five kept per kind, pruned only after a successful delete.

Backups

  • Unique attachment names: copies are named by attachment ID (no more collisions between same-named files).
  • Accurate reporting: per-file outcomes are reported, and one bad file no longer aborts the backup.
  • Originals kept: HTML/TXT copy byte-for-byte; HEIC is no longer converted to JPEG.
  • Fresh folder per run: the JSON exporter used to append to existing files.
  • External drives are now allowed as destinations.

Orphan cleanup

  • Folder-level detection: files are judged orphaned by folder, so companion files such as Live Photo videos are kept. Symlinks are never followed.
  • Path normalization before matching: case, //, symlinked or renamed home folders, and resolved roots, which fixed the failure on macOS's /var → /private/var.
  • Age guards: files and records from the last 24h are skipped.
  • Only what was reviewed: the clean refuses unless the data still matches the scan the user reviewed.
  • Safety gates: a chat.db snapshot is taken first, Messages must be quit, and removing records needs the iCloud acknowledgement.

Security / build

  • Entitlements: removed disable-library-validation and allow-unsigned-executable-memory.
  • Content Security Policy: strict, and checked against the production build in headless Chromium.
  • Unused shell plugin removed.
  • Pinned dependency: imessage-database is pinned to an exact git commit, and CI no longer clones the fork.

Tests

  • 103 Rust tests, including end-to-end tests (src-tauri/src/e2e_tests.rs) that run backup → delete → restore → orphan cleanup in a throwaway home whose chat.db uses Apple's real schema and triggers (src-tauri/tests/fixtures/chat-db-schema.sql).
  • CI's macOS job runs the suite. It caught the /private/var path issue that the Linux runs couldn't.
  • docs/MAC-TEST-CHECKLIST.md lists what still needs a person and a Mac: signing, WebKit CSP, real hdiutil encryption, real iCloud sync behaviour.

Known follow-ups

  • public/index.html "Download for Mac" buttons still use an iOS-only itms-services:// link.
  • quick-xml advisory via plist in the fork.
  • Snapshots aren't encrypted.
  • The remaining items in SECURITY-TODO.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36

claude added 14 commits October 22, 2026 18:02
@claude
Harden delete and orphan-clean paths against data loss
4872a6b 
Orphan cleanup:
- Refuse while Messages.app is running; snapshot chat.db before deleting rows
- Only delete the exact orphan set the user reviewed (scan fingerprint)
- Don't follow symlinks when walking attachment folders
- Treat a file as orphaned only if no referenced attachment shares its
  folder (keeps Live Photo videos, previews, etc.), compare paths
  case-insensitively, and skip files changed in the last 24 hours
- Never unlink a db-orphan file still referenced by a kept row
- Skip attachments the user kept via a messages-only delete

Delete:
- Backup gate now checks backend-side receipts of completed backups for the
  same filter, and requires attachments to have been copied when attachment
  files will be unlinked
- Require iCloud acknowledgement unless sync is detected as disabled; the
  frontend no longer auto-acknowledges
- Re-check Messages.app right before writing; safety_status treats an
  unknown state as running, matching run_delete
- Only unlink regular files inside the Messages attachment folders

Snapshots get a unique folder so back-to-back runs can't overwrite each
other. Remaining security/hardening items are tracked in SECURITY-TODO.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Fix date filtering, stale delete confirmations and other review issues
8d6ca3f 
- Date filter: validate full dates in local time (every YYYY-MM-DD was
  rejected east of UTC), and block preview/backup/delete on any date error
  instead of silently dropping the filter and targeting every message
- Make the end date inclusive of the whole end day
- Delete: require the preview to match the current filter, and have the
  backend refuse if its plan differs from the counts the user confirmed
- Keep attachments (and files) still referenced by messages or rows
  outside the delete plan
- Allow backup and snapshot destinations on external volumes; refuse
  ~/Library/Messages
- Store snapshots in the app data folder instead of ~/Documents (which may
  sync to iCloud Drive) and keep only the 5 most recent
- Detect missing Full Disk Access via the metadata error kind instead of
  exists(), which reports a TCC denial as "not found"
- Contacts: don't match group chat ids as phone numbers; match national
  vs international formats by unambiguous number suffix
- Retained attachments: roll back on a failed delete, prune stale ids, and
  let the orphan cleaner optionally include them

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Back up attachments safely and fix remaining review issues in the app
47c0cbc 
Backups:
- Copy attachments ourselves as attachments/<chat>/<rowid>-<name>; the
  library's copy named files by basename and silently skipped (while
  counting as copied) every later same-named file, e.g. all voice memos
- Count per-file outcomes (copied / not on this Mac / no file / failed)
  instead of aborting the whole export on one unreadable file, report
  them in the UI, and only let a backup with zero failures vouch for
  attachments in the delete gate
- JSON: run the export in reference mode and fill copiedTo/manifest
  counts from our copy pass
- HTML/TXT: let the runtime do its (rowid-named) copying once and verify
  what landed, instead of copying everything twice
- Write each run into a fresh timestamped subfolder; the JSON exporter
  appended to existing files, duplicating messages on re-runs

Delete:
- Require an explicit acknowledgement before deleting attachment records
  whose files exist only in iCloud
- Also delete tapbacks whose target message is deleted
- Correct the Messages in iCloud warning: direct chat.db edits aren't
  tracked by sync, so deletes may not propagate and may be re-downloaded

Other:
- Separate snapshot folders (and pruning) for deletes and orphan cleanups
- Writable connections wait for locks and use BEGIN IMMEDIATE
- Merge chat rows that are the same conversation (iMessage + SMS…) in the
  picker and filter by all of them
- Run SQLite / filesystem work off the async runtime
- Report Full Disk Access denials as such in every command

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Serialize destructive operations and fix remaining review issues
0e18a35 
- Refuse a backup, delete, orphan scan or orphan clean while another one
  is running; an orphan scan during a messages-only delete could drop the
  kept attachments from the retained list
- Only forget retained ids whose attachment record is gone (not ones that
  are linked right now)
- Reset the "not on this Mac" acknowledgement when the filter changes and
  the iCloud acknowledgement when the detected state changes; abort a
  delete if the iCloud state changed since the user acknowledged it
- Persist backup receipts across restarts; a receipt only counts while the
  backup folder still exists
- Remove the unused shell plugin and its capability
- Keep original modification dates on copied attachments
- Show how many tapbacks a delete adds beyond the filter
- Clear cache_has_attachments on messages left without attachments by an
  attachments-only delete
- Remove attachment folders emptied by a delete or cleanup
- Count folder-type attachments when verifying HTML/TXT exports
- Salvage a damaged retained-attachments list instead of blocking cleanups
  and messages-only deletes
- Warn that backups are unencrypted (more strongly for external drives and
  iCloud-synced folders), point out snapshots left in ~/Documents by
  earlier builds, and document restoring from a snapshot

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Tie backup coverage to exact content and keep original attachments
9a7bd3d 
- Backups record a fingerprint of the exact message and attachment ids
  their filter matched when the run started; a delete only counts as
  backed up if its filter matches the same content now. Before, a backup of
  an open-ended filter (no end date, or chats only) vouched for messages
  that arrived after it, across restarts
- HTML/TXT backups copy attachments byte-for-byte instead of re-encoding
  HEIC photos and stickers as JPEG, so deletes never leave only a lossy copy
- New backup_coverage command; the delete step uses it instead of the last
  in-session backup result, so backups from earlier sessions count and the
  UI explains when new messages have broken coverage

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Account for chat.db's real triggers; tighten backup coverage
bc859dc 
chat.db has delete triggers and cascading foreign keys the test fixtures
didn't model; the fixture now includes them and tests run through the
app's real writable connection.

- Remove the "messages only" delete: the triggers delete an attachment's
  record when its last message link goes, so kept files were left
  untracked and later wiped by the orphan cleaner. The backend refuses
  the scope; the UI always deletes attachments with their messages
- Correct the Messages in iCloud warning: the triggers queue every
  deletion in sync_deleted_messages/attachments, so deletes are expected
  to reach iCloud and other devices
- Backup receipts record which attachment files were actually copied; a
  file downloaded after the backup means it no longer covers attachments
- HTML/TXT copy check compares sizes with the source, so a partial copy
  counts as a failure
- Tapbacks a delete adds are part of the backup fingerprint and are saved
  to tapbacks-outside-filter.json with the backup
- Orphan clean only unlinks files (and un-retains ids) for rows it
  actually deleted, checked directly rather than via the change count
- Reject contact and attachment-type filters, which the library ignores
  and which would otherwise widen a delete to every message
- Snapshot pruning orders by write time (not local-time names, with
  numeric suffixes) and never removes the snapshot just taken

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Add end-to-end tests on Apple's real schema; harden the build
9b155a8 
End-to-end tests (src-tauri/src/e2e_tests.rs) run the real backup →
coverage → delete → orphan-cleanup code against a throwaway home whose
chat.db uses Apple's actual tables, triggers and cascading foreign keys
(tests/fixtures/chat-db-schema.sql). They cover a full year delete,
same-named attachments, shared attachments, tapbacks, coverage breaking on
new messages or later downloads, attachments-only and refused scopes,
orphan cleanup, and byte-for-byte HTML attachment copies.

To make that possible without touching real data:
- home_dir() with a per-thread test override replaces direct $HOME reads
- run_backup / run_delete / preview / coverage bodies are plain functions
  (backup_now, delete_now, …); Messages/iCloud checks are injectable
- backup receipts are shared as Arc state

Security:
- Remove disable-library-validation and allow-unsigned-executable-memory
  entitlements
- Set a strict Content Security Policy (and devCsp); verified against the
  production build in headless Chromium with no violations
- Pin imessage-database to an exact git commit; CI and Xcode Cloud no longer
  clone the fork's develop branch

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Fix issues from the final review pass
0c2ac38 
Backups and deletes:
- PDF backups no longer count toward the delete step's backup check: the
  PDF writer turns every non-Latin-1 character (emoji, smart quotes,
  Cyrillic, CJK…) into "?"
- HTML/TXT backups save tapbacks and poll votes whose target message is
  outside the filter (the exporter renders them nowhere) to
  tapbacks-without-target.json
- De-duplicate attachments the library lists once per 800-message chunk,
  so counts and sizes in the preview are right
- Prune snapshots only after a delete or cleanup commits, and remove the
  snapshot of an attempt that failed, so retries can't push out the real
  rollback

Orphan cleanup:
- Require an iCloud acknowledgement before removing records (chat.db
  queues them for iCloud deletion) and skip records created in the last 24h
- Normalise stored paths (// and . components, symlinked parents, and
  paths under a renamed home) before deciding a file is unreferenced
- Never treat a file any record points at as an orphan, including files
  directly in Attachments/ or StickerCache/
- File removal checks containment on the real location, so a symlinked
  folder inside the Messages folders can't redirect it elsewhere

UI:
- Typed "DELETE" confirmation, now actually entered by the user
- A hidden "delete without a backup" tick is never sent, and one-off
  acknowledgements reset after a backup or delete
- Refresh the preview after a refused delete instead of retrying stale
  counts; never show counts for a different filter
- Say which conversations a delete covers, including ones hidden by the
  search; warn when a delete ran without a verified backup
- Explain why PDF isn't a full backup; iCloud warning in the cleanup dialog

Website: correct the safety claims (snapshot location, attachment files
not in the snapshot, backup check, conversation filters).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Add manual macOS test checklist
d5e4a2b 
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Copy folder-type attachments whole in JSON/PDF backups
e90a84f 
Bundle-style attachments are folders. The app's copy pass used to report
them as failed copies, so a JSON or PDF backup that included one could
never satisfy the delete step's backup check. Copy them recursively
(skipping, never following, symlinks inside), check every file arrived,
keep original dates, and point copiedTo at the folder.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Check free space before backups and snapshots
90eff43 
Backups, delete snapshots and orphan-cleanup snapshots now refuse up
front, with the sizes in the message, when the target volume can't hold
them plus headroom (10%, at least 100 MB), instead of failing halfway and
leaving a partial backup or no rollback copy.

- Backup estimate: the real on-disk size of attachment files to copy plus
  twice the matched share of chat.db for the exported text (min 10 MB)
- Snapshot estimate: chat.db plus its write-ahead log
- Free-space lookup is injectable (SafetyProbe / SpaceProbe) so tests can
  simulate a full disk

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Resolve attachment roots wherever resolved paths are compared
7a94f37 
CI's first macOS run failed: the temp folder there is under /var, a
symlink to /private/var. Stored paths are now resolved before matching,
but functions given unresolved roots compared them against those roots
(and walked the unresolved tree), so every file looked unreferenced and
removals were refused.

The app always passed resolved roots, but that made correctness depend on
every caller. find_orphans and remove_attachment_file now resolve the
roots they're given (canonical_roots) and walk the resolved root, and
stored paths resolve their longest existing prefix rather than giving up
when the last folders don't exist exactly as spelled.

Reproduced on Linux by running the suite with TMPDIR pointing at a
symlink: 4 failures before, 0 after.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Add restore from snapshot
a72e54b 
The delete step now lists the snapshots taken before deletes, orphan
cleanups and restores, and can restore one, replacing the "swap chat.db by
hand" instructions.

- Only folders this app created inside its own snapshot folders can be
  restored; anything else (including arbitrary paths over IPC) is refused
- The snapshot must pass SQLite's integrity check and be a Messages
  database
- Messages must be quit; Messages in iCloud needs an acknowledgement (the
  confirm dialog names the risk)
- The current database is snapshotted first (space-checked), so a restore
  can itself be undone
- Contents are copied with SQLite's backup API into the live chat.db, which
  keeps its write-ahead log consistent, rather than swapping files
- Attachment files aren't in snapshots; the UI says to restore them from the
  backup

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@claude
Add encrypted backups
bd70c94 
Backups can now go into an AES-256 encrypted APFS sparse bundle that opens
in Finder with the user's password — no extra software.

- Created and mounted with macOS's hdiutil; the password goes over stdin
  (-stdinpass), never on the command line, and is redacted from Debug
- Mounted only while the backup is written: a guard unmounts it on every
  exit path, so a failed export never leaves it mounted and readable
- The image (not the temporary mount) is what the backup check records, so
  an encrypted backup counts for the delete step while it still exists
- Minimum 8-character password, confirmed in the UI; the page clears the
  password after each run and warns that a forgotten password can't be
  recovered
- hdiutil sits behind a small trait with a test stand-in, so the whole flow
  is tested end to end on Linux; the real hdiutil path is in the Mac
  checklist

Also: the typed-DELETE field and password fields use the page's form
styling.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GJnqoukR4qZMYyayKJqr36
@dbpecka dbpecka changed the title Harden backup, delete and orphan cleanup against data loss Harden backup/delete against data loss; add encrypted backups and snapshot restore Oct 23, 2026
@dbpecka
dbpecka marked this pull request as ready for review October 23, 2026 19:21
@dbpecka
dbpecka merged commit b326f33 into main Oct 26, 2026
2 checks passed
@dbpecka
dbpecka deleted the claude/blissful-dijkstra-5r6y4o branch October 28, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dbpecka @claude