Harden backup/delete against data loss; add encrypted backups and snapshot restore

.github/workflows/ci.yml

@@ -17,20 +17,8 @@ jobs:
         with:
           submodules: false
 
-      # imessage-database is a sibling path dep (../../imessage-exporter).
-      # We check out the fork into that expected layout so cargo resolves
-      # the dep the same way it does locally.
-      - name: Check out imessage-exporter fork
-        uses: actions/checkout@v4
-        with:
-          repository: dbpecka/imessage-exporter
-          ref: develop
-          path: imessage-exporter-checkout
-
-      - name: Restage sibling layout
-        run: |
-          mv imessage-exporter-checkout ../imessage-exporter
-          ls -la ../imessage-exporter
+      # imessage-database is a git dependency pinned to an exact commit in
+      # src-tauri/Cargo.toml; cargo fetches it, no separate checkout needed.
 
       - uses: dtolnay/rust-toolchain@stable
         with:

README.md

@@ -34,10 +34,10 @@ Tauri IPC bridge
 Rust backend (src-tauri/)
   ├── commands/                 — fda, discover, preview, backup, delete, orphans, safety
   ├── core/                     — db_path, filter, contacts, icloud, snapshot, progress
-  └── imessage-database         — path dep: ../imessage-exporter (fork)
+  └── imessage-database         — git dep: dbpecka/imessage-exporter, pinned commit
 ```
 
-The Rust backend delegates all iMessage parsing and export logic to a local fork of [imessage-exporter](https://github.com/ReagentX/imessage-exporter) (`../imessage-exporter`), built with two custom feature flags:
+The Rust backend delegates all iMessage parsing and export logic to a fork of [imessage-exporter](https://github.com/ReagentX/imessage-exporter) ([dbpecka/imessage-exporter](https://github.com/dbpecka/imessage-exporter), pinned to an exact commit in `src-tauri/Cargo.toml`), built with two custom feature flags:
 
 - `export` — enables the HTML/TXT/JSON/PDF export runtime
 - `write` — enables destructive SQL operations (two-phase delete planning and execution)
@@ -52,11 +52,11 @@ The single-page UI walks through five numbered cards in sequence:
 
 3. **Preview** — live preview (400 ms debounce) of how many messages and attachments match the current filters, with breakdown of on-disk vs. not-on-this-Mac (iCloud metadata without local content) vs. missing.
 
-4. **Backup** — format selector (JSON / HTML / PDF / TXT), folder picker, attachments toggle. Progress is driven by real-time events from the Rust side. Result shows message count, conversation count, attachments copied, total bytes, and manifest path.
+4. **Backup** — format selector (JSON / HTML / PDF / TXT), folder picker (home folder or an external drive), attachments toggle. Each run writes into a new `Bubble Wrap <timestamp>` subfolder; copied attachments are named `<rowid>-<original name>` so same-named files (e.g. every `Audio Message.caf`) never collide. Progress is driven by real-time events from the Rust side. Result shows message count, conversation count, per-attachment copy outcomes (copied / not on this Mac / failed), and the output path. Backups can be encrypted (an AES-256 `.sparsebundle` opened in Finder with a password), and the app checks there's enough free space before starting. PDF exports don't count as a backup for the delete step: the PDF writer can only store Latin-1 text, so emoji, smart quotes and non-Latin scripts become `?`.
 
-5. **Delete** (opt-in, behind a toggle) — shows Messages.app running status and iCloud sync warning. Scope selector: delete messages + attachments, messages only, or attachments only. Requires typed confirmation before executing.
+5. **Delete** (opt-in, behind a toggle) — shows Messages.app running status and a Messages in iCloud warning (chat.db's own triggers queue every deletion for iCloud sync, so with Messages in iCloud on, deletes are expected to reach your other devices). Scope: delete messages together with their attachments, or attachments only — deleting messages while keeping their attachments isn't offered, because chat.db's triggers delete an attachment's record along with its last message. Requires typed confirmation before executing. Snapshots taken before deletes and cleanups can be restored from the same step (the current database is snapshotted first, so a restore can be undone too).
 
-A **File > Clean Orphaned Data…** menu item opens a modal that scans for and removes two categories of orphaned data: attachment DB rows with no linked message, and files in `~/Library/Messages/Attachments` and `StickerCache` with no DB record.
+A **File > Clean Orphaned Data…** menu item opens a modal that scans for and removes two categories of orphaned data: attachment DB rows with no linked message, and files in `~/Library/Messages/Attachments` and `StickerCache` with no DB record. Records and files created in the last 24 hours are skipped, and removing records needs an iCloud acknowledgement unless Messages in iCloud is detected as off (chat.db queues those deletions for iCloud).
 
 ### Delete safety contract
 
@@ -67,7 +67,7 @@ A **File > Clean Orphaned Data…** menu item opens a modal that scans for and r
 3. `pgrep -x Messages` must return empty (Messages.app must be closed)
 4. If Messages-in-iCloud is detected as **Enabled**, `acknowledge_icloud_sync=true` is required
 5. `PRAGMA wal_checkpoint(TRUNCATE)` flushes the WAL on the source DB
-6. Atomic snapshot of `chat.db` via SQLite's online backup API written to `~/Documents/iMessage Backups/snapshots/<ISO-timestamp>/`
+6. Atomic snapshot of `chat.db` via SQLite's online backup API written to `~/Library/Application Support/io.pecka.apps.bubblewrap/snapshots/delete/<ISO-timestamp>/` (orphan cleanups use `snapshots/orphan-clean/`; the 5 most recent of each are kept)
 7. Delete plan built on a read-only connection, then applied on a separate writable connection
 8. Writable connection released immediately after commit
 9. Attachment files unlinked after the SQL transaction; failures are logged to stderr with path + reason but are non-fatal
@@ -96,7 +96,7 @@ iCloud Messages sync state is detected via `defaults read com.apple.Messages ICl
 
 - Rust (stable)
 - Node.js + npm
-- The `imessage-exporter` fork checked out at `../imessage-exporter` on the `develop` branch
+- Nothing else: cargo fetches the pinned `imessage-exporter` fork itself. To work on the fork locally, see the `[patch]` note in `src-tauri/Cargo.toml`.
 
 ### Run in development
 
@@ -119,6 +119,17 @@ cargo test --manifest-path src-tauri/Cargo.toml --lib
 npm run check
 ```
 
+The suite includes end-to-end tests (`src-tauri/src/e2e_tests.rs`) that run
+backup → delete → orphan cleanup against a throwaway home folder whose
+`chat.db` uses Apple's real schema and triggers
+(`src-tauri/tests/fixtures/chat-db-schema.sql`). They never touch your real
+Messages data.
+
+```sh
+# just the end-to-end tests
+cargo test --manifest-path src-tauri/Cargo.toml --lib e2e
+```
+
 ### Build
 
 ```sh
@@ -158,8 +169,25 @@ src-tauri/
       progress.rs          — TauriProgress: ProgressReporter → Tauri events
       snapshot.rs          — WAL checkpoint + timestamped file copy
 
-../imessage-exporter/      — sibling fork (path dependency)
+dbpecka/imessage-exporter  — the fork (git dependency, pinned)
   imessage-database/
     src/exporters/         — html, txt, json, pdf export engines
     src/tables/write/      — delete.rs (two-phase plan + execute), orphans.rs
 ```
+
+---
+
+## Restoring from a snapshot
+
+Every delete and orphan cleanup first saves a copy of `chat.db` to
+`~/Library/Application Support/io.pecka.apps.bubblewrap/snapshots/<delete|orphan-clean>/<timestamp>/`.
+To roll back:
+
+1. Quit Messages (and make sure it stays quit).
+2. Replace `~/Library/Messages/chat.db` with the snapshot's `chat.db`.
+3. Delete `chat.db-wal` and `chat.db-shm` next to it in `~/Library/Messages/`.
+4. Reopen Messages.
+
+Snapshots only contain the database. Attachment files removed by a delete
+are not in the snapshot — restore them from your backup's `attachments/`
+folder. Backups and snapshots are not encrypted.

SECURITY-TODO.md

@@ -0,0 +1,84 @@
+# Security TODO
+
+Security / hardening items found during code review, with their status.
+Data-loss bugs from the same reviews have been fixed in code.
+
+## 1. Done: risky hardened-runtime entitlements removed
+
+`com.apple.security.cs.disable-library-validation` and
+`com.apple.security.cs.allow-unsigned-executable-memory` are gone from
+`src-tauri/entitlements.plist`. With Full Disk Access granted, library
+validation off would have let any dylib the process could be induced to load
+run with the app's TCC grants.
+
+**Still to verify on a Mac:** a signed, notarized build launches, exports
+and deletes normally. If something fails, re-add only the specific
+entitlement it needs and document why here.
+
+## 2. Done: Content Security Policy set
+
+`src-tauri/tauri.conf.json` now sets `app.security.csp` (and `devCsp`, which
+also allows the Vite dev server's websocket): scripts only from the app
+itself (Tauri adds hashes for SvelteKit's inline bootstrap script), no
+plugins, frames, forms or `<base>` tags, and network access limited to
+Tauri's IPC. `style-src` allows inline styles because the UI uses `style`
+attributes; `dangerousDisableAssetCspModification: ["style-src"]` stops
+Tauri adding a nonce there, which would otherwise make browsers ignore
+`'unsafe-inline'`.
+
+Checked by serving the production build in headless Chromium with the same
+policy: the page renders with no violations, and the same harness does
+report a violation when the inline-script hash is left out.
+
+**Still to verify on a Mac:** `npm run tauri dev` and a release build show
+no CSP violations in the WebKit inspector.
+
+## 3. Done: `imessage-database` pinned
+
+`src-tauri/Cargo.toml` now depends on the fork by git URL at an exact commit
+(`rev = "cffac6a…"`), and `Cargo.lock` records it. CI and the Xcode Cloud
+script no longer clone the fork's `develop` (or derive the owner from the
+origin URL); cargo fetches the pinned commit. Bump `rev` deliberately after
+reviewing the fork's changes. For local work on the fork, use the `[patch]`
+override described in `Cargo.toml`.
+
+## 4. Don't publish the whole `public/` directory
+
+**File:** `public/docker-compose.yml`
+
+nginx mounts `.` (all of `public/`) as the web root, which publishes
+`public/.idea/` and `docker-compose.yml` itself.
+
+**Fix:** move site assets into a dedicated subfolder (e.g. `public/site/`)
+and mount only that, or add an nginx `location ~ /\. { deny all; }` rule and
+remove `public/.idea/` from the repo.
+
+## 5. Repo hygiene
+
+Committed files that shouldn't be: `errors.txt`, `.idea/`, `public/.idea/`,
+`BubbleWrap.xcodeproj/project.xcworkspace/xcuserdata/`. Remove them and add
+to `.gitignore` so local paths/usernames and dev logs don't leak.
+
+## 6. Backups: encryption available; snapshots still unencrypted
+
+Backups and `chat.db` snapshots are full copies of the user's message
+history.
+
+**Done:** backups can be encrypted — an AES-256 encrypted APFS sparse bundle
+created with `hdiutil` (password passed on stdin, never as an argument;
+redacted from `Debug` output), mounted only while the backup is written and
+unmounted on every exit path. Unencrypted backups say so, with a stronger
+warning for `/Volumes/…`, iCloud Drive, and Desktop/Documents destinations.
+Snapshots live in `~/Library/Application Support/<app>/` (not synced), five
+kept per kind.
+
+**Still to do:** snapshots are not encrypted (they're needed for restore
+without a password prompt); consider encrypting them at rest or expiring
+them after a configurable period. Verify the real `hdiutil` flow on a Mac
+(`docs/MAC-TEST-CHECKLIST.md`, section 4) — tests use a stand-in.
+
+## 7. Removed: unused shell plugin
+
+`tauri-plugin-shell` and the `shell:allow-open` capability were registered
+but never used by the frontend; both have been removed to shrink the attack
+surface. Don't re-add them without a concrete need and a scoped permission.

ci_scripts/ci_post_clone.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-# Xcode Cloud post-clone hook — installs toolchain, fetches sibling dep, builds Tauri.
+# Xcode Cloud post-clone hook — installs toolchain, builds Tauri.
 set -euo pipefail
 
 REPO_DIR="/Volumes/workspace/repository"
@@ -21,18 +21,9 @@ if ! command -v node &>/dev/null; then
     brew install node
 fi
 
-# --- Sibling repo: imessage-exporter fork ---
-# Cargo.toml has a path dep: path = "../../imessage-exporter/imessage-database"
-# Relative to src-tauri/ that resolves to /Volumes/workspace/imessage-exporter/.
-EXPORTER_DIR="$WORKSPACE_DIR/imessage-exporter"
-if [ ! -d "$EXPORTER_DIR" ]; then
-    # Derive the GitHub user from this repo's origin URL (works for both HTTPS and SSH remotes).
-    ORIGIN=$(git -C "$REPO_DIR" remote get-url origin)
-    GITHUB_USER=$(echo "$ORIGIN" | sed -E 's|.*github\.com[:/]([^/]+)/.*|\1|')
-    git clone --depth 1 --branch develop \
-        "https://github.com/$GITHUB_USER/imessage-exporter.git" \
-        "$EXPORTER_DIR"
-fi
+# --- imessage-exporter fork ---
+# A git dependency pinned to an exact commit in src-tauri/Cargo.toml; cargo
+# fetches it from GitHub during the build, so no separate clone is needed.
 
 # --- Build ---
 cd "$REPO_DIR"

docs/MAC-TEST-CHECKLIST.md

@@ -0,0 +1,131 @@
+# Manual test checklist (macOS)
+
+Things the automated tests can't cover: they run on Linux against a
+throwaway `chat.db`, so anything involving macOS permissions, code signing,
+WebKit, real Messages data or iCloud needs a person and a Mac.
+
+**Use test data.** Run this on a spare Mac user account (or a VM) signed in
+to a **throwaway Apple ID**, with a second device on the same Apple ID for
+the iCloud steps. Never run the delete steps against messages you care
+about.
+
+Record for each run: macOS version, Apple Silicon or Intel, build (dev,
+signed, notarized), and the result of each step.
+
+---
+
+## 0. Setup
+
+- [ ] Send yourself a mix of messages from the second device: plain text,
+      emoji and smart quotes (`it’s`), a non-Latin script (e.g. `Привет`),
+      two voice memos, a HEIC photo, a Live Photo, a sticker, a tapback on an
+      older message, and a group chat.
+- [ ] Note the total message count shown by Messages for one conversation.
+- [ ] Quit Messages.
+
+## 1. Full Disk Access
+
+- [ ] First launch **without** Full Disk Access: the app shows the Full Disk
+      Access guide (not "chat.db not found").
+- [ ] "Open System Settings" lands on Privacy & Security → Full Disk Access,
+      with Bubble Wrap already in the list.
+- [ ] Turn it on and return to the app: it asks to relaunch, relaunches, and
+      shows the conversation list with contact names.
+- [ ] Revoke access while the app is open, then try a preview: the error is
+      understandable (known gap: it doesn't route back to the guide).
+
+## 2. Signed build (entitlements removed)
+
+- [ ] Build, sign and notarize a release (`npm run tauri build`, then your
+      usual notarization). `spctl -a -vv` accepts it.
+- [ ] `codesign -d --entitlements - "Bubble Wrap.app"` shows **no**
+      `disable-library-validation` and **no**
+      `allow-unsigned-executable-memory`.
+- [ ] The signed build launches, lists conversations, previews, backs up
+      (JSON, HTML, PDF, TXT) and deletes. If anything fails, note the
+      crash log — re-add only the specific entitlement it needs.
+
+## 3. Content Security Policy
+
+- [ ] `npm run tauri dev`: open the WebKit inspector (right-click → Inspect
+      Element) → Console. No "Refused to…" CSP messages while you click
+      through every step, including the orphan-cleanup dialog.
+- [ ] Same check in the signed release build (enable the inspector with
+      `defaults write io.pecka.apps.bubblewrap WebKitDeveloperExtras -bool true`
+      if needed).
+
+## 4. Backups
+
+For each format (JSON, HTML, TXT; PDF last):
+
+- [ ] Back up one conversation with "Copy attachment files" on, to a folder
+      in your home folder. A new `Bubble Wrap <date>` subfolder appears.
+- [ ] Both voice memos are present and play, with different contents.
+- [ ] The HEIC photo and Live Photo are present byte-for-byte (compare with
+      `shasum` against the files under `~/Library/Messages/Attachments`).
+- [ ] Copied attachments keep their original dates in Finder.
+- [ ] HTML: the conversation renders; note whether HEIC images display
+      inline (known: browsers may not show HEIC).
+- [ ] PDF: emoji/non-Latin text shows as `?`, and the result says PDF
+      doesn't count as a backup for deleting.
+- [ ] Back up the same conversation to an **external drive**: accepted, and
+      the UI warns that the drive isn't protected.
+- [ ] Back up into a folder inside `~/Library/Messages`: refused.
+- [ ] Encrypted backup: tick "Encrypt", set a password, back up. A
+      `.sparsebundle` appears; double-clicking it asks for the password and
+      the backup is inside. Without the password it can't be opened.
+- [ ] Not enough space: back up to a nearly full USB stick; the app refuses
+      up front with a clear message instead of failing halfway.
+
+## 5. Delete
+
+- [ ] With no backup for the filter, Run delete is disabled until you tick
+      "delete without a backup" **and** type `DELETE`.
+- [ ] Back up (JSON), then preview a delete of one conversation for a past
+      year. The step shows "Backed up in …" and names the conversation.
+- [ ] Send a new message into that conversation (if the filter has no end
+      date), then click Run delete: it's refused, the preview refreshes, and
+      the new count is shown.
+- [ ] Delete. Messages for that range are gone in Messages.app after
+      relaunch; attachment files are gone from `~/Library/Messages/Attachments`;
+      the result shows the snapshot path.
+- [ ] Reopen Messages *during* a delete's snapshot step (large database
+      helps): the delete is refused and no extra snapshot is left behind.
+- [ ] "Attachments only" delete: the messages stay, the attachments are
+      gone, and Messages shows the messages without broken bubbles (or note
+      what it shows).
+
+## 6. Restore from snapshot
+
+- [ ] After a delete, open "Restore from snapshot", pick the snapshot from
+      that delete and restore (Messages quit). Relaunch Messages: the
+      deleted messages are back.
+- [ ] A pre-restore snapshot of the state just before the restore exists.
+- [ ] Restoring while Messages is running is refused.
+
+## 7. Messages in iCloud (two devices, throwaway Apple ID)
+
+This is the one behaviour the app can only infer from chat.db's triggers.
+
+- [ ] With Messages in iCloud **on**, delete a few old messages on the Mac.
+      Open Messages, wait for it to sync, and check the second device:
+      **are the messages gone there too?** Record the answer — the app's
+      warnings currently say deletions are "expected to" sync.
+- [ ] Do they ever come back on the Mac after a sync?
+- [ ] Orphan cleanup with iCloud on asks for the iCloud acknowledgement
+      before removing records.
+
+## 8. Orphan cleanup
+
+- [ ] File → Clean Orphaned Data… with Messages quit: the scan finishes, the
+      counts look plausible, and Live Photo videos next to their photos are
+      *not* listed.
+- [ ] Clean: the listed items are removed, nothing else in Messages breaks
+      (spot-check a few conversations with attachments).
+- [ ] With Messages running, cleaning is refused.
+
+## 9. Large real database
+
+- [ ] On a Mac with years of real history (read-only steps only: preview,
+      backup, orphan **scan**): note how long the delete preview takes to
+      update after changing the filter, and how long the orphan scan takes.