diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 554457b..93c2bfd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,20 +17,8 @@ jobs: with: submodules: false - # imessage-database is a sibling path dep (../../imessage-exporter). - # We check out the fork into that expected layout so cargo resolves - # the dep the same way it does locally. - - name: Check out imessage-exporter fork - uses: actions/checkout@v4 - with: - repository: dbpecka/imessage-exporter - ref: develop - path: imessage-exporter-checkout - - - name: Restage sibling layout - run: | - mv imessage-exporter-checkout ../imessage-exporter - ls -la ../imessage-exporter + # imessage-database is a git dependency pinned to an exact commit in + # src-tauri/Cargo.toml; cargo fetches it, no separate checkout needed. - uses: dtolnay/rust-toolchain@stable with: diff --git a/README.md b/README.md index 0a568ca..668bde9 100644 --- a/README.md +++ b/README.md @@ -34,10 +34,10 @@ Tauri IPC bridge Rust backend (src-tauri/) ├── commands/ — fda, discover, preview, backup, delete, orphans, safety ├── core/ — db_path, filter, contacts, icloud, snapshot, progress - └── imessage-database — path dep: ../imessage-exporter (fork) + └── imessage-database — git dep: dbpecka/imessage-exporter, pinned commit ``` -The Rust backend delegates all iMessage parsing and export logic to a local fork of [imessage-exporter](https://github.com/ReagentX/imessage-exporter) (`../imessage-exporter`), built with two custom feature flags: +The Rust backend delegates all iMessage parsing and export logic to a fork of [imessage-exporter](https://github.com/ReagentX/imessage-exporter) ([dbpecka/imessage-exporter](https://github.com/dbpecka/imessage-exporter), pinned to an exact commit in `src-tauri/Cargo.toml`), built with two custom feature flags: - `export` — enables the HTML/TXT/JSON/PDF export runtime - `write` — enables destructive SQL operations (two-phase delete planning and execution) @@ -52,11 +52,11 @@ The single-page UI walks through five numbered cards in sequence: 3. **Preview** — live preview (400 ms debounce) of how many messages and attachments match the current filters, with breakdown of on-disk vs. not-on-this-Mac (iCloud metadata without local content) vs. missing. -4. **Backup** — format selector (JSON / HTML / PDF / TXT), folder picker, attachments toggle. Progress is driven by real-time events from the Rust side. Result shows message count, conversation count, attachments copied, total bytes, and manifest path. +4. **Backup** — format selector (JSON / HTML / PDF / TXT), folder picker (home folder or an external drive), attachments toggle. Each run writes into a new `Bubble Wrap ` subfolder; copied attachments are named `-` so same-named files (e.g. every `Audio Message.caf`) never collide. Progress is driven by real-time events from the Rust side. Result shows message count, conversation count, per-attachment copy outcomes (copied / not on this Mac / failed), and the output path. Backups can be encrypted (an AES-256 `.sparsebundle` opened in Finder with a password), and the app checks there's enough free space before starting. PDF exports don't count as a backup for the delete step: the PDF writer can only store Latin-1 text, so emoji, smart quotes and non-Latin scripts become `?`. -5. **Delete** (opt-in, behind a toggle) — shows Messages.app running status and iCloud sync warning. Scope selector: delete messages + attachments, messages only, or attachments only. Requires typed confirmation before executing. +5. **Delete** (opt-in, behind a toggle) — shows Messages.app running status and a Messages in iCloud warning (chat.db's own triggers queue every deletion for iCloud sync, so with Messages in iCloud on, deletes are expected to reach your other devices). Scope: delete messages together with their attachments, or attachments only — deleting messages while keeping their attachments isn't offered, because chat.db's triggers delete an attachment's record along with its last message. Requires typed confirmation before executing. Snapshots taken before deletes and cleanups can be restored from the same step (the current database is snapshotted first, so a restore can be undone too). -A **File > Clean Orphaned Data…** menu item opens a modal that scans for and removes two categories of orphaned data: attachment DB rows with no linked message, and files in `~/Library/Messages/Attachments` and `StickerCache` with no DB record. +A **File > Clean Orphaned Data…** menu item opens a modal that scans for and removes two categories of orphaned data: attachment DB rows with no linked message, and files in `~/Library/Messages/Attachments` and `StickerCache` with no DB record. Records and files created in the last 24 hours are skipped, and removing records needs an iCloud acknowledgement unless Messages in iCloud is detected as off (chat.db queues those deletions for iCloud). ### Delete safety contract @@ -67,7 +67,7 @@ A **File > Clean Orphaned Data…** menu item opens a modal that scans for and r 3. `pgrep -x Messages` must return empty (Messages.app must be closed) 4. If Messages-in-iCloud is detected as **Enabled**, `acknowledge_icloud_sync=true` is required 5. `PRAGMA wal_checkpoint(TRUNCATE)` flushes the WAL on the source DB -6. Atomic snapshot of `chat.db` via SQLite's online backup API written to `~/Documents/iMessage Backups/snapshots//` +6. Atomic snapshot of `chat.db` via SQLite's online backup API written to `~/Library/Application Support/io.pecka.apps.bubblewrap/snapshots/delete//` (orphan cleanups use `snapshots/orphan-clean/`; the 5 most recent of each are kept) 7. Delete plan built on a read-only connection, then applied on a separate writable connection 8. Writable connection released immediately after commit 9. Attachment files unlinked after the SQL transaction; failures are logged to stderr with path + reason but are non-fatal @@ -96,7 +96,7 @@ iCloud Messages sync state is detected via `defaults read com.apple.Messages ICl - Rust (stable) - Node.js + npm -- The `imessage-exporter` fork checked out at `../imessage-exporter` on the `develop` branch +- Nothing else: cargo fetches the pinned `imessage-exporter` fork itself. To work on the fork locally, see the `[patch]` note in `src-tauri/Cargo.toml`. ### Run in development @@ -119,6 +119,17 @@ cargo test --manifest-path src-tauri/Cargo.toml --lib npm run check ``` +The suite includes end-to-end tests (`src-tauri/src/e2e_tests.rs`) that run +backup → delete → orphan cleanup against a throwaway home folder whose +`chat.db` uses Apple's real schema and triggers +(`src-tauri/tests/fixtures/chat-db-schema.sql`). They never touch your real +Messages data. + +```sh +# just the end-to-end tests +cargo test --manifest-path src-tauri/Cargo.toml --lib e2e +``` + ### Build ```sh @@ -158,8 +169,25 @@ src-tauri/ progress.rs — TauriProgress: ProgressReporter → Tauri events snapshot.rs — WAL checkpoint + timestamped file copy -../imessage-exporter/ — sibling fork (path dependency) +dbpecka/imessage-exporter — the fork (git dependency, pinned) imessage-database/ src/exporters/ — html, txt, json, pdf export engines src/tables/write/ — delete.rs (two-phase plan + execute), orphans.rs ``` + +--- + +## Restoring from a snapshot + +Every delete and orphan cleanup first saves a copy of `chat.db` to +`~/Library/Application Support/io.pecka.apps.bubblewrap/snapshots///`. +To roll back: + +1. Quit Messages (and make sure it stays quit). +2. Replace `~/Library/Messages/chat.db` with the snapshot's `chat.db`. +3. Delete `chat.db-wal` and `chat.db-shm` next to it in `~/Library/Messages/`. +4. Reopen Messages. + +Snapshots only contain the database. Attachment files removed by a delete +are not in the snapshot — restore them from your backup's `attachments/` +folder. Backups and snapshots are not encrypted. diff --git a/SECURITY-TODO.md b/SECURITY-TODO.md new file mode 100644 index 0000000..65b794f --- /dev/null +++ b/SECURITY-TODO.md @@ -0,0 +1,84 @@ +# Security TODO + +Security / hardening items found during code review, with their status. +Data-loss bugs from the same reviews have been fixed in code. + +## 1. Done: risky hardened-runtime entitlements removed + +`com.apple.security.cs.disable-library-validation` and +`com.apple.security.cs.allow-unsigned-executable-memory` are gone from +`src-tauri/entitlements.plist`. With Full Disk Access granted, library +validation off would have let any dylib the process could be induced to load +run with the app's TCC grants. + +**Still to verify on a Mac:** a signed, notarized build launches, exports +and deletes normally. If something fails, re-add only the specific +entitlement it needs and document why here. + +## 2. Done: Content Security Policy set + +`src-tauri/tauri.conf.json` now sets `app.security.csp` (and `devCsp`, which +also allows the Vite dev server's websocket): scripts only from the app +itself (Tauri adds hashes for SvelteKit's inline bootstrap script), no +plugins, frames, forms or `` tags, and network access limited to +Tauri's IPC. `style-src` allows inline styles because the UI uses `style` +attributes; `dangerousDisableAssetCspModification: ["style-src"]` stops +Tauri adding a nonce there, which would otherwise make browsers ignore +`'unsafe-inline'`. + +Checked by serving the production build in headless Chromium with the same +policy: the page renders with no violations, and the same harness does +report a violation when the inline-script hash is left out. + +**Still to verify on a Mac:** `npm run tauri dev` and a release build show +no CSP violations in the WebKit inspector. + +## 3. Done: `imessage-database` pinned + +`src-tauri/Cargo.toml` now depends on the fork by git URL at an exact commit +(`rev = "cffac6a…"`), and `Cargo.lock` records it. CI and the Xcode Cloud +script no longer clone the fork's `develop` (or derive the owner from the +origin URL); cargo fetches the pinned commit. Bump `rev` deliberately after +reviewing the fork's changes. For local work on the fork, use the `[patch]` +override described in `Cargo.toml`. + +## 4. Don't publish the whole `public/` directory + +**File:** `public/docker-compose.yml` + +nginx mounts `.` (all of `public/`) as the web root, which publishes +`public/.idea/` and `docker-compose.yml` itself. + +**Fix:** move site assets into a dedicated subfolder (e.g. `public/site/`) +and mount only that, or add an nginx `location ~ /\. { deny all; }` rule and +remove `public/.idea/` from the repo. + +## 5. Repo hygiene + +Committed files that shouldn't be: `errors.txt`, `.idea/`, `public/.idea/`, +`BubbleWrap.xcodeproj/project.xcworkspace/xcuserdata/`. Remove them and add +to `.gitignore` so local paths/usernames and dev logs don't leak. + +## 6. Backups: encryption available; snapshots still unencrypted + +Backups and `chat.db` snapshots are full copies of the user's message +history. + +**Done:** backups can be encrypted — an AES-256 encrypted APFS sparse bundle +created with `hdiutil` (password passed on stdin, never as an argument; +redacted from `Debug` output), mounted only while the backup is written and +unmounted on every exit path. Unencrypted backups say so, with a stronger +warning for `/Volumes/…`, iCloud Drive, and Desktop/Documents destinations. +Snapshots live in `~/Library/Application Support//` (not synced), five +kept per kind. + +**Still to do:** snapshots are not encrypted (they're needed for restore +without a password prompt); consider encrypting them at rest or expiring +them after a configurable period. Verify the real `hdiutil` flow on a Mac +(`docs/MAC-TEST-CHECKLIST.md`, section 4) — tests use a stand-in. + +## 7. Removed: unused shell plugin + +`tauri-plugin-shell` and the `shell:allow-open` capability were registered +but never used by the frontend; both have been removed to shrink the attack +surface. Don't re-add them without a concrete need and a scoped permission. diff --git a/ci_scripts/ci_post_clone.sh b/ci_scripts/ci_post_clone.sh index de870c3..3b61260 100755 --- a/ci_scripts/ci_post_clone.sh +++ b/ci_scripts/ci_post_clone.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Xcode Cloud post-clone hook — installs toolchain, fetches sibling dep, builds Tauri. +# Xcode Cloud post-clone hook — installs toolchain, builds Tauri. set -euo pipefail REPO_DIR="/Volumes/workspace/repository" @@ -21,18 +21,9 @@ if ! command -v node &>/dev/null; then brew install node fi -# --- Sibling repo: imessage-exporter fork --- -# Cargo.toml has a path dep: path = "../../imessage-exporter/imessage-database" -# Relative to src-tauri/ that resolves to /Volumes/workspace/imessage-exporter/. -EXPORTER_DIR="$WORKSPACE_DIR/imessage-exporter" -if [ ! -d "$EXPORTER_DIR" ]; then - # Derive the GitHub user from this repo's origin URL (works for both HTTPS and SSH remotes). - ORIGIN=$(git -C "$REPO_DIR" remote get-url origin) - GITHUB_USER=$(echo "$ORIGIN" | sed -E 's|.*github\.com[:/]([^/]+)/.*|\1|') - git clone --depth 1 --branch develop \ - "https://github.com/$GITHUB_USER/imessage-exporter.git" \ - "$EXPORTER_DIR" -fi +# --- imessage-exporter fork --- +# A git dependency pinned to an exact commit in src-tauri/Cargo.toml; cargo +# fetches it from GitHub during the build, so no separate clone is needed. # --- Build --- cd "$REPO_DIR" diff --git a/docs/MAC-TEST-CHECKLIST.md b/docs/MAC-TEST-CHECKLIST.md new file mode 100644 index 0000000..1cc6687 --- /dev/null +++ b/docs/MAC-TEST-CHECKLIST.md @@ -0,0 +1,131 @@ +# Manual test checklist (macOS) + +Things the automated tests can't cover: they run on Linux against a +throwaway `chat.db`, so anything involving macOS permissions, code signing, +WebKit, real Messages data or iCloud needs a person and a Mac. + +**Use test data.** Run this on a spare Mac user account (or a VM) signed in +to a **throwaway Apple ID**, with a second device on the same Apple ID for +the iCloud steps. Never run the delete steps against messages you care +about. + +Record for each run: macOS version, Apple Silicon or Intel, build (dev, +signed, notarized), and the result of each step. + +--- + +## 0. Setup + +- [ ] Send yourself a mix of messages from the second device: plain text, + emoji and smart quotes (`it’s`), a non-Latin script (e.g. `Привет`), + two voice memos, a HEIC photo, a Live Photo, a sticker, a tapback on an + older message, and a group chat. +- [ ] Note the total message count shown by Messages for one conversation. +- [ ] Quit Messages. + +## 1. Full Disk Access + +- [ ] First launch **without** Full Disk Access: the app shows the Full Disk + Access guide (not "chat.db not found"). +- [ ] "Open System Settings" lands on Privacy & Security → Full Disk Access, + with Bubble Wrap already in the list. +- [ ] Turn it on and return to the app: it asks to relaunch, relaunches, and + shows the conversation list with contact names. +- [ ] Revoke access while the app is open, then try a preview: the error is + understandable (known gap: it doesn't route back to the guide). + +## 2. Signed build (entitlements removed) + +- [ ] Build, sign and notarize a release (`npm run tauri build`, then your + usual notarization). `spctl -a -vv` accepts it. +- [ ] `codesign -d --entitlements - "Bubble Wrap.app"` shows **no** + `disable-library-validation` and **no** + `allow-unsigned-executable-memory`. +- [ ] The signed build launches, lists conversations, previews, backs up + (JSON, HTML, PDF, TXT) and deletes. If anything fails, note the + crash log — re-add only the specific entitlement it needs. + +## 3. Content Security Policy + +- [ ] `npm run tauri dev`: open the WebKit inspector (right-click → Inspect + Element) → Console. No "Refused to…" CSP messages while you click + through every step, including the orphan-cleanup dialog. +- [ ] Same check in the signed release build (enable the inspector with + `defaults write io.pecka.apps.bubblewrap WebKitDeveloperExtras -bool true` + if needed). + +## 4. Backups + +For each format (JSON, HTML, TXT; PDF last): + +- [ ] Back up one conversation with "Copy attachment files" on, to a folder + in your home folder. A new `Bubble Wrap ` subfolder appears. +- [ ] Both voice memos are present and play, with different contents. +- [ ] The HEIC photo and Live Photo are present byte-for-byte (compare with + `shasum` against the files under `~/Library/Messages/Attachments`). +- [ ] Copied attachments keep their original dates in Finder. +- [ ] HTML: the conversation renders; note whether HEIC images display + inline (known: browsers may not show HEIC). +- [ ] PDF: emoji/non-Latin text shows as `?`, and the result says PDF + doesn't count as a backup for deleting. +- [ ] Back up the same conversation to an **external drive**: accepted, and + the UI warns that the drive isn't protected. +- [ ] Back up into a folder inside `~/Library/Messages`: refused. +- [ ] Encrypted backup: tick "Encrypt", set a password, back up. A + `.sparsebundle` appears; double-clicking it asks for the password and + the backup is inside. Without the password it can't be opened. +- [ ] Not enough space: back up to a nearly full USB stick; the app refuses + up front with a clear message instead of failing halfway. + +## 5. Delete + +- [ ] With no backup for the filter, Run delete is disabled until you tick + "delete without a backup" **and** type `DELETE`. +- [ ] Back up (JSON), then preview a delete of one conversation for a past + year. The step shows "Backed up in …" and names the conversation. +- [ ] Send a new message into that conversation (if the filter has no end + date), then click Run delete: it's refused, the preview refreshes, and + the new count is shown. +- [ ] Delete. Messages for that range are gone in Messages.app after + relaunch; attachment files are gone from `~/Library/Messages/Attachments`; + the result shows the snapshot path. +- [ ] Reopen Messages *during* a delete's snapshot step (large database + helps): the delete is refused and no extra snapshot is left behind. +- [ ] "Attachments only" delete: the messages stay, the attachments are + gone, and Messages shows the messages without broken bubbles (or note + what it shows). + +## 6. Restore from snapshot + +- [ ] After a delete, open "Restore from snapshot", pick the snapshot from + that delete and restore (Messages quit). Relaunch Messages: the + deleted messages are back. +- [ ] A pre-restore snapshot of the state just before the restore exists. +- [ ] Restoring while Messages is running is refused. + +## 7. Messages in iCloud (two devices, throwaway Apple ID) + +This is the one behaviour the app can only infer from chat.db's triggers. + +- [ ] With Messages in iCloud **on**, delete a few old messages on the Mac. + Open Messages, wait for it to sync, and check the second device: + **are the messages gone there too?** Record the answer — the app's + warnings currently say deletions are "expected to" sync. +- [ ] Do they ever come back on the Mac after a sync? +- [ ] Orphan cleanup with iCloud on asks for the iCloud acknowledgement + before removing records. + +## 8. Orphan cleanup + +- [ ] File → Clean Orphaned Data… with Messages quit: the scan finishes, the + counts look plausible, and Live Photo videos next to their photos are + *not* listed. +- [ ] Clean: the listed items are removed, nothing else in Messages breaks + (spot-check a few conversations with attachments). +- [ ] With Messages running, cleaning is refused. + +## 9. Large real database + +- [ ] On a Mac with years of real history (read-only steps only: preview, + backup, orphan **scan**): note how long the delete preview takes to + update after changing the filter, and how long the orphan scan takes. diff --git a/package-lock.json b/package-lock.json index 3422f55..8c250a3 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,8 +9,7 @@ "version": "0.1.0", "dependencies": { "@tauri-apps/api": "^2.1.1", - "@tauri-apps/plugin-dialog": "^2.2.0", - "@tauri-apps/plugin-shell": "^2.2.0" + "@tauri-apps/plugin-dialog": "^2.2.0" }, "devDependencies": { "@sveltejs/adapter-static": "^3.0.8", @@ -1218,15 +1217,6 @@ "@tauri-apps/api": "^2.10.1" } }, - "node_modules/@tauri-apps/plugin-shell": { - "version": "2.3.5", - "resolved": "https://registry.npmjs.org/@tauri-apps/plugin-shell/-/plugin-shell-2.3.5.tgz", - "integrity": "sha512-jewtULhiQ7lI7+owCKAjc8tYLJr92U16bPOeAa472LHJdgaibLP83NcfAF2e+wkEcA53FxKQAZ7byDzs2eeizg==", - "license": "MIT OR Apache-2.0", - "dependencies": { - "@tauri-apps/api": "^2.10.1" - } - }, "node_modules/@types/cookie": { "version": "0.6.0", "resolved": "https://registry.npmjs.org/@types/cookie/-/cookie-0.6.0.tgz", diff --git a/package.json b/package.json index 7a73ad5..e3ff808 100644 --- a/package.json +++ b/package.json @@ -12,8 +12,7 @@ }, "dependencies": { "@tauri-apps/api": "^2.1.1", - "@tauri-apps/plugin-dialog": "^2.2.0", - "@tauri-apps/plugin-shell": "^2.2.0" + "@tauri-apps/plugin-dialog": "^2.2.0" }, "devDependencies": { "@sveltejs/adapter-static": "^3.0.8", diff --git a/public/index.html b/public/index.html index 902c3d4..f8abc37 100644 --- a/public/index.html +++ b/public/index.html @@ -707,7 +707,7 @@ "featureList": [ "Export iMessages to HTML, PDF, plain text, or JSON", "Back up photos and attachments", - "Filter by date range or contact", + "Filter by date range or conversation", "Preview before deleting", "Automatic database snapshot before deletion", "Reclaim gigabytes of disk space" @@ -749,7 +749,7 @@ "name": "Is it safe to delete messages with Bubble Wrap?", "acceptedAnswer": { "@type": "Answer", - "text": "Yes. Bubble Wrap uses a multi-step safety process: first you preview exactly what will be affected, then the app automatically saves a complete snapshot of your Messages database to your Documents folder, and finally you must type a confirmation word before any deletion runs." + "text": "Bubble Wrap uses a multi-step safety process: you preview exactly what will be affected, the app checks that a backup covers exactly those messages and attachments, it saves a snapshot of your Messages database, and you must type DELETE before anything is removed. The snapshot only covers the database, so deleted attachment files can only be restored from your backup. If Messages in iCloud is on, deletions are expected to sync to your other devices." } }, { @@ -858,7 +858,7 @@

Everything you need,
nothing you don't.

🔎
-
Filter by date or contact
+
Filter by date or conversation

Choose any date range and pick specific conversations — archive just the messages that matter to you.

@@ -906,7 +906,7 @@

Three steps.
Zero surprises.

Built for peace of mind

We take "safely" seriously.

- Deleting messages can feel scary. That's why Bubble Wrap uses a multi-step safety process — nothing is removed until every check passes, and you can always roll back. + Deleting messages can feel scary. That's why Bubble Wrap uses a multi-step safety process — nothing is removed until every check passes. Deleting is permanent for attachment files, so keep your backup.

@@ -917,13 +917,13 @@

We take "safely" seriously.

Step 2
-
Automatic snapshot
-

A complete copy of your Messages database is saved to your Documents folder before any deletion runs.

+
Backup check & snapshot
+

Bubble Wrap checks that a backup holds exactly what you're about to delete, then saves a copy of your Messages database so the deletion can be undone. Attachment files are restored from your backup.

Step 3
Typed confirmation
-

You must type a confirmation word before the deletion runs — no accidental taps or clicks.

+

You must type DELETE before the deletion runs — no accidental taps or clicks.

diff --git a/public/llms.txt b/public/llms.txt index 83558c6..3f04c81 100644 --- a/public/llms.txt +++ b/public/llms.txt @@ -8,7 +8,7 @@ Bubble Wrap lets Mac users export their iMessage conversations — messages, pho - **Export messages**: Save iMessage conversations as HTML pages, PDFs, plain text files, or JSON - **Save attachments**: Photos, voice memos, and files are copied to a local folder alongside messages -- **Filter by date or contact**: Choose any date range and specific conversations to back up +- **Filter by date or conversation**: Choose any date range and specific conversations to back up - **Preview before deleting**: See the exact count of messages and files affected before anything changes - **Reclaim disk space**: Remove old messages and attachments from the Mac's Messages database after a successful backup - **Real names**: Contacts appear with names from your address book, not raw phone numbers or email addresses @@ -21,11 +21,14 @@ Bubble Wrap lets Mac users export their iMessage conversations — messages, pho ## Safety model -Bubble Wrap uses a three-step safety process before any deletion runs: +Bubble Wrap uses a multi-step safety process before any deletion runs: 1. **Preview first** — Shows the exact count of messages and files that will be affected -2. **Automatic snapshot** — Saves a complete copy of the Messages database to your Documents folder -3. **Typed confirmation** — Requires you to type a confirmation word; no accidental clicks +2. **Backup check** — Confirms a backup holds exactly the messages and attachment files being deleted (PDF exports don't count: they can't store emoji or non-Latin text) +3. **Automatic snapshot** — Saves a copy of the Messages database in the app's own folder (`~/Library/Application Support`); it covers the database only, so deleted attachment files are restored from the backup +4. **Typed confirmation** — Requires you to type DELETE; no accidental clicks + +If Messages in iCloud is on, deletions are expected to sync to iCloud and your other devices. ## Technical details diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index 63e9709..370a52a 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -1715,6 +1715,7 @@ name = "imessage-backup" version = "0.1.0" dependencies = [ "chrono", + "fs2", "imessage-database", "rusqlite", "serde", @@ -1722,7 +1723,6 @@ dependencies = [ "tauri", "tauri-build", "tauri-plugin-dialog", - "tauri-plugin-shell", "tempfile", "thiserror 2.0.18", "tokio", @@ -1732,6 +1732,7 @@ dependencies = [ [[package]] name = "imessage-database" version = "0.0.0" +source = "git+https://github.com/dbpecka/imessage-exporter?rev=cffac6a59099f8c1398c4089bd20fe222d983fa0#cffac6a59099f8c1398c4089bd20fe222d983fa0" dependencies = [ "base64 0.22.1", "chrono", @@ -1807,25 +1808,6 @@ dependencies = [ "serde", ] -[[package]] -name = "is-docker" -version = "0.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "928bae27f42bc99b60d9ac7334e3a21d10ad8f1835a4e12ec3ec0464765ed1b3" -dependencies = [ - "once_cell", -] - -[[package]] -name = "is-wsl" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "173609498df190136aa7dea1a91db051746d339e18476eed5ca40521f02d7aa5" -dependencies = [ - "is-docker", - "once_cell", -] - [[package]] name = "itoa" version = "1.0.18" @@ -2420,34 +2402,12 @@ version = "1.21.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" -[[package]] -name = "open" -version = "5.3.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9f3bab717c29a857abf75fcef718d441ec7cb2725f937343c734740a985d37fd" -dependencies = [ - "dunce", - "is-wsl", - "libc", - "pathdiff", -] - [[package]] name = "option-ext" version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" -[[package]] -name = "os_pipe" -version = "1.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7d8fae84b431384b68627d0f9b3b1245fcf9f46f6c0e3dc902e9dce64edd1967" -dependencies = [ - "libc", - "windows-sys 0.61.2", -] - [[package]] name = "owned_ttf_parser" version = "0.12.1" @@ -2516,12 +2476,6 @@ dependencies = [ "subtle", ] -[[package]] -name = "pathdiff" -version = "0.2.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "df94ce210e5bc13cb6651479fa48d14f601d9858cfe0467f43ae157023b938d3" - [[package]] name = "pbkdf2" version = "0.12.2" @@ -3520,54 +3474,12 @@ dependencies = [ "digest", ] -[[package]] -name = "shared_child" -version = "1.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e362d9935bc50f019969e2f9ecd66786612daae13e8f277be7bfb66e8bed3f7" -dependencies = [ - "libc", - "sigchld", - "windows-sys 0.60.2", -] - [[package]] name = "shlex" version = "1.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" -[[package]] -name = "sigchld" -version = "0.2.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "47106eded3c154e70176fc83df9737335c94ce22f821c32d17ed1db1f83badb1" -dependencies = [ - "libc", - "os_pipe", - "signal-hook", -] - -[[package]] -name = "signal-hook" -version = "0.3.18" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d881a16cf4426aa584979d30bd82cb33429027e42122b169753d6ef1085ed6e2" -dependencies = [ - "libc", - "signal-hook-registry", -] - -[[package]] -name = "signal-hook-registry" -version = "1.4.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" -dependencies = [ - "errno", - "libc", -] - [[package]] name = "simd-adler32" version = "0.3.9" @@ -4029,27 +3941,6 @@ dependencies = [ "url", ] -[[package]] -name = "tauri-plugin-shell" -version = "2.3.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8457dbf9e2bab1edd8df22bb2c20857a59a9868e79cb3eac5ed639eec4d0c73b" -dependencies = [ - "encoding_rs", - "log", - "open", - "os_pipe", - "regex", - "schemars 0.8.22", - "serde", - "serde_json", - "shared_child", - "tauri", - "tauri-plugin", - "thiserror 2.0.18", - "tokio", -] - [[package]] name = "tauri-runtime" version = "2.10.1" diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index 967925f..db26dce 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -20,24 +20,24 @@ tauri-build = { version = "2.0", features = [] } [dependencies] tauri = { version = "2.1", features = [] } tauri-plugin-dialog = "2.2" -tauri-plugin-shell = "2.2" serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" thiserror = "2.0" tokio = { version = "1.42", features = ["rt-multi-thread", "macros", "sync"] } tokio-util = "0.7" chrono = "0.4" +fs2 = "0.4" -# Library crate from the sibling repository. Uses path dep against the user's -# fork (dbpecka/imessage-exporter); features are added as the app grows. +# Library crate from the fork (dbpecka/imessage-exporter), pinned to an exact +# commit so a signed build can't silently pick up whatever is on `develop`. +# To bump it, update `rev` after reviewing the fork's changes. # -# TODO(C3): once the Phase 5b vendoring is committed in the fork, add a -# pinned `git` + `rev` (or a versioned `version = "x.y"`) alongside `path` -# so reproducible builds don't silently change when the fork's working tree -# moves. Today the fork's develop branch HEAD (b60ecc9) predates the -# vendored `exporters` and `write` modules this app requires, so there is -# no rev to pin — pinning is blocked on committing the fork first. -imessage-database = { path = "../../imessage-exporter/imessage-database", features = ["export", "write"] } +# To develop against a local checkout of the fork instead, create +# `src-tauri/.cargo/config.toml` (git-ignored) with: +# +# [patch."https://github.com/dbpecka/imessage-exporter"] +# imessage-database = { path = "../../imessage-exporter/imessage-database" } +imessage-database = { git = "https://github.com/dbpecka/imessage-exporter", rev = "cffac6a59099f8c1398c4089bd20fe222d983fa0", features = ["export", "write"] } rusqlite = { version = "0.39", features = ["backup"] } [dev-dependencies] diff --git a/src-tauri/capabilities/default.json b/src-tauri/capabilities/default.json index 361e87e..d83b213 100644 --- a/src-tauri/capabilities/default.json +++ b/src-tauri/capabilities/default.json @@ -2,10 +2,11 @@ "$schema": "../gen/schemas/desktop-schema.json", "identifier": "default", "description": "Capabilities for the main window", - "windows": ["main"], + "windows": [ + "main" + ], "permissions": [ "core:default", - "dialog:default", - "shell:allow-open" + "dialog:default" ] } diff --git a/src-tauri/entitlements.plist b/src-tauri/entitlements.plist index 0201dea..2aa9add 100644 --- a/src-tauri/entitlements.plist +++ b/src-tauri/entitlements.plist @@ -6,9 +6,5 @@ com.apple.security.files.user-selected.read-write - com.apple.security.cs.allow-unsigned-executable-memory - - com.apple.security.cs.disable-library-validation - diff --git a/src-tauri/src/commands/backup.rs b/src-tauri/src/commands/backup.rs index c7a8897..374336f 100644 --- a/src-tauri/src/commands/backup.rs +++ b/src-tauri/src/commands/backup.rs @@ -1,6 +1,6 @@ use imessage_database::{ exporters::{ - attachments::copy_attachments, + compatibility::attachment_manager::AttachmentManagerMode, config::{AttachmentMode, ExportConfig}, export_type::ExportType, json::run_json_export, @@ -9,16 +9,121 @@ use imessage_database::{ progress::ProgressReporter, runtime::Config, }, - tables::{messages::message::Message, table::get_connection}, + tables::messages::message::Message, }; +use std::collections::HashSet; +use std::sync::Arc; + use serde::{Deserialize, Serialize}; -use tauri::AppHandle; +use tauri::{AppHandle, State}; -use crate::core::db_path::default_chat_db_path; +use crate::commands::delete::filter_content; +use crate::core::attachment_copy::source_size; +use crate::core::attachment_copy::{ + annotate_json_export, copy_attachments, verify_runtime_copies, CopySummary, +}; +use crate::core::db_path::{blocking, check_chat_db_access, default_chat_db_path, open_read}; +use crate::core::disk_image::{DiskImageTool, Hdiutil, MountedImage, Password}; use crate::core::filter::FilterSpec; +use crate::core::op_lock::OperationLock; use crate::core::progress::TauriProgress; +use crate::core::receipts::{BackupReceipts, ContentFingerprint}; +use crate::core::snapshot::create_unique_dir; +use crate::core::space::{ensure_space, snapshot_size, SpaceProbe}; use crate::error::AppError; +/// Write the tapbacks in `ids` to `tapbacks-outside-filter.json`. +fn write_extra_tapbacks( + conn: &rusqlite::Connection, + export_path: &std::path::Path, + file_name: &str, + ids: &[i64], +) -> Result<(), AppError> { + let mut rows = Vec::new(); + for chunk in ids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT ROWID, guid, associated_message_guid, associated_message_type, date, \ + handle_id, is_from_me, text \ + FROM message WHERE ROWID IN ({placeholders})" + ))?; + let mapped = stmt.query_map(rusqlite::params_from_iter(chunk.iter()), |r| { + Ok(serde_json::json!({ + "rowid": r.get::<_, i64>(0)?, + "guid": r.get::<_, Option>(1)?, + "associatedMessageGuid": r.get::<_, Option>(2)?, + "associatedMessageType": r.get::<_, Option>(3)?, + "date": r.get::<_, Option>(4)?, + "handleId": r.get::<_, Option>(5)?, + "isFromMe": r.get::<_, Option>(6)?, + "text": r.get::<_, Option>(7)?, + })) + })?; + for row in mapped { + rows.push(row?); + } + } + let json = serde_json::to_vec_pretty(&rows).map_err(|e| AppError::Other(e.to_string()))?; + std::fs::write(export_path.join(file_name), json)?; + Ok(()) +} + +/// Tapbacks, poll votes and poll updates matching `query` whose target +/// message doesn't match it. +fn children_without_target( + conn: &rusqlite::Connection, + query: &imessage_database::util::query_context::QueryContext, +) -> Result, AppError> { + use imessage_database::tables::table::Table; + let mut guids: HashSet = HashSet::new(); + let mut children: Vec<(i64, String)> = Vec::new(); + let mut stmt = Message::stream_rows(conn, query)?; + let rows = stmt.query_map([], |row| Ok(Message::from_row(row)))?; + for row in rows { + let m = Message::extract(row)?; + guids.insert(m.guid.clone()); + if m.is_tapback() || m.is_poll_vote() || m.is_poll_update() { + if let Some(target) = &m.associated_message_guid { + children.push((i64::from(m.rowid), target.clone())); + } + } + } + Ok(children + .into_iter() + .filter(|(_, t)| !guids.contains(crate::commands::delete::reaction_target(t))) + .map(|(id, _)| id) + .collect()) +} + +/// Rough upper bound on what a backup will write: the attachment files to +/// copy (their real sizes on disk) plus the exported text, estimated as +/// twice the matched share of chat.db (HTML adds markup; chat.db's indexes +/// make it a generous base), at least 10 MB. +fn estimate_backup_size( + conn: &rusqlite::Connection, + db_path: &std::path::Path, + plan: &imessage_database::tables::write::delete::DeletePlan, + copy: bool, +) -> Result { + let total: i64 = conn.query_row("SELECT COUNT(*) FROM message", [], |r| r.get(0))?; + let share = plan.message_rowids.len() as f64 / (total.max(1) as f64); + let text = ((snapshot_size(db_path) as f64) * share.min(1.0) * 2.0) as u64; + let files: u64 = if copy { + plan.attachment_files_on_disk + .iter() + .filter_map(|p| source_size(p)) + .sum() + } else { + 0 + }; + Ok(files + text.max(10 * 1024 * 1024)) +} + +fn count(r: Result) -> Result { + r.map(|n| n.max(0) as u64) + .map_err(|e| AppError::Other(e.to_string())) +} + pub const BACKUP_PROGRESS_EVENT: &str = "backup-progress"; #[derive(Debug, Deserialize)] @@ -29,6 +134,10 @@ pub struct RunBackupArgs { pub destination: String, #[serde(default)] pub copy_attachments: bool, + /// Write the backup into a new AES-256 encrypted disk image protected + /// by this password, instead of a plain folder. + #[serde(default)] + pub encrypt_password: Option, } #[derive(Debug, Serialize)] @@ -39,110 +148,209 @@ pub struct BackupResult { pub attachment_bytes_copied: u64, pub conversation_count: u64, pub manifest_path: String, + /// The folder this run wrote into (a fresh subfolder of the chosen + /// destination). pub export_path: String, pub format: String, + /// Whether every attachment on this Mac was copied into the export. A + /// backup that only references attachments, or where some copies + /// failed, doesn't protect them from a delete. + pub includes_attachments: bool, + /// Per-attachment copy outcomes; `None` when attachments weren't copied. + pub attachment_copy: Option, + /// Whether this backup counts toward the delete step's backup check. + /// PDF backups don't: the PDF writer can only store Latin-1 text, so + /// emoji, smart quotes and non-Latin scripts come out as `?`. + pub counts_for_delete: bool, + /// Whether the backup is inside an encrypted disk image (`export_path` + /// is then the `.sparsebundle`; open it in Finder with the password). + pub encrypted: bool, } -/// Run a backup with the supplied filter and format to the supplied -/// destination directory. Emits progress events on `backup-progress`. +/// Run a backup with the supplied filter and format into a new, timestamped +/// subfolder of the supplied destination. Emits progress events on +/// `backup-progress`. #[tauri::command] -pub async fn run_backup(app: AppHandle, args: RunBackupArgs) -> Result { +pub async fn run_backup( + app: AppHandle, + receipts: State<'_, Arc>, + lock: State<'_, OperationLock>, + args: RunBackupArgs, +) -> Result { + // A delete or cleanup running alongside could remove files mid-copy. + let _guard = lock.try_acquire("a backup")?; + let db_path = default_chat_db_path()?; + let receipts = Arc::clone(&receipts); + let progress = TauriProgress::new(app.clone(), BACKUP_PROGRESS_EVENT); + blocking(move || { + backup_now( + &db_path, + &receipts, + args, + &progress, + crate::core::space::system_available, + &Hdiutil, + ) + }) + .await +} + +/// The whole backup as a plain function (see [`run_backup`]): export, +/// attachment copy, and the receipt that lets a later delete rely on it. +pub(crate) fn backup_now( + db_path: &std::path::Path, + receipts: &BackupReceipts, + args: RunBackupArgs, + progress: &dyn ProgressReporter, + space: SpaceProbe, + images: &dyn DiskImageTool, +) -> Result { let format = ExportType::from_cli(&args.format) .ok_or_else(|| AppError::Other(format!("unsupported format: {}", args.format)))?; - - let db_path = default_chat_db_path()?; + check_chat_db_access(db_path)?; + let db_path = db_path.to_path_buf(); let destination = crate::core::paths::validate_user_path(&args.destination, "destination")?; + let filter_key = args.filter.key(); + let copy = args.copy_attachments; + let query = args.filter.to_query_context()?; + let format_label = format.to_string(); - let config = ExportConfig { - db_path, - export_path: destination.clone(), + // Fingerprint what the filter matches *before* exporting. Anything + // that arrives during or after the run changes the fingerprint, so + // this backup can never vouch for messages it might not contain. + let (content, planned_attachments, reaction_ids, estimate) = { + let conn = open_read(&db_path)?; + let fc = filter_content(&conn, &query, &db_path)?; + let estimate = estimate_backup_size(&conn, &db_path, &fc.plan, copy)?; + ( + fc.content, + fc.plan.attachment_rowids, + fc.reaction_ids, + estimate, + ) + }; + // Refuse up front rather than leave a half-written backup on a full + // drive. + ensure_space(&destination, estimate, "this backup", space)?; + + // Every run gets its own folder: the JSON exporter appends to + // existing files (re-running into the same folder duplicated every + // message), and stale files from an earlier run must never be + // mistaken for this run's copies. + std::fs::create_dir_all(&destination)?; + let stamp = chrono::Local::now().format("%Y-%m-%d %H.%M.%S").to_string(); + let name = format!("Bubble Wrap {stamp}"); + + // Encrypted: export into a freshly created, mounted encrypted image. + // The guard unmounts it on every exit path, so a failed export never + // leaves the backup mounted and readable. + let mounted = match &args.encrypt_password { + Some(password) => Some(MountedImage::create_and_mount( + images, + &destination, + &name, + password, + estimate + .saturating_add(estimate / 2) + .max(1024 * 1024 * 1024), + )?), + None => None, + }; + let export_root = mounted + .as_ref() + .map_or_else(|| destination.clone(), |m| m.mountpoint.clone()); + let export_path = create_unique_dir(&export_root, &name)?; + + // The library's own attachment copying is collision-prone (see + // `attachment_copy`), so JSON/PDF always run in reference mode and + // we copy separately. The HTML/TXT runtime names copies safely by + // rowid and links to them, so it keeps doing its own copying and we + // verify the result instead of copying everything twice. + let config = |mode: AttachmentMode| ExportConfig { + db_path: db_path.clone(), + export_path: export_path.clone(), format, - query: args.filter.to_query_context()?, - attachments: if args.copy_attachments { - AttachmentMode::Copy - } else { - AttachmentMode::Reference - }, + query: query.clone(), + attachments: mode, attachment_root: None, custom_owner_name: None, }; + // What the user is told, and what the receipt checks still exists: the + // image itself when encrypted (the mount goes away). + let export_label = mounted.as_ref().map_or_else( + || export_path.display().to_string(), + |m| m.image.display().to_string(), + ); - let progress = TauriProgress::new(app.clone(), BACKUP_PROGRESS_EVENT); - let format_label = format.to_string(); - - let result = tokio::task::spawn_blocking(move || -> Result { + let (message_count, attachment_count, conversation_count, manifest_path, copy_summary) = match format { ExportType::Json => { - let summary = run_json_export(&config, &progress) + let summary = run_json_export(&config(AttachmentMode::Reference), progress) .map_err(|e| AppError::Other(e.to_string()))?; - Ok(BackupResult { - message_count: summary.message_count, - attachment_count: summary.attachment_count, - attachment_bytes_copied: summary.attachment_bytes_copied, - conversation_count: summary.conversation_count, - manifest_path: summary.manifest_path.display().to_string(), - export_path: config.export_path.display().to_string(), - format: format_label, - }) + let copy_summary = if copy { + let conn = open_read(&db_path)?; + let (cs, copied) = + copy_attachments(&conn, &db_path, &query, &export_path, progress)?; + annotate_json_export(&export_path, &summary.manifest_path, &copied, &cs)?; + Some(cs) + } else { + None + }; + ( + summary.message_count, + summary.attachment_count, + summary.conversation_count, + summary.manifest_path.display().to_string(), + copy_summary, + ) } ExportType::Pdf => { - // Copy attachment files into {export}/attachments// - // using the same helper JSON uses, so the PDF run leaves a - // browsable attachment archive next to the PDF. - let att_bytes = copy_attachments(&config, &progress) - .map_err(|e| AppError::Other(format!("attachment copy failed: {e}")))? - .bytes; - let summary = run_pdf_export(&config, &progress) + // Copy first so the PDF run leaves a browsable attachment + // archive next to the PDF. + let copy_summary = if copy { + let conn = open_read(&db_path)?; + Some(copy_attachments(&conn, &db_path, &query, &export_path, progress)?.0) + } else { + None + }; + let summary = run_pdf_export(&config(AttachmentMode::Reference), progress) .map_err(|e| AppError::Other(e.to_string()))?; - Ok(BackupResult { - message_count: summary.message_count, - attachment_count: summary.attachment_count, - attachment_bytes_copied: att_bytes, - conversation_count: summary.conversation_count, - manifest_path: String::new(), - export_path: config.export_path.display().to_string(), - format: format_label, - }) + ( + summary.message_count, + summary.attachment_count, + summary.conversation_count, + String::new(), + copy_summary, + ) } ExportType::Html | ExportType::Txt => { - // HTML / TXT run through the full library runtime. That - // path owns its own stderr-based progress bar — migrating - // it to `ProgressReporter` is a follow-up; for now we - // flip the UI into a generic "running" state and emit a - // finish event when start() returns. - - // The library's HTML/TXT runtime doesn't return a summary, - // so compute counts from the same filter before running. - // Open a scoped readonly connection; drop it before the - // library opens its own. - let (message_count, conversation_count, attachment_count) = { - let conn = get_connection(&config.db_path) - .map_err(|e| AppError::Other(format!("failed to open chat.db: {e}")))?; - let messages = Message::get_count(&conn, &config.query) - .map_err(|e| AppError::Other(e.to_string()))? - .max(0) as u64; - let conversations = Message::get_conversation_count(&conn, &config.query) - .map_err(|e| AppError::Other(e.to_string()))? - .max(0) as u64; - let attachments = Message::get_attachment_count(&conn, &config.query) - .map_err(|e| AppError::Other(e.to_string()))? - .max(0) as u64; - (messages, conversations, attachments) + // The library's HTML/TXT runtime doesn't return a + // summary, so compute counts from the same filter first. + let (messages, conversations, attachments) = { + let conn = open_read(&db_path)?; + ( + count(Message::get_count(&conn, &query))?, + count(Message::get_conversation_count(&conn, &query))?, + count(Message::get_attachment_count(&conn, &query))?, + ) }; - // Copy attachment files into {export}/attachments// - // using the same helper JSON uses. The HTML/TXT runtime's - // own AttachmentManager has historically lost files for - // iCloud-offloaded sources and logged noisy "not found" - // output; this pass is the reliable one. - let att_summary = copy_attachments(&config, &progress) - .map_err(|e| AppError::Other(format!("attachment copy failed: {e}")))?; - progress.start(0); progress.set_message("Running HTML/TXT export…"); - - let export_path = config.export_path.clone(); - let options: Options = config.into(); + let mode = if copy { + AttachmentMode::Copy + } else { + AttachmentMode::Reference + }; + let mut options: Options = config(mode).into(); + if copy { + // The library's default copy mode re-encodes HEIC + // photos (and stickers) as JPEG via `sips`. A backup + // must hold the originals: a delete removes them + // from the Mac. Browsers may not show HEIC inline in + // the HTML, but the files are byte-for-byte intact. + options.attachment_manager.mode = AttachmentManagerMode::Clone; + } let mut runtime = Config::new(options).map_err(|e| { AppError::Other(format!("failed to initialise export runtime: {e}")) })?; @@ -151,20 +359,121 @@ pub async fn run_backup(app: AppHandle, args: RunBackupArgs) -> Result = planned_attachments.into_iter().collect(); + let copied_ids: Vec = copy_summary + .as_ref() + .map(|c| { + c.copied_ids + .iter() + .copied() + .filter(|id| planned.contains(id)) + .collect() + }) + .unwrap_or_default(); + let copied = ContentFingerprint::of(&[], &copied_ids); + + // Inside an image, paths under the temporary mount mean nothing later; + // report the manifest relative to the image. + let manifest_path = match &mounted { + Some(m) => std::path::Path::new(&manifest_path) + .strip_prefix(&m.mountpoint) + .map(|rel| rel.display().to_string()) + .unwrap_or(manifest_path), + None => manifest_path, + }; + let encrypted = mounted.is_some(); + if let Some(m) = mounted { + let image = m.image.clone(); + m.detach().map_err(|e| { + AppError::Other(format!( + "The backup was written to {}, but the image couldn't be unmounted ({e:?}). \ + Eject it in Finder, then run the backup again so it can be used for deleting.", + image.display() + )) + })?; + } + + let result = BackupResult { + encrypted, + message_count, + attachment_count, + attachment_bytes_copied: copy_summary.as_ref().map_or(0, |c| c.bytes), + conversation_count, + manifest_path, + export_path: export_label, + format: format_label, + includes_attachments: copy_summary.as_ref().is_some_and(|c| c.failed == 0), + attachment_copy: copy_summary, + counts_for_delete: !matches!(format, ExportType::Pdf), + }; + + // Only a backup that ran to completion counts toward `run_delete`'s + // backup gate, and it only vouches for attachments if every copy that + // could be made was made. + if result.counts_for_delete { + receipts.record( + filter_key, + content, + copied, + result.includes_attachments, + std::path::Path::new(&result.export_path), + ); + } Ok(result) } diff --git a/src-tauri/src/commands/delete.rs b/src-tauri/src/commands/delete.rs index 25f2ab7..fc9a21b 100644 --- a/src-tauri/src/commands/delete.rs +++ b/src-tauri/src/commands/delete.rs @@ -1,19 +1,31 @@ -use std::fs; - -use imessage_database::tables::{ - table::{get_connection, get_writable_connection}, - write::{ +use imessage_database::{ + tables::write::{ attachment_filter::AttachmentFilter, delete::{execute_delete, preview_delete as lib_preview_delete, DeletePlan}, }, + util::query_context::QueryContext, }; +use std::collections::{HashMap, HashSet}; +use std::sync::Arc; + +use rusqlite::{params_from_iter, Connection}; use serde::{Deserialize, Serialize}; +use tauri::{AppHandle, State}; -use crate::core::db_path::default_chat_db_path; +use crate::core::attachment_files::{ + attachment_roots, expand_tilde, path_key, remove_attachment_file, RemoveOutcome, +}; +use crate::core::db_path::{blocking, default_chat_db_path, open_read, open_write}; use crate::core::filter::FilterSpec; use crate::core::icloud::{detect_icloud_messages, ICloudState}; use crate::core::messages_app::is_messages_running; -use crate::core::snapshot::{checkpoint_wal, default_snapshot_root, snapshot_chat_db}; +use crate::core::op_lock::OperationLock; +use crate::core::receipts::{BackupReceipts, ContentFingerprint, Coverage}; +use crate::core::snapshot::{ + checkpoint_wal, default_snapshot_root, prune_snapshots, snapshot_chat_db, SnapshotKind, + SNAPSHOTS_TO_KEEP, +}; +use crate::core::space::{ensure_space, snapshot_size, SpaceProbe}; use crate::error::AppError; pub const DELETE_CONFIRMATION_PHRASE: &str = "DELETE"; @@ -21,20 +33,349 @@ pub const DELETE_CONFIRMATION_PHRASE: &str = "DELETE"; /// Apply a `DeleteScope` to a freshly-computed plan, zeroing out the parts /// the caller wants to keep. Pulled out of `run_delete` so the slicing /// logic is covered by unit tests without needing a real chat.db. -pub(crate) fn apply_scope(plan: &mut DeletePlan, scope: &DeleteScope) { +pub(crate) fn apply_scope(plan: &mut DeletePlan, scope: &DeleteScope) -> Result<(), AppError> { match scope { - DeleteScope::MessagesOnly => { - // Keep attachment records and files on disk; only remove messages. - plan.attachment_rowids.clear(); - plan.attachment_files_on_disk.clear(); - plan.attachment_bytes = 0; - } + DeleteScope::MessagesOnly => Err(AppError::Other( + "Deleting messages while keeping their attachments isn't possible: chat.db's own \ + triggers delete an attachment's record as soon as its last message is gone, which \ + would leave the files untracked. Delete messages together with their attachments, \ + or attachments only." + .into(), + )), DeleteScope::AttachmentsOnly => { // Keep messages; only remove attachment records and files. plan.message_rowids.clear(); + Ok(()) + } + DeleteScope::Both => Ok(()), + } +} + +/// Drop attachments from `plan` that a message outside the plan still links +/// to, and drop on-disk files that an attachment row outside the plan still +/// points at. The library plans every attachment joined to a matched +/// message; without this, deleting one message could take an attachment (or +/// its file) out from under a message the user kept. +/// +/// Must run before [`apply_scope`], against the full matched message set. +pub(crate) fn exclude_shared_attachments( + conn: &Connection, + plan: &mut DeletePlan, +) -> Result<(), AppError> { + if plan.attachment_rowids.is_empty() { + return Ok(()); + } + let planned_messages: HashSet = plan.message_rowids.iter().copied().collect(); + + let mut shared: HashSet = HashSet::new(); + for chunk in plan.attachment_rowids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT attachment_id, message_id FROM message_attachment_join \ + WHERE attachment_id IN ({placeholders})" + ))?; + let rows = stmt.query_map(params_from_iter(chunk.iter()), |r| { + Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)) + })?; + for row in rows { + let (attachment_id, message_id) = row?; + if !planned_messages.contains(&message_id) { + shared.insert(attachment_id); + } } - DeleteScope::Both => {} } + + let mut stmt = + conn.prepare("SELECT ROWID, filename, COALESCE(total_bytes, 0) FROM attachment")?; + let rows: Vec<(i64, Option, i64)> = stmt + .query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))? + .collect::>()?; + + plan.attachment_rowids.retain(|id| !shared.contains(id)); + let deleting: HashSet = plan.attachment_rowids.iter().copied().collect(); + + let mut bytes_by_row: HashMap = HashMap::new(); + let mut kept_files: HashSet = HashSet::new(); + for (rowid, filename, bytes) in rows { + bytes_by_row.insert(rowid, bytes); + if !deleting.contains(&rowid) { + if let Some(f) = filename { + kept_files.insert(path_key(&expand_tilde(&f))); + } + } + } + + for id in &shared { + let bytes = bytes_by_row.get(id).copied().unwrap_or(0).max(0) as u64; + plan.attachment_bytes = plan.attachment_bytes.saturating_sub(bytes); + } + plan.attachment_files_on_disk + .retain(|p| !kept_files.contains(&path_key(p))); + Ok(()) +} + +/// What a delete means on an iCloud-synced account. chat.db's own delete +/// triggers record every removed message and attachment in +/// `sync_deleted_messages` / `sync_deleted_attachments` — the queue Messages +/// uses to push deletions to iCloud — so the delete is expected to reach the +/// user's other devices the next time Messages syncs. +const ICLOUD_RISK: &str = "Deleting here records each deletion for Messages in iCloud, so when \ + Messages next syncs, these messages and attachments are expected to be deleted from iCloud \ + and your other devices too — not just this Mac."; + +/// Extract the target message guid from an `associated_message_guid` +/// (`p:0/`, `bp:`, or a bare guid). +pub(crate) fn reaction_target(associated: &str) -> &str { + if let Some(rest) = associated.strip_prefix("bp:") { + rest + } else if let Some(rest) = associated.strip_prefix("p:") { + rest.split_once('/').map_or(rest, |(_, g)| g) + } else { + associated + } +} + +fn is_missing_column(e: &rusqlite::Error) -> bool { + // Reported as `SqlInputError` or `SqliteFailure` depending on the + // rusqlite build; the message is the stable part. + e.to_string().contains("no such column") +} + +/// Add tapbacks (and tapback removals) whose target message is being +/// deleted, so they aren't left pointing at nothing. Reactions that carry +/// attachments (sticker reactions) are left alone rather than orphaning +/// their attachment. +pub(crate) fn add_orphaned_reactions( + conn: &Connection, + plan: &mut DeletePlan, +) -> Result, AppError> { + if plan.message_rowids.is_empty() { + return Ok(Vec::new()); + } + let mut guids: HashSet = HashSet::new(); + for chunk in plan.message_rowids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT guid FROM message WHERE ROWID IN ({placeholders})" + ))?; + for g in stmt.query_map(params_from_iter(chunk.iter()), |r| r.get::<_, String>(0))? { + guids.insert(g?); + } + } + + let stmt = conn.prepare( + "SELECT ROWID, associated_message_guid FROM message \ + WHERE associated_message_guid IS NOT NULL \ + AND associated_message_type BETWEEN 2000 AND 3999 \ + AND COALESCE(cache_has_attachments, 0) = 0", + ); + let mut stmt = match stmt { + Ok(s) => s, + // Very old (or test) schemas don't track reactions this way. + Err(e) if is_missing_column(&e) => return Ok(Vec::new()), + Err(e) => return Err(e.into()), + }; + let planned: HashSet = plan.message_rowids.iter().copied().collect(); + let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, String>(1)?)))?; + let mut added = Vec::new(); + for row in rows { + let (rowid, associated) = row?; + if !planned.contains(&rowid) && guids.contains(reaction_target(&associated)) { + plan.message_rowids.push(rowid); + added.push(rowid); + } + } + plan.message_rowids.sort_unstable(); + plan.message_rowids.dedup(); + Ok(added) +} + +/// Messages linked to any of `attachment_ids`. +fn messages_linked_to(conn: &Connection, attachment_ids: &[i64]) -> Result, AppError> { + let mut out = Vec::new(); + for chunk in attachment_ids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT DISTINCT message_id FROM message_attachment_join \ + WHERE attachment_id IN ({placeholders})" + ))?; + for id in stmt.query_map(params_from_iter(chunk.iter()), |r| r.get::<_, i64>(0))? { + out.push(id?); + } + } + out.sort_unstable(); + out.dedup(); + Ok(out) +} + +/// After an attachments-only delete, messages that no longer have any +/// attachment still carry `cache_has_attachments = 1`, which makes Messages +/// and exporters look for attachments that are gone. Best-effort: runs in +/// its own transaction after the delete committed, and a failure only means +/// the flag stays stale. +pub(crate) fn clear_stale_attachment_flags( + conn: &mut Connection, + message_ids: &[i64], +) -> Result { + if message_ids.is_empty() { + return Ok(0); + } + let tx = conn.transaction()?; + let mut updated = 0u64; + for chunk in message_ids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let sql = format!( + "UPDATE message SET cache_has_attachments = 0 \ + WHERE ROWID IN ({placeholders}) \ + AND NOT EXISTS (SELECT 1 FROM message_attachment_join j \ + WHERE j.message_id = message.ROWID)" + ); + match tx.execute(&sql, params_from_iter(chunk.iter())) { + Ok(n) => updated += n as u64, + Err(e) if is_missing_column(&e) => return Ok(0), + Err(e) => return Err(e.into()), + } + } + tx.commit()?; + Ok(updated) +} + +/// Planned attachments whose file isn't on this Mac (typically offloaded to +/// iCloud). No backup can include them, and deleting their records removes +/// the only way to download them again. +pub(crate) fn count_not_on_mac(conn: &Connection, plan: &DeletePlan) -> Result { + let mut n = 0u64; + for chunk in plan.attachment_rowids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT filename FROM attachment WHERE ROWID IN ({placeholders})" + ))?; + for f in stmt.query_map(params_from_iter(chunk.iter()), |r| { + r.get::<_, Option>(0) + })? { + if let Some(f) = f? { + if !expand_tilde(&f).exists() { + n += 1; + } + } + } + } + Ok(n) +} + +/// The full (unscoped) plan for `ctx`, shared by the preview and the delete +/// so the counts the user confirms are the counts that get deleted. +/// What a filter matches, in the form a backup records and a delete +/// checks: the filter's messages and attachments plus the tapbacks (outside +/// the filter) that deleting those messages would also remove. +pub(crate) struct FilterContent { + /// The library plan with the tapbacks added; shared attachments not yet + /// excluded. + pub plan: DeletePlan, + pub reaction_ids: Vec, + /// Everything that would be deleted or backed up. + pub content: ContentFingerprint, + /// Which of the matched attachments have their file on this Mac right + /// now. A backup only protects the files that were present (and copied) + /// when it ran; anything downloaded afterwards changes this. + pub present: ContentFingerprint, +} + +/// Attachment ids (of `ids`) whose file is on this Mac. +pub(crate) fn present_attachments(conn: &Connection, ids: &[i64]) -> Result, AppError> { + let mut out = Vec::new(); + for chunk in ids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT ROWID, filename FROM attachment WHERE ROWID IN ({placeholders})" + ))?; + let rows = stmt.query_map(params_from_iter(chunk.iter()), |r| { + Ok((r.get::<_, i64>(0)?, r.get::<_, Option>(1)?)) + })?; + for row in rows { + let (id, f) = row?; + if f.is_some_and(|f| expand_tilde(&f).exists()) { + out.push(id); + } + } + } + Ok(out) +} + +/// The library collects attachments per chunk of 800 messages with a +/// per-chunk `DISTINCT`, so an attachment linked from messages in two +/// chunks is listed (and its size counted) twice. De-duplicate the ids and +/// files and recompute the total from the database. +fn dedup_attachments(conn: &Connection, plan: &mut DeletePlan) -> Result<(), AppError> { + let before = plan.attachment_rowids.len(); + plan.attachment_rowids.sort_unstable(); + plan.attachment_rowids.dedup(); + let mut seen: HashSet = HashSet::new(); + plan.attachment_files_on_disk + .retain(|p| seen.insert(path_key(p))); + if plan.attachment_rowids.len() == before { + return Ok(()); + } + let mut total: u64 = 0; + for chunk in plan.attachment_rowids.chunks(800) { + let placeholders = vec!["?"; chunk.len()].join(","); + let mut stmt = conn.prepare(&format!( + "SELECT COALESCE(total_bytes, 0) FROM attachment WHERE ROWID IN ({placeholders})" + ))?; + for b in stmt.query_map(params_from_iter(chunk.iter()), |r| r.get::<_, i64>(0))? { + total = total.saturating_add(b?.max(0) as u64); + } + } + plan.attachment_bytes = total; + Ok(()) +} + +pub(crate) fn filter_content( + conn: &Connection, + ctx: &QueryContext, + db_path: &std::path::Path, +) -> Result { + let mut plan = lib_preview_delete(conn, ctx, &AttachmentFilter::any(), db_path) + .map_err(|e| AppError::Database(format!("failed to compute delete plan: {e}")))?; + dedup_attachments(conn, &mut plan)?; + let reaction_ids = add_orphaned_reactions(conn, &mut plan)?; + let content = ContentFingerprint::of(&plan.message_rowids, &plan.attachment_rowids); + let present = ContentFingerprint::of(&[], &present_attachments(conn, &plan.attachment_rowids)?); + Ok(FilterContent { + plan, + reaction_ids, + content, + present, + }) +} + +pub(crate) struct BuiltPlan { + pub plan: DeletePlan, + /// Tapbacks outside the filter added because their target message is + /// being deleted. + pub reactions: u64, + pub content: ContentFingerprint, + pub present: ContentFingerprint, +} + +pub(crate) fn build_plan( + conn: &Connection, + ctx: &QueryContext, + db_path: &std::path::Path, +) -> Result { + let FilterContent { + mut plan, + reaction_ids, + content, + present, + } = filter_content(conn, ctx, db_path)?; + exclude_shared_attachments(conn, &mut plan)?; + Ok(BuiltPlan { + plan, + reactions: reaction_ids.len() as u64, + content, + present, + }) } #[derive(Debug, Serialize)] @@ -44,34 +385,51 @@ pub struct DeletePreview { pub attachment_count: u64, pub attachment_bytes: u64, pub on_disk_file_count: u64, + /// Matched attachments whose file isn't on this Mac. + pub not_on_mac_count: u64, + /// Of `message_count`: tapbacks outside the filter whose target message + /// is being deleted (so they'd otherwise point at nothing). + pub reaction_count: u64, } /// Compute what a delete with the given filter would remove. Read-only. #[tauri::command] pub async fn preview_delete(filter: FilterSpec) -> Result { let db_path = default_chat_db_path()?; - let conn = get_connection(&db_path)?; - let ctx = filter.to_query_context()?; - - let plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &db_path) - .map_err(|e| AppError::Database(format!("failed to compute delete plan: {e}")))?; + blocking(move || preview_delete_now(&db_path, &filter)).await +} +pub(crate) fn preview_delete_now( + db_path: &std::path::Path, + filter: &FilterSpec, +) -> Result { + let ctx = filter.to_query_context()?; + let conn = open_read(db_path)?; + let BuiltPlan { + plan, + reactions: reaction_count, + .. + } = build_plan(&conn, &ctx, db_path)?; Ok(DeletePreview { + reaction_count, message_count: plan.message_rowids.len() as u64, attachment_count: plan.attachment_rowids.len() as u64, attachment_bytes: plan.attachment_bytes, on_disk_file_count: plan.attachment_files_on_disk.len() as u64, + not_on_mac_count: count_not_on_mac(&conn, &plan)?, }) } -#[derive(Debug, Deserialize, Default)] +#[derive(Debug, Deserialize, Default, Clone, Copy)] #[serde(rename_all = "snake_case")] pub enum DeleteScope { /// Delete both messages and their attachment records/files (default). #[default] Both, - /// Delete message rows and join tables only; leave attachment records - /// and files on disk untouched. + /// No longer supported (always refused): chat.db's triggers delete an + /// attachment's record when its last message link goes, so attachments + /// can't be kept without their messages. Kept on the wire so an old + /// caller gets a clear error rather than a parse failure. MessagesOnly, /// Delete attachment records and files only; leave message rows intact /// so conversation history is preserved. @@ -83,28 +441,53 @@ pub enum DeleteScope { pub struct RunDeleteArgs { pub filter: FilterSpec, pub confirmation_phrase: String, - /// Set to `true` when the caller has already run a matching `run_backup` - /// in this session. If `false`, `acknowledge_skip_backup` must be - /// explicitly `true` — the backend no longer silently permits - /// backup-less deletes. - #[serde(default)] - pub backup_verified: bool, - /// Required when `backup_verified` is `false`. A direct-IPC caller must - /// actively opt out of the backup recommendation rather than default - /// into it. + /// Required when no backup with exactly this filter ran in this session + /// (or, when attachment files will be deleted, when that backup didn't + /// copy all of them). Whether a backup ran is checked against the + /// backend's own receipts — the frontend can't assert it. #[serde(default)] pub acknowledge_skip_backup: bool, /// Optional override for where snapshots are stored. Defaults to - /// `~/Documents/iMessage Backups/snapshots//`. + /// `/snapshots/delete//`. pub snapshot_root: Option, /// Controls which parts of the matched records are removed. #[serde(default)] pub delete_scope: DeleteScope, - /// Must be `true` when Messages in iCloud is detected as enabled. A - /// local delete on a synced account can be re-synced from cloud state, - /// so the caller has to prove they understand the consequences. + /// Must be `true` unless Messages in iCloud is positively detected as + /// disabled. #[serde(default)] pub acknowledge_icloud_sync: bool, + /// Required when attachment records will be deleted and some of their + /// files aren't on this Mac: those can't be backed up and become + /// unrecoverable. + #[serde(default)] + pub acknowledge_not_on_mac: bool, + /// Messages the user was shown and confirmed (0 when the scope keeps + /// messages). When set, the delete refuses if the plan differs — e.g. + /// the filter changed after the preview was computed. + pub expected_message_count: Option, + /// Attachments the user was shown and confirmed (0 when the scope keeps + /// attachments). Checked like `expected_message_count`. + pub expected_attachment_count: Option, +} + +/// Refuse when the scoped plan doesn't match what the user confirmed. +pub(crate) fn check_expected_counts( + plan: &DeletePlan, + expected_messages: Option, + expected_attachments: Option, +) -> Result<(), AppError> { + let messages = plan.message_rowids.len() as u64; + let attachments = plan.attachment_rowids.len() as u64; + let mismatch = expected_messages.is_some_and(|n| n != messages) + || expected_attachments.is_some_and(|n| n != attachments); + if mismatch { + return Err(AppError::Other(format!( + "What would be deleted changed since the preview (now {messages} messages and \ + {attachments} attachments). Nothing was deleted — review the preview and try again." + ))); + } + Ok(()) } #[derive(Debug, Serialize)] @@ -119,110 +502,313 @@ pub struct DeleteResult { pub snapshot_path: String, pub on_disk_files_removed: u64, pub on_disk_files_failed: u64, + /// Files left alone because they weren't regular files inside the + /// Messages attachment folders. + pub on_disk_files_skipped: u64, pub backup_verified: bool, } -/// Execute a delete. Enforces the full safety contract: typed -/// confirmation, Messages.app-not-running, mandatory snapshot with a WAL -/// checkpoint, then a transactional delete against a writable connection. -/// Attachment files on disk are unlinked **after** the SQL transaction -/// commits; individual unlink failures are counted but do not roll back. -#[tauri::command] -pub async fn run_delete(args: RunDeleteArgs) -> Result { - // Gate 1: typed confirmation. - if args.confirmation_phrase != DELETE_CONFIRMATION_PHRASE { - return Err(AppError::Other(format!( - "confirmation phrase must be exactly '{DELETE_CONFIRMATION_PHRASE}'" - ))); +/// Returns whether a matching backup exists, or errors if none does and +/// the caller didn't explicitly opt out. The snapshot only covers chat.db, +/// so when attachment files will be unlinked the backup must have copied +/// them. +pub(crate) fn check_backup_gate( + coverage: &Coverage, + scope: &DeleteScope, + acknowledge_skip_backup: bool, +) -> Result { + let deletes_files = !matches!(scope, DeleteScope::MessagesOnly); + let verified = coverage.covered && (coverage.with_attachments || !deletes_files); + if verified || acknowledge_skip_backup { + return Ok(verified); } + let msg = if coverage.covered { + "The backup of these messages didn't copy every attachment file, and this delete \ + removes them permanently (the safety snapshot only covers chat.db). Re-run the backup \ + with attachments, or pass acknowledgeSkipBackup=true to proceed anyway." + } else { + "No backup covers exactly the messages this delete removes (new messages may have \ + arrived since the last one). Run a backup first, or pass acknowledgeSkipBackup=true \ + to proceed without one." + }; + Err(AppError::Other(msg.into())) +} - // Gate 1a: backup gate. If the caller didn't back up first, they must - // explicitly acknowledge the skip. This turns the previously-advisory - // flag into a real check that survives a frontend regression or a - // direct-IPC caller. - if !args.backup_verified && !args.acknowledge_skip_backup { - return Err(AppError::Other( - "No backup has been run for this filter. Run a backup first, or pass \ - acknowledgeSkipBackup=true to proceed without one." - .into(), - )); - } +/// Whether completed backups cover what `filter` currently matches. Lets +/// the UI show the real state (including backups from earlier sessions) +/// instead of guessing from the last backup it ran. +#[tauri::command] +pub async fn backup_coverage( + receipts: State<'_, Arc>, + filter: FilterSpec, +) -> Result { + let db_path = default_chat_db_path()?; + let receipts = Arc::clone(&receipts); + blocking(move || coverage_now(&db_path, &receipts, &filter)).await +} + +pub(crate) fn coverage_now( + db_path: &std::path::Path, + receipts: &BackupReceipts, + filter: &FilterSpec, +) -> Result { + let ctx = filter.to_query_context()?; + let conn = open_read(db_path)?; + let fc = filter_content(&conn, &ctx, db_path)?; + Ok(receipts.coverage(&filter.key(), &fc.content, &fc.present)) +} - // Gate 2: Messages.app must be quit. We err on the side of refusing - // when we can't tell (pgrep unavailable → treat as "could be running"). - if is_messages_running().unwrap_or(true) { +fn ensure_messages_not_running(probe: &SafetyProbe) -> Result<(), AppError> { + if (probe.messages_running)() { return Err(AppError::Other( "Quit Messages.app before running a delete — it holds a write lock on chat.db.".into(), )); } + Ok(()) +} - // Gate 2a: Messages-in-iCloud refuses-to-proceed without explicit ack. - // When detection is Unknown we don't block (false positives would be - // worse than a soft UI warning), but Enabled is hard-gated. - if detect_icloud_messages().unwrap_or(ICloudState::Unknown) == ICloudState::Enabled - && !args.acknowledge_icloud_sync - { - return Err(AppError::Other( - "Messages in iCloud is enabled. A local delete may be re-synced from cloud. \ - Acknowledge the iCloud warning in the UI (or pass acknowledgeIcloudSync=true) \ - before retrying." - .into(), - )); +/// Everything `run_delete` works out from chat.db before deciding whether +/// to go ahead. +struct Prepared { + /// The scoped plan. + plan: DeletePlan, + scope: DeleteScope, + /// What the filter matches, and which attachment files are present, + /// for the backup gate. + content: ContentFingerprint, + present: ContentFingerprint, + /// Messages whose attachment flag may need clearing afterwards. + flag_candidates: Vec, + not_on_mac: u64, +} + +fn prepare_delete( + db_path: &std::path::Path, + ctx: &QueryContext, + scope: &DeleteScope, + (expected_messages, expected_attachments): (Option, Option), +) -> Result { + // Build the delete plan against a read-only connection — even though + // we'll execute it against a writable one, there's no reason to hold + // two writers, and the read-only path is more conservative. + let conn = open_read(db_path)?; + let BuiltPlan { + mut plan, + content, + present, + .. + } = build_plan(&conn, ctx, db_path)?; + + apply_scope(&mut plan, scope)?; + + // Gate 3: the plan must match what the user confirmed. + check_expected_counts(&plan, expected_messages, expected_attachments)?; + + let not_on_mac = count_not_on_mac(&conn, &plan)?; + let flag_candidates = if matches!(scope, DeleteScope::AttachmentsOnly) { + messages_linked_to(&conn, &plan.attachment_rowids)? + } else { + Vec::new() + }; + Ok(Prepared { + plan, + scope: *scope, + content, + present, + flag_candidates, + not_on_mac, + }) +} + +/// Execute a delete. Enforces the full safety contract: typed +/// confirmation, backup (or explicit skip), Messages.app-not-running, +/// iCloud acknowledgement, a plan matching what the user confirmed, +/// mandatory snapshot with a WAL checkpoint, then a transactional delete +/// against a writable connection. Attachment files on disk are unlinked +/// **after** the SQL transaction commits; individual unlink failures are +/// counted but do not roll back. +/// Host state a delete depends on. Injectable so end-to-end tests don't +/// depend on whether Messages happens to be running on the test machine. +pub(crate) struct SafetyProbe { + pub messages_running: fn() -> bool, + pub icloud: fn() -> ICloudState, + pub available_space: SpaceProbe, +} + +impl SafetyProbe { + pub fn system() -> Self { + Self { + // We err on the side of refusing when we can't tell (pgrep + // unavailable → treat as "could be running"). + messages_running: || is_messages_running().unwrap_or(true), + icloud: || detect_icloud_messages().unwrap_or(ICloudState::Unknown), + available_space: crate::core::space::system_available, + } } +} +#[tauri::command] +pub async fn run_delete( + app: AppHandle, + receipts: State<'_, Arc>, + lock: State<'_, OperationLock>, + args: RunDeleteArgs, +) -> Result { + let _guard = lock.try_acquire("a delete")?; let db_path = default_chat_db_path()?; - - // Gate 3: mandatory WAL checkpoint + snapshot. - let snapshot_root = match args.snapshot_root.as_deref() { - Some(p) if !p.is_empty() => crate::core::paths::validate_user_path(p, "snapshot root")?, - _ => default_snapshot_root()?, + let (snapshot_root, managed_root) = match args.snapshot_root.as_deref() { + Some(p) if !p.is_empty() => ( + crate::core::paths::validate_user_path(p, "snapshot root")?, + false, + ), + _ => (default_snapshot_root(&app, SnapshotKind::Delete)?, true), }; + let receipts = Arc::clone(&receipts); + blocking(move || { + delete_now( + &db_path, + &receipts, + args, + &snapshot_root, + managed_root, + &SafetyProbe::system(), + ) + }) + .await +} - checkpoint_wal(&db_path)?; - let snapshot = snapshot_chat_db(&db_path, &snapshot_root)?; +/// The whole delete, gates included, as a plain function (see +/// [`run_delete`]). +pub(crate) fn delete_now( + db_path: &std::path::Path, + receipts: &BackupReceipts, + args: RunDeleteArgs, + snapshot_root: &std::path::Path, + managed_root: bool, + probe: &SafetyProbe, +) -> Result { + // Gate 1: typed confirmation. + if args.confirmation_phrase != DELETE_CONFIRMATION_PHRASE { + return Err(AppError::Other(format!( + "confirmation phrase must be exactly '{DELETE_CONFIRMATION_PHRASE}'" + ))); + } + + // Gate 2: Messages.app must be quit. + ensure_messages_not_running(probe)?; + + // Gate 2a: Messages in iCloud. Detection is best-effort, so anything + // other than a positive "disabled" needs an explicit acknowledgement. + match (probe.icloud)() { + ICloudState::Disabled => {} + ICloudState::Enabled if !args.acknowledge_icloud_sync => { + return Err(AppError::Other(format!( + "Messages in iCloud is enabled. {ICLOUD_RISK} Acknowledge the iCloud warning \ + in the UI (or pass acknowledgeIcloudSync=true) before retrying." + ))); + } + ICloudState::Unknown if !args.acknowledge_icloud_sync => { + return Err(AppError::Other(format!( + "Couldn't determine whether Messages in iCloud is enabled. If it is: \ + {ICLOUD_RISK} Acknowledge the iCloud warning in the UI (or pass \ + acknowledgeIcloudSync=true) before retrying." + ))); + } + _ => {} + } - // Build the delete plan against a read-only connection — even though - // we'll execute it against a writable one, there's no reason to hold - // two writers, and the read-only path is more conservative. - let read_conn = get_connection(&db_path)?; let ctx = args.filter.to_query_context()?; - let mut plan = lib_preview_delete(&read_conn, &ctx, &AttachmentFilter::any(), &db_path) - .map_err(|e| AppError::Database(format!("failed to compute delete plan: {e}")))?; - drop(read_conn); + let filter_key = args.filter.key(); + let scope = args.delete_scope; + let expected = (args.expected_message_count, args.expected_attachment_count); + let prepared = prepare_delete(db_path, &ctx, &scope, expected)?; - apply_scope(&mut plan, &args.delete_scope); + // Gate 3b: backup gate, against the exact content being deleted — a + // backup of the same filter taken before newer messages arrived doesn't + // cover them. + let coverage = receipts.coverage(&filter_key, &prepared.content, &prepared.present); + let backup_verified = + check_backup_gate(&coverage, &prepared.scope, args.acknowledge_skip_backup)?; - // Now execute against a writable connection. - let mut write_conn = get_writable_connection(&db_path)?; - let report = match execute_delete(&mut write_conn, &plan) { - Ok(r) => r, - Err(e) => { - return Err(AppError::Database(format!( + // Gate 3c: attachments that exist only in iCloud can't be backed up + // and become unrecoverable once their records are gone. + if prepared.not_on_mac > 0 && !args.acknowledge_not_on_mac { + return Err(AppError::Other(format!( + "{} attachments in this delete aren't downloaded to this Mac, so no backup can \ + include them and they can't be recovered afterwards. Download them in Messages \ + first, or pass acknowledgeNotOnMac=true to delete them anyway.", + prepared.not_on_mac + ))); + } + + let Prepared { + plan, + flag_candidates, + .. + } = prepared; + + // Gate 4: mandatory WAL checkpoint + snapshot — refused up front if + // the snapshot folder's volume can't hold it. + ensure_space( + snapshot_root, + snapshot_size(db_path), + "the safety snapshot", + probe.available_space, + )?; + checkpoint_wal(db_path)?; + let snapshot = snapshot_chat_db(db_path, snapshot_root)?; + + // Everything that can still fail before the delete commits. If any of + // it does, nothing changed, so the snapshot just taken is redundant: + // remove it rather than let it push an older (real) rollback out. + let committed = (|| { + // Re-check right before writing: the snapshot of a large chat.db + // can take long enough for the user to reopen Messages. + ensure_messages_not_running(probe)?; + let mut write_conn = open_write(db_path)?; + let report = execute_delete(&mut write_conn, &plan).map_err(|e| { + AppError::Database(format!( "delete transaction failed (rolled back; no changes made): {e}" - ))); + )) + })?; + Ok::<_, AppError>((write_conn, report)) + })(); + let (mut write_conn, report) = match committed { + Ok(c) => c, + Err(e) => { + if let Err(re) = std::fs::remove_dir_all(&snapshot.dir) { + eprintln!("delete: couldn't remove unused snapshot: {re}"); + } + return Err(e); } }; - // Release the writable handle immediately so Messages.app can reopen. + if managed_root { + // Only after a successful delete, and only in the folder we own; a + // caller-chosen root is theirs. + prune_snapshots(snapshot_root, SNAPSHOTS_TO_KEEP, &snapshot.dir); + } + if let Err(e) = clear_stale_attachment_flags(&mut write_conn, &flag_candidates) { + eprintln!("delete: couldn't clear attachment flags on kept messages: {e:?}"); + } + // Release the writable handle immediately so Messages.app can + // reopen. drop(write_conn); - // Unlink attachment files from disk. Failures are non-fatal — the DB is - // the source of truth, so a stale file on disk is a minor janitor issue, - // not a data-integrity event. Still, we emit the path + reason to stderr - // so the user has something to grep after the fact. + // Unlink attachment files from disk. Failures are non-fatal — the + // DB is the source of truth, so a stale file on disk is a minor + // janitor issue, not a data-integrity event. Paths come from + // chat.db, so only regular files inside the Messages attachment + // folders are ever removed. + let roots = attachment_roots(); let mut removed = 0u64; let mut failed = 0u64; + let mut skipped = 0u64; for p in &plan.attachment_files_on_disk { - if p.exists() { - match fs::remove_file(p) { - Ok(()) => removed += 1, - Err(e) => { - eprintln!( - "delete: failed to remove attachment file {}: {e}", - p.display() - ); - failed += 1; - } - } + match remove_attachment_file(p, &roots, "delete") { + RemoveOutcome::Removed => removed += 1, + RemoveOutcome::Missing => {} + RemoveOutcome::Refused => skipped += 1, + RemoveOutcome::Failed => failed += 1, } } @@ -236,7 +822,8 @@ pub async fn run_delete(args: RunDeleteArgs) -> Result { snapshot_path: snapshot.dir.display().to_string(), on_disk_files_removed: removed, on_disk_files_failed: failed, - backup_verified: args.backup_verified, + on_disk_files_skipped: skipped, + backup_verified, }) } @@ -263,7 +850,9 @@ mod tests { service TEXT, handle_id INTEGER, date INTEGER NOT NULL, - is_from_me INTEGER DEFAULT 0 + is_from_me INTEGER DEFAULT 0, + ck_record_id TEXT, + balloon_bundle_id TEXT ); CREATE TABLE chat ( ROWID INTEGER PRIMARY KEY AUTOINCREMENT, @@ -284,7 +873,9 @@ mod tests { transfer_name TEXT, uti TEXT, is_sticker INTEGER DEFAULT 0, - hide_attachment INTEGER DEFAULT 0 + hide_attachment INTEGER DEFAULT 0, + guid TEXT, + ck_record_id TEXT ); CREATE TABLE chat_message_join ( chat_id INTEGER NOT NULL, @@ -292,15 +883,48 @@ mod tests { PRIMARY KEY (chat_id, message_id) ); CREATE TABLE message_attachment_join ( - message_id INTEGER NOT NULL, - attachment_id INTEGER NOT NULL, - PRIMARY KEY (message_id, attachment_id) + message_id INTEGER REFERENCES message (ROWID) ON DELETE CASCADE, + attachment_id INTEGER REFERENCES attachment (ROWID) ON DELETE CASCADE, + UNIQUE (message_id, attachment_id) ); CREATE TABLE chat_handle_join ( chat_id INTEGER NOT NULL, handle_id INTEGER NOT NULL, PRIMARY KEY (chat_id, handle_id) - );", + ); + -- Apple's own bookkeeping and triggers (copied from a real + -- chat.db). They change what a delete actually does, so tests + -- must run with them. + CREATE TABLE sync_deleted_messages (guid TEXT, recordID TEXT); + CREATE TABLE sync_deleted_attachments (guid TEXT, recordID TEXT); + CREATE TABLE deleted_messages (guid TEXT); + CREATE TRIGGER after_delete_on_message_attachment_join + AFTER DELETE ON message_attachment_join BEGIN + DELETE FROM attachment WHERE attachment.ROWID = OLD.attachment_id + AND (SELECT 1 FROM message_attachment_join + WHERE attachment_id = OLD.attachment_id LIMIT 1) IS NULL; + END; + CREATE TRIGGER before_delete_on_attachment BEFORE DELETE ON attachment BEGIN + SELECT before_delete_attachment_path(OLD.ROWID, OLD.guid); + END; + CREATE TRIGGER after_delete_on_attachment AFTER DELETE ON attachment BEGIN + SELECT delete_attachment_path(OLD.filename); + END; + CREATE TRIGGER add_to_sync_deleted_attachments AFTER DELETE ON attachment BEGIN + INSERT INTO sync_deleted_attachments (guid, recordID) + VALUES (OLD.guid, OLD.ck_record_id); + END; + CREATE TRIGGER add_to_sync_deleted_messages AFTER DELETE ON message BEGIN + INSERT INTO sync_deleted_messages (guid, recordID) + VALUES (OLD.guid, OLD.ck_record_id); + END; + CREATE TRIGGER add_to_deleted_messages AFTER DELETE ON message BEGIN + INSERT INTO deleted_messages (guid) VALUES (OLD.guid); + END; + CREATE TRIGGER after_delete_on_message_plugin AFTER DELETE ON message + WHEN OLD.balloon_bundle_id IS NOT NULL BEGIN + SELECT after_delete_message_plugin(OLD.ROWID, OLD.guid); + END;", ) .unwrap(); } @@ -336,6 +960,12 @@ mod tests { att_path } + /// How the app opens chat.db for writing: foreign keys on and Apple's + /// trigger functions stubbed, so cascades and triggers behave for real. + fn writable(db_path: &std::path::Path) -> Connection { + crate::core::db_path::open_write(db_path).unwrap() + } + fn fresh_fixture() -> (TempDir, PathBuf, PathBuf) { let tmp = TempDir::new().unwrap(); let db_path = tmp.path().join("chat.db"); @@ -347,19 +977,14 @@ mod tests { } #[test] - fn apply_scope_messages_only_keeps_attachments() { + fn messages_only_scope_is_refused() { let mut plan = DeletePlan { message_rowids: vec![1, 2], attachment_rowids: vec![10], - attachment_files_on_disk: vec![PathBuf::from("/tmp/x.jpg")], - attachment_bytes: 1024, - cleanup_orphans: true, + ..Default::default() }; - apply_scope(&mut plan, &DeleteScope::MessagesOnly); - assert_eq!(plan.message_rowids, vec![1, 2]); - assert!(plan.attachment_rowids.is_empty()); - assert!(plan.attachment_files_on_disk.is_empty()); - assert_eq!(plan.attachment_bytes, 0); + let err = apply_scope(&mut plan, &DeleteScope::MessagesOnly).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("isn't possible"))); } #[test] @@ -371,7 +996,7 @@ mod tests { attachment_bytes: 1024, cleanup_orphans: true, }; - apply_scope(&mut plan, &DeleteScope::AttachmentsOnly); + apply_scope(&mut plan, &DeleteScope::AttachmentsOnly).unwrap(); assert!(plan.message_rowids.is_empty()); assert_eq!(plan.attachment_rowids, vec![10]); assert_eq!(plan.attachment_files_on_disk.len(), 1); @@ -388,16 +1013,306 @@ mod tests { cleanup_orphans: true, }; let mut plan = original.clone(); - apply_scope(&mut plan, &DeleteScope::Both); + apply_scope(&mut plan, &DeleteScope::Both).unwrap(); assert_eq!(plan.message_rowids, original.message_rowids); assert_eq!(plan.attachment_rowids, original.attachment_rowids); assert_eq!(plan.attachment_bytes, original.attachment_bytes); } + #[test] + fn backup_gate_requires_matching_backup_with_attachments_when_deleting_files() { + let none = Coverage::default(); + let refs_only = Coverage { + covered: true, + with_attachments: false, + export_path: None, + }; + let full = Coverage { + covered: true, + with_attachments: true, + export_path: None, + }; + + assert!(check_backup_gate(&none, &DeleteScope::Both, false).is_err()); + assert!(!check_backup_gate(&none, &DeleteScope::Both, true).unwrap()); + + // A references-only backup covers a messages-only delete… + assert!(check_backup_gate(&refs_only, &DeleteScope::MessagesOnly, false).unwrap()); + // …but not one that unlinks attachment files. + let err = check_backup_gate(&refs_only, &DeleteScope::AttachmentsOnly, false).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("didn't copy every attachment"))); + + assert!(check_backup_gate(&full, &DeleteScope::Both, false).unwrap()); + } + + #[test] + fn backup_stops_covering_once_new_messages_match_the_filter() { + let (_tmp, db_path, _att) = fresh_fixture(); + let conn = writable(&db_path); + let backup_dir = tempfile::TempDir::new().unwrap(); + let receipts = BackupReceipts::default(); + let filter = FilterSpec::default(); + let ctx = filter.to_query_context().unwrap(); + + // Back up: attachment 1's file is on the Mac and gets copied. + let fc = filter_content(&conn, &ctx, &db_path).unwrap(); + let copied = ContentFingerprint::of(&[], &[1]); + receipts.record(filter.key(), fc.content, copied, true, backup_dir.path()); + let p = build_plan(&conn, &ctx, &db_path).unwrap(); + assert!( + receipts + .coverage(&filter.key(), &p.content, &p.present) + .with_attachments + ); + + // A message arrives after the backup; same filter, new content. + conn.execute_batch( + "INSERT INTO message (ROWID, guid, date) VALUES (3, 'g3', 0); + INSERT INTO chat_message_join VALUES (1, 3);", + ) + .unwrap(); + let p = build_plan(&conn, &ctx, &db_path).unwrap(); + assert!( + !receipts + .coverage(&filter.key(), &p.content, &p.present) + .covered + ); + } + + #[test] + fn attachment_downloaded_after_backup_is_not_covered() { + let (tmp, db_path, _att) = fresh_fixture(); + let conn = writable(&db_path); + let offloaded = tmp.path().join("offloaded.jpg"); + conn.execute( + "INSERT INTO attachment (ROWID, filename) VALUES (2, ?1)", + params![offloaded.to_str().unwrap()], + ) + .unwrap(); + conn.execute("INSERT INTO message_attachment_join VALUES (1, 2)", []) + .unwrap(); + let backup_dir = tempfile::TempDir::new().unwrap(); + let receipts = BackupReceipts::default(); + let filter = FilterSpec::default(); + let ctx = filter.to_query_context().unwrap(); + + // Backup copies attachment 1; attachment 2 is still in iCloud. + let fc = filter_content(&conn, &ctx, &db_path).unwrap(); + receipts.record( + filter.key(), + fc.content, + ContentFingerprint::of(&[], &[1]), + true, + backup_dir.path(), + ); + let p = build_plan(&conn, &ctx, &db_path).unwrap(); + assert!( + receipts + .coverage(&filter.key(), &p.content, &p.present) + .with_attachments + ); + + // The user downloads it; the backup doesn't have it. + std::fs::write(&offloaded, b"now here").unwrap(); + let p = build_plan(&conn, &ctx, &db_path).unwrap(); + let c = receipts.coverage(&filter.key(), &p.content, &p.present); + assert!(c.covered && !c.with_attachments); + } + + #[test] + fn late_tapback_on_an_in_range_message_breaks_coverage() { + let (_tmp, db_path, _att) = fresh_fixture(); + let conn = writable(&db_path); + conn.execute_batch( + "ALTER TABLE message ADD COLUMN associated_message_guid TEXT; + ALTER TABLE message ADD COLUMN associated_message_type INTEGER DEFAULT 0; + ALTER TABLE message ADD COLUMN cache_has_attachments INTEGER DEFAULT 0; + INSERT INTO chat (ROWID, chat_identifier) VALUES (2, 'other');", + ) + .unwrap(); + let mut ctx = QueryContext::default(); + ctx.set_selected_chat_ids(std::collections::BTreeSet::from([1])); + let at_backup = filter_content(&conn, &ctx, &db_path).unwrap().content; + + // Someone reacts later, from a chat outside the filter. + conn.execute_batch( + "INSERT INTO message (ROWID, guid, date, associated_message_guid, associated_message_type) + VALUES (10, 'r1', 0, 'p:0/g1', 2000); + INSERT INTO chat_message_join VALUES (2, 10);", + ) + .unwrap(); + let now = filter_content(&conn, &ctx, &db_path).unwrap(); + assert_eq!(now.reaction_ids, vec![10]); + assert_ne!(now.content, at_backup); + } + + #[test] + fn shared_attachments_and_files_are_excluded_from_plan() { + let (tmp, db_path, att1) = fresh_fixture(); + let conn = writable(&db_path); + // Message 3 lives in another chat and is NOT being deleted. + let shared_file = tmp.path().join("shared.jpg"); + std::fs::write(&shared_file, b"x").unwrap(); + conn.execute_batch( + "INSERT INTO chat (ROWID, chat_identifier) VALUES (2, 'other'); + INSERT INTO message (ROWID, guid, handle_id, date) VALUES (3, 'g3', 1, 0); + INSERT INTO chat_message_join VALUES (2, 3); + -- Attachment 2 is linked from a deleted message and a kept one. + INSERT INTO attachment (ROWID, filename, total_bytes) VALUES (2, '/nowhere/a2', 100); + INSERT INTO message_attachment_join VALUES (1, 2), (3, 2);", + ) + .unwrap(); + // Attachments 3 (deleted message) and 4 (kept message) share a file. + conn.execute( + "INSERT INTO attachment (ROWID, filename, total_bytes) VALUES (3, ?1, 7), (4, ?1, 7)", + params![shared_file.to_str().unwrap()], + ) + .unwrap(); + conn.execute_batch("INSERT INTO message_attachment_join VALUES (1, 3), (3, 4);") + .unwrap(); + + let mut ctx = QueryContext::default(); + ctx.set_selected_chat_ids(std::collections::BTreeSet::from([1])); + let mut plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &db_path).unwrap(); + let mut planned = plan.attachment_rowids.clone(); + planned.sort(); + assert_eq!(planned, vec![1, 2, 3]); + + exclude_shared_attachments(&conn, &mut plan).unwrap(); + let mut kept = plan.attachment_rowids.clone(); + kept.sort(); + // Attachment 2 is still linked to message 3, so it stays. + assert_eq!(kept, vec![1, 3]); + assert_eq!(plan.attachment_bytes, 15 + 7); + // Row 3 goes, but its file is still used by row 4. + assert!(plan.attachment_files_on_disk.contains(&att1)); + assert!(!plan.attachment_files_on_disk.contains(&shared_file)); + } + + #[test] + fn attachments_only_delete_clears_stale_flags() { + let (_tmp, db_path, _att) = fresh_fixture(); + let mut conn = writable(&db_path); + conn.execute_batch( + "ALTER TABLE message ADD COLUMN cache_has_attachments INTEGER DEFAULT 0; + UPDATE message SET cache_has_attachments = 1 WHERE ROWID = 2; + -- Message 1 has two attachments; only one is deleted. + INSERT INTO attachment (ROWID, filename) VALUES (2, '/x/a'), (3, '/x/b'); + INSERT INTO message_attachment_join VALUES (1, 2), (1, 3); + UPDATE message SET cache_has_attachments = 1 WHERE ROWID = 1;", + ) + .unwrap(); + let plan = DeletePlan { + attachment_rowids: vec![1, 2], + ..Default::default() + }; + let candidates = messages_linked_to(&conn, &plan.attachment_rowids).unwrap(); + assert_eq!(candidates, vec![1, 2]); + execute_delete(&mut conn, &plan).unwrap(); + + assert_eq!( + clear_stale_attachment_flags(&mut conn, &candidates).unwrap(), + 1 + ); + let flag = |id: i64| -> i64 { + conn.query_row( + "SELECT cache_has_attachments FROM message WHERE ROWID = ?1", + [id], + |r| r.get(0), + ) + .unwrap() + }; + assert_eq!(flag(2), 0, "lost its only attachment"); + assert_eq!(flag(1), 1, "still has attachment 3"); + } + + #[test] + fn reaction_targets_are_parsed() { + assert_eq!(reaction_target("p:0/ABC-123"), "ABC-123"); + assert_eq!(reaction_target("p:12/ABC"), "ABC"); + assert_eq!(reaction_target("bp:ABC"), "ABC"); + assert_eq!(reaction_target("ABC"), "ABC"); + } + + #[test] + fn tapbacks_on_deleted_messages_are_deleted_too() { + let (_tmp, db_path, _att) = fresh_fixture(); + let conn = writable(&db_path); + conn.execute_batch( + "ALTER TABLE message ADD COLUMN associated_message_guid TEXT; + ALTER TABLE message ADD COLUMN associated_message_type INTEGER DEFAULT 0; + ALTER TABLE message ADD COLUMN cache_has_attachments INTEGER DEFAULT 0; + INSERT INTO chat (ROWID, chat_identifier) VALUES (2, 'other'); + -- Tapback in another chat on message g1 (being deleted): goes. + INSERT INTO message (ROWID, guid, date, associated_message_guid, associated_message_type) + VALUES (10, 'r1', 0, 'p:0/g1', 2000); + -- Sticker reaction with an attachment on g1: stays. + INSERT INTO message (ROWID, guid, date, associated_message_guid, associated_message_type, cache_has_attachments) + VALUES (11, 'r2', 0, 'p:0/g1', 2007, 1); + -- Tapback on a message that is kept: stays. + INSERT INTO message (ROWID, guid, date) VALUES (12, 'kept', 0); + INSERT INTO message (ROWID, guid, date, associated_message_guid, associated_message_type) + VALUES (13, 'r3', 0, 'p:0/kept', 2001); + -- A reply (not a reaction) to g1: stays. + INSERT INTO message (ROWID, guid, date, associated_message_guid, associated_message_type) + VALUES (14, 'r4', 0, 'g1', 0); + INSERT INTO chat_message_join VALUES (2, 10), (2, 11), (2, 12), (2, 13), (2, 14);", + ) + .unwrap(); + + let mut ctx = QueryContext::default(); + ctx.set_selected_chat_ids(std::collections::BTreeSet::from([1])); + let BuiltPlan { + plan, reactions, .. + } = build_plan(&conn, &ctx, &db_path).unwrap(); + assert_eq!(plan.message_rowids, vec![1, 2, 10]); + assert_eq!(reactions, 1); + } + + #[test] + fn reactions_are_skipped_on_schemas_without_them() { + let (_tmp, db_path, _att) = fresh_fixture(); + let conn = writable(&db_path); + let plan = build_plan(&conn, &QueryContext::default(), &db_path) + .unwrap() + .plan; + assert_eq!(plan.message_rowids, vec![1, 2]); + } + + #[test] + fn not_on_mac_counts_missing_files_only() { + let (_tmp, db_path, _att) = fresh_fixture(); + let conn = writable(&db_path); + conn.execute_batch( + "INSERT INTO attachment (ROWID, filename) VALUES (2, '/nowhere/offloaded.jpg'), (3, NULL); + INSERT INTO message_attachment_join VALUES (1, 2), (1, 3);", + ) + .unwrap(); + let plan = build_plan(&conn, &QueryContext::default(), &db_path) + .unwrap() + .plan; + // Row 1's file exists, row 2's doesn't, row 3 has no file at all. + assert_eq!(count_not_on_mac(&conn, &plan).unwrap(), 1); + } + + #[test] + fn expected_counts_must_match_scoped_plan() { + let plan = DeletePlan { + message_rowids: vec![1, 2], + attachment_rowids: vec![10], + ..Default::default() + }; + assert!(check_expected_counts(&plan, None, None).is_ok()); + assert!(check_expected_counts(&plan, Some(2), Some(1)).is_ok()); + let err = check_expected_counts(&plan, Some(3), Some(1)).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("changed since the preview"))); + assert!(check_expected_counts(&plan, Some(2), Some(0)).is_err()); + } + #[test] fn preview_on_fixture_counts_all_messages() { let (_tmp, db_path, _att) = fresh_fixture(); - let conn = Connection::open(&db_path).unwrap(); + let conn = writable(&db_path); let ctx = QueryContext::default(); let plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &db_path).unwrap(); assert_eq!(plan.message_rowids.len(), 2); @@ -410,7 +1325,7 @@ mod tests { let (tmp, db_path, att) = fresh_fixture(); assert!(att.exists()); - let mut conn = Connection::open(&db_path).unwrap(); + let mut conn = writable(&db_path); let ctx = QueryContext::default(); let plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &db_path).unwrap(); let report = execute_delete(&mut conn, &plan).unwrap(); @@ -437,27 +1352,27 @@ mod tests { } #[test] - fn scoped_messages_only_delete_leaves_attachment_rows_and_file_intact() { - let (tmp, db_path, att) = fresh_fixture(); - let mut conn = Connection::open(&db_path).unwrap(); - let ctx = QueryContext::default(); - let mut plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &db_path).unwrap(); - apply_scope(&mut plan, &DeleteScope::MessagesOnly); + fn deleting_messages_takes_their_attachment_records_with_them() { + // Why "messages only" was removed: with chat.db's real triggers, + // removing a message's last link deletes the attachment record. + let (_tmp, db_path, _att) = fresh_fixture(); + let mut conn = writable(&db_path); + let plan = DeletePlan { + message_rowids: vec![1, 2], + ..Default::default() + }; execute_delete(&mut conn, &plan).unwrap(); - - // Messages gone… - let n_msg: i64 = conn - .query_row("SELECT COUNT(*) FROM message", [], |r| r.get(0)) - .unwrap(); - assert_eq!(n_msg, 0); - // …but attachment row and file survive. let n_att: i64 = conn .query_row("SELECT COUNT(*) FROM attachment", [], |r| r.get(0)) .unwrap(); - assert_eq!(n_att, 1); - assert!(att.exists()); - - drop(tmp); + assert_eq!(n_att, 0); + // …and every deletion is queued for iCloud sync. + let synced: i64 = conn + .query_row("SELECT COUNT(*) FROM sync_deleted_messages", [], |r| { + r.get(0) + }) + .unwrap(); + assert_eq!(synced, 2); } #[test] @@ -468,7 +1383,7 @@ mod tests { let snap = crate::core::snapshot::snapshot_chat_db(&db_path, &snap_root).unwrap(); // Execute the delete on the source db. - let mut conn = Connection::open(&db_path).unwrap(); + let mut conn = writable(&db_path); let ctx = QueryContext::default(); let plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &db_path).unwrap(); execute_delete(&mut conn, &plan).unwrap(); diff --git a/src-tauri/src/commands/discover.rs b/src-tauri/src/commands/discover.rs index 10f45a3..7edf9f1 100644 --- a/src-tauri/src/commands/discover.rs +++ b/src-tauri/src/commands/discover.rs @@ -1,4 +1,4 @@ -use crate::core::db_path::default_chat_db_path; +use crate::core::db_path::{blocking, check_chat_db_access, default_chat_db_path, open_read}; use crate::error::AppError; use imessage_database::tables::{ chat::Chat, @@ -6,6 +6,7 @@ use imessage_database::tables::{ table::{get_connection, Cacheable}, }; use serde::Serialize; +use std::collections::HashMap; #[derive(Debug, Serialize)] #[serde(rename_all = "camelCase")] @@ -20,16 +21,13 @@ pub struct ProbeResult { /// can't open it — the typical signal that the user hasn't granted FDA yet. #[tauri::command] pub async fn probe_db() -> Result { - let path = default_chat_db_path()?; - - if !path.exists() { - return Err(AppError::Other(format!( - "chat.db not found at {}", - path.display() - ))); - } + blocking(probe_db_blocking).await +} +fn probe_db_blocking() -> Result { + let path = default_chat_db_path()?; let path_str = path.display().to_string(); + check_chat_db_access(&path)?; let conn = match get_connection(&path) { Ok(c) => c, @@ -63,6 +61,51 @@ pub struct ChatSummary { pub participant_count: usize, pub participant_handles: Vec, pub message_count: i64, + /// Every chat row that makes up this conversation. Messages often keeps + /// separate rows for the same conversation (iMessage vs SMS/RCS, or + /// after re-registration); filtering by just one would back up or + /// delete only part of the history. + pub chat_ids: Vec, +} + +/// Fold chat rows that share a `chat_identifier` (case-insensitively) into +/// one conversation. The row with the most messages provides the `rowid` +/// and labels; counts, participants and services are combined. +pub(crate) fn merge_chats(rows: Vec) -> Vec { + let mut sorted = rows; + sorted.sort_by_key(|s| std::cmp::Reverse(s.message_count)); + + let mut by_key: HashMap = HashMap::new(); + let mut out: Vec = Vec::new(); + for row in sorted { + let key = row.chat_identifier.to_lowercase(); + let Some(&i) = by_key.get(&key).filter(|_| !key.is_empty()) else { + by_key.insert(key, out.len()); + out.push(row); + continue; + }; + let merged = &mut out[i]; + merged.chat_ids.extend(row.chat_ids); + merged.message_count += row.message_count; + merged.display_name = merged.display_name.take().or(row.display_name); + merged.contact_name = merged.contact_name.take().or(row.contact_name); + merged.service_name = match (merged.service_name.take(), row.service_name) { + (Some(a), Some(b)) if !a.split(", ").any(|s| s == b) => Some(format!("{a}, {b}")), + (a, b) => a.or(b), + }; + merged.participant_handles.extend(row.participant_handles); + merged.participant_handles.sort(); + merged.participant_handles.dedup(); + merged.participant_count = merged + .participant_count + .max(row.participant_count) + .max(merged.participant_handles.len()); + } + for c in &mut out { + c.chat_ids.sort_unstable(); + } + out.sort_by_key(|s| std::cmp::Reverse(s.message_count)); + out } /// List all conversations in chat.db, sorted by message count (descending) so @@ -75,9 +118,17 @@ pub async fn list_chats() -> Result, AppError> { let contact_map = tokio::task::spawn_blocking(crate::core::contacts::fetch_contact_map) .await .map_err(|e| AppError::Other(format!("contact lookup task failed: {e}")))?; + #[cfg(not(target_os = "macos"))] + let contact_map = crate::core::contacts::ContactMap::default(); + blocking(move || list_chats_blocking(&contact_map)).await +} + +fn list_chats_blocking( + contact_map: &crate::core::contacts::ContactMap, +) -> Result, AppError> { let path = default_chat_db_path()?; - let conn = get_connection(&path)?; + let conn = open_read(&path)?; let chats = Chat::cache(&conn)?; @@ -137,14 +188,10 @@ pub async fn list_chats() -> Result, AppError> { } } - let mut summaries: Vec = chats + let summaries: Vec = chats .into_iter() .map(|(rowid, c)| { - #[cfg(target_os = "macos")] - let contact_name = - crate::core::contacts::lookup_contact_name(&c.chat_identifier, &contact_map); - #[cfg(not(target_os = "macos"))] - let contact_name: Option = None; + let contact_name = contact_map.lookup(&c.chat_identifier); ChatSummary { rowid, @@ -155,12 +202,12 @@ pub async fn list_chats() -> Result, AppError> { participant_count: participant_counts.get(&rowid).copied().unwrap_or(0), participant_handles: participant_handles.remove(&rowid).unwrap_or_default(), message_count: message_counts.get(&rowid).copied().unwrap_or(0), + chat_ids: vec![rowid], } }) .collect(); - summaries.sort_by_key(|s| std::cmp::Reverse(s.message_count)); - Ok(summaries) + Ok(merge_chats(summaries)) } #[derive(Debug, Serialize)] @@ -177,8 +224,12 @@ pub struct ContactSummary { /// given contact and the library will resolve the full group. #[tauri::command] pub async fn list_contacts() -> Result, AppError> { + blocking(list_contacts_blocking).await +} + +fn list_contacts_blocking() -> Result, AppError> { let path = default_chat_db_path()?; - let conn = get_connection(&path)?; + let conn = open_read(&path)?; let handles = Handle::cache(&conn)?; let mut contacts: Vec = handles @@ -190,3 +241,45 @@ pub async fn list_contacts() -> Result, AppError> { contacts.sort_by(|a, b| a.id.cmp(&b.id)); Ok(contacts) } + +#[cfg(test)] +mod tests { + use super::*; + + fn chat(rowid: i32, id: &str, service: &str, count: i64, handles: &[&str]) -> ChatSummary { + ChatSummary { + rowid, + chat_identifier: id.into(), + display_name: None, + contact_name: None, + service_name: Some(service.into()), + participant_count: handles.len(), + participant_handles: handles.iter().map(|h| h.to_string()).collect(), + message_count: count, + chat_ids: vec![rowid], + } + } + + #[test] + fn merges_rows_for_the_same_conversation() { + let merged = merge_chats(vec![ + chat(1, "+15551234567", "SMS", 10, &["+15551234567"]), + chat( + 2, + "other@example.com", + "iMessage", + 5, + &["other@example.com"], + ), + chat(3, "+15551234567", "iMessage", 40, &["+15551234567"]), + ]); + assert_eq!(merged.len(), 2); + let first = &merged[0]; + assert_eq!(first.rowid, 3, "row with most messages is the primary"); + assert_eq!(first.chat_ids, vec![1, 3]); + assert_eq!(first.message_count, 50); + assert_eq!(first.service_name.as_deref(), Some("iMessage, SMS")); + assert_eq!(first.participant_handles, vec!["+15551234567"]); + assert_eq!(merged[1].chat_ids, vec![2]); + } +} diff --git a/src-tauri/src/commands/fda.rs b/src-tauri/src/commands/fda.rs index 6d9db77..4343a14 100644 --- a/src-tauri/src/commands/fda.rs +++ b/src-tauri/src/commands/fda.rs @@ -21,8 +21,20 @@ pub async fn check_fda() -> Result { let path = default_chat_db_path()?; let path_str = path.display().to_string(); - if !path.exists() { - return Err(AppError::Other(format!("chat.db not found at {path_str}"))); + // Don't use `path.exists()`: without FDA, TCC refuses even to stat files + // in ~/Library/Messages, and `exists()` reports that as "not found". + match std::fs::metadata(&path) { + Ok(_) => {} + Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => { + return Ok(FdaStatus { + granted: false, + path: path_str, + }); + } + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + return Err(AppError::Other(format!("chat.db not found at {path_str}"))); + } + Err(e) => return Err(AppError::from(e)), } match File::open(&path) { diff --git a/src-tauri/src/commands/mod.rs b/src-tauri/src/commands/mod.rs index 156a86b..c8b8f8d 100644 --- a/src-tauri/src/commands/mod.rs +++ b/src-tauri/src/commands/mod.rs @@ -4,4 +4,5 @@ pub mod discover; pub mod fda; pub mod orphans; pub mod preview; +pub mod restore; pub mod safety; diff --git a/src-tauri/src/commands/orphans.rs b/src-tauri/src/commands/orphans.rs index 5d0f063..b3a1b36 100644 --- a/src-tauri/src/commands/orphans.rs +++ b/src-tauri/src/commands/orphans.rs @@ -1,17 +1,31 @@ -use std::collections::HashSet; -use std::fs; -use std::path::PathBuf; +use std::collections::{BTreeSet, HashSet}; +use std::hash::{DefaultHasher, Hash, Hasher}; +use std::path::{Path, PathBuf}; +use std::time::{Duration, SystemTime}; -use imessage_database::tables::table::{get_connection, get_writable_connection}; -use serde::Serialize; +use rusqlite::Connection; +use serde::{Deserialize, Serialize}; +use tauri::{AppHandle, State}; -use crate::core::db_path::default_chat_db_path; +use crate::commands::delete::SafetyProbe; +use crate::core::attachment_files::{ + attachment_roots, canonical_roots, expand_tilde, is_within_roots, path_key, + remove_attachment_file, walk_files, RemoveOutcome, +}; +use crate::core::db_path::{blocking, default_chat_db_path, open_read, open_write}; +use crate::core::icloud::ICloudState; +use crate::core::op_lock::OperationLock; +use crate::core::retained::RetainedAttachments; +use crate::core::snapshot::{ + checkpoint_wal, default_snapshot_root, prune_snapshots, snapshot_chat_db, SnapshotKind, + SNAPSHOTS_TO_KEEP, +}; use crate::error::AppError; -const ATTACHMENT_DIRS: &[&str] = &[ - "~/Library/Messages/Attachments", - "~/Library/Messages/StickerCache", -]; +/// Files newer than this are never treated as filesystem orphans: Messages +/// can write an attachment's file before the row that references it is +/// committed (e.g. an in-flight download). +const MIN_FS_ORPHAN_AGE: Duration = Duration::from_secs(24 * 60 * 60); #[derive(Debug, Serialize)] #[serde(rename_all = "camelCase")] @@ -20,6 +34,30 @@ pub struct OrphanScan { pub db_orphan_bytes: u64, pub fs_orphan_count: u64, pub fs_orphan_bytes: u64, + /// Unlinked attachments the user kept via a messages-only delete. Not + /// included in the counts above unless the scan asked for them. + pub retained_count: u64, + pub retained_bytes: u64, + /// Identifies the exact set of orphans found. `clean_orphans` refuses to + /// run unless it finds the same set, so it can only ever delete what the + /// user reviewed. + pub fingerprint: String, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct CleanOrphansArgs { + /// `fingerprint` from the `scan_orphans` result the user confirmed. + pub scan_fingerprint: String, + /// Also remove attachments kept by earlier messages-only deletes. Must + /// match the `include_retained` the scan ran with. + #[serde(default)] + pub include_retained: bool, + /// Required (when records will be deleted) unless Messages in iCloud + /// is positively detected as disabled: chat.db's triggers queue every + /// deleted attachment record for iCloud deletion. + #[serde(default)] + pub acknowledge_icloud_sync: bool, } #[derive(Debug, Serialize)] @@ -30,212 +68,474 @@ pub struct OrphanCleanResult { pub db_files_failed: u64, pub fs_files_removed: u64, pub fs_files_failed: u64, + /// Files left alone because they weren't regular files inside the + /// Messages attachment folders. + pub files_skipped: u64, + /// Snapshot of chat.db taken before deleting rows; `None` when no rows + /// were deleted. + pub snapshot_path: Option, +} + +#[derive(Debug)] +struct DbOrphan { + rowid: i64, + /// File to unlink with the row. `None` when the row has no file, the + /// file is shared with a row that's being kept, or it lies outside the + /// Messages attachment folders. + path: Option, +} + +#[derive(Debug, Default)] +struct Orphans { + db: Vec, + db_bytes: u64, + fs: Vec, + fs_bytes: u64, + /// Retained (kept) unlinked attachments left out of `db`. + retained_count: u64, + retained_bytes: u64, + /// Retained ids included in `db` because the caller asked for them. + included_retained: Vec, + /// Retained ids whose attachment record no longer exists and can be + /// dropped from the list. + stale_retained: Vec, +} + +impl Orphans { + fn fingerprint(&self) -> String { + let mut h = DefaultHasher::new(); + for o in &self.db { + o.rowid.hash(&mut h); + o.path.hash(&mut h); + } + self.fs.hash(&mut h); + format!("{}:{}:{:016x}", self.db.len(), self.fs.len(), h.finish()) + } } -/// Expand a leading `~` to the user's home directory, matching the library's -/// own `gen_macos_attachment` behaviour. -fn expand_tilde(path: &str) -> PathBuf { - if path.starts_with('~') { - if let Ok(home) = std::env::var("HOME") { - return PathBuf::from(path.replacen('~', &home, 1)); +/// `true` if `path` sits in a directory (below `root`) that also holds a +/// file referenced by chat.db. Messages keeps companion files next to the +/// referenced one (Live Photo videos, previews, …) that have no row of +/// their own, so we only ever treat whole unreferenced folders as orphaned. +fn in_referenced_dir(path: &Path, root: &Path, referenced_dirs: &HashSet) -> bool { + let root_key = path_key(root); + let mut dir = path.parent(); + while let Some(d) = dir { + let key = path_key(d); + if key.len() <= root_key.len() { + break; + } + if referenced_dirs.contains(&key) { + return true; } + dir = d.parent(); + } + false +} + +fn is_recent(path: &Path, now: SystemTime) -> bool { + let Ok(modified) = std::fs::metadata(path).and_then(|m| m.modified()) else { + // Can't tell how old it is — don't risk it. + return true; + }; + match now.duration_since(modified) { + Ok(age) => age < MIN_FS_ORPHAN_AGE, + // Modified "in the future" (clock skew): treat as recent. + Err(_) => true, } - PathBuf::from(path) } -fn walk_files(dir: &std::path::Path, out: &mut Vec) { - let Ok(entries) = fs::read_dir(dir) else { - return; +fn file_len(path: &Path) -> u64 { + std::fs::metadata(path).map(|m| m.len()).unwrap_or(0) +} + +/// Resolve how chat.db spells a path into how the filesystem walk will see +/// it, so the same file can't look unreferenced because of spelling: +/// - rebuild from components (drops `//` and `.`), +/// - resolve symlinks in the parent folder (e.g. an old home path left as a +/// link after the home folder moved), +/// - if it still isn't under an attachment root but clearly belongs to one +/// (`…/Library/Messages/Attachments/…` under some other home), map its +/// tail onto the current root. +fn normalize_stored_path(path: &Path, roots: &[PathBuf]) -> PathBuf { + let rebuilt: PathBuf = path.components().collect(); + // Resolve the longest part of the parent folder that exists, so a + // symlinked prefix is resolved even if the last folders don't exist + // exactly as spelled (e.g. a case difference on a case-sensitive + // volume). + let resolved = match (rebuilt.parent(), rebuilt.file_name()) { + (Some(parent), Some(name)) => crate::core::paths::canonical_prefix(parent).join(name), + _ => rebuilt.clone(), }; - for entry in entries.flatten() { - let path = entry.path(); - if path.is_dir() { - walk_files(&path, out); - } else if path.is_file() { - out.push(path); + if is_within_roots(&resolved, roots) { + return resolved; + } + let text = resolved.to_string_lossy(); + let lower = text.to_lowercase(); + for root in roots { + let Some(name) = root.file_name() else { + continue; + }; + let marker = format!( + "/library/messages/{}/", + name.to_string_lossy().to_lowercase() + ); + if let Some(i) = lower.rfind(&marker) { + let candidate = root.join(&text[i + marker.len()..]); + if is_within_roots(&candidate, roots) { + return candidate; + } } } + resolved +} + +/// chat.db `created_date`: seconds since 2001, or nanoseconds on newer +/// macOS. `None` when unset. +fn created_at(apple_date: i64) -> Option { + if apple_date <= 0 { + return None; + } + let secs = if apple_date > 100_000_000_000 { + apple_date / 1_000_000_000 + } else { + apple_date + }; + let unix = u64::try_from(secs).ok()? + 978_307_200; + Some(SystemTime::UNIX_EPOCH + Duration::from_secs(unix)) } -/// Scan for two categories of orphaned data: +/// Find two categories of orphaned data: /// /// - **DB orphans**: rows in `attachment` with no `message_attachment_join` -/// entry (the attachment is not linked to any message). +/// entry, excluding attachments the user kept via a messages-only delete +/// and records created in the last [`MIN_FS_ORPHAN_AGE`] (their message +/// may simply not have synced to this Mac yet). /// - **FS orphans**: files under the Messages Attachments/StickerCache -/// directories that have no corresponding row in `attachment` at all. -#[tauri::command] -pub async fn scan_orphans() -> Result { - let db_path = default_chat_db_path()?; - let conn = get_connection(&db_path)?; - - // DB orphans ------------------------------------------------------- - // Prefer actual on-disk size; total_bytes in the DB is often 0 or NULL. +/// directories whose folder holds no file referenced by any `attachment` +/// row, and that are at least [`MIN_FS_ORPHAN_AGE`] old. +fn find_orphans( + conn: &Connection, + retained: &BTreeSet, + include_retained: bool, + roots: &[PathBuf], + now: SystemTime, +) -> Result { + // Stored paths are resolved below; compare (and walk) resolved roots. + let roots = &canonical_roots(roots); let mut stmt = conn.prepare( - "SELECT a.filename, COALESCE(a.total_bytes, 0) \ + "SELECT a.ROWID, a.filename, \ + EXISTS (SELECT 1 FROM message_attachment_join j WHERE j.attachment_id = a.ROWID), \ + COALESCE(a.created_date, 0) \ FROM attachment a \ - LEFT JOIN message_attachment_join j ON j.attachment_id = a.ROWID \ - WHERE j.message_id IS NULL", + ORDER BY a.ROWID", )?; - let mut db_orphan_count = 0u64; - let mut db_orphan_bytes = 0u64; - let db_rows = stmt.query_map([], |row| { - Ok((row.get::<_, Option>(0)?, row.get::<_, i64>(1)?)) - })?; - for row in db_rows.flatten() { - let (filename, db_bytes) = row; - db_orphan_count += 1; - let bytes = filename + let rows: Vec<(i64, Option, bool, i64)> = stmt + .query_map([], |row| { + Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)) + })? + .collect::>()?; + + let mut referenced_dirs: HashSet = HashSet::new(); + // Every file any record points at. Never a filesystem orphan, even when + // it sits directly in a root folder (where folder matching stops). + let mut referenced_files: HashSet = HashSet::new(); + let mut kept_files: HashSet = HashSet::new(); + let mut candidates: Vec<(i64, Option)> = Vec::new(); + let mut existing: HashSet = HashSet::new(); + let mut out = Orphans::default(); + + for (rowid, filename, linked, created) in rows { + let path = filename .as_deref() - .map(expand_tilde) - .and_then(|p| fs::metadata(&p).ok()) - .map(|m| m.len()) - .unwrap_or(db_bytes.max(0) as u64); - db_orphan_bytes = db_orphan_bytes.saturating_add(bytes); - } - - // Known paths (ALL attachment rows, orphaned or not) --------------- - let mut path_stmt = - conn.prepare("SELECT filename FROM attachment WHERE filename IS NOT NULL")?; - let known_paths: HashSet = path_stmt - .query_map([], |row| row.get::<_, String>(0))? - .flatten() - .map(|p| expand_tilde(&p)) + .map(|f| normalize_stored_path(&expand_tilde(f), roots)); + if let Some(p) = &path { + referenced_files.insert(path_key(p)); + if let Some(parent) = p.parent() { + referenced_dirs.insert(path_key(parent)); + } + } + existing.insert(rowid); + let is_retained = !linked && retained.contains(&rowid); + // Unset dates count as old; a date in the future (clock skew) + // counts as new. + let too_new = created_at(created).is_some_and(|t| { + now.duration_since(t) + .map_or(true, |age| age < MIN_FS_ORPHAN_AGE) + }); + if too_new && !linked { + // Treat like a linked record: keep it and its file. + if let Some(p) = &path { + kept_files.insert(path_key(p)); + } + continue; + } + if linked || (is_retained && !include_retained) { + if let Some(p) = &path { + kept_files.insert(path_key(p)); + if is_retained && is_within_roots(p, roots) { + out.retained_bytes = out.retained_bytes.saturating_add(file_len(p)); + } + } + if is_retained { + out.retained_count += 1; + } + } else { + if is_retained { + out.included_retained.push(rowid); + } + candidates.push((rowid, path)); + } + } + // Only forget ids whose record is gone. A retained id that is linked + // right now may belong to a messages-only delete that is about to + // commit; dropping it would leave the attachment unprotected. + out.stale_retained = retained + .iter() + .copied() + .filter(|id| !existing.contains(id)) .collect(); - // FS orphans ------------------------------------------------------- - let mut fs_orphan_count = 0u64; - let mut fs_orphan_bytes = 0u64; - for dir_str in ATTACHMENT_DIRS { - let dir = expand_tilde(dir_str); - if !dir.exists() { - continue; + for (rowid, path) in candidates { + let path = path.filter(|p| is_within_roots(p, roots) && !kept_files.contains(&path_key(p))); + if let Some(p) = &path { + out.db_bytes = out.db_bytes.saturating_add(file_len(p)); } + out.db.push(DbOrphan { rowid, path }); + } + + for root in roots { let mut files = Vec::new(); - walk_files(&dir, &mut files); + walk_files(root, &mut files); for path in files { - if !known_paths.contains(&path) { - fs_orphan_count += 1; - fs_orphan_bytes = fs_orphan_bytes - .saturating_add(fs::metadata(&path).map(|m| m.len()).unwrap_or(0)); + if referenced_files.contains(&path_key(&path)) + || in_referenced_dir(&path, root, &referenced_dirs) + || is_recent(&path, now) + { + continue; } + out.fs_bytes = out.fs_bytes.saturating_add(file_len(&path)); + out.fs.push(path); } } + out.fs.sort(); - Ok(OrphanScan { - db_orphan_count, - db_orphan_bytes, - fs_orphan_count, - fs_orphan_bytes, - }) + Ok(out) } -/// Delete all orphaned data found by [`scan_orphans`]. -/// -/// Execution order: -/// 1. Collect orphan rowids + paths with read connection (then drop it). -/// 2. Delete DB orphan rows via a writable transaction. -/// 3. Remove DB orphan files from disk. -/// 4. Remove FS orphan files from disk. #[tauri::command] -pub async fn clean_orphans() -> Result { +pub async fn scan_orphans( + app: AppHandle, + lock: State<'_, OperationLock>, + include_retained: bool, +) -> Result { + let _guard = lock.try_acquire("an orphan scan")?; + let store = RetainedAttachments::for_app(&app)?; + blocking(move || scan_orphans_blocking(&store, include_retained)).await +} + +pub(crate) fn scan_orphans_blocking( + store: &RetainedAttachments, + include_retained: bool, +) -> Result { let db_path = default_chat_db_path()?; + let conn = open_read(&db_path)?; + let retained = store.load()?; + let orphans = find_orphans( + &conn, + &retained, + include_retained, + &attachment_roots(), + SystemTime::now(), + )?; + // Housekeeping: forget kept attachments that were since deleted or + // linked to a message again, so the list doesn't grow forever. + store.remove(&orphans.stale_retained)?; - // Phase 1: collect everything while holding only a read connection. - let (db_orphan_rowids, db_orphan_paths, fs_orphan_paths) = { - let conn = get_connection(&db_path)?; + Ok(OrphanScan { + db_orphan_count: orphans.db.len() as u64, + db_orphan_bytes: orphans.db_bytes, + fs_orphan_count: orphans.fs.len() as u64, + fs_orphan_bytes: orphans.fs_bytes, + retained_count: orphans.retained_count, + retained_bytes: orphans.retained_bytes, + fingerprint: orphans.fingerprint(), + }) +} - let mut stmt = conn.prepare( - "SELECT a.ROWID, a.filename \ - FROM attachment a \ - LEFT JOIN message_attachment_join j ON j.attachment_id = a.ROWID \ - WHERE j.message_id IS NULL", +/// Delete the given orphan rows in one transaction, re-checking the orphan +/// condition so a row that gained a link since the scan is left alone. +/// Returns the rowids whose record is actually gone. +fn delete_orphan_rows(conn: &mut Connection, rows: &[DbOrphan]) -> Result, AppError> { + let mut deleted = HashSet::new(); + let tx = conn.transaction()?; + for o in rows { + tx.execute( + "DELETE FROM attachment WHERE ROWID = ?1 \ + AND NOT EXISTS (SELECT 1 FROM message_attachment_join WHERE attachment_id = ?1)", + rusqlite::params![o.rowid], )?; - let pairs: Vec<(i64, Option)> = stmt - .query_map([], |row| Ok((row.get(0)?, row.get(1)?)))? - .flatten() - .collect(); + // Check directly rather than trusting the affected-row count: + // chat.db's delete triggers can make it unreliable. + let still_there: bool = tx.query_row( + "SELECT EXISTS (SELECT 1 FROM attachment WHERE ROWID = ?1)", + rusqlite::params![o.rowid], + |r| r.get(0), + )?; + if !still_there { + deleted.insert(o.rowid); + } + } + tx.commit()?; + Ok(deleted) +} - let db_orphan_rowids: Vec = pairs.iter().map(|(id, _)| *id).collect(); - let db_orphan_paths: Vec> = pairs - .iter() - .map(|(_, f)| f.as_deref().map(expand_tilde)) - .collect(); +fn ensure_messages_not_running(messages_running: fn() -> bool) -> Result<(), AppError> { + if messages_running() { + return Err(AppError::Other( + "Quit Messages.app before cleaning orphaned data — it may be writing new \ + attachments." + .into(), + )); + } + Ok(()) +} - let mut path_stmt = - conn.prepare("SELECT filename FROM attachment WHERE filename IS NOT NULL")?; - let known_paths: HashSet = path_stmt - .query_map([], |row| row.get::<_, String>(0))? - .flatten() - .map(|p| expand_tilde(&p)) - .collect(); +/// Delete the orphaned data found by [`scan_orphans`]. +/// +/// Execution order: +/// 1. Refuse if Messages.app is running. +/// 2. Recompute orphans and refuse unless they match the reviewed scan. +/// 3. If rows will be deleted: WAL checkpoint + chat.db snapshot. +/// 4. Delete DB orphan rows via a writable transaction. +/// 5. Remove DB orphan files, then FS orphan files, from disk. +#[tauri::command] +pub async fn clean_orphans( + app: AppHandle, + lock: State<'_, OperationLock>, + args: CleanOrphansArgs, +) -> Result { + let _guard = lock.try_acquire("a cleanup")?; + let store = RetainedAttachments::for_app(&app)?; + let snapshot_root = default_snapshot_root(&app, SnapshotKind::OrphanClean)?; + blocking(move || clean_orphans_blocking(&store, &snapshot_root, &args, &SafetyProbe::system())) + .await +} - let mut fs_orphan_paths: Vec = Vec::new(); - for dir_str in ATTACHMENT_DIRS { - let dir = expand_tilde(dir_str); - if !dir.exists() { - continue; - } - let mut files = Vec::new(); - walk_files(&dir, &mut files); - for path in files { - if !known_paths.contains(&path) { - fs_orphan_paths.push(path); - } - } - } +pub(crate) fn clean_orphans_blocking( + store: &RetainedAttachments, + snapshot_root: &Path, + args: &CleanOrphansArgs, + probe: &SafetyProbe, +) -> Result { + ensure_messages_not_running(probe.messages_running)?; + + let db_path = default_chat_db_path()?; + let roots = attachment_roots(); + let retained = store.load()?; - (db_orphan_rowids, db_orphan_paths, fs_orphan_paths) + let orphans = { + let conn = open_read(&db_path)?; + find_orphans( + &conn, + &retained, + args.include_retained, + &roots, + SystemTime::now(), + )? // conn dropped here }; + if orphans.fingerprint() != args.scan_fingerprint { + return Err(AppError::Other( + "Orphaned data changed since the scan. Run the scan again and review the new \ + results before cleaning." + .into(), + )); + } + if !orphans.db.is_empty() + && (probe.icloud)() != ICloudState::Disabled + && !args.acknowledge_icloud_sync + { + return Err(AppError::Other( + "Removing attachment records queues them for deletion in Messages in iCloud, which \ + can remove them from your other devices too. Acknowledge the iCloud warning in the \ + cleanup dialog (or pass acknowledgeIcloudSync=true) before retrying." + .into(), + )); + } - // Phase 2: delete DB orphan rows in a single transaction. let mut db_rows_deleted = 0u64; - if !db_orphan_rowids.is_empty() { - let mut conn = get_writable_connection(&db_path)?; - let tx = conn.transaction()?; - for &rowid in &db_orphan_rowids { - if tx.execute( - "DELETE FROM attachment WHERE ROWID = ?1", - rusqlite::params![rowid], - )? > 0 - { - db_rows_deleted += 1; + let mut snapshot_path = None; + // Rowids whose record really was deleted. Only their files are removed: + // a row the re-check below kept (it gained a link since the scan) must + // keep its file too. + let mut deleted: HashSet = HashSet::new(); + if !orphans.db.is_empty() { + crate::core::space::ensure_space( + snapshot_root, + crate::core::space::snapshot_size(&db_path), + "the safety snapshot", + probe.available_space, + )?; + checkpoint_wal(&db_path)?; + let snapshot = snapshot_chat_db(&db_path, snapshot_root)?; + + // If anything fails before the rows are deleted, nothing changed: + // drop the redundant snapshot instead of letting it push an older + // one out, and only prune after a successful clean. + let result = (|| { + ensure_messages_not_running(probe.messages_running)?; + let mut conn = open_write(&db_path)?; + delete_orphan_rows(&mut conn, &orphans.db) + })(); + deleted = match result { + Ok(d) => d, + Err(e) => { + if let Err(re) = std::fs::remove_dir_all(&snapshot.dir) { + eprintln!("orphan-clean: couldn't remove unused snapshot: {re}"); + } + return Err(e); } - } - tx.commit()?; + }; + prune_snapshots(snapshot_root, SNAPSHOTS_TO_KEEP, &snapshot.dir); + snapshot_path = Some(snapshot.dir.display().to_string()); + db_rows_deleted = deleted.len() as u64; + // The included kept attachments that are gone now; stop tracking + // them. + let gone: Vec = orphans + .included_retained + .iter() + .copied() + .filter(|id| deleted.contains(id)) + .collect(); + store.remove(&gone)?; } - // Phase 3: remove DB orphan files from disk. + let mut files_skipped = 0u64; let mut db_files_removed = 0u64; let mut db_files_failed = 0u64; - for path in db_orphan_paths.iter().flatten() { - if path.exists() { - match fs::remove_file(path) { - Ok(()) => db_files_removed += 1, - Err(e) => { - eprintln!( - "orphan-clean: failed to remove db-orphan file {}: {e}", - path.display() - ); - db_files_failed += 1; - } - } + let deleted_files = orphans + .db + .iter() + .filter(|o| deleted.contains(&o.rowid)) + .filter_map(|o| o.path.as_deref()); + for path in deleted_files { + match remove_attachment_file(path, &roots, "orphan-clean") { + RemoveOutcome::Removed => db_files_removed += 1, + RemoveOutcome::Missing => {} + RemoveOutcome::Refused => files_skipped += 1, + RemoveOutcome::Failed => db_files_failed += 1, } } - // Phase 4: remove FS orphan files. let mut fs_files_removed = 0u64; let mut fs_files_failed = 0u64; - for path in &fs_orphan_paths { - match fs::remove_file(path) { - Ok(()) => fs_files_removed += 1, - Err(e) => { - eprintln!( - "orphan-clean: failed to remove fs-orphan file {}: {e}", - path.display() - ); - fs_files_failed += 1; - } + for path in &orphans.fs { + match remove_attachment_file(path, &roots, "orphan-clean") { + RemoveOutcome::Removed => fs_files_removed += 1, + RemoveOutcome::Missing => {} + RemoveOutcome::Refused => files_skipped += 1, + RemoveOutcome::Failed => fs_files_failed += 1, } } @@ -245,5 +545,196 @@ pub async fn clean_orphans() -> Result { db_files_failed, fs_files_removed, fs_files_failed, + files_skipped, + snapshot_path, }) } + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use tempfile::TempDir; + + fn fixture(tmp: &TempDir) -> (Connection, PathBuf) { + // Resolved, like the app's real roots (macOS's temp folder is under + // the `/var` → `/private/var` symlink). + let root = std::fs::canonicalize(tmp.path()) + .unwrap() + .join("Attachments"); + fs::create_dir_all(&root).unwrap(); + let conn = Connection::open(tmp.path().join("chat.db")).unwrap(); + conn.execute_batch( + "CREATE TABLE attachment (ROWID INTEGER PRIMARY KEY, filename TEXT, created_date INTEGER DEFAULT 0); + CREATE TABLE message_attachment_join (message_id INTEGER, attachment_id INTEGER);", + ) + .unwrap(); + (conn, root) + } + + fn file(path: &Path) -> PathBuf { + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, b"data").unwrap(); + path.to_path_buf() + } + + fn add_row(conn: &Connection, rowid: i64, path: &Path, linked: bool) { + conn.execute( + "INSERT INTO attachment (ROWID, filename) VALUES (?1, ?2)", + rusqlite::params![rowid, path.to_str().unwrap()], + ) + .unwrap(); + if linked { + conn.execute( + "INSERT INTO message_attachment_join VALUES (1, ?1)", + rusqlite::params![rowid], + ) + .unwrap(); + } + } + + fn later() -> SystemTime { + SystemTime::now() + MIN_FS_ORPHAN_AGE + Duration::from_secs(60) + } + + #[test] + fn companion_files_in_referenced_folders_are_kept() { + let tmp = TempDir::new().unwrap(); + let (conn, root) = fixture(&tmp); + let heic = file(&root.join("ab/01/GUID-1/IMG_1.HEIC")); + file(&root.join("ab/01/GUID-1/IMG_1.MOV")); + file(&root.join("ab/01/GUID-1/sub/preview.png")); + let stray = file(&root.join("cd/02/GUID-2/old.jpg")); + add_row(&conn, 1, &heic, true); + + let o = find_orphans(&conn, &BTreeSet::new(), false, &[root], later()).unwrap(); + assert!(o.db.is_empty()); + assert_eq!(o.fs, vec![stray]); + } + + #[test] + fn path_case_differences_do_not_create_orphans() { + let tmp = TempDir::new().unwrap(); + let (conn, root) = fixture(&tmp); + let real = file(&root.join("ab/01/GUID-1/a.jpg")); + let db_spelling = PathBuf::from(real.to_str().unwrap().replace("GUID-1", "guid-1")); + add_row(&conn, 1, &db_spelling, true); + + let o = find_orphans(&conn, &BTreeSet::new(), false, &[root], later()).unwrap(); + assert!(o.fs.is_empty()); + } + + #[test] + fn recent_files_are_not_fs_orphans() { + let tmp = TempDir::new().unwrap(); + let (conn, root) = fixture(&tmp); + file(&root.join("ab/01/GUID-NEW/incoming.jpg")); + + let o = find_orphans(&conn, &BTreeSet::new(), false, &[root], SystemTime::now()).unwrap(); + assert!(o.fs.is_empty()); + } + + #[test] + fn db_orphans_respect_retained_shared_and_out_of_root_files() { + let tmp = TempDir::new().unwrap(); + let (conn, root) = fixture(&tmp); + let shared = file(&root.join("ab/01/G/shared.jpg")); + let own = file(&root.join("ab/02/G/own.jpg")); + let outside = file(&tmp.path().join("Documents/important.pdf")); + add_row(&conn, 1, &shared, true); + add_row(&conn, 2, &shared, false); // orphan row sharing a kept file + add_row(&conn, 3, &own, false); // genuine orphan + add_row(&conn, 4, &outside, false); // orphan row pointing outside + add_row(&conn, 5, &own, false); + let solo = file(&root.join("ab/03/G/solo.jpg")); + add_row(&conn, 6, &solo, false); // orphan with its own file + let retained = BTreeSet::from([5]); + + let o = find_orphans( + &conn, + &retained, + false, + std::slice::from_ref(&root), + later(), + ) + .unwrap(); + let got: Vec<(i64, Option)> = + o.db.iter().map(|d| (d.rowid, d.path.clone())).collect(); + // Row 5 is retained, so its file (shared with orphan row 3) is kept. + assert_eq!(got, vec![(2, None), (3, None), (4, None), (6, Some(solo))]); + assert_eq!(o.retained_count, 1); + assert!(o.included_retained.is_empty()); + + // Opting in turns the kept attachment (and the file it shares with + // row 3) into cleanup candidates. + let o = find_orphans(&conn, &retained, true, &[root], later()).unwrap(); + let ids: Vec = o.db.iter().map(|d| d.rowid).collect(); + assert_eq!(ids, vec![2, 3, 4, 5, 6]); + assert_eq!(o.included_retained, vec![5]); + assert_eq!(o.retained_count, 0); + assert!(o + .db + .iter() + .any(|d| d.rowid == 3 && d.path == Some(own.clone()))); + } + + #[test] + fn rows_linked_since_the_scan_are_not_deleted() { + let tmp = TempDir::new().unwrap(); + let (mut conn, root) = fixture(&tmp); + let a = file(&root.join("ab/01/G/a.jpg")); + let b = file(&root.join("ab/02/G/b.jpg")); + add_row(&conn, 1, &a, false); + add_row(&conn, 2, &b, false); + let o = find_orphans( + &conn, + &BTreeSet::new(), + false, + std::slice::from_ref(&root), + later(), + ) + .unwrap(); + assert_eq!(o.db.len(), 2); + + // Row 2 gets linked to a message after the scan. + conn.execute("INSERT INTO message_attachment_join VALUES (5, 2)", []) + .unwrap(); + let deleted = delete_orphan_rows(&mut conn, &o.db).unwrap(); + assert_eq!(deleted, HashSet::from([1])); + } + + #[test] + fn stale_retained_ids_are_reported() { + let tmp = TempDir::new().unwrap(); + let (conn, root) = fixture(&tmp); + let a = file(&root.join("ab/01/G/a.jpg")); + add_row(&conn, 1, &a, true); // linked (e.g. a delete mid-commit): keep tracking + add_row(&conn, 2, &a, false); // still kept & unlinked + let retained = BTreeSet::from([1, 2, 99]); // 99 no longer exists + + let o = find_orphans(&conn, &retained, false, &[root], later()).unwrap(); + assert_eq!(o.stale_retained, vec![99]); + } + + #[test] + fn fingerprint_changes_when_orphans_change() { + let tmp = TempDir::new().unwrap(); + let (conn, root) = fixture(&tmp); + let a = file(&root.join("ab/01/G/a.jpg")); + add_row(&conn, 1, &a, false); + let roots = [root]; + let f1 = find_orphans(&conn, &BTreeSet::new(), false, &roots, later()) + .unwrap() + .fingerprint(); + let f2 = find_orphans(&conn, &BTreeSet::new(), false, &roots, later()) + .unwrap() + .fingerprint(); + assert_eq!(f1, f2); + + add_row(&conn, 2, &a, false); + let f3 = find_orphans(&conn, &BTreeSet::new(), false, &roots, later()) + .unwrap() + .fingerprint(); + assert_ne!(f1, f3); + } +} diff --git a/src-tauri/src/commands/preview.rs b/src-tauri/src/commands/preview.rs index 4224ff4..356b3d8 100644 --- a/src-tauri/src/commands/preview.rs +++ b/src-tauri/src/commands/preview.rs @@ -1,9 +1,8 @@ -use crate::core::db_path::default_chat_db_path; +use crate::core::db_path::{blocking, default_chat_db_path, open_read}; use crate::core::filter::FilterSpec; use crate::error::AppError; -use imessage_database::tables::{ - table::get_connection, - write::{attachment_filter::AttachmentFilter, delete::preview_delete as lib_preview_delete}, +use imessage_database::tables::write::{ + attachment_filter::AttachmentFilter, delete::preview_delete as lib_preview_delete, }; use serde::Serialize; @@ -31,11 +30,18 @@ pub struct BackupPreview { #[tauri::command] pub async fn preview_backup(filter: FilterSpec) -> Result { let path = default_chat_db_path()?; - let conn = get_connection(&path)?; let ctx = filter.to_query_context()?; + blocking(move || preview_blocking(&path, &ctx)).await +} + +fn preview_blocking( + path: &std::path::Path, + ctx: &imessage_database::util::query_context::QueryContext, +) -> Result { + let conn = open_read(path)?; let has_filters = ctx.has_filters(); - let plan = lib_preview_delete(&conn, &ctx, &AttachmentFilter::any(), &path) + let plan = lib_preview_delete(&conn, ctx, &AttachmentFilter::any(), path) .map_err(|e| AppError::Database(format!("failed to count messages: {e}")))?; let mut on_disk_count = 0u64; diff --git a/src-tauri/src/commands/restore.rs b/src-tauri/src/commands/restore.rs new file mode 100644 index 0000000..d920fb9 --- /dev/null +++ b/src-tauri/src/commands/restore.rs @@ -0,0 +1,193 @@ +//! Restore chat.db from a snapshot taken before a delete or cleanup. + +use std::path::{Path, PathBuf}; + +use rusqlite::{backup::Backup, Connection, OpenFlags}; +use serde::{Deserialize, Serialize}; +use tauri::{AppHandle, State}; + +use crate::commands::delete::SafetyProbe; +use crate::core::db_path::{blocking, default_chat_db_path, open_write}; +use crate::core::icloud::ICloudState; +use crate::core::op_lock::OperationLock; +use crate::core::snapshot::{ + checkpoint_wal, default_snapshot_root, list_snapshots as find_snapshots, prune_snapshots, + snapshot_chat_db, SnapshotInfo, SnapshotKind, SNAPSHOTS_TO_KEEP, +}; +use crate::core::space::{ensure_space, snapshot_size}; +use crate::error::AppError; + +fn snapshot_roots(app: &AppHandle) -> Result, AppError> { + SnapshotKind::ALL + .iter() + .map(|k| Ok((*k, default_snapshot_root(app, *k)?))) + .collect() +} + +/// Snapshots available to restore, newest first. +#[tauri::command] +pub async fn list_snapshots(app: AppHandle) -> Result, AppError> { + let roots = snapshot_roots(&app)?; + blocking(move || Ok(find_snapshots(&roots))).await +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct RestoreArgs { + /// A `path` from [`list_snapshots`]. + pub snapshot_path: String, + /// Required unless Messages in iCloud is positively detected as off: + /// how iCloud reconciles a database rolled back underneath it is + /// unpredictable. + #[serde(default)] + pub acknowledge_icloud_sync: bool, +} + +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct RestoreResult { + pub restored_from: String, + /// The database as it was just before the restore, so the restore + /// itself can be undone. + pub pre_restore_snapshot: String, +} + +#[tauri::command] +pub async fn restore_snapshot( + app: AppHandle, + lock: State<'_, OperationLock>, + args: RestoreArgs, +) -> Result { + let _guard = lock.try_acquire("a restore")?; + let db_path = default_chat_db_path()?; + let roots = snapshot_roots(&app)?; + blocking(move || restore_now(&db_path, &roots, &args, &SafetyProbe::system())).await +} + +/// The snapshot folder, if it is one this app created: directly inside one +/// of `roots`, holding a `chat.db`. Anything else is refused — this command +/// overwrites the live database, so it must never take an arbitrary path. +fn resolve_snapshot( + snapshot_path: &str, + roots: &[(SnapshotKind, PathBuf)], +) -> Result { + let refuse = || { + AppError::Other(format!( + "{snapshot_path} isn't a Bubble Wrap snapshot. Pick one from the list." + )) + }; + let dir = std::fs::canonicalize(snapshot_path).map_err(|_| refuse())?; + let in_a_root = roots.iter().any(|(_, root)| { + std::fs::canonicalize(root).is_ok_and(|root| dir.parent() == Some(root.as_path())) + }); + let listed = find_snapshots(roots) + .iter() + .any(|s| std::fs::canonicalize(&s.path).is_ok_and(|p| p == dir)); + if !in_a_root || !listed { + return Err(refuse()); + } + Ok(dir) +} + +/// Refuse a snapshot SQLite doesn't consider intact, or that isn't a +/// Messages database at all. +fn check_snapshot(db: &Path) -> Result<(), AppError> { + let conn = Connection::open_with_flags( + db, + OpenFlags::SQLITE_OPEN_READ_ONLY | OpenFlags::SQLITE_OPEN_NO_MUTEX, + ) + .map_err(|e| AppError::Other(format!("can't open snapshot: {e}")))?; + let result: String = conn + .query_row("PRAGMA integrity_check", [], |r| r.get(0)) + .map_err(|e| AppError::Other(format!("snapshot is damaged: {e}")))?; + if result != "ok" { + return Err(AppError::Other(format!("snapshot is damaged: {result}"))); + } + let has_messages: bool = conn + .query_row( + "SELECT EXISTS (SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'message')", + [], + |r| r.get(0), + ) + .map_err(|e| AppError::Other(format!("can't read snapshot: {e}")))?; + if !has_messages { + return Err(AppError::Other( + "that snapshot isn't a Messages database".into(), + )); + } + Ok(()) +} + +/// Replace the live chat.db's contents with a snapshot's (see +/// [`restore_snapshot`]). Attachment files deleted since aren't in the +/// snapshot and stay deleted. +pub(crate) fn restore_now( + db_path: &Path, + roots: &[(SnapshotKind, PathBuf)], + args: &RestoreArgs, + probe: &SafetyProbe, +) -> Result { + let snapshot_dir = resolve_snapshot(&args.snapshot_path, roots)?; + let snapshot_db = snapshot_dir.join("chat.db"); + check_snapshot(&snapshot_db)?; + + if (probe.messages_running)() { + return Err(AppError::Other( + "Quit Messages.app before restoring a snapshot.".into(), + )); + } + if (probe.icloud)() != ICloudState::Disabled && !args.acknowledge_icloud_sync { + return Err(AppError::Other( + "Messages in iCloud may be on. Restoring rolls this Mac's database back underneath \ + it, and how iCloud reconciles that is unpredictable. Acknowledge the iCloud warning \ + (or pass acknowledgeIcloudSync=true) before retrying." + .into(), + )); + } + + // Keep the current state first, so the restore can be undone too. + let pre_root = roots + .iter() + .find(|(k, _)| *k == SnapshotKind::PreRestore) + .map(|(_, r)| r.clone()) + .ok_or_else(|| AppError::Other("no pre-restore snapshot folder".into()))?; + ensure_space( + &pre_root, + snapshot_size(db_path), + "a snapshot of the current database", + probe.available_space, + )?; + checkpoint_wal(db_path)?; + let pre = snapshot_chat_db(db_path, &pre_root)?; + + if (probe.messages_running)() { + return Err(AppError::Other( + "Messages.app was opened during the restore — quit it and try again. Nothing was \ + changed." + .into(), + )); + } + + // SQLite's backup API copies the snapshot page by page into the live + // database under its own locks, which keeps the write-ahead log + // consistent (unlike swapping files underneath it). + let src = Connection::open_with_flags( + &snapshot_db, + OpenFlags::SQLITE_OPEN_READ_ONLY | OpenFlags::SQLITE_OPEN_NO_MUTEX, + )?; + let mut live = open_write(db_path)?; + { + let backup = Backup::new(&src, &mut live) + .map_err(|e| AppError::Other(format!("couldn't start restore: {e}")))?; + backup + .run_to_completion(1024, std::time::Duration::from_millis(0), None) + .map_err(|e| AppError::Other(format!("restore failed: {e}")))?; + } + drop(live); + prune_snapshots(&pre_root, SNAPSHOTS_TO_KEEP, &pre.dir); + + Ok(RestoreResult { + restored_from: snapshot_dir.display().to_string(), + pre_restore_snapshot: pre.dir.display().to_string(), + }) +} diff --git a/src-tauri/src/commands/safety.rs b/src-tauri/src/commands/safety.rs index fbeb053..d691bdc 100644 --- a/src-tauri/src/commands/safety.rs +++ b/src-tauri/src/commands/safety.rs @@ -8,16 +8,35 @@ use serde::Serialize; pub struct SafetyStatus { pub messages_running: bool, pub icloud_messages: ICloudState, + /// Where earlier builds kept full copies of chat.db (under + /// `~/Documents`, which may sync to iCloud Drive), if any are still + /// there. Shown so the user can remove them; never deleted for them. + pub legacy_snapshot_dir: Option, +} + +fn legacy_snapshot_dir() -> Option { + let dir = crate::core::db_path::home_dir() + .ok()? + .join("Documents/iMessage Backups/snapshots"); + std::fs::read_dir(&dir) + .ok()? + .next() + .is_some() + .then(|| dir.display().to_string()) } /// Collect the safety signals the frontend needs to decide whether to allow /// a delete operation to proceed. Never performs any mutation. #[tauri::command] pub async fn safety_status() -> Result { - let messages_running = is_messages_running().unwrap_or(false); + // Same conservative default as `run_delete`: if we can't tell, report + // "running" so the UI doesn't show a delete as safe that the backend + // will then refuse. + let messages_running = is_messages_running().unwrap_or(true); let icloud_messages = detect_icloud_messages().unwrap_or(ICloudState::Unknown); Ok(SafetyStatus { messages_running, icloud_messages, + legacy_snapshot_dir: legacy_snapshot_dir(), }) } diff --git a/src-tauri/src/core/attachment_copy.rs b/src-tauri/src/core/attachment_copy.rs new file mode 100644 index 0000000..2d94693 --- /dev/null +++ b/src-tauri/src/core/attachment_copy.rs @@ -0,0 +1,559 @@ +//! Attachment copying for backups. +//! +//! The library's `copy_single_attachment` names copies after the source +//! file's basename and silently skips (while counting as copied) any name +//! already present — so every voice memo after the first (`Audio +//! Message.caf`) or a second `IMG_0001.HEIC` in a chat was never backed up. +//! It also aborts the whole export on the first unreadable file. This pass +//! names each copy uniquely by attachment rowid and counts per-file outcomes +//! instead. + +use std::collections::{HashMap, HashSet}; +use std::fs; +use std::io::{BufRead, BufReader, BufWriter, Write}; +use std::path::{Path, PathBuf}; + +use imessage_database::{ + exporters::progress::ProgressReporter, + tables::{attachment::Attachment, messages::message::Message, table::Table}, + util::{platform::Platform, query_context::QueryContext}, +}; +use rusqlite::Connection; +use serde::Serialize; + +use crate::core::attachment_files::walk_files; +use crate::error::AppError; + +const ATTACHMENTS_DIR: &str = "attachments"; +/// How many individual failures to report back to the UI. +const MAX_REPORTED_FAILURES: usize = 20; + +#[derive(Debug, Default, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct CopySummary { + /// Distinct attachments matched by the filter. + pub attempted: u64, + pub copied: u64, + pub bytes: u64, + /// The record points at a file that isn't on this Mac (typically + /// offloaded to iCloud). Nothing could be copied. + pub not_on_mac: u64, + /// The record has no file path at all. + pub unresolvable: u64, + /// The file is on this Mac but copying it failed. + pub failed: u64, + /// The first few failures, as "path: reason". + pub failures: Vec, + /// Rowids of the attachments actually copied, for the backup receipt. + #[serde(skip)] + pub copied_ids: Vec, +} + +impl CopySummary { + fn fail(&mut self, what: &Path, why: impl std::fmt::Display) { + self.failed += 1; + if self.failures.len() < MAX_REPORTED_FAILURES { + self.failures.push(format!("{}: {why}", what.display())); + } + } +} + +/// Visit every distinct attachment of every message matching `query`, with +/// the chat it belongs to and its resolved source path. +fn for_each_attachment( + conn: &Connection, + db_path: &Path, + query: &QueryContext, + mut f: impl FnMut(Option, &Attachment, Option), +) -> Result<(), AppError> { + let mut seen: HashSet = HashSet::new(); + let mut stmt = Message::stream_rows(conn, query)?; + let rows = stmt.query_map([], |row| Ok(Message::from_row(row)))?; + for row in rows { + let message = Message::extract(row)?; + if !message.has_attachments() { + continue; + } + for attachment in Attachment::from_message(conn, &message)? { + if !seen.insert(attachment.rowid) { + continue; + } + let src = attachment + .resolved_attachment_path(&Platform::macOS, db_path, None) + .map(PathBuf::from); + f(message.chat_id, &attachment, src); + } + } + Ok(()) +} + +fn total_attachments(conn: &Connection, query: &QueryContext) -> u64 { + Message::get_attachment_count(conn, query) + .map(|n| n.max(0) as u64) + .unwrap_or(0) +} + +/// Messages stamps balloon-plugin payloads with a synthetic +/// `.pluginPayloadAttachment` extension; use the MIME type's instead when +/// it's a common one so the copy opens normally. +fn extension_for_mime(mime: Option<&str>) -> Option<&'static str> { + Some(match mime? { + "image/jpeg" => "jpeg", + "image/png" => "png", + "image/gif" => "gif", + "image/heic" => "heic", + "image/heif" => "heif", + "image/webp" => "webp", + "image/tiff" => "tiff", + "video/mp4" => "mp4", + "video/quicktime" => "mov", + "audio/x-m4a" | "audio/mp4" => "m4a", + "audio/mpeg" => "mp3", + "audio/x-caf" => "caf", + _ => return None, + }) +} + +/// `-`: unique per attachment, still recognisable. +pub fn copy_file_name(attachment: &Attachment, src: &Path) -> String { + let raw = src + .file_name() + .map(|n| n.to_string_lossy().into_owned()) + .unwrap_or_else(|| "attachment".to_string()); + let mut name: String = raw + .chars() + .map(|c| { + if matches!(c, '/' | ':' | '\0') { + '_' + } else { + c + } + }) + .collect(); + let path = Path::new(&name); + let is_plugin = path + .extension() + .and_then(|e| e.to_str()) + .is_some_and(|e| e.eq_ignore_ascii_case("pluginPayloadAttachment")); + if is_plugin { + if let Some(ext) = extension_for_mime(attachment.mime_type.as_deref()) { + name = Path::new(&name) + .with_extension(ext) + .to_string_lossy() + .into_owned(); + } + } + // Stay well under the 255-byte file name limit. + while name.len() > 200 { + name.pop(); + } + format!("{}-{name}", attachment.rowid) +} + +/// Copy every attachment matching `query` into +/// `/attachments//-`. Returns the summary +/// and, per attachment rowid, the copy's path relative to `export_path`. +pub fn copy_attachments( + conn: &Connection, + db_path: &Path, + query: &QueryContext, + export_path: &Path, + progress: &dyn ProgressReporter, +) -> Result<(CopySummary, HashMap), AppError> { + progress.start(total_attachments(conn, query)); + progress.set_message("Copying attachments"); + + let mut summary = CopySummary::default(); + let mut copied: HashMap = HashMap::new(); + + for_each_attachment(conn, db_path, query, |chat_id, attachment, src| { + summary.attempted += 1; + progress.set_position(summary.attempted); + + let Some(src) = src else { + summary.unresolvable += 1; + return; + }; + let meta = match fs::metadata(&src) { + Ok(m) => m, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + summary.not_on_mac += 1; + return; + } + Err(e) => return summary.fail(&src, e), + }; + if !meta.is_file() && !meta.is_dir() { + return summary.fail(&src, "not a regular file or folder"); + } + + let chat_dir = chat_id.map_or_else(|| "_orphaned".to_string(), |id| id.to_string()); + let rel = format!( + "{ATTACHMENTS_DIR}/{chat_dir}/{}", + copy_file_name(attachment, &src) + ); + let dest = export_path.join(&rel); + if let Some(dir) = dest.parent() { + if let Err(e) = fs::create_dir_all(dir) { + return summary.fail(&src, e); + } + } + // Some attachments (bundles) are folders; copy those whole. + let result = if meta.is_dir() { + copy_tree(&src, &dest).inspect_err(|_| { + let _ = fs::remove_dir_all(&dest); + }) + } else { + copy_one_file(&src, &dest, &meta).inspect_err(|_| { + let _ = fs::remove_file(&dest); + }) + }; + match result { + Ok(n) => { + summary.copied += 1; + summary.bytes += n; + summary.copied_ids.push(i64::from(attachment.rowid)); + copied.insert(attachment.rowid, rel); + } + Err(why) => summary.fail(&src, why), + } + })?; + + progress.finish(); + Ok((summary, copied)) +} + +/// Copy one file, check the whole thing arrived, and keep its original +/// modification date (so the archive sorts by when the attachment was +/// received, not when it was backed up). Returns the bytes copied. +fn copy_one_file(src: &Path, dest: &Path, meta: &fs::Metadata) -> Result { + let n = fs::copy(src, dest).map_err(|e| e.to_string())?; + if n != meta.len() { + return Err(format!("short copy ({n} of {} bytes)", meta.len())); + } + if let Ok(modified) = meta.modified() { + let set = fs::File::options() + .write(true) + .open(dest) + .and_then(|f| f.set_modified(modified)); + if let Err(e) = set { + eprintln!("backup: couldn't keep date on {}: {e}", dest.display()); + } + } + Ok(n) +} + +/// Copy a folder attachment recursively. Symlinks inside it are skipped, +/// never followed — the same rule [`source_size`] uses, so the sizes +/// compare like for like. +fn copy_tree(src: &Path, dest: &Path) -> Result { + fs::create_dir_all(dest).map_err(|e| e.to_string())?; + let mut total = 0u64; + for entry in fs::read_dir(src).map_err(|e| e.to_string())? { + let entry = entry.map_err(|e| e.to_string())?; + let kind = entry.file_type().map_err(|e| e.to_string())?; + let to = dest.join(entry.file_name()); + if kind.is_dir() { + total += copy_tree(&entry.path(), &to)?; + } else if kind.is_file() { + let meta = entry.metadata().map_err(|e| e.to_string())?; + total += copy_one_file(&entry.path(), &to, &meta) + .map_err(|e| format!("{}: {e}", entry.path().display()))?; + } + } + Ok(total) +} + +/// Sizes of what the runtime wrote, keyed by file stem (the rowid): +/// `attachments//[.]` — a file, or a folder for +/// bundle-type attachments. +fn runtime_copy_sizes(export_path: &Path) -> HashMap { + let mut by_stem: HashMap = HashMap::new(); + if let Ok(chats) = fs::read_dir(export_path.join(ATTACHMENTS_DIR)) { + for chat in chats.flatten() { + if !chat.file_type().is_ok_and(|t| t.is_dir()) { + continue; + } + let Ok(entries) = fs::read_dir(chat.path()) else { + continue; + }; + for entry in entries.flatten() { + let path = entry.path(); + let Some(stem) = path.file_stem().and_then(|s| s.to_str()) else { + continue; + }; + let len = match entry.file_type() { + Ok(t) if t.is_file() => fs::metadata(&path).map(|m| m.len()).unwrap_or(0), + Ok(t) if t.is_dir() => { + let mut files = Vec::new(); + walk_files(&path, &mut files); + files + .iter() + .map(|f| fs::metadata(f).map(|m| m.len()).unwrap_or(0)) + .sum() + } + _ => continue, + }; + *by_stem.entry(stem.to_string()).or_default() += len; + } + } + } + + by_stem +} + +/// The HTML/TXT runtime does its own copying (to +/// `attachments//.`, possibly converted) but only logs +/// failures. Work out what actually landed in the export so the result is +/// truthful. +pub fn verify_runtime_copies( + conn: &Connection, + db_path: &Path, + query: &QueryContext, + export_path: &Path, +) -> Result { + let by_stem = runtime_copy_sizes(export_path); + + let mut summary = CopySummary::default(); + for_each_attachment(conn, db_path, query, |_, attachment, src| { + summary.attempted += 1; + let Some(src) = src else { + summary.unresolvable += 1; + return; + }; + let Some(expected) = source_size(&src) else { + summary.not_on_mac += 1; + return; + }; + match by_stem.get(&attachment.rowid.to_string()) { + // The runtime copies byte-for-byte (`Clone` mode), so a size + // difference means a copy that failed partway; the library only + // logs those and leaves the partial file behind. + Some(&bytes) if bytes == expected => { + summary.copied += 1; + summary.bytes += bytes; + summary.copied_ids.push(i64::from(attachment.rowid)); + } + Some(&bytes) => summary.fail( + &src, + format!("incomplete copy ({bytes} of {expected} bytes)"), + ), + None => summary.fail(&src, "not found in the export"), + } + })?; + Ok(summary) +} + +/// Size of an attachment source: a file's length, or the total of a +/// folder's files. `None` if it isn't on this Mac. +pub(crate) fn source_size(src: &Path) -> Option { + let meta = fs::metadata(src).ok()?; + if meta.is_dir() { + let mut files = Vec::new(); + walk_files(src, &mut files); + Some( + files + .iter() + .map(|f| fs::metadata(f).map(|m| m.len()).unwrap_or(0)) + .sum(), + ) + } else { + Some(meta.len()) + } +} + +/// The JSON export runs in reference mode (so the library doesn't do its +/// own, collision-prone copying); fill each attachment's `copiedTo` in the +/// `.ndjson` files from our copy pass, and record the real counts in +/// `manifest.json`. +pub fn annotate_json_export( + export_path: &Path, + manifest_path: &Path, + copied: &HashMap, + summary: &CopySummary, +) -> Result<(), AppError> { + let entries = fs::read_dir(export_path)?; + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().and_then(|e| e.to_str()) == Some("ndjson") { + rewrite_ndjson(&path, copied)?; + } + } + + if let Ok(bytes) = fs::read(manifest_path) { + if let Ok(mut manifest) = serde_json::from_slice::(&bytes) { + if let Some(counts) = manifest.get_mut("counts").and_then(|c| c.as_object_mut()) { + counts.insert("attachmentBytesCopied".into(), summary.bytes.into()); + counts.insert("attachmentsCopied".into(), summary.copied.into()); + counts.insert("attachmentsNotOnMac".into(), summary.not_on_mac.into()); + counts.insert( + "attachmentsUnresolvable".into(), + summary.unresolvable.into(), + ); + counts.insert("attachmentsFailed".into(), summary.failed.into()); + } + let json = + serde_json::to_vec_pretty(&manifest).map_err(|e| AppError::Other(e.to_string()))?; + write_atomic(manifest_path, &json)?; + } + } + Ok(()) +} + +fn rewrite_ndjson(path: &Path, copied: &HashMap) -> Result<(), AppError> { + let tmp = path.with_extension("ndjson.tmp"); + { + let reader = BufReader::new(fs::File::open(path)?); + let mut out = BufWriter::new(fs::File::create(&tmp)?); + for line in reader.lines() { + let line = line?; + let rewritten = serde_json::from_str::(&line) + .ok() + .and_then(|mut v| set_copied_to(&mut v, copied).then_some(v)); + match rewritten { + Some(v) => serde_json::to_writer(&mut out, &v) + .map_err(|e| AppError::Other(e.to_string()))?, + None => out.write_all(line.as_bytes())?, + } + out.write_all(b"\n")?; + } + out.flush()?; + } + fs::rename(&tmp, path)?; + Ok(()) +} + +/// Returns whether anything changed. +fn set_copied_to(message: &mut serde_json::Value, copied: &HashMap) -> bool { + let Some(atts) = message + .get_mut("attachments") + .and_then(|a| a.as_array_mut()) + else { + return false; + }; + let mut changed = false; + for att in atts { + let rel = att + .get("rowid") + .and_then(|r| r.as_i64()) + .and_then(|r| i32::try_from(r).ok()) + .and_then(|r| copied.get(&r)); + if let (Some(rel), Some(obj)) = (rel, att.as_object_mut()) { + obj.insert("copiedTo".into(), rel.clone().into()); + changed = true; + } + } + changed +} + +fn write_atomic(path: &Path, bytes: &[u8]) -> Result<(), AppError> { + let tmp = path.with_extension("tmp"); + fs::write(&tmp, bytes)?; + fs::rename(&tmp, path)?; + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use tempfile::TempDir; + + fn att(rowid: i32, mime: Option<&str>) -> Attachment { + Attachment { + rowid, + filename: None, + uti: None, + mime_type: mime.map(str::to_string), + transfer_name: None, + total_bytes: 0, + is_sticker: false, + hide_attachment: 0, + emoji_description: None, + copied_path: None, + } + } + + #[test] + fn copy_names_are_unique_per_attachment() { + let src = Path::new("/x/Audio Message.caf"); + assert_eq!(copy_file_name(&att(1, None), src), "1-Audio Message.caf"); + assert_eq!(copy_file_name(&att(2, None), src), "2-Audio Message.caf"); + } + + #[test] + fn plugin_payload_gets_real_extension() { + let src = Path::new("/x/IMG_1.pluginPayloadAttachment"); + assert_eq!( + copy_file_name(&att(5, Some("image/heic")), src), + "5-IMG_1.heic" + ); + assert_eq!( + copy_file_name(&att(5, None), src), + "5-IMG_1.pluginPayloadAttachment" + ); + } + + #[test] + fn runtime_copies_are_found_as_files_or_folders() { + let tmp = TempDir::new().unwrap(); + let chat = tmp.path().join("attachments/7"); + fs::create_dir_all(chat.join("12")).unwrap(); + fs::write(chat.join("12/inner.bin"), b"abcd").unwrap(); + fs::write(chat.join("11.jpeg"), b"xy").unwrap(); + + let stems = runtime_copy_sizes(tmp.path()); + assert_eq!(stems.get("11"), Some(&2)); + assert_eq!(stems.get("12"), Some(&4)); + } + + #[test] + fn source_size_sums_folder_contents() { + let tmp = TempDir::new().unwrap(); + let f = tmp.path().join("a.jpg"); + fs::write(&f, b"abc").unwrap(); + let d = tmp.path().join("bundle"); + fs::create_dir_all(d.join("sub")).unwrap(); + fs::write(d.join("x"), b"12").unwrap(); + fs::write(d.join("sub/y"), b"345").unwrap(); + assert_eq!(source_size(&f), Some(3)); + assert_eq!(source_size(&d), Some(5)); + assert_eq!(source_size(&tmp.path().join("missing")), None); + } + + #[test] + fn ndjson_copied_to_is_filled_in() { + let tmp = TempDir::new().unwrap(); + let file = tmp.path().join("chat.ndjson"); + fs::write( + &file, + "{\"rowid\":1,\"attachments\":[{\"rowid\":7,\"copiedTo\":null},{\"rowid\":8,\"copiedTo\":null}]}\n\ + {\"rowid\":2,\"attachments\":[]}\n", + ) + .unwrap(); + let manifest = tmp.path().join("manifest.json"); + fs::write(&manifest, r#"{"counts":{"attachments":2}}"#).unwrap(); + let copied = HashMap::from([(7, "attachments/1/7-a.jpg".to_string())]); + let summary = CopySummary { + copied: 1, + not_on_mac: 1, + bytes: 10, + ..Default::default() + }; + + annotate_json_export(tmp.path(), &manifest, &copied, &summary).unwrap(); + + let lines: Vec = fs::read_to_string(&file) + .unwrap() + .lines() + .map(|l| serde_json::from_str(l).unwrap()) + .collect(); + assert_eq!(lines.len(), 2); + assert_eq!( + lines[0]["attachments"][0]["copiedTo"], + "attachments/1/7-a.jpg" + ); + assert!(lines[0]["attachments"][1]["copiedTo"].is_null()); + let m: serde_json::Value = serde_json::from_slice(&fs::read(&manifest).unwrap()).unwrap(); + assert_eq!(m["counts"]["attachmentsCopied"], 1); + assert_eq!(m["counts"]["attachmentsNotOnMac"], 1); + } +} diff --git a/src-tauri/src/core/attachment_files.rs b/src-tauri/src/core/attachment_files.rs new file mode 100644 index 0000000..d654dd9 --- /dev/null +++ b/src-tauri/src/core/attachment_files.rs @@ -0,0 +1,289 @@ +use std::fs; +use std::path::{Component, Path, PathBuf}; + +/// Directories Messages stores attachment payloads in. Anything this app +/// unlinks from disk must live under one of these. +const ATTACHMENT_DIRS: &[&str] = &[ + "~/Library/Messages/Attachments", + "~/Library/Messages/StickerCache", +]; + +/// Expand a leading `~` to the user's home directory, matching the library's +/// own `gen_macos_attachment` behaviour. +pub fn expand_tilde(path: &str) -> PathBuf { + if path.starts_with('~') { + if let Ok(home) = crate::core::db_path::home_dir() { + return PathBuf::from(path.replacen('~', &home.to_string_lossy(), 1)); + } + } + PathBuf::from(path) +} + +/// `roots` with symlinks resolved where they exist (e.g. macOS's `/var` → +/// `/private/var`). Everything that compares a resolved path against the +/// roots resolves the roots too, so callers can pass them either way. +pub fn canonical_roots(roots: &[PathBuf]) -> Vec { + roots + .iter() + .map(|r| fs::canonicalize(r).unwrap_or_else(|_| r.clone())) + .collect() +} + +/// The attachment folders, with symlinks resolved when they exist, so +/// walked paths and normalised chat.db paths compare like for like. +pub fn attachment_roots() -> Vec { + ATTACHMENT_DIRS + .iter() + .map(|d| { + let p = expand_tilde(d); + fs::canonicalize(&p).unwrap_or(p) + }) + .collect() +} + +/// Case-insensitive comparison key for a path. APFS is case-insensitive by +/// default, and chat.db paths don't always match on-disk casing. +pub fn path_key(path: &Path) -> String { + path.to_string_lossy().to_lowercase() +} + +/// `true` when `path` is lexically inside one of `roots` and contains no +/// `..` components that could walk back out. +pub fn is_within_roots(path: &Path, roots: &[PathBuf]) -> bool { + if path.components().any(|c| matches!(c, Component::ParentDir)) { + return false; + } + let key = path_key(path); + roots.iter().any(|r| { + let root = path_key(r); + key.len() > root.len() && key.starts_with(&root) && key.as_bytes()[root.len()] == b'/' + }) +} + +/// Recursively collect regular files under `dir`. Symlinks are never +/// followed or returned, so a link inside the Messages folders can't steer +/// the walk (or a later delete) outside of it, and link cycles can't recurse +/// forever. +pub fn walk_files(dir: &Path, out: &mut Vec) { + let Ok(entries) = fs::read_dir(dir) else { + return; + }; + for entry in entries.flatten() { + let Ok(ft) = entry.file_type() else { + continue; + }; + if ft.is_dir() { + walk_files(&entry.path(), out); + } else if ft.is_file() { + out.push(entry.path()); + } + } +} + +#[derive(Debug, PartialEq, Eq)] +pub enum RemoveOutcome { + Removed, + /// Nothing to do: the file is already gone. + Missing, + /// Refused: outside the Messages attachment folders, or not a regular + /// file (symlink, directory, …). + Refused, + Failed, +} + +/// Unlink an attachment file, but only if it is a regular file inside one of +/// `roots`. chat.db is the source of these paths; a row pointing somewhere +/// else on disk must never turn into deleting an unrelated file. +pub fn remove_attachment_file(path: &Path, roots: &[PathBuf], context: &str) -> RemoveOutcome { + let roots = &canonical_roots(roots); + let meta = match fs::symlink_metadata(path) { + Ok(m) => m, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => return RemoveOutcome::Missing, + Err(e) => { + eprintln!("{context}: cannot stat {}: {e}", path.display()); + return RemoveOutcome::Failed; + } + }; + // Check containment on the real location: resolve symlinks in the + // parent folders (the file itself must not be a link — checked above via + // `symlink_metadata`), so a link anywhere up the path can't point the + // removal outside the Messages folders, and a symlinked home still + // matches the canonical roots. + let real = match (path.parent().map(fs::canonicalize), path.file_name()) { + (Some(Ok(parent)), Some(name)) => parent.join(name), + _ => path.to_path_buf(), + }; + if !meta.file_type().is_file() || !is_within_roots(&real, roots) { + eprintln!( + "{context}: refusing to remove {} (not a regular file inside the Messages attachment folders)", + path.display() + ); + return RemoveOutcome::Refused; + } + match fs::remove_file(&real) { + Ok(()) => { + remove_empty_parents(&real, roots); + RemoveOutcome::Removed + } + Err(e) => { + eprintln!("{context}: failed to remove {}: {e}", path.display()); + RemoveOutcome::Failed + } + } +} + +/// After unlinking `path`, remove the folders it leaves empty, walking up +/// to (but never including) the attachment root. `remove_dir` refuses +/// non-empty folders, so this can only ever remove empty ones. +fn remove_empty_parents(path: &Path, roots: &[PathBuf]) { + let mut dir = path.parent(); + while let Some(d) = dir { + if !is_within_roots(d, roots) || fs::remove_dir(d).is_err() { + break; + } + dir = d.parent(); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use tempfile::TempDir; + + #[test] + fn within_roots_rejects_escapes_and_prefix_tricks() { + let roots = vec![PathBuf::from("/Users/a/Library/Messages/Attachments")]; + assert!(is_within_roots( + Path::new("/Users/a/Library/Messages/Attachments/ab/01/G/x.jpg"), + &roots + )); + assert!(is_within_roots( + Path::new("/users/A/library/messages/attachments/ab/x.jpg"), + &roots + )); + assert!(!is_within_roots( + Path::new("/Users/a/Library/Messages/Attachments/../chat.db"), + &roots + )); + assert!(!is_within_roots( + Path::new("/Users/a/Library/Messages/AttachmentsEvil/x.jpg"), + &roots + )); + assert!(!is_within_roots( + Path::new("/Users/a/Library/Messages/Attachments"), + &roots + )); + assert!(!is_within_roots(Path::new("/Users/a/Documents/x"), &roots)); + } + + #[test] + fn remove_refuses_outside_roots_and_symlinks() { + let tmp = TempDir::new().unwrap(); + let root = tmp.path().join("Attachments"); + fs::create_dir_all(&root).unwrap(); + let outside = tmp.path().join("keep.txt"); + fs::write(&outside, b"x").unwrap(); + let roots = vec![root.clone()]; + + assert_eq!( + remove_attachment_file(&outside, &roots, "test"), + RemoveOutcome::Refused + ); + assert!(outside.exists()); + + #[cfg(unix)] + { + let link = root.join("link"); + std::os::unix::fs::symlink(&outside, &link).unwrap(); + assert_eq!( + remove_attachment_file(&link, &roots, "test"), + RemoveOutcome::Refused + ); + assert!(outside.exists()); + } + + let inside = root.join("a.jpg"); + fs::write(&inside, b"x").unwrap(); + assert_eq!( + remove_attachment_file(&inside, &roots, "test"), + RemoveOutcome::Removed + ); + assert_eq!( + remove_attachment_file(&inside, &roots, "test"), + RemoveOutcome::Missing + ); + } + + #[cfg(unix)] + #[test] + fn remove_follows_the_real_location_of_parent_folders() { + let tmp = TempDir::new().unwrap(); + let real_root = tmp.path().join("Attachments"); + fs::create_dir_all(real_root.join("ab")).unwrap(); + let roots = vec![fs::canonicalize(&real_root).unwrap()]; + let outside = tmp.path().join("outside"); + fs::create_dir_all(&outside).unwrap(); + fs::write(outside.join("keep.jpg"), b"x").unwrap(); + + // A folder link inside the root that points outside it. + std::os::unix::fs::symlink(&outside, real_root.join("ab/escape")).unwrap(); + assert_eq!( + remove_attachment_file(&real_root.join("ab/escape/keep.jpg"), &roots, "test"), + RemoveOutcome::Refused + ); + assert!(outside.join("keep.jpg").exists()); + + // The root reached through a linked alias still works. + let alias = tmp.path().join("alias"); + std::os::unix::fs::symlink(&real_root, &alias).unwrap(); + fs::write(real_root.join("ab/a.jpg"), b"x").unwrap(); + assert_eq!( + remove_attachment_file(&alias.join("ab/a.jpg"), &roots, "test"), + RemoveOutcome::Removed + ); + } + + #[test] + fn removing_last_file_prunes_empty_folders_but_not_root() { + let tmp = TempDir::new().unwrap(); + let root = tmp.path().join("Attachments"); + let roots = vec![root.clone()]; + let a = root.join("ab/01/GUID-1/a.jpg"); + let b1 = root.join("ab/02/GUID-2/b.jpg"); + let b2 = root.join("ab/02/GUID-2/b.mov"); + for f in [&a, &b1, &b2] { + fs::create_dir_all(f.parent().unwrap()).unwrap(); + fs::write(f, b"x").unwrap(); + } + + remove_attachment_file(&a, &roots, "test"); + assert!(!root.join("ab/01").exists(), "emptied folders are removed"); + assert!(root.join("ab").exists(), "non-empty parent stays"); + + remove_attachment_file(&b1, &roots, "test"); + assert!(b2.exists() && root.join("ab/02/GUID-2").exists()); + + remove_attachment_file(&b2, &roots, "test"); + assert!(root.exists(), "the root itself is never removed"); + assert!(!root.join("ab").exists()); + } + + #[cfg(unix)] + #[test] + fn walk_does_not_follow_symlinks() { + let tmp = TempDir::new().unwrap(); + let root = tmp.path().join("root"); + let elsewhere = tmp.path().join("elsewhere"); + fs::create_dir_all(root.join("sub")).unwrap(); + fs::create_dir_all(&elsewhere).unwrap(); + fs::write(root.join("sub/a.jpg"), b"x").unwrap(); + fs::write(elsewhere.join("secret.txt"), b"x").unwrap(); + std::os::unix::fs::symlink(&elsewhere, root.join("dirlink")).unwrap(); + std::os::unix::fs::symlink(&root, root.join("sub/loop")).unwrap(); + std::os::unix::fs::symlink(elsewhere.join("secret.txt"), root.join("filelink")).unwrap(); + + let mut out = Vec::new(); + walk_files(&root, &mut out); + assert_eq!(out, vec![root.join("sub/a.jpg")]); + } +} diff --git a/src-tauri/src/core/contacts.rs b/src-tauri/src/core/contacts.rs index 5655e58..b4e9cee 100644 --- a/src-tauri/src/core/contacts.rs +++ b/src-tauri/src/core/contacts.rs @@ -1,18 +1,99 @@ -#![cfg(target_os = "macos")] - use std::collections::HashMap; use std::path::PathBuf; use rusqlite::{Connection, OpenFlags}; -/// Fetch contacts from all local AddressBook sources and return a map of -/// normalized phone/email → display name. +/// Phone numbers are matched on their last this-many digits when an exact +/// match fails, so a contact saved in national format (`07700 900123`) +/// still matches the international handle (`+447700900123`). +const PHONE_SUFFIX_LEN: usize = 9; + +/// Normalized phone/email → display name. +#[derive(Debug, Default)] +pub struct ContactMap { + exact: HashMap, + /// Last [`PHONE_SUFFIX_LEN`] digits → name, or `None` when different + /// contacts share that suffix (ambiguous, so never used). + suffix: HashMap>, +} + +impl ContactMap { + fn insert_phone(&mut self, phone: &str, name: &str) { + let digits: String = phone.chars().filter(|c| c.is_ascii_digit()).collect(); + if digits.is_empty() { + return; + } + for key in phone_lookup_keys(&digits) { + self.exact.entry(key).or_insert_with(|| name.to_string()); + } + if let Some(sfx) = phone_suffix(&digits) { + self.suffix + .entry(sfx) + .and_modify(|existing| { + if existing.as_deref() != Some(name) { + *existing = None; + } + }) + .or_insert_with(|| Some(name.to_string())); + } + } + + fn insert_email(&mut self, email: &str, name: &str) { + let email_lower = email.trim().to_lowercase(); + if !email_lower.is_empty() { + self.exact + .entry(email_lower) + .or_insert_with(|| name.to_string()); + } + } + + /// Resolve a Messages `chat_identifier` / handle (phone or email) to a + /// contact name. + pub fn lookup(&self, identifier: &str) -> Option { + let id = identifier.strip_prefix("tel:").unwrap_or(identifier).trim(); + + if id.contains('@') { + return self.exact.get(&id.to_lowercase()).cloned(); + } + + // Only treat it as a phone number if it looks like one. Group chats + // use opaque identifiers like `chat123456789`, whose digits must not + // be matched against contacts' numbers. + if id.is_empty() + || !id + .chars() + .all(|c| c.is_ascii_digit() || matches!(c, '+' | ' ' | '-' | '(' | ')' | '.')) + { + return None; + } + let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect(); + if digits.is_empty() { + return None; + } + + for key in phone_lookup_keys(&digits) { + if let Some(name) = self.exact.get(&key) { + return Some(name.clone()); + } + } + phone_suffix(&digits) + .and_then(|sfx| self.suffix.get(&sfx)) + .and_then(|n| n.clone()) + } +} + +fn phone_suffix(digits: &str) -> Option { + (digits.len() >= PHONE_SUFFIX_LEN) + .then(|| digits[digits.len() - PHONE_SUFFIX_LEN..].to_string()) +} + +/// Fetch contacts from all local AddressBook sources. /// /// Uses Full Disk Access (already required by the app) to read the /// AddressBook SQLite databases directly. No separate Contacts TCC /// permission is needed. -pub fn fetch_contact_map() -> HashMap { - let mut map = HashMap::new(); +pub fn fetch_contact_map() -> ContactMap { + let mut map = ContactMap::default(); for path in addressbook_db_paths() { if !path.exists() { continue; @@ -45,7 +126,7 @@ fn addressbook_db_paths() -> Vec { paths } -fn load_phones(conn: &Connection, map: &mut HashMap) { +fn load_phones(conn: &Connection, map: &mut ContactMap) { let mut stmt = match conn.prepare( "SELECT r.ZFIRSTNAME, r.ZLASTNAME, r.ZNICKNAME, r.ZORGANIZATION, p.ZFULLNUMBER FROM ZABCDRECORD r @@ -78,16 +159,11 @@ fn load_phones(conn: &Connection, map: &mut HashMap) { ) else { continue; }; - let digits: String = phone.chars().filter(|c| c.is_ascii_digit()).collect(); - if !digits.is_empty() { - for key in phone_lookup_keys(&digits) { - map.entry(key).or_insert_with(|| name.clone()); - } - } + map.insert_phone(&phone, &name); } } -fn load_emails(conn: &Connection, map: &mut HashMap) { +fn load_emails(conn: &Connection, map: &mut ContactMap) { let mut stmt = match conn.prepare( "SELECT r.ZFIRSTNAME, r.ZLASTNAME, r.ZNICKNAME, r.ZORGANIZATION, e.ZADDRESS FROM ZABCDRECORD r @@ -120,10 +196,7 @@ fn load_emails(conn: &Connection, map: &mut HashMap) { ) else { continue; }; - let email_lower = email.to_lowercase(); - if !email_lower.is_empty() { - map.entry(email_lower).or_insert_with(|| name.clone()); - } + map.insert_email(&email, &name); } } @@ -161,23 +234,50 @@ fn phone_lookup_keys(digits: &str) -> Vec { keys } -/// Resolve a Messages `chat_identifier` (phone or email) to a contact name. -pub fn lookup_contact_name(identifier: &str, map: &HashMap) -> Option { - let id = identifier.strip_prefix("tel:").unwrap_or(identifier); +#[cfg(test)] +mod tests { + use super::*; - if id.contains('@') { - return map.get(&id.to_lowercase()).cloned(); + fn map() -> ContactMap { + let mut m = ContactMap::default(); + m.insert_phone("(555) 123-4567", "Ana US"); + m.insert_phone("07700 900123", "Ben UK"); + m.insert_email("Cara@Example.com", "Cara"); + m } - let digits: String = id.chars().filter(|c| c.is_ascii_digit()).collect(); - if digits.is_empty() { - return None; + #[test] + fn matches_us_numbers_with_or_without_country_code() { + let m = map(); + assert_eq!(m.lookup("+15551234567").as_deref(), Some("Ana US")); + assert_eq!(m.lookup("tel:5551234567").as_deref(), Some("Ana US")); } - for key in phone_lookup_keys(&digits) { - if let Some(name) = map.get(&key) { - return Some(name.clone()); - } + #[test] + fn matches_national_format_against_international_handle() { + assert_eq!(map().lookup("+447700900123").as_deref(), Some("Ben UK")); + } + + #[test] + fn emails_are_case_insensitive() { + assert_eq!(map().lookup("cara@example.COM").as_deref(), Some("Cara")); + } + + #[test] + fn group_chat_identifiers_never_match_numbers() { + let mut m = map(); + m.insert_phone("123456789012", "Digits Person"); + assert_eq!(m.lookup("chat123456789012"), None); + } + + #[test] + fn ambiguous_suffixes_are_not_used() { + let mut m = ContactMap::default(); + m.insert_phone("+44 7700 900123", "One"); + m.insert_phone("+61 7700 900123", "Two"); + // Exact matches still work… + assert_eq!(m.lookup("+447700900123").as_deref(), Some("One")); + // …but a third number sharing only the suffix resolves to nobody. + assert_eq!(m.lookup("+3307700900123"), None); } - None } diff --git a/src-tauri/src/core/db_path.rs b/src-tauri/src/core/db_path.rs index 409a818..5ea4a29 100644 --- a/src-tauri/src/core/db_path.rs +++ b/src-tauri/src/core/db_path.rs @@ -1,9 +1,89 @@ +use std::path::{Path, PathBuf}; +use std::time::Duration; + +use imessage_database::tables::table::{get_connection, get_writable_connection}; +use rusqlite::{Connection, TransactionBehavior}; + use crate::error::AppError; -use std::path::PathBuf; + +/// How long a write waits for a lock held by Messages' background +/// processes before giving up. +const BUSY_TIMEOUT: Duration = Duration::from_secs(10); + +#[cfg(test)] +thread_local! { + /// Per-thread stand-in for `$HOME`, so end-to-end tests can run against + /// a throwaway Messages folder in parallel without touching the real + /// one (or each other). + static TEST_HOME: std::cell::RefCell> = const { std::cell::RefCell::new(None) }; +} + +/// Point [`home_dir`] at `dir` for the current thread (tests only). +#[cfg(test)] +pub fn set_test_home(dir: Option) { + TEST_HOME.with(|h| *h.borrow_mut() = dir); +} + +/// The user's home folder. Every path the app derives from it (chat.db, +/// attachment folders, path validation) goes through here. +pub fn home_dir() -> Result { + #[cfg(test)] + if let Some(dir) = TEST_HOME.with(|h| h.borrow().clone()) { + return Ok(dir); + } + std::env::var("HOME") + .map(PathBuf::from) + .map_err(|_| AppError::Other("HOME environment variable is not set".into())) +} /// Default macOS chat.db location: `~/Library/Messages/chat.db`. pub fn default_chat_db_path() -> Result { - let home = std::env::var("HOME") - .map_err(|_| AppError::Other("HOME environment variable is not set".into()))?; - Ok(PathBuf::from(home).join("Library/Messages/chat.db")) + Ok(home_dir()?.join("Library/Messages/chat.db")) +} + +/// Check that `path` is reachable, telling a Full Disk Access denial apart +/// from a missing file. `Path::exists()` (which the library uses) reports a +/// TCC denial as "not found". +pub fn check_chat_db_access(path: &Path) -> Result<(), AppError> { + match std::fs::metadata(path) { + Ok(_) => Ok(()), + Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => { + Err(AppError::FullDiskAccess { + path: path.display().to_string(), + }) + } + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Err(AppError::Other(format!( + "chat.db not found at {}", + path.display() + ))), + Err(e) => Err(AppError::from(e)), + } +} + +/// Open `path` read-only, with Full Disk Access errors reported as such. +pub fn open_read(path: &Path) -> Result { + check_chat_db_access(path)?; + Ok(get_connection(path)?) +} + +/// Open `path` for writing. Waits up to [`BUSY_TIMEOUT`] for locks, and +/// makes `transaction()` take the write lock up front (`BEGIN IMMEDIATE`) +/// so a delete can't fail halfway through on a lock upgrade. +pub fn open_write(path: &Path) -> Result { + check_chat_db_access(path)?; + let mut conn = get_writable_connection(path)?; + conn.busy_timeout(BUSY_TIMEOUT)?; + conn.set_transaction_behavior(TransactionBehavior::Immediate); + Ok(conn) +} + +/// Run blocking work (SQLite, filesystem walks) off the async runtime. +pub async fn blocking(f: F) -> Result +where + F: FnOnce() -> Result + Send + 'static, + T: Send + 'static, +{ + tokio::task::spawn_blocking(f) + .await + .map_err(|e| AppError::Other(format!("background task failed: {e}")))? } diff --git a/src-tauri/src/core/disk_image.rs b/src-tauri/src/core/disk_image.rs new file mode 100644 index 0000000..c930450 --- /dev/null +++ b/src-tauri/src/core/disk_image.rs @@ -0,0 +1,312 @@ +//! Encrypted backups: an AES-256 encrypted APFS sparse bundle, created and +//! mounted with macOS's own `hdiutil`, so the user can open it later in +//! Finder with just the password — no extra software. + +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::process::{Command, Stdio}; + +use serde::Deserialize; + +use crate::error::AppError; + +/// A backup password. Deserialises from a plain string but never prints: +/// `Debug` is redacted so it can't end up in a log by accident. +#[derive(Clone, Deserialize)] +#[serde(transparent)] +pub struct Password(String); + +impl Password { + #[cfg(test)] + pub fn new(s: &str) -> Self { + Self(s.to_string()) + } + + pub fn expose(&self) -> &str { + &self.0 + } +} + +impl std::fmt::Debug for Password { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("Password()") + } +} + +pub const MIN_PASSWORD_LEN: usize = 8; + +/// The disk-image operations a backup needs. `hdiutil` in the app; a fake +/// in tests (there's no `hdiutil` off macOS). +pub trait DiskImageTool { + /// Create an encrypted image at `image` that can grow to `max_bytes`. + fn create( + &self, + image: &Path, + volume_name: &str, + password: &Password, + max_bytes: u64, + ) -> Result<(), AppError>; + /// Mount `image` at `mountpoint` (an empty, existing folder). + fn attach(&self, image: &Path, mountpoint: &Path, password: &Password) -> Result<(), AppError>; + fn detach(&self, mountpoint: &Path) -> Result<(), AppError>; +} + +/// macOS's `hdiutil`. The password is passed on stdin (`-stdinpass`), never +/// as an argument, so it doesn't show up in `ps`. +pub struct Hdiutil; + +fn run_hdiutil(args: &[&std::ffi::OsStr], password: Option<&Password>) -> Result<(), AppError> { + let mut child = Command::new("/usr/bin/hdiutil") + .args(args) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .map_err(|e| AppError::Other(format!("couldn't run hdiutil: {e}")))?; + if let Some(pw) = password { + let mut stdin = child + .stdin + .take() + .ok_or_else(|| AppError::Other("couldn't pass the password to hdiutil".into()))?; + stdin + .write_all(pw.expose().as_bytes()) + .map_err(|e| AppError::Other(format!("couldn't pass the password to hdiutil: {e}")))?; + // Dropping stdin closes it, which ends the password. + } + let out = child + .wait_with_output() + .map_err(|e| AppError::Other(format!("hdiutil failed: {e}")))?; + if !out.status.success() { + return Err(AppError::Other(format!( + "hdiutil {} failed: {}", + args.first() + .map(|a| a.to_string_lossy()) + .unwrap_or_default(), + String::from_utf8_lossy(&out.stderr).trim() + ))); + } + Ok(()) +} + +impl DiskImageTool for Hdiutil { + fn create( + &self, + image: &Path, + volume_name: &str, + password: &Password, + max_bytes: u64, + ) -> Result<(), AppError> { + // A sparse bundle only takes the space its contents need; -size is + // just the most it can grow to. + let size = format!("{}m", max_bytes.div_ceil(1024 * 1024).max(100)); + let args: Vec<&std::ffi::OsStr> = vec![ + "create".as_ref(), + "-size".as_ref(), + size.as_ref(), + "-type".as_ref(), + "SPARSEBUNDLE".as_ref(), + "-fs".as_ref(), + "APFS".as_ref(), + "-encryption".as_ref(), + "AES-256".as_ref(), + "-stdinpass".as_ref(), + "-volname".as_ref(), + volume_name.as_ref(), + image.as_os_str(), + ]; + run_hdiutil(&args, Some(password)) + } + + fn attach(&self, image: &Path, mountpoint: &Path, password: &Password) -> Result<(), AppError> { + let args: Vec<&std::ffi::OsStr> = vec![ + "attach".as_ref(), + "-stdinpass".as_ref(), + "-nobrowse".as_ref(), + "-noautoopen".as_ref(), + "-owners".as_ref(), + "on".as_ref(), + "-mountpoint".as_ref(), + mountpoint.as_os_str(), + image.as_os_str(), + ]; + run_hdiutil(&args, Some(password)) + } + + fn detach(&self, mountpoint: &Path) -> Result<(), AppError> { + let plain: Vec<&std::ffi::OsStr> = vec!["detach".as_ref(), mountpoint.as_os_str()]; + if run_hdiutil(&plain, None).is_ok() { + return Ok(()); + } + // Something (Spotlight, Finder) may still hold a file open briefly. + std::thread::sleep(std::time::Duration::from_secs(2)); + let forced: Vec<&std::ffi::OsStr> = + vec!["detach".as_ref(), "-force".as_ref(), mountpoint.as_os_str()]; + run_hdiutil(&forced, None) + } +} + +/// A mounted encrypted image. Detaches (and removes the empty mount folder) +/// when dropped, so an export that fails halfway never leaves the backup +/// mounted and readable. +pub struct MountedImage<'a> { + tool: &'a dyn DiskImageTool, + pub image: PathBuf, + pub mountpoint: PathBuf, + detached: bool, +} + +impl<'a> MountedImage<'a> { + /// Create a new encrypted image next to `destination` and mount it. + pub fn create_and_mount( + tool: &'a dyn DiskImageTool, + destination: &Path, + name: &str, + password: &Password, + max_bytes: u64, + ) -> Result { + if password.expose().chars().count() < MIN_PASSWORD_LEN { + return Err(AppError::Other(format!( + "The backup password must be at least {MIN_PASSWORD_LEN} characters." + ))); + } + let image = unique_path(destination, name, "sparsebundle"); + tool.create(&image, name, password, max_bytes)?; + let mountpoint = std::env::temp_dir().join(format!( + "bubblewrap-mount-{}-{}", + std::process::id(), + image + .file_stem() + .map(|s| s.to_string_lossy().replace(' ', "_")) + .unwrap_or_default() + )); + std::fs::create_dir_all(&mountpoint)?; + if let Err(e) = tool.attach(&image, &mountpoint, password) { + let _ = std::fs::remove_dir(&mountpoint); + return Err(e); + } + Ok(Self { + tool, + image, + mountpoint, + detached: false, + }) + } + + /// Unmount now, reporting failure (dropping only logs it). + pub fn detach(mut self) -> Result<(), AppError> { + self.detached = true; + let result = self.tool.detach(&self.mountpoint); + let _ = std::fs::remove_dir(&self.mountpoint); + result + } +} + +impl Drop for MountedImage<'_> { + fn drop(&mut self) { + if !self.detached { + if let Err(e) = self.tool.detach(&self.mountpoint) { + eprintln!( + "backup: couldn't unmount {}: {e:?}", + self.mountpoint.display() + ); + } + let _ = std::fs::remove_dir(&self.mountpoint); + } + } +} + +/// `dir/name.ext`, or `dir/name-1.ext`, … — a path that doesn't exist yet. +fn unique_path(dir: &Path, name: &str, ext: &str) -> PathBuf { + let mut p = dir.join(format!("{name}.{ext}")); + let mut n = 1; + while p.exists() { + p = dir.join(format!("{name}-{n}.{ext}")); + n += 1; + } + p +} + +/// Test stand-in for `hdiutil`: the "image" is a folder holding the +/// password and a `volume/` folder; "mounting" makes the mountpoint a +/// symlink to that volume, after checking the password. +#[cfg(test)] +pub struct FakeImageTool; + +#[cfg(test)] +impl DiskImageTool for FakeImageTool { + fn create(&self, image: &Path, _: &str, password: &Password, _: u64) -> Result<(), AppError> { + assert!(!image.exists(), "hdiutil create refuses existing paths"); + std::fs::create_dir_all(image.join("volume"))?; + std::fs::write(image.join("password"), password.expose())?; + Ok(()) + } + + fn attach(&self, image: &Path, mountpoint: &Path, password: &Password) -> Result<(), AppError> { + if std::fs::read_to_string(image.join("password"))? != password.expose() { + return Err(AppError::Other( + "hdiutil attach failed: Authentication error".into(), + )); + } + std::fs::remove_dir(mountpoint)?; + #[cfg(unix)] + std::os::unix::fs::symlink(image.join("volume"), mountpoint)?; + Ok(()) + } + + fn detach(&self, mountpoint: &Path) -> Result<(), AppError> { + std::fs::remove_file(mountpoint)?; + std::fs::create_dir(mountpoint)?; + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn password_never_prints() { + let pw = Password::new("hunter2-secret"); + assert!(!format!("{pw:?}").contains("hunter2")); + } + + #[test] + fn short_passwords_are_refused() { + let tmp = tempfile::TempDir::new().unwrap(); + let err = MountedImage::create_and_mount( + &FakeImageTool, + tmp.path(), + "b", + &Password::new("short"), + 1, + ) + .err() + .unwrap(); + assert!(matches!(err, AppError::Other(m) if m.contains("at least 8"))); + } + + #[cfg(unix)] + #[test] + fn dropping_a_mounted_image_unmounts_it() { + let tmp = tempfile::TempDir::new().unwrap(); + let mountpoint; + { + let m = MountedImage::create_and_mount( + &FakeImageTool, + tmp.path(), + "Backup", + &Password::new("long enough"), + 1, + ) + .unwrap(); + mountpoint = m.mountpoint.clone(); + std::fs::write(m.mountpoint.join("x"), b"inside").unwrap(); + // An error mid-export: the guard goes out of scope. + } + assert!(!mountpoint.exists(), "unmounted and mount folder removed"); + assert_eq!( + std::fs::read(tmp.path().join("Backup.sparsebundle/volume/x")).unwrap(), + b"inside" + ); + } +} diff --git a/src-tauri/src/core/filter.rs b/src-tauri/src/core/filter.rs index 44abe02..e163826 100644 --- a/src-tauri/src/core/filter.rs +++ b/src-tauri/src/core/filter.rs @@ -1,9 +1,9 @@ use crate::error::AppError; use imessage_database::util::query_context::QueryContext; -use serde::Deserialize; +use serde::{Deserialize, Serialize}; use std::collections::BTreeSet; -#[derive(Debug, Deserialize, Default)] +#[derive(Debug, Deserialize, Default, Clone)] #[serde(rename_all = "camelCase")] pub struct FilterSpec { pub date_range: Option, @@ -15,14 +15,14 @@ pub struct FilterSpec { pub attachments: Option, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Deserialize, Clone)] #[serde(rename_all = "camelCase")] pub struct DateRange { pub start: Option, pub end: Option, } -#[derive(Debug, Deserialize)] +#[derive(Debug, Deserialize, Clone)] #[serde(rename_all = "camelCase")] #[allow(dead_code)] pub struct AttachmentFilterSpec { @@ -31,8 +31,70 @@ pub struct AttachmentFilterSpec { pub max_bytes: Option, } +/// Canonical form of a [`FilterSpec`], used to check whether a backup taken +/// earlier in the session covers exactly the records a delete will remove. +/// Empty strings and empty id lists are folded to `None`, matching how +/// `to_query_context` (and the library's `QueryContext`) treat them. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct FilterKey { + start: Option, + end: Option, + chat_ids: Option>, + handle_ids: Option>, +} + +fn non_empty_str(v: Option<&str>) -> Option { + v.map(str::trim) + .filter(|s| !s.is_empty()) + .map(str::to_string) +} + +fn non_empty_set(ids: Option<&Vec>) -> Option> { + ids.map(|v| v.iter().copied().collect::>()) + .filter(|s| !s.is_empty()) +} + +/// The library filters with `date <= end`, where `end` is local midnight at +/// the *start* of the given day — so a plain `set_end("2020-12-31")` would +/// leave out everything sent on Dec 31. Users mean "through the end of that +/// day", so set the bound to one nanosecond before the following midnight. +fn set_inclusive_end(ctx: &mut QueryContext, end: &str) -> Result<(), AppError> { + let invalid = |e: &dyn std::fmt::Display| AppError::Other(format!("invalid end date: {e}")); + let day = + chrono::NaiveDate::parse_from_str(end.trim(), "%Y-%m-%d").map_err(|_| invalid(&end))?; + let next = day.succ_opt().ok_or_else(|| invalid(&end))?; + ctx.set_end(&next.format("%Y-%m-%d").to_string()) + .map_err(|e| invalid(&e))?; + ctx.end = ctx.end.map(|ns| ns - 1); + Ok(()) +} + impl FilterSpec { + pub fn key(&self) -> FilterKey { + let range = self.date_range.as_ref(); + FilterKey { + start: non_empty_str(range.and_then(|r| r.start.as_deref())), + end: non_empty_str(range.and_then(|r| r.end.as_deref())), + chat_ids: non_empty_set(self.chat_ids.as_ref()), + handle_ids: non_empty_set(self.handle_ids.as_ref()), + } + } + pub fn to_query_context(&self) -> Result { + // The library marks a handle filter as active but its SQL ignores + // it, so a contact-only filter would quietly match every message — + // and a delete would target the whole database. Refuse filters + // that aren't actually applied rather than widen the scope. + if self.handle_ids.as_ref().is_some_and(|v| !v.is_empty()) { + return Err(AppError::Other( + "Filtering by contact isn't supported yet; filter by conversation instead.".into(), + )); + } + if self.attachments.is_some() { + return Err(AppError::Other( + "Filtering by attachment type or size isn't supported yet.".into(), + )); + } let mut ctx = QueryContext::default(); if let Some(range) = &self.date_range { @@ -41,8 +103,7 @@ impl FilterSpec { .map_err(|e| AppError::Other(format!("invalid start date: {e}")))?; } if let Some(e) = range.end.as_deref().filter(|v| !v.is_empty()) { - ctx.set_end(e) - .map_err(|e| AppError::Other(format!("invalid end date: {e}")))?; + set_inclusive_end(&mut ctx, e)?; } } @@ -51,11 +112,98 @@ impl FilterSpec { ctx.set_selected_chat_ids(set); } - if let Some(ids) = &self.handle_ids { - let set: BTreeSet = ids.iter().copied().collect(); - ctx.set_selected_handle_ids(set); + Ok(ctx) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn spec(start: Option<&str>, chats: Option>) -> FilterSpec { + FilterSpec { + date_range: Some(DateRange { + start: start.map(str::to_string), + end: None, + }), + chat_ids: chats, + ..Default::default() } + } - Ok(ctx) + #[test] + fn key_ignores_order_duplicates_and_empties() { + assert_eq!( + spec(Some("2020"), Some(vec![3, 1, 3])).key(), + spec(Some("2020"), Some(vec![1, 3])).key() + ); + assert_eq!( + spec(Some(""), Some(vec![])).key(), + FilterSpec::default().key() + ); + } + + fn ctx_for(start: Option<&str>, end: Option<&str>) -> QueryContext { + FilterSpec { + date_range: Some(DateRange { + start: start.map(str::to_string), + end: end.map(str::to_string), + }), + ..Default::default() + } + .to_query_context() + .unwrap() + } + + #[test] + fn end_date_includes_the_whole_end_day() { + // End of 2020-05-15 is one nanosecond before the start of 2020-05-16. + let end = ctx_for(None, Some("2020-05-15")).end.unwrap(); + let next_start = ctx_for(Some("2020-05-16"), None).start.unwrap(); + let same_start = ctx_for(Some("2020-05-15"), None).start.unwrap(); + assert_eq!(end, next_start - 1); + assert!(end > same_start); + } + + #[test] + fn end_date_rolls_over_month_and_year() { + let end = ctx_for(None, Some("2020-12-31")).end.unwrap(); + let next_start = ctx_for(Some("2021-01-01"), None).start.unwrap(); + assert_eq!(end, next_start - 1); + } + + #[test] + fn unapplied_filters_are_rejected_not_ignored() { + let by_contact = FilterSpec { + handle_ids: Some(vec![3]), + ..Default::default() + }; + assert!(by_contact.to_query_context().is_err()); + let empty_contacts = FilterSpec { + handle_ids: Some(vec![]), + ..Default::default() + }; + assert!(!empty_contacts.to_query_context().unwrap().has_filters()); + } + + #[test] + fn invalid_end_date_is_rejected() { + let spec = FilterSpec { + date_range: Some(DateRange { + start: None, + end: Some("2020-02-30".into()), + }), + ..Default::default() + }; + assert!(spec.to_query_context().is_err()); + } + + #[test] + fn key_distinguishes_different_filters() { + assert_ne!( + spec(Some("2020"), None).key(), + spec(Some("2021"), None).key() + ); + assert_ne!(spec(None, Some(vec![1])).key(), spec(None, None).key()); } } diff --git a/src-tauri/src/core/mod.rs b/src-tauri/src/core/mod.rs index ae25519..e53a5f1 100644 --- a/src-tauri/src/core/mod.rs +++ b/src-tauri/src/core/mod.rs @@ -1,9 +1,18 @@ -#[cfg(target_os = "macos")] +pub mod attachment_copy; +pub mod attachment_files; +// Reading AddressBook only happens on macOS, but the matching logic is +// platform-independent so it can be unit-tested anywhere. +#[cfg_attr(not(target_os = "macos"), allow(dead_code))] pub mod contacts; pub mod db_path; +pub mod disk_image; pub mod filter; pub mod icloud; pub mod messages_app; +pub mod op_lock; pub mod paths; pub mod progress; +pub mod receipts; +pub mod retained; pub mod snapshot; +pub mod space; diff --git a/src-tauri/src/core/op_lock.rs b/src-tauri/src/core/op_lock.rs new file mode 100644 index 0000000..a702a71 --- /dev/null +++ b/src-tauri/src/core/op_lock.rs @@ -0,0 +1,39 @@ +use std::sync::Arc; + +use tokio::sync::{Mutex, OwnedMutexGuard}; + +use crate::error::AppError; + +/// Serialises operations that change chat.db, attachment files or the +/// retained-attachment list (backups, deletes, orphan scans and cleans). +/// Running two at once can corrupt each other's assumptions — e.g. an +/// orphan scan during a messages-only delete — so a second one is refused +/// rather than queued. +#[derive(Debug, Default, Clone)] +pub struct OperationLock(Arc>); + +impl OperationLock { + pub fn try_acquire(&self, what: &str) -> Result, AppError> { + self.0.clone().try_lock_owned().map_err(|_| { + AppError::Other(format!( + "Another backup, delete or cleanup is still running. Wait for it to finish \ + before starting {what}." + )) + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn second_operation_is_refused_until_first_finishes() { + let lock = OperationLock::default(); + let guard = lock.try_acquire("a delete").unwrap(); + let err = lock.try_acquire("a cleanup").unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("still running"))); + drop(guard); + assert!(lock.try_acquire("a cleanup").is_ok()); + } +} diff --git a/src-tauri/src/core/paths.rs b/src-tauri/src/core/paths.rs index 722a1b8..888bcf5 100644 --- a/src-tauri/src/core/paths.rs +++ b/src-tauri/src/core/paths.rs @@ -7,7 +7,11 @@ use crate::error::AppError; /// - path must not be empty /// - path must be absolute (Tauri's native file picker always produces /// absolute paths; a relative path on the IPC wire is suspicious) -/// - path must resolve under the user's `$HOME` +/// - path must resolve under the user's `$HOME` or onto a mounted volume +/// under `/Volumes//` (external drives are the natural place for an +/// archive) +/// - path must not be inside `~/Library/Messages`, where an export would +/// mix with live data (and look like orphaned attachments) /// /// Returns the canonicalised path on success. Non-existent destinations are /// permitted — we canonicalise the longest existing prefix and append the @@ -24,14 +28,20 @@ pub fn validate_user_path(raw: &str, purpose: &str) -> Result ))); } - let home = std::env::var("HOME") - .map_err(|_| AppError::Other("HOME environment variable is not set".into()))?; - let home = canonical_or_self(Path::new(&home)); + let home = canonical_or_self(&crate::core::db_path::home_dir()?); let canonical = canonical_prefix(&p); - if !canonical.starts_with(&home) { + let messages_dir = home.join("Library/Messages"); + if canonical.starts_with(&messages_dir) { return Err(AppError::Other(format!( - "{purpose} must be inside your home directory ({}): {raw}", + "{purpose} can't be inside {}: {raw}", + messages_dir.display() + ))); + } + + if !canonical.starts_with(&home) && !is_on_mounted_volume(&canonical) { + return Err(AppError::Other(format!( + "{purpose} must be inside your home directory ({}) or on an external drive: {raw}", home.display() ))); } @@ -39,10 +49,20 @@ pub fn validate_user_path(raw: &str, purpose: &str) -> Result Ok(canonical) } +/// `true` for paths inside a volume mounted at `/Volumes/`. The boot +/// volume's `/Volumes/Macintosh HD` entry is a symlink to `/`, so it +/// canonicalises away and is not accepted here. +fn is_on_mounted_volume(canonical: &Path) -> bool { + canonical + .strip_prefix("/Volumes") + .map(|rest| rest.components().count() >= 1) + .unwrap_or(false) +} + /// Canonicalise as much of `path` as already exists; preserve the rest. /// Needed because `fs::canonicalize` errors when the target doesn't yet /// exist (typical for new backup destinations). -fn canonical_prefix(path: &Path) -> PathBuf { +pub(crate) fn canonical_prefix(path: &Path) -> PathBuf { let mut existing = path.to_path_buf(); let mut trailing: Vec = Vec::new(); while !existing.exists() { @@ -99,6 +119,27 @@ mod tests { assert!(resolved.ends_with("nested/path")); } + #[test] + fn accepts_external_volume() { + let resolved = + validate_user_path("/Volumes/BubbleWrapTestDrive/archive", "destination").unwrap(); + assert!(resolved.ends_with("BubbleWrapTestDrive/archive")); + } + + #[test] + fn rejects_volumes_root() { + let err = validate_user_path("/Volumes", "destination").unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("external drive"))); + } + + #[test] + fn rejects_messages_library() { + let home = std::env::var("HOME").unwrap(); + let target = format!("{home}/Library/Messages/Attachments/export"); + let err = validate_user_path(&target, "destination").unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("can't be inside"))); + } + #[test] fn rejects_traversal_escaping_home() { let home = std::env::var("HOME").unwrap(); diff --git a/src-tauri/src/core/receipts.rs b/src-tauri/src/core/receipts.rs new file mode 100644 index 0000000..811207e --- /dev/null +++ b/src-tauri/src/core/receipts.rs @@ -0,0 +1,270 @@ +use std::fs; +use std::path::{Path, PathBuf}; +use std::sync::Mutex; + +use serde::{Deserialize, Serialize}; + +use crate::core::filter::FilterKey; + +/// How many receipts to keep on disk. +const MAX_RECEIPTS: usize = 100; + +/// Completed backups, so `run_delete` can verify a matching backup actually +/// ran instead of trusting a flag sent by the frontend. Persisted to the +/// app data directory so a relaunch (e.g. the forced one after granting +/// Full Disk Access) doesn't force another full backup. +#[derive(Debug, Default)] +pub struct BackupReceipts { + inner: Mutex>, + /// Where receipts are saved; `None` keeps them in memory only. + path: Option, +} + +/// Identifies exactly which messages and attachments a filter matched at a +/// point in time. A filter alone isn't enough: with no end date (or a +/// chats-only filter) the same filter matches new messages that arrived +/// after the backup. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +pub struct ContentFingerprint { + pub messages: u64, + pub attachments: u64, + pub hash: u64, +} + +impl ContentFingerprint { + pub fn of(message_rowids: &[i64], attachment_rowids: &[i64]) -> Self { + // FNV-1a: stable across runs and Rust versions, unlike + // `DefaultHasher`, which matters because receipts are persisted. + let mut hash: u64 = 0xcbf2_9ce4_8422_2325; + let mut feed = |bytes: &[u8]| { + for b in bytes { + hash ^= u64::from(*b); + hash = hash.wrapping_mul(0x0100_0000_01b3); + } + }; + for ids in [message_rowids, attachment_rowids] { + let mut sorted = ids.to_vec(); + sorted.sort_unstable(); + sorted.dedup(); + for id in sorted { + feed(&id.to_le_bytes()); + } + feed(b"|"); + } + Self { + messages: message_rowids.len() as u64, + attachments: attachment_rowids.len() as u64, + hash, + } + } +} + +/// Whether completed backups cover a given filter's current content. +#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct Coverage { + /// A backup of exactly these messages exists and its folder is still there. + pub covered: bool, + /// …and it copied every attachment file. + pub with_attachments: bool, + /// The covering backup's folder (the one with attachments, if any). + pub export_path: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +struct Receipt { + filter: FilterKey, + /// What the filter matched when the backup started. + content: ContentFingerprint, + /// The attachments whose files were actually copied. Files that + /// weren't on the Mac then (and were downloaded later) aren't in it. + copied: ContentFingerprint, + includes_attachments: bool, + /// The run's output folder. A receipt only counts while this still + /// exists — a deleted backup protects nothing. + export_path: PathBuf, +} + +impl BackupReceipts { + /// Receipts backed by `path`, loading any saved earlier. An unreadable + /// file just starts empty: a missing receipt only means the user is + /// asked to back up again. + pub fn persistent(path: PathBuf) -> Self { + let saved = fs::read(&path) + .ok() + .and_then(|b| serde_json::from_slice::>(&b).ok()) + .unwrap_or_default(); + Self { + inner: Mutex::new(saved), + path: Some(path), + } + } + + pub fn record( + &self, + filter: FilterKey, + content: ContentFingerprint, + copied: ContentFingerprint, + includes_attachments: bool, + export_path: &Path, + ) { + let mut g = self.inner.lock().unwrap_or_else(|p| p.into_inner()); + g.push(Receipt { + filter, + content, + copied, + includes_attachments, + export_path: export_path.to_path_buf(), + }); + let excess = g.len().saturating_sub(MAX_RECEIPTS); + g.drain(..excess); + if let Some(path) = &self.path { + if let Err(e) = save(path, &g) { + eprintln!("receipts: failed to save {}: {e}", path.display()); + } + } + } + + /// Whether a successful backup with this filter, of exactly this + /// content, still exists on disk — and whether it holds a copy of every + /// attachment file that is on this Mac now (`present`). + pub fn coverage( + &self, + filter: &FilterKey, + content: &ContentFingerprint, + present: &ContentFingerprint, + ) -> Coverage { + let g = self.inner.lock().unwrap_or_else(|p| p.into_inner()); + let matching = || { + g.iter() + .rev() + .filter(|r| &r.filter == filter && &r.content == content && r.export_path.is_dir()) + }; + let has_files = |r: &Receipt| r.includes_attachments && &r.copied == present; + let best = matching() + .find(|r| has_files(r)) + .or_else(|| matching().next()); + match best { + Some(r) => Coverage { + covered: true, + with_attachments: has_files(r), + export_path: Some(r.export_path.display().to_string()), + }, + None => Coverage::default(), + } + } +} + +fn save(path: &Path, receipts: &[Receipt]) -> std::io::Result<()> { + if let Some(dir) = path.parent() { + fs::create_dir_all(dir)?; + } + let tmp = path.with_extension("json.tmp"); + fs::write(&tmp, serde_json::to_vec(receipts)?)?; + fs::rename(&tmp, path) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::core::filter::FilterSpec; + use tempfile::TempDir; + + fn files(ids: &[i64]) -> ContentFingerprint { + ContentFingerprint::of(&[], ids) + } + + #[test] + fn coverage_requires_matching_filter_content_and_attachments() { + let tmp = TempDir::new().unwrap(); + let receipts = BackupReceipts::default(); + let key = FilterSpec::default().key(); + let content = ContentFingerprint::of(&[1, 2], &[10]); + let present = files(&[10]); + assert!(!receipts.coverage(&key, &content, &present).covered); + + receipts.record(key.clone(), content, files(&[]), false, tmp.path()); + let c = receipts.coverage(&key, &content, &present); + assert!(c.covered && !c.with_attachments); + + receipts.record(key.clone(), content, files(&[10]), true, tmp.path()); + assert!(receipts.coverage(&key, &content, &present).with_attachments); + } + + #[test] + fn files_downloaded_after_the_backup_are_not_covered() { + let tmp = TempDir::new().unwrap(); + let receipts = BackupReceipts::default(); + let key = FilterSpec::default().key(); + let content = ContentFingerprint::of(&[1], &[10, 11]); + // At backup time only attachment 10 was on the Mac (11 was in iCloud). + receipts.record(key.clone(), content, files(&[10]), true, tmp.path()); + assert!( + receipts + .coverage(&key, &content, &files(&[10])) + .with_attachments + ); + + // The user downloads 11; the backup doesn't have it. + let c = receipts.coverage(&key, &content, &files(&[10, 11])); + assert!(c.covered && !c.with_attachments); + } + + #[test] + fn new_messages_since_the_backup_break_coverage() { + let tmp = TempDir::new().unwrap(); + let receipts = BackupReceipts::default(); + let key = FilterSpec::default().key(); + let f = files(&[10]); + receipts.record( + key.clone(), + ContentFingerprint::of(&[1, 2], &[10]), + f, + true, + tmp.path(), + ); + // Same filter, but a message (or attachment) arrived afterwards. + assert!( + !receipts + .coverage(&key, &ContentFingerprint::of(&[1, 2, 3], &[10]), &f) + .covered + ); + assert!( + !receipts + .coverage(&key, &ContentFingerprint::of(&[1, 2], &[10, 11]), &f) + .covered + ); + // Order doesn't matter. + assert!( + receipts + .coverage(&key, &ContentFingerprint::of(&[2, 1], &[10]), &f) + .covered + ); + } + + #[test] + fn fingerprint_distinguishes_message_and_attachment_ids() { + assert_ne!( + ContentFingerprint::of(&[1], &[2]), + ContentFingerprint::of(&[2], &[1]) + ); + } + + #[test] + fn receipts_survive_a_restart_but_not_a_deleted_backup() { + let tmp = TempDir::new().unwrap(); + let file = tmp.path().join("receipts.json"); + let export = tmp.path().join("Bubble Wrap 1"); + fs::create_dir_all(&export).unwrap(); + let key = FilterSpec::default().key(); + let content = ContentFingerprint::of(&[1], &[]); + let f = files(&[]); + + BackupReceipts::persistent(file.clone()).record(key.clone(), content, f, true, &export); + let reloaded = BackupReceipts::persistent(file.clone()); + assert!(reloaded.coverage(&key, &content, &f).with_attachments); + + fs::remove_dir_all(&export).unwrap(); + assert!(!reloaded.coverage(&key, &content, &f).covered); + } +} diff --git a/src-tauri/src/core/retained.rs b/src-tauri/src/core/retained.rs new file mode 100644 index 0000000..246ed3b --- /dev/null +++ b/src-tauri/src/core/retained.rs @@ -0,0 +1,151 @@ +use std::collections::BTreeSet; +use std::fs; +use std::path::{Path, PathBuf}; + +use crate::error::AppError; + +/// Attachment rows the user chose to keep when running a "messages only" +/// delete. Deleting the messages also removes their +/// `message_attachment_join` rows, which makes these attachments look like +/// orphans; the orphan cleaner must skip them or it would delete exactly +/// what the user asked to keep. +pub struct RetainedAttachments { + path: PathBuf, +} + +const FILE_NAME: &str = "retained_attachments.json"; + +impl RetainedAttachments { + pub fn in_dir(dir: &Path) -> Self { + Self { + path: dir.join(FILE_NAME), + } + } + + /// The store for this app install, under the app data directory. + pub fn for_app(app: &tauri::AppHandle) -> Result { + use tauri::Manager; + let dir = app + .path() + .app_data_dir() + .map_err(|e| AppError::Other(format!("cannot locate app data directory: {e}")))?; + Ok(Self::in_dir(&dir)) + } + + pub fn load(&self) -> Result, AppError> { + let bytes = match fs::read(&self.path) { + Ok(b) => b, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(BTreeSet::new()), + Err(e) => return Err(e.into()), + }; + if let Ok(set) = serde_json::from_slice(&bytes) { + return Ok(set); + } + // Damaged (shouldn't happen with atomic writes, but a failing disk + // or a hand edit can do it). Refusing to load would block every + // cleanup and messages-only delete for good; starting empty would + // silently unprotect kept attachments. Instead keep every number + // we can still find — protecting a few extra ids is harmless — and + // set the damaged file aside for inspection. + let salvaged = salvage_ids(&bytes); + let aside = self.path.with_extension("json.damaged"); + eprintln!( + "retained: {} was unreadable; recovered {} ids, original kept at {}", + self.path.display(), + salvaged.len(), + aside.display() + ); + fs::copy(&self.path, &aside)?; + self.save(&salvaged)?; + Ok(salvaged) + } + + /// Merge `ids` into the persisted set and return the ones that weren't + /// already in it, so a caller can undo exactly its own addition with + /// [`Self::remove`]. + // Nothing adds ids any more: "messages only" deletes were removed + // because chat.db's triggers delete the attachment records anyway. The + // list is still read so ids saved by earlier builds keep working. + #[cfg(test)] + pub fn add(&self, ids: &[i64]) -> Result, AppError> { + let mut set = self.load()?; + let added: Vec = ids.iter().copied().filter(|id| set.insert(*id)).collect(); + if !added.is_empty() { + self.save(&set)?; + } + Ok(added) + } + + /// Remove `ids` from the persisted set. + pub fn remove(&self, ids: &[i64]) -> Result<(), AppError> { + if ids.is_empty() { + return Ok(()); + } + let mut set = self.load()?; + let before = set.len(); + for id in ids { + set.remove(id); + } + if set.len() != before { + self.save(&set)?; + } + Ok(()) + } + + /// Written via a temp file + rename so a crash can't leave a truncated + /// list behind. + fn save(&self, set: &BTreeSet) -> Result<(), AppError> { + if let Some(dir) = self.path.parent() { + fs::create_dir_all(dir)?; + } + let tmp = self.path.with_extension("json.tmp"); + let json = serde_json::to_vec(set).map_err(|e| AppError::Other(e.to_string()))?; + fs::write(&tmp, json)?; + fs::rename(&tmp, &self.path)?; + Ok(()) + } +} + +/// Every integer in `bytes`, however mangled the surrounding JSON is. +fn salvage_ids(bytes: &[u8]) -> BTreeSet { + String::from_utf8_lossy(bytes) + .split(|c: char| !c.is_ascii_digit() && c != '-') + .filter_map(|t| t.parse::().ok()) + .collect() +} + +#[cfg(test)] +mod tests { + use super::*; + use tempfile::TempDir; + + #[test] + fn round_trips_and_merges() { + let tmp = TempDir::new().unwrap(); + let r = RetainedAttachments::in_dir(&tmp.path().join("nested")); + assert!(r.load().unwrap().is_empty()); + assert_eq!(r.add(&[3, 1]).unwrap(), vec![3, 1]); + assert_eq!(r.add(&[2, 3]).unwrap(), vec![2]); + assert_eq!(r.load().unwrap(), BTreeSet::from([1, 2, 3])); + + // Undoing the second add leaves ids that were already there. + r.remove(&[2]).unwrap(); + assert_eq!(r.load().unwrap(), BTreeSet::from([1, 3])); + } + + #[test] + fn damaged_list_is_salvaged_not_dropped() { + let tmp = TempDir::new().unwrap(); + let r = RetainedAttachments::in_dir(tmp.path()); + fs::write(tmp.path().join(FILE_NAME), b"[12,34,5\x00garbage").unwrap(); + assert_eq!(r.load().unwrap(), BTreeSet::from([5, 12, 34])); + assert!(tmp + .path() + .join("retained_attachments.json.damaged") + .exists()); + // Rewritten as valid JSON. + assert_eq!(r.load().unwrap(), BTreeSet::from([5, 12, 34])); + r.add(&[7]).unwrap(); + assert_eq!(r.load().unwrap(), BTreeSet::from([5, 7, 12, 34])); + } +} diff --git a/src-tauri/src/core/snapshot.rs b/src-tauri/src/core/snapshot.rs index dcb32e0..8054154 100644 --- a/src-tauri/src/core/snapshot.rs +++ b/src-tauri/src/core/snapshot.rs @@ -32,8 +32,7 @@ pub fn snapshot_chat_db(chat_db: &Path, snapshot_root: &Path) -> Result Result` directory, appending `-1`, `-2`, … if it +/// already exists. Two snapshots in the same second must never share a +/// folder, or the second would overwrite the only pre-delete copy taken by +/// the first. +pub fn create_unique_dir(root: &Path, base: &str) -> Result { + fs::create_dir_all(root)?; + let mut dir = root.join(base); + let mut n = 1u32; + loop { + match fs::create_dir(&dir) { + Ok(()) => return Ok(dir), + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists && n < 1000 => { + dir = root.join(format!("{base}-{n}")); + n += 1; + } + Err(e) => return Err(e.into()), + } + } +} + /// Run `PRAGMA wal_checkpoint(TRUNCATE)` against `chat_db`. Retained as a /// defensive pre-step so readers of the *source* database find a clean WAL; /// the snapshot itself no longer depends on it (we use the SQLite backup /// API, which is WAL-safe). pub fn checkpoint_wal(chat_db: &Path) -> Result<(), AppError> { - use imessage_database::tables::table::get_writable_connection; - let conn = get_writable_connection(chat_db) + let conn = crate::core::db_path::open_write(chat_db) .map_err(|e| AppError::Other(format!("failed to open for checkpoint: {e}")))?; conn.pragma_update(None, "wal_checkpoint", "TRUNCATE") .map_err(|e| AppError::Other(format!("wal_checkpoint failed: {e}")))?; Ok(()) } -/// Default snapshot root under the user's home directory: -/// `~/Documents/iMessage Backups/snapshots`. -pub fn default_snapshot_root() -> Result { - let home = std::env::var("HOME") - .map_err(|_| AppError::Other("HOME environment variable is not set".into()))?; - Ok(PathBuf::from(home).join("Documents/iMessage Backups/snapshots")) +/// What a snapshot was taken for. Each kind gets its own folder and is +/// pruned on its own, so routine orphan cleanups can never push out the +/// snapshot taken before a message delete. +#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)] +#[serde(rename_all = "camelCase")] +pub enum SnapshotKind { + Delete, + OrphanClean, + /// The live database just before a restore, so a restore can be undone. + PreRestore, +} + +impl SnapshotKind { + pub const ALL: [SnapshotKind; 3] = [ + SnapshotKind::Delete, + SnapshotKind::OrphanClean, + SnapshotKind::PreRestore, + ]; + + pub fn dir_name(self) -> &'static str { + match self { + SnapshotKind::Delete => "delete", + SnapshotKind::OrphanClean => "orphan-clean", + SnapshotKind::PreRestore => "pre-restore", + } + } +} + +/// Default snapshot root: `/snapshots/`, i.e. +/// `~/Library/Application Support/io.pecka.apps.bubblewrap/snapshots/delete`. +/// Deliberately not under `~/Documents`, which "Desktop & Documents in +/// iCloud" uploads to iCloud Drive and may evict from local disk. +pub fn default_snapshot_root( + app: &tauri::AppHandle, + kind: SnapshotKind, +) -> Result { + use tauri::Manager; + let dir = app + .path() + .app_data_dir() + .map_err(|e| AppError::Other(format!("cannot locate app data directory: {e}")))?; + Ok(dir.join("snapshots").join(kind.dir_name())) +} + +/// How many snapshots [`prune_snapshots`] keeps per default root. +pub const SNAPSHOTS_TO_KEEP: usize = 5; + +/// Remove all but the `keep` newest snapshot folders under `root`. Only +/// folders this module created (timestamp-named, containing `chat.db`) are +/// considered. Returns how many were removed; failures are logged and +/// skipped since pruning is housekeeping, not part of the safety contract. +pub fn prune_snapshots(root: &Path, keep: usize, just_taken: &Path) -> usize { + let Ok(entries) = fs::read_dir(root) else { + return 0; + }; + let mut dirs: Vec<(SortKey, PathBuf)> = entries + .flatten() + .filter(|e| e.file_type().map(|t| t.is_dir()).unwrap_or(false)) + .map(|e| e.path()) + .filter_map(|p| { + let name = p.file_name()?.to_str()?.to_string(); + let suffix = snapshot_suffix(&name)?; + // Order by when the snapshot was actually written, not by its + // name: names are local time, so after a DST fall-back or a + // westward time-zone change a newer snapshot can sort as older. + let written = fs::metadata(p.join("chat.db")).ok()?.modified().ok()?; + Some(((written, name[..19].to_string(), suffix), p)) + }) + .collect(); + dirs.sort(); + let excess = dirs.len().saturating_sub(keep); + let mut removed = 0; + for (_, dir) in dirs.into_iter().take(excess) { + // Never prune the snapshot protecting the operation in progress. + if dir == just_taken { + continue; + } + match fs::remove_dir_all(&dir) { + Ok(()) => removed += 1, + Err(e) => eprintln!("snapshot: failed to prune {}: {e}", dir.display()), + } + } + removed +} + +/// (write time, timestamp part of the name, numeric `-N` suffix). +type SortKey = (std::time::SystemTime, String, u32); + +/// For a snapshot folder name (`YYYY-MM-DDTHH-MM-SS` plus an optional +/// `-N`), the numeric suffix (0 if none); `None` for other names. +/// A snapshot folder found on disk. +#[derive(Debug, Clone, serde::Serialize)] +#[serde(rename_all = "camelCase")] +pub struct SnapshotInfo { + pub path: String, + pub kind: SnapshotKind, + /// When its chat.db was written, in Unix seconds. + pub taken_at_unix: u64, + pub bytes: u64, +} + +/// Every snapshot this app created under the given per-kind roots, newest +/// first. +pub fn list_snapshots(roots: &[(SnapshotKind, PathBuf)]) -> Vec { + let mut out = Vec::new(); + for (kind, root) in roots { + let Ok(entries) = fs::read_dir(root) else { + continue; + }; + for entry in entries.flatten() { + let path = entry.path(); + let named = path + .file_name() + .and_then(|n| n.to_str()) + .and_then(snapshot_suffix) + .is_some(); + let Ok(meta) = fs::metadata(path.join("chat.db")) else { + continue; + }; + if !named || !entry.file_type().is_ok_and(|t| t.is_dir()) || !meta.is_file() { + continue; + } + let taken_at_unix = meta + .modified() + .ok() + .and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok()) + .map_or(0, |d| d.as_secs()); + out.push(SnapshotInfo { + path: path.display().to_string(), + kind: *kind, + taken_at_unix, + bytes: meta.len(), + }); + } + } + out.sort_by(|a, b| { + b.taken_at_unix + .cmp(&a.taken_at_unix) + .then(b.path.cmp(&a.path)) + }); + out +} + +fn snapshot_suffix(name: &str) -> Option { + if !is_snapshot_name(name) { + return None; + } + match name.get(19..) { + Some("") | None => Some(0), + Some(rest) => rest.strip_prefix('-')?.parse().ok(), + } +} + +fn is_snapshot_name(name: &str) -> bool { + let (base, suffix) = name.split_at(name.len().min(19)); + chrono::NaiveDateTime::parse_from_str(base, "%Y-%m-%dT%H-%M-%S").is_ok() + && (suffix.is_empty() + || suffix + .strip_prefix('-') + .is_some_and(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()))) } #[cfg(test)] @@ -149,6 +321,102 @@ mod tests { assert_eq!(n, 2, "snapshot should preserve the pre-delete state"); } + #[test] + fn back_to_back_snapshots_do_not_overwrite_each_other() { + let tmp = TempDir::new().unwrap(); + let src = tmp.path().join("chat.db"); + make_fixture_db(&src); + let root = tmp.path().join("snaps"); + + let first = snapshot_chat_db(&src, &root).unwrap(); + let w = Connection::open(&src).unwrap(); + w.execute("DELETE FROM message", []).unwrap(); + drop(w); + let second = snapshot_chat_db(&src, &root).unwrap(); + + assert_ne!(first.dir, second.dir); + let conn = Connection::open(first.dir.join("chat.db")).unwrap(); + let n: i64 = conn + .query_row("SELECT COUNT(*) FROM message", [], |r| r.get(0)) + .unwrap(); + assert_eq!(n, 2, "first snapshot must survive a second one"); + } + + #[test] + fn prune_keeps_newest_and_ignores_foreign_folders() { + let tmp = TempDir::new().unwrap(); + let root = tmp.path(); + let names = [ + "2024-01-01T10-00-00", + "2024-01-01T10-00-00-1", + "2024-01-02T09-00-00", + "2025-06-01T00-00-00", + ]; + for n in names { + fs::create_dir_all(root.join(n)).unwrap(); + fs::write(root.join(n).join("chat.db"), b"x").unwrap(); + } + // Not ours: wrong name, or no chat.db. + fs::create_dir_all(root.join("my-notes")).unwrap(); + fs::write(root.join("my-notes/chat.db"), b"x").unwrap(); + fs::create_dir_all(root.join("2020-01-01T00-00-00")).unwrap(); + + assert_eq!( + prune_snapshots(root, 2, &root.join("2025-06-01T00-00-00")), + 2 + ); + assert!(!root.join("2024-01-01T10-00-00").exists()); + assert!(!root.join("2024-01-01T10-00-00-1").exists()); + assert!(root.join("2024-01-02T09-00-00").exists()); + assert!(root.join("2025-06-01T00-00-00").exists()); + assert!(root.join("my-notes").exists()); + assert!(root.join("2020-01-01T00-00-00").exists()); + } + + #[test] + fn prune_orders_by_write_time_and_spares_the_new_snapshot() { + let tmp = TempDir::new().unwrap(); + let root = tmp.path(); + let base = std::time::SystemTime::now() - std::time::Duration::from_secs(10_000); + // Taken in this order, but the clock went back an hour before the + // last one (DST fall-back), so its name sorts first. + let taken = [ + "2024-11-03T01-10-00", + "2024-11-03T01-20-00", + "2024-11-03T01-30-00", + "2024-11-03T01-05-00", + ]; + for (i, n) in taken.iter().enumerate() { + let db = root.join(n).join("chat.db"); + fs::create_dir_all(db.parent().unwrap()).unwrap(); + fs::write(&db, b"x").unwrap(); + let f = fs::File::options().write(true).open(&db).unwrap(); + f.set_modified(base + std::time::Duration::from_secs(i as u64 * 60)) + .unwrap(); + } + let newest = root.join("2024-11-03T01-05-00"); + + assert_eq!(prune_snapshots(root, 2, &newest), 2); + assert!(newest.exists(), "the snapshot just taken survives"); + assert!(root.join("2024-11-03T01-30-00").exists()); + assert!(!root.join("2024-11-03T01-10-00").exists()); + assert!(!root.join("2024-11-03T01-20-00").exists()); + + // Even with keep = 0 the just-taken snapshot is never removed. + assert_eq!(prune_snapshots(root, 0, &newest), 1); + assert!(newest.exists()); + } + + #[test] + fn numeric_suffixes_sort_numerically() { + assert_eq!(snapshot_suffix("2024-01-01T00-00-00"), Some(0)); + assert_eq!(snapshot_suffix("2024-01-01T00-00-00-10"), Some(10)); + assert!( + snapshot_suffix("2024-01-01T00-00-00-2") < snapshot_suffix("2024-01-01T00-00-00-10") + ); + assert_eq!(snapshot_suffix("notes"), None); + } + #[test] fn snapshot_missing_source_errors() { let tmp = TempDir::new().unwrap(); diff --git a/src-tauri/src/core/space.rs b/src-tauri/src/core/space.rs new file mode 100644 index 0000000..cea7c66 --- /dev/null +++ b/src-tauri/src/core/space.rs @@ -0,0 +1,113 @@ +use std::path::{Path, PathBuf}; + +use crate::error::AppError; + +/// Free space (bytes) on the volume holding a path. Injectable so tests can +/// simulate a full disk. +pub type SpaceProbe = fn(&Path) -> std::io::Result; + +/// The real probe: free space on the volume of `path`, or of its nearest +/// existing ancestor (backup and snapshot folders often don't exist yet). +pub fn system_available(path: &Path) -> std::io::Result { + fs2::available_space(nearest_existing(path)) +} + +fn nearest_existing(path: &Path) -> PathBuf { + let mut p = path.to_path_buf(); + while !p.exists() { + match p.parent() { + Some(parent) => p = parent.to_path_buf(), + None => break, + } + } + p +} + +/// Extra room demanded beyond the estimate: 10%, at least 100 MB. Running a +/// volume completely full can break other apps (Messages included). +fn headroom(needed: u64) -> u64 { + (needed / 10).max(100 * 1024 * 1024) +} + +/// Refuse up front when `dir`'s volume can't hold `needed` bytes (plus +/// headroom), rather than failing halfway through a backup or snapshot. +pub fn ensure_space( + dir: &Path, + needed: u64, + what: &str, + probe: SpaceProbe, +) -> Result<(), AppError> { + let available = probe(dir).map_err(|e| { + AppError::Other(format!( + "couldn't check free space for {what} at {}: {e}", + dir.display() + )) + })?; + let required = needed.saturating_add(headroom(needed)); + if available < required { + return Err(AppError::Other(format!( + "Not enough free space for {what}: about {} is needed at {} but only {} is free. \ + Free up space or choose another location.", + human(required), + dir.display(), + human(available) + ))); + } + Ok(()) +} + +pub fn human(bytes: u64) -> String { + const UNITS: [&str; 5] = ["B", "KB", "MB", "GB", "TB"]; + let mut v = bytes as f64; + let mut i = 0; + while v >= 1024.0 && i < UNITS.len() - 1 { + v /= 1024.0; + i += 1; + } + if i == 0 { + format!("{bytes} B") + } else { + format!("{v:.1} {}", UNITS[i]) + } +} + +/// Bytes a snapshot of `chat_db` will take: the database plus its +/// write-ahead log (the backup API copies both into one file). +pub fn snapshot_size(chat_db: &Path) -> u64 { + let len = |p: PathBuf| std::fs::metadata(p).map(|m| m.len()).unwrap_or(0); + let mut wal = chat_db.as_os_str().to_owned(); + wal.push("-wal"); + len(chat_db.to_path_buf()) + len(PathBuf::from(wal)) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn refuses_when_too_little_space_and_says_how_much() { + let err = ensure_space(Path::new("/x"), 1024 * 1024 * 1024, "the backup", |_| { + Ok(500 * 1024 * 1024) + }) + .unwrap_err(); + assert!( + matches!(&err, AppError::Other(m) if m.contains("1.1 GB") && m.contains("500.0 MB")), + "{err:?}" + ); + } + + #[test] + fn allows_when_estimate_plus_headroom_fits() { + assert!(ensure_space(Path::new("/x"), 10, "x", |_| Ok(200 * 1024 * 1024)).is_ok()); + // 100 MB headroom minimum. + assert!(ensure_space(Path::new("/x"), 10, "x", |_| Ok(50 * 1024 * 1024)).is_err()); + } + + #[test] + fn nearest_existing_walks_up_to_a_real_folder() { + let tmp = tempfile::TempDir::new().unwrap(); + let missing = tmp.path().join("a/b/c"); + assert_eq!(nearest_existing(&missing), tmp.path()); + assert!(system_available(&missing).unwrap() > 0); + } +} diff --git a/src-tauri/src/e2e_tests.rs b/src-tauri/src/e2e_tests.rs new file mode 100644 index 0000000..4099007 --- /dev/null +++ b/src-tauri/src/e2e_tests.rs @@ -0,0 +1,1129 @@ +//! End-to-end tests: the real backup → coverage → delete → orphan-cleanup +//! flow, run against a real `chat.db` schema (Apple's tables, triggers and +//! cascading foreign keys, in `tests/fixtures/chat-db-schema.sql`) inside a +//! throwaway home folder. +//! +//! Each test gets its own home via [`set_test_home`], which is per-thread, +//! so tests run in parallel and never touch the real `~/Library/Messages`. +//! Attachment paths are stored absolute (the library expands `~` from the +//! real `$HOME`, which the override can't reach); after `~` expansion that +//! is exactly what a real chat.db holds. + +use std::fs; +use std::path::{Path, PathBuf}; +use std::time::{Duration, SystemTime}; + +use imessage_database::exporters::progress::NoopProgress; +use rusqlite::{params, Connection}; +use tempfile::TempDir; + +use crate::commands::backup::{backup_now, BackupResult, RunBackupArgs}; +use crate::commands::delete::{ + coverage_now, delete_now, preview_delete_now, DeletePreview, DeleteResult, DeleteScope, + RunDeleteArgs, SafetyProbe, DELETE_CONFIRMATION_PHRASE, +}; +use crate::commands::orphans::{clean_orphans_blocking, scan_orphans_blocking, CleanOrphansArgs}; +use crate::core::db_path::{open_write, set_test_home}; +use crate::core::filter::{DateRange, FilterSpec}; +use crate::core::icloud::ICloudState; +use crate::core::receipts::BackupReceipts; +use crate::core::retained::RetainedAttachments; +use crate::error::AppError; + +const REAL_SCHEMA: &str = include_str!("../tests/fixtures/chat-db-schema.sql"); + +/// Seconds between the Unix epoch and Apple's (2001-01-01). +const APPLE_EPOCH_OFFSET: i64 = 978_307_200; + +/// Noon UTC on the given day, in chat.db's nanoseconds-since-2001. +fn apple_date(y: i32, m: u32, d: u32) -> i64 { + let unix = chrono::NaiveDate::from_ymd_opt(y, m, d) + .unwrap() + .and_hms_opt(12, 0, 0) + .unwrap() + .and_utc() + .timestamp(); + (unix - APPLE_EPOCH_OFFSET) * 1_000_000_000 +} + +fn quiet_mac() -> SafetyProbe { + SafetyProbe { + messages_running: || false, + icloud: || ICloudState::Disabled, + available_space: crate::core::space::system_available, + } +} + +/// One throwaway home with a seeded Messages folder. +struct Env { + _tmp: TempDir, + home: PathBuf, + db: PathBuf, + attachments: PathBuf, + receipts: BackupReceipts, + retained: RetainedAttachments, +} + +impl Drop for Env { + fn drop(&mut self) { + set_test_home(None); + } +} + +impl Env { + fn new() -> Self { + let tmp = TempDir::new().unwrap(); + let home = fs::canonicalize(tmp.path()).unwrap().join("home"); + let messages = home.join("Library/Messages"); + let attachments = messages.join("Attachments"); + fs::create_dir_all(&attachments).unwrap(); + let db = messages.join("chat.db"); + Connection::open(&db) + .unwrap() + .execute_batch(REAL_SCHEMA) + .unwrap(); + set_test_home(Some(home.clone())); + + let env = Env { + retained: RetainedAttachments::in_dir(&home.join("app-data")), + receipts: BackupReceipts::default(), + _tmp: tmp, + home, + db, + attachments, + }; + env.seed(); + env + } + + fn conn(&self) -> Connection { + open_write(&self.db).unwrap() + } + + /// Write an attachment file under the Messages folder. + fn file(&self, rel: &str, bytes: &[u8]) -> PathBuf { + let p = self.attachments.join(rel); + fs::create_dir_all(p.parent().unwrap()).unwrap(); + fs::write(&p, bytes).unwrap(); + p + } + + fn add_attachment(&self, rowid: i64, path: &Path, mime: &str, msg: Option) { + let c = self.conn(); + let len = fs::metadata(path).map(|m| m.len() as i64).unwrap_or(0); + c.execute( + "INSERT INTO attachment (ROWID, guid, original_guid, filename, mime_type, \ + total_bytes, transfer_name) VALUES (?1, ?2, ?2, ?3, ?4, ?5, ?6)", + params![ + rowid, + format!("att-{rowid}"), + path.to_str().unwrap(), + mime, + len, + path.file_name().unwrap().to_str().unwrap(), + ], + ) + .unwrap(); + if let Some(m) = msg { + c.execute( + "INSERT INTO message_attachment_join (message_id, attachment_id) VALUES (?1, ?2)", + params![m, rowid], + ) + .unwrap(); + } + } + + fn add_message(&self, rowid: i64, chat: i64, handle: i64, date: i64, text: &str) { + let c = self.conn(); + c.execute( + "INSERT INTO message (ROWID, guid, text, handle_id, date, service, is_from_me) \ + VALUES (?1, ?2, ?3, ?4, ?5, 'iMessage', 0)", + params![rowid, format!("g{rowid}"), text, handle, date], + ) + .unwrap(); + c.execute( + "INSERT INTO chat_message_join (chat_id, message_id, message_date) VALUES (?1, ?2, ?3)", + params![chat, rowid, date], + ) + .unwrap(); + } + + /// Two chats: + /// - chat 1 (+15550001): m1 (2020), m2 (2020, two voice memos both named + /// `Audio Message.caf`), m3 (2023, a photo). Attachment 5 is shared + /// by m1 and m3. + /// - chat 2 (friend@example.com): m4 (2020, a photo), m5 (2023, a + /// tapback on m1). + fn seed(&self) { + let c = self.conn(); + c.execute_batch( + "INSERT INTO handle (ROWID, id, service) VALUES + (1, '+15550001', 'iMessage'), (2, 'friend@example.com', 'iMessage'); + INSERT INTO chat (ROWID, guid, chat_identifier, service_name) VALUES + (1, 'iMessage;-;+15550001', '+15550001', 'iMessage'), + (2, 'iMessage;-;friend@example.com', 'friend@example.com', 'iMessage'); + INSERT INTO chat_handle_join VALUES (1, 1), (2, 2);", + ) + .unwrap(); + drop(c); + + let y2020 = apple_date(2020, 6, 1); + let y2023 = apple_date(2023, 6, 1); + // Like real messages, each attachment has a U+FFFC placeholder in + // the text; the HTML/TXT exporter only handles attachments it finds + // that way. + self.add_message(1, 1, 1, y2020, "old one \u{FFFC}"); + self.add_message(2, 1, 1, y2020, "\u{FFFC}\u{FFFC}"); + self.add_message(3, 1, 1, y2023, "\u{FFFC}\u{FFFC}"); + self.add_message(4, 2, 2, y2020, "\u{FFFC}"); + + let memo1 = self.file("a1/01/GUID-1/Audio Message.caf", b"first memo"); + let memo2 = self.file("a2/02/GUID-2/Audio Message.caf", b"second memo!"); + let recent = self.file("a3/03/GUID-3/IMG_0001.HEIC", b"recent heic bytes"); + let other = self.file("a4/04/GUID-4/IMG_0001.HEIC", b"other chat heic"); + let shared = self.file("a5/05/GUID-5/shared.jpg", b"shared jpeg"); + self.add_attachment(1, &memo1, "audio/x-caf", Some(2)); + self.add_attachment(2, &memo2, "audio/x-caf", Some(2)); + self.add_attachment(3, &recent, "image/heic", Some(3)); + self.add_attachment(4, &other, "image/heic", Some(4)); + self.add_attachment(5, &shared, "image/jpeg", Some(1)); + self.conn() + .execute( + "INSERT INTO message_attachment_join (message_id, attachment_id) VALUES (3, 5)", + [], + ) + .unwrap(); + + // m5: a tapback (in chat 2, in 2023) on m1. + let c = self.conn(); + c.execute( + "INSERT INTO message (ROWID, guid, text, handle_id, date, associated_message_guid, \ + associated_message_type) VALUES (5, 'g5', 'Loved “old one”', 2, ?1, 'p:0/g1', 2000)", + params![y2023], + ) + .unwrap(); + c.execute( + "INSERT INTO chat_message_join (chat_id, message_id, message_date) VALUES (2, 5, ?1)", + params![y2023], + ) + .unwrap(); + } + + fn count(&self, sql: &str) -> i64 { + self.conn().query_row(sql, [], |r| r.get(0)).unwrap() + } + + fn backup(&self, filter: FilterSpec, format: &str) -> BackupResult { + backup_now( + &self.db, + &self.receipts, + RunBackupArgs { + filter, + format: format.into(), + destination: self.home.join("Backups").display().to_string(), + copy_attachments: true, + encrypt_password: None, + }, + &NoopProgress, + crate::core::space::system_available, + &crate::core::disk_image::FakeImageTool, + ) + .unwrap() + } + + fn preview(&self, filter: &FilterSpec) -> DeletePreview { + preview_delete_now(&self.db, filter).unwrap() + } + + fn delete( + &self, + filter: FilterSpec, + scope: DeleteScope, + skip_backup: bool, + ) -> Result { + let p = self.preview(&filter); + let deletes_messages = matches!(scope, DeleteScope::Both); + delete_now( + &self.db, + &self.receipts, + RunDeleteArgs { + filter, + confirmation_phrase: DELETE_CONFIRMATION_PHRASE.into(), + acknowledge_skip_backup: skip_backup, + snapshot_root: None, + delete_scope: scope, + acknowledge_icloud_sync: false, + acknowledge_not_on_mac: false, + expected_message_count: Some(if deletes_messages { p.message_count } else { 0 }), + expected_attachment_count: Some(p.attachment_count), + }, + &self.home.join("snapshots"), + true, + &quiet_mac(), + ) + } +} + +fn year_2020() -> FilterSpec { + FilterSpec { + date_range: Some(DateRange { + start: Some("2020-01-01".into()), + end: Some("2020-12-31".into()), + }), + ..Default::default() + } +} + +fn chat(id: i32) -> FilterSpec { + FilterSpec { + chat_ids: Some(vec![id]), + ..Default::default() + } +} + +fn find_files(dir: &Path) -> Vec { + let mut out = Vec::new(); + crate::core::attachment_files::walk_files(dir, &mut out); + out.sort(); + out +} + +#[test] +fn backup_then_delete_a_year_end_to_end() { + let env = Env::new(); + let filter = year_2020(); + + // Preview: m1, m2, m4 plus the 2023 tapback on m1. Attachment 5 is also + // used by the kept 2023 message, so it stays out of the plan. + let p = env.preview(&filter); + assert_eq!(p.message_count, 4); + assert_eq!(p.reaction_count, 1); + assert_eq!(p.attachment_count, 3); + assert_eq!(p.not_on_mac_count, 0); + assert!( + !coverage_now(&env.db, &env.receipts, &filter) + .unwrap() + .covered + ); + + // Back up as JSON with attachments. + let b = env.backup(filter.clone(), "json"); + let copy = b.attachment_copy.as_ref().unwrap(); + assert_eq!(copy.failed, 0, "{:?}", copy.failures); + assert!(b.includes_attachments); + let export = PathBuf::from(&b.export_path); + assert!(export.starts_with(env.home.join("Backups"))); + + // Both voice memos are in the backup, with their own contents. + let copies = find_files(&export.join("attachments")); + let memos: Vec = copies + .iter() + .filter(|p| p.to_string_lossy().ends_with("Audio Message.caf")) + .map(|p| fs::read_to_string(p).unwrap()) + .collect(); + assert_eq!(memos.len(), 2, "{copies:?}"); + assert!(memos.contains(&"first memo".to_string())); + assert!(memos.contains(&"second memo!".to_string())); + + // The JSON points at those copies, and the tapback is saved alongside. + let ndjson: String = fs::read_dir(&export) + .unwrap() + .flatten() + .filter(|e| e.path().extension().is_some_and(|x| x == "ndjson")) + .map(|e| fs::read_to_string(e.path()).unwrap()) + .collect(); + assert!(ndjson.contains("\"copiedTo\":\"attachments/"), "{ndjson}"); + let tapbacks = fs::read_to_string(export.join("tapbacks-outside-filter.json")).unwrap(); + assert!(tapbacks.contains("p:0/g1")); + + // The backup now covers exactly this delete. + let cov = coverage_now(&env.db, &env.receipts, &filter).unwrap(); + assert!(cov.covered && cov.with_attachments); + + let r = env + .delete(filter.clone(), DeleteScope::Both, false) + .unwrap(); + assert!(r.backup_verified); + assert_eq!(r.messages_deleted, 4); + assert_eq!(r.on_disk_files_removed, 3); + + // 2023 message, its photo and the shared attachment survive. + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 1); + assert_eq!( + env.count("SELECT COUNT(*) FROM attachment ORDER BY ROWID"), + 2 + ); + assert!(env.attachments.join("a3/03/GUID-3/IMG_0001.HEIC").exists()); + assert!(env.attachments.join("a5/05/GUID-5/shared.jpg").exists()); + assert!(!env + .attachments + .join("a1/01/GUID-1/Audio Message.caf") + .exists()); + assert!( + !env.attachments.join("a1").exists(), + "emptied folders removed" + ); + // Apple's triggers queued the deletions for iCloud. + assert_eq!(env.count("SELECT COUNT(*) FROM sync_deleted_messages"), 4); + + // The snapshot still has everything. + let snap = Connection::open(Path::new(&r.snapshot_path).join("chat.db")).unwrap(); + let n: i64 = snap + .query_row("SELECT COUNT(*) FROM message", [], |r| r.get(0)) + .unwrap(); + assert_eq!(n, 5); + + // Nothing left to delete for that year. + assert_eq!(env.preview(&filter).message_count, 0); +} + +#[test] +fn delete_without_backup_needs_explicit_skip() { + let env = Env::new(); + let err = env.delete(chat(2), DeleteScope::Both, false).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("No backup covers"))); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 5); + + let r = env.delete(chat(2), DeleteScope::Both, true).unwrap(); + assert!(!r.backup_verified); + assert_eq!( + env.count("SELECT COUNT(*) FROM message WHERE ROWID IN (4, 5)"), + 0 + ); +} + +#[test] +fn message_arriving_after_backup_breaks_coverage() { + let env = Env::new(); + env.backup(chat(2), "json"); + assert!( + coverage_now(&env.db, &env.receipts, &chat(2)) + .unwrap() + .covered + ); + + env.add_message(6, 2, 2, apple_date(2024, 1, 1), "new since the backup"); + assert!( + !coverage_now(&env.db, &env.receipts, &chat(2)) + .unwrap() + .covered + ); + let err = env.delete(chat(2), DeleteScope::Both, false).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("No backup covers"))); + assert_eq!(env.count("SELECT COUNT(*) FROM message WHERE ROWID = 6"), 1); +} + +#[test] +fn attachment_downloaded_after_backup_is_not_vouched_for() { + let env = Env::new(); + // Attachment 6 on m4 is still in iCloud when the backup runs. + let offloaded = env.attachments.join("a6/06/GUID-6/later.jpg"); + env.add_attachment(6, &offloaded, "image/jpeg", Some(4)); + let b = env.backup(chat(2), "json"); + assert_eq!(b.attachment_copy.as_ref().unwrap().not_on_mac, 1); + + // The user downloads it, then tries to delete. + env.file("a6/06/GUID-6/later.jpg", b"downloaded later"); + let cov = coverage_now(&env.db, &env.receipts, &chat(2)).unwrap(); + assert!(cov.covered && !cov.with_attachments); + let err = env.delete(chat(2), DeleteScope::Both, false).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("didn't copy every attachment"))); + assert!(offloaded.exists()); +} + +#[test] +fn attachments_only_delete_keeps_messages() { + let env = Env::new(); + let filter = chat(2); + env.backup(filter.clone(), "json"); + let r = env + .delete(filter, DeleteScope::AttachmentsOnly, false) + .unwrap(); + assert_eq!(r.messages_deleted, 0); + assert_eq!(r.on_disk_files_removed, 1); + assert_eq!(env.count("SELECT COUNT(*) FROM message WHERE ROWID = 4"), 1); + assert_eq!( + env.count("SELECT COUNT(*) FROM attachment WHERE ROWID = 4"), + 0 + ); + assert_eq!( + env.count("SELECT cache_has_attachments FROM message WHERE ROWID = 4"), + 0 + ); + assert!(!env.attachments.join("a4/04/GUID-4/IMG_0001.HEIC").exists()); +} + +#[test] +fn messages_only_delete_is_refused() { + let env = Env::new(); + let err = env + .delete(chat(2), DeleteScope::MessagesOnly, true) + .unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("isn't possible"))); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 5); +} + +#[test] +fn contact_filter_is_refused_rather_than_deleting_everything() { + let env = Env::new(); + let filter = FilterSpec { + handle_ids: Some(vec![2]), + ..Default::default() + }; + assert!(preview_delete_now(&env.db, &filter).is_err()); + let err = delete_now( + &env.db, + &env.receipts, + RunDeleteArgs { + filter, + confirmation_phrase: DELETE_CONFIRMATION_PHRASE.into(), + acknowledge_skip_backup: true, + snapshot_root: None, + delete_scope: DeleteScope::Both, + acknowledge_icloud_sync: true, + acknowledge_not_on_mac: true, + expected_message_count: None, + expected_attachment_count: None, + }, + &env.home.join("snapshots"), + true, + &quiet_mac(), + ) + .unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("contact"))); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 5); +} + +#[test] +fn orphan_cleanup_removes_only_true_orphans() { + let env = Env::new(); + let two_days_ago = SystemTime::now() - Duration::from_secs(2 * 24 * 60 * 60); + let age = |p: &Path| { + fs::File::options() + .write(true) + .open(p) + .unwrap() + .set_modified(two_days_ago) + .unwrap(); + }; + // An unlinked attachment record with its file. + let unlinked = env.file("a7/07/GUID-7/unlinked.jpg", b"unlinked"); + env.add_attachment(7, &unlinked, "image/jpeg", None); + // An old file nothing refers to. + let stray = env.file("a8/08/GUID-8/stray.jpg", b"stray"); + age(&stray); + // A Live Photo video next to a referenced photo. + let companion = env.file("a3/03/GUID-3/IMG_0001.MOV", b"live photo video"); + age(&companion); + // A brand-new untracked file (could be an in-flight download). + let fresh = env.file("a9/09/GUID-9/incoming.jpg", b"incoming"); + + let scan = scan_orphans_blocking(&env.retained, false).unwrap(); + assert_eq!(scan.db_orphan_count, 1); + assert_eq!(scan.fs_orphan_count, 1); + + let r = clean_orphans_blocking( + &env.retained, + &env.home.join("orphan-snapshots"), + &CleanOrphansArgs { + scan_fingerprint: scan.fingerprint, + include_retained: false, + acknowledge_icloud_sync: false, + }, + &quiet_mac(), + ) + .unwrap(); + assert_eq!(r.db_rows_deleted, 1); + assert_eq!(r.db_files_removed + r.fs_files_removed, 2); + assert!(r.snapshot_path.is_some()); + assert!(!unlinked.exists() && !stray.exists()); + assert!( + companion.exists(), + "companion of a referenced photo is kept" + ); + assert!(fresh.exists(), "recent files are never orphans"); + assert_eq!(env.count("SELECT COUNT(*) FROM attachment"), 5); +} + +#[test] +fn orphan_clean_refuses_if_anything_changed_since_the_scan() { + let env = Env::new(); + let unlinked = env.file("a7/07/GUID-7/unlinked.jpg", b"unlinked"); + env.add_attachment(7, &unlinked, "image/jpeg", None); + let scan = scan_orphans_blocking(&env.retained, false).unwrap(); + + // It gets linked to a message before the user confirms. + env.conn() + .execute( + "INSERT INTO message_attachment_join (message_id, attachment_id) VALUES (3, 7)", + [], + ) + .unwrap(); + let err = clean_orphans_blocking( + &env.retained, + &env.home.join("orphan-snapshots"), + &CleanOrphansArgs { + scan_fingerprint: scan.fingerprint, + include_retained: false, + acknowledge_icloud_sync: false, + }, + &quiet_mac(), + ) + .unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("changed since the scan"))); + assert!(unlinked.exists()); +} + +#[test] +fn html_backup_keeps_original_attachment_bytes() { + let env = Env::new(); + let b = env.backup(chat(1), "html"); + let copy = b.attachment_copy.as_ref().unwrap(); + assert_eq!(copy.failed, 0, "{:?}", copy.failures); + assert_eq!(copy.copied, 4); + // The HEIC photo is copied byte-for-byte, not converted to JPEG. + let export = PathBuf::from(&b.export_path); + let heic: Vec = find_files(&export.join("attachments")) + .into_iter() + .filter(|p| p.file_stem().is_some_and(|s| s == "3")) + .collect(); + assert_eq!(heic.len(), 1, "{heic:?}"); + assert_eq!(fs::read(&heic[0]).unwrap(), b"recent heic bytes"); + assert!(b.includes_attachments); +} + +// ---- Regression tests from the final review ---- + +/// Age a file so the 24-hour guard doesn't protect it. +fn make_old(p: &Path) { + fs::File::options() + .write(true) + .open(p) + .unwrap() + .set_modified(SystemTime::now() - Duration::from_secs(2 * 24 * 60 * 60)) + .unwrap(); +} + +fn scan_and_clean( + env: &Env, + icloud_ack: bool, + probe: &SafetyProbe, +) -> Result { + let scan = scan_orphans_blocking(&env.retained, false).unwrap(); + clean_orphans_blocking( + &env.retained, + &env.home.join("orphan-snapshots"), + &CleanOrphansArgs { + scan_fingerprint: scan.fingerprint, + include_retained: false, + acknowledge_icloud_sync: icloud_ack, + }, + probe, + ) +} + +#[test] +fn pdf_backup_does_not_count_for_delete() { + let env = Env::new(); + // The PDF writer can only store Latin-1: this would come out as `?`. + env.add_message(6, 2, 2, apple_date(2020, 7, 1), "Привет 😀 it’s"); + let b = env.backup(chat(2), "pdf"); + assert!(!b.counts_for_delete); + assert!( + !coverage_now(&env.db, &env.receipts, &chat(2)) + .unwrap() + .covered + ); + let err = env.delete(chat(2), DeleteScope::Both, false).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("No backup covers"))); + + // A JSON backup of the same thing does count. + let b = env.backup(chat(2), "json"); + assert!(b.counts_for_delete); + assert!( + env.delete(chat(2), DeleteScope::Both, false) + .unwrap() + .backup_verified + ); +} + +#[test] +fn html_backup_saves_tapbacks_whose_target_is_outside_the_filter() { + let env = Env::new(); + // m5 (chat 2) is a tapback on m1 (chat 1): the HTML exporter only + // renders it under m1, which isn't in a chat-2 export. + let b = env.backup(chat(2), "html"); + let saved = + fs::read_to_string(Path::new(&b.export_path).join("tapbacks-without-target.json")).unwrap(); + assert!(saved.contains("\"guid\": \"g5\""), "{saved}"); +} + +#[test] +fn failed_delete_attempts_keep_the_earlier_rollback_snapshot() { + let env = Env::new(); + let snaps = env.home.join("snapshots"); + let first = env.delete(chat(2), DeleteScope::Both, true).unwrap(); + let rollback = PathBuf::from(&first.snapshot_path); + + // Messages reopens during every later attempt, after the snapshot. + static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + fn reopens() -> bool { + CALLS.fetch_add(1, std::sync::atomic::Ordering::SeqCst) % 2 == 1 + } + for _ in 0..(crate::core::snapshot::SNAPSHOTS_TO_KEEP + 2) { + std::thread::sleep(Duration::from_millis(1100)); + let p = env.preview(&chat(1)); + let r = delete_now( + &env.db, + &env.receipts, + RunDeleteArgs { + filter: chat(1), + confirmation_phrase: DELETE_CONFIRMATION_PHRASE.into(), + acknowledge_skip_backup: true, + snapshot_root: None, + delete_scope: DeleteScope::Both, + acknowledge_icloud_sync: false, + acknowledge_not_on_mac: false, + expected_message_count: Some(p.message_count), + expected_attachment_count: Some(p.attachment_count), + }, + &snaps, + true, + &SafetyProbe { + messages_running: reopens, + icloud: || ICloudState::Disabled, + available_space: crate::core::space::system_available, + }, + ); + assert!(r.is_err()); + } + assert!( + rollback.exists(), + "failed attempts pruned the real rollback" + ); + // …and they left no snapshots of their own behind. + assert_eq!(fs::read_dir(&snaps).unwrap().count(), 1); +} + +#[test] +fn attachments_shared_across_chunks_are_counted_once() { + let env = Env::new(); + // 900 messages, so the library splits them into two chunks of ≤800; + // attachment 9 is linked from one message in each. + { + let c = env.conn(); + for i in 0..900i64 { + let id = 100 + i; + c.execute( + "INSERT INTO message (ROWID, guid, text, handle_id, date, service) \ + VALUES (?1, ?2, '\u{FFFC}', 2, ?3, 'iMessage')", + params![id, format!("bulk{id}"), apple_date(2021, 1, 1) + i], + ) + .unwrap(); + c.execute( + "INSERT INTO chat_message_join (chat_id, message_id, message_date) VALUES (2, ?1, 0)", + params![id], + ) + .unwrap(); + } + } + let f = env.file("b9/09/G/x.jpg", b"jpeg"); + env.add_attachment(9, &f, "image/jpeg", Some(100)); + env.conn() + .execute( + "INSERT INTO message_attachment_join (message_id, attachment_id) VALUES (999, 9)", + [], + ) + .unwrap(); + let p = env.preview(&chat(2)); + // Attachment 4 (m4) and attachment 9, each once. + assert_eq!(p.attachment_count, 2); + assert_eq!(p.attachment_bytes, 15 + 4); +} + +#[test] +fn orphan_clean_requires_icloud_acknowledgement() { + let env = Env::new(); + let f = env.file("a7/07/GUID-7/unlinked.jpg", b"unlinked"); + env.add_attachment(7, &f, "image/jpeg", None); + let icloud_on = SafetyProbe { + messages_running: || false, + icloud: || ICloudState::Enabled, + available_space: crate::core::space::system_available, + }; + let err = scan_and_clean(&env, false, &icloud_on).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("iCloud"))); + assert!(f.exists()); + + let r = scan_and_clean(&env, true, &icloud_on).unwrap(); + assert_eq!(r.db_rows_deleted, 1); +} + +#[test] +fn recently_created_unlinked_record_is_not_an_orphan() { + let env = Env::new(); + let f = env.file("a7/07/GUID-7/just-arrived.jpg", b"new"); + env.add_attachment(7, &f, "image/jpeg", None); + // Created a minute ago — its message may not have synced yet. + let now_apple = SystemTime::now() + .duration_since(SystemTime::UNIX_EPOCH) + .unwrap() + .as_secs() as i64 + - APPLE_EPOCH_OFFSET + - 60; + env.conn() + .execute( + "UPDATE attachment SET created_date = ?1 WHERE ROWID = 7", + params![now_apple], + ) + .unwrap(); + let scan = scan_orphans_blocking(&env.retained, false).unwrap(); + assert_eq!(scan.db_orphan_count, 0); + assert!(f.exists()); +} + +#[cfg(unix)] +#[test] +fn linked_file_stored_via_symlinked_home_is_kept() { + let env = Env::new(); + let alias = env.home.parent().unwrap().join("old-home-name"); + std::os::unix::fs::symlink(&env.home, &alias).unwrap(); + let real = env + .home + .join("Library/Messages/StickerCache/8e-GUID/GUID.heic"); + fs::create_dir_all(real.parent().unwrap()).unwrap(); + fs::write(&real, b"sticker").unwrap(); + make_old(&real); + let stored = alias.join("Library/Messages/StickerCache/8e-GUID/GUID.heic"); + env.add_attachment(20, &stored, "image/heic", Some(3)); + + let r = scan_and_clean(&env, false, &quiet_mac()).unwrap(); + assert_eq!(r.fs_files_removed, 0); + assert!(real.exists()); +} + +#[test] +fn linked_file_stored_under_a_renamed_home_is_kept() { + let env = Env::new(); + // chat.db still says /Users//…, which no longer exists. + let real = env.file("c1/01/GUID-C/photo.jpg", b"photo"); + make_old(&real); + let stored = PathBuf::from( + "/Users/old-name-that-does-not-exist/Library/Messages/Attachments/c1/01/GUID-C/photo.jpg", + ); + env.add_attachment(23, &stored, "image/jpeg", Some(3)); + let r = scan_and_clean(&env, false, &quiet_mac()).unwrap(); + assert_eq!(r.fs_files_removed, 0); + assert!(real.exists()); +} + +#[test] +fn linked_file_with_double_slash_path_is_kept() { + let env = Env::new(); + let f = env.file("b1/01/GUID-B/pic.jpg", b"pic"); + make_old(&f); + let stored = f.to_str().unwrap().replace("/b1/01/", "/b1//01/"); + env.add_attachment(22, Path::new(&stored), "image/jpeg", Some(3)); + let r = scan_and_clean(&env, false, &quiet_mac()).unwrap(); + assert_eq!(r.fs_files_removed, 0); + assert!(f.exists()); +} + +#[test] +fn linked_file_directly_in_a_root_folder_is_kept() { + let env = Env::new(); + let f = env.home.join("Library/Messages/StickerCache/sticker.heic"); + fs::create_dir_all(f.parent().unwrap()).unwrap(); + fs::write(&f, b"sticker").unwrap(); + make_old(&f); + env.add_attachment(21, &f, "image/heic", Some(3)); + let r = scan_and_clean(&env, false, &quiet_mac()).unwrap(); + assert_eq!(r.fs_files_removed, 0); + assert!(f.exists()); +} + +#[test] +fn folder_attachment_is_copied_whole_in_json_backups() { + let env = Env::new(); + // A bundle-style attachment: a folder with nested files (and a symlink, + // which is skipped rather than followed). + let bundle = env.attachments.join("d1/01/GUID-D/Pass.pkpass"); + fs::create_dir_all(bundle.join("en.lproj")).unwrap(); + fs::write(bundle.join("pass.json"), b"{\"pass\":1}").unwrap(); + fs::write(bundle.join("en.lproj/strings"), b"hello").unwrap(); + #[cfg(unix)] + std::os::unix::fs::symlink("/etc/hosts", bundle.join("link")).unwrap(); + env.add_attachment(30, &bundle, "application/vnd.apple.pkpass", Some(4)); + + let b = env.backup(chat(2), "json"); + let copy = b.attachment_copy.as_ref().unwrap(); + assert_eq!(copy.failed, 0, "{:?}", copy.failures); + assert!(b.includes_attachments); + + let export = PathBuf::from(&b.export_path); + let copied = export.join("attachments/2/30-Pass.pkpass"); + assert_eq!(fs::read(copied.join("pass.json")).unwrap(), b"{\"pass\":1}"); + assert_eq!(fs::read(copied.join("en.lproj/strings")).unwrap(), b"hello"); + assert!(!copied.join("link").exists(), "symlinks are not copied"); + let ndjson: String = fs::read_dir(&export) + .unwrap() + .flatten() + .filter(|e| e.path().extension().is_some_and(|x| x == "ndjson")) + .map(|e| fs::read_to_string(e.path()).unwrap()) + .collect(); + assert!(ndjson.contains("attachments/2/30-Pass.pkpass"), "{ndjson}"); + + // The backup now vouches for the folder, so the delete is verified. + let cov = coverage_now(&env.db, &env.receipts, &chat(2)).unwrap(); + assert!(cov.with_attachments); +} + +#[test] +fn backup_and_delete_refuse_up_front_when_the_disk_is_full() { + let env = Env::new(); + let almost_full: crate::core::space::SpaceProbe = |_| Ok(1024); + let err = backup_now( + &env.db, + &env.receipts, + RunBackupArgs { + filter: chat(1), + format: "json".into(), + destination: env.home.join("Backups").display().to_string(), + copy_attachments: true, + encrypt_password: None, + }, + &NoopProgress, + almost_full, + &crate::core::disk_image::FakeImageTool, + ) + .unwrap_err(); + assert!( + matches!(&err, AppError::Other(m) if m.contains("Not enough free space")), + "{err:?}" + ); + // Nothing half-written was left behind. + assert!( + !env.home.join("Backups").exists() + || fs::read_dir(env.home.join("Backups")).unwrap().count() == 0 + ); + + let p = env.preview(&chat(2)); + let err = delete_now( + &env.db, + &env.receipts, + RunDeleteArgs { + filter: chat(2), + confirmation_phrase: DELETE_CONFIRMATION_PHRASE.into(), + acknowledge_skip_backup: true, + snapshot_root: None, + delete_scope: DeleteScope::Both, + acknowledge_icloud_sync: false, + acknowledge_not_on_mac: false, + expected_message_count: Some(p.message_count), + expected_attachment_count: Some(p.attachment_count), + }, + &env.home.join("snapshots"), + true, + &SafetyProbe { + available_space: almost_full, + ..quiet_mac() + }, + ) + .unwrap_err(); + assert!( + matches!(&err, AppError::Other(m) if m.contains("safety snapshot")), + "{err:?}" + ); + assert_eq!( + env.count("SELECT COUNT(*) FROM message"), + 5, + "nothing deleted" + ); +} + +fn snapshot_roots(env: &Env) -> Vec<(crate::core::snapshot::SnapshotKind, PathBuf)> { + use crate::core::snapshot::SnapshotKind; + vec![ + (SnapshotKind::Delete, env.home.join("snapshots")), + (SnapshotKind::OrphanClean, env.home.join("orphan-snapshots")), + (SnapshotKind::PreRestore, env.home.join("pre-restore")), + ] +} + +fn restore( + env: &Env, + snapshot: &str, + probe: &SafetyProbe, +) -> Result { + crate::commands::restore::restore_now( + &env.db, + &snapshot_roots(env), + &crate::commands::restore::RestoreArgs { + snapshot_path: snapshot.into(), + acknowledge_icloud_sync: false, + }, + probe, + ) +} + +#[test] +fn restore_brings_back_deleted_messages_and_can_itself_be_undone() { + let env = Env::new(); + let r = env.delete(chat(2), DeleteScope::Both, true).unwrap(); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 3); + + let listed = crate::core::snapshot::list_snapshots(&snapshot_roots(&env)); + assert_eq!(listed.len(), 1); + assert_eq!(listed[0].kind, crate::core::snapshot::SnapshotKind::Delete); + + let restored = restore(&env, &r.snapshot_path, &quiet_mac()).unwrap(); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 5); + assert_eq!( + env.count("SELECT COUNT(*) FROM attachment WHERE ROWID = 4"), + 1 + ); + + // The state just before the restore was kept, and restoring it undoes + // the restore. + let pre = Connection::open(Path::new(&restored.pre_restore_snapshot).join("chat.db")).unwrap(); + let n: i64 = pre + .query_row("SELECT COUNT(*) FROM message", [], |r| r.get(0)) + .unwrap(); + assert_eq!(n, 3); + drop(pre); + restore(&env, &restored.pre_restore_snapshot, &quiet_mac()).unwrap(); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 3); +} + +#[test] +fn restore_refuses_anything_that_isnt_one_of_our_snapshots() { + let env = Env::new(); + // A folder that looks like a snapshot but lives elsewhere. + let elsewhere = env.home.join("Downloads/2024-01-01T00-00-00"); + fs::create_dir_all(&elsewhere).unwrap(); + fs::copy(&env.db, elsewhere.join("chat.db")).unwrap(); + let err = restore(&env, elsewhere.to_str().unwrap(), &quiet_mac()).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("isn't a Bubble Wrap snapshot"))); + // The live database itself. + assert!(restore( + &env, + env.db.parent().unwrap().to_str().unwrap(), + &quiet_mac() + ) + .is_err()); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 5); +} + +#[test] +fn restore_refuses_a_damaged_snapshot() { + let env = Env::new(); + let r = env.delete(chat(2), DeleteScope::Both, true).unwrap(); + let snap_db = Path::new(&r.snapshot_path).join("chat.db"); + // Corrupt it past the header. + let mut bytes = fs::read(&snap_db).unwrap(); + for b in bytes.iter_mut().skip(4096).take(8192) { + *b = 0xAB; + } + fs::write(&snap_db, bytes).unwrap(); + let _ = fs::remove_file(Path::new(&r.snapshot_path).join("chat.db-wal")); + assert!(restore(&env, &r.snapshot_path, &quiet_mac()).is_err()); + assert_eq!( + env.count("SELECT COUNT(*) FROM message"), + 3, + "live db untouched" + ); +} + +#[test] +fn restore_refuses_while_messages_runs_or_without_icloud_ack() { + let env = Env::new(); + let r = env.delete(chat(2), DeleteScope::Both, true).unwrap(); + let running = SafetyProbe { + messages_running: || true, + ..quiet_mac() + }; + assert!(restore(&env, &r.snapshot_path, &running).is_err()); + let icloud = SafetyProbe { + icloud: || ICloudState::Enabled, + ..quiet_mac() + }; + let err = restore(&env, &r.snapshot_path, &icloud).unwrap_err(); + assert!(matches!(err, AppError::Other(m) if m.contains("iCloud"))); + assert_eq!(env.count("SELECT COUNT(*) FROM message"), 3); +} + +#[cfg(unix)] +#[test] +fn encrypted_backup_goes_into_an_image_that_counts_for_delete() { + use crate::core::disk_image::{DiskImageTool, FakeImageTool, Password}; + let env = Env::new(); + let b = backup_now( + &env.db, + &env.receipts, + RunBackupArgs { + filter: chat(2), + format: "json".into(), + destination: env.home.join("Backups").display().to_string(), + copy_attachments: true, + encrypt_password: Some(Password::new("correct horse battery")), + }, + &NoopProgress, + crate::core::space::system_available, + &FakeImageTool, + ) + .unwrap(); + assert!(b.encrypted); + let image = PathBuf::from(&b.export_path); + assert_eq!(image.extension().unwrap(), "sparsebundle"); + assert!(image.is_dir()); + + // Nothing readable outside the image, and nothing left mounted. + let outside: Vec<_> = fs::read_dir(env.home.join("Backups")) + .unwrap() + .flatten() + .map(|e| e.file_name()) + .collect(); + assert_eq!(outside, vec![image.file_name().unwrap().to_owned()]); + let leftover_mounts: Vec<_> = fs::read_dir(std::env::temp_dir()) + .unwrap() + .flatten() + .filter(|e| { + e.file_name().to_string_lossy().contains( + &*image + .file_stem() + .unwrap() + .to_string_lossy() + .replace(' ', "_"), + ) + }) + .collect(); + assert!(leftover_mounts.is_empty(), "{leftover_mounts:?}"); + + // The backup is inside, attachments included. + let inside = find_files(&image.join("volume")); + assert!( + inside + .iter() + .any(|p| p.to_string_lossy().ends_with("IMG_0001.HEIC")), + "{inside:?}" + ); + assert!(inside + .iter() + .any(|p| p.extension().is_some_and(|e| e == "ndjson"))); + assert!( + !b.manifest_path.starts_with('/'), + "manifest reported relative to the image" + ); + + // It counts for the delete step. + let cov = coverage_now(&env.db, &env.receipts, &chat(2)).unwrap(); + assert!(cov.with_attachments); + assert_eq!(cov.export_path.as_deref(), Some(b.export_path.as_str())); + + // The wrong password can't open it. + let mp = env.home.join("mnt"); + fs::create_dir_all(&mp).unwrap(); + assert!(FakeImageTool + .attach(&image, &mp, &Password::new("wrong password")) + .is_err()); +} diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index e231a2e..a6c7926 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -2,6 +2,9 @@ mod commands; mod core; mod error; +#[cfg(test)] +mod e2e_tests; + pub use error::AppError; use tauri::menu::{MenuBuilder, MenuItemBuilder, SubmenuBuilder}; @@ -11,8 +14,17 @@ use tauri::Emitter; pub fn run() { tauri::Builder::default() .plugin(tauri_plugin_dialog::init()) - .plugin(tauri_plugin_shell::init()) + .manage(core::op_lock::OperationLock::default()) .setup(|app| { + use tauri::Manager; + let receipts = match app.path().app_data_dir() { + Ok(dir) => { + core::receipts::BackupReceipts::persistent(dir.join("backup_receipts.json")) + } + Err(_) => core::receipts::BackupReceipts::default(), + }; + app.manage(std::sync::Arc::new(receipts)); + let clean_item = MenuItemBuilder::with_id("clean-orphans", "Clean Orphaned Data…").build(app)?; @@ -68,9 +80,12 @@ pub fn run() { commands::backup::run_backup, commands::delete::preview_delete, commands::delete::run_delete, + commands::delete::backup_coverage, commands::safety::safety_status, commands::orphans::scan_orphans, commands::orphans::clean_orphans, + commands::restore::list_snapshots, + commands::restore::restore_snapshot, ]) .run(tauri::generate_context!()) .expect("error while running tauri application"); diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json index 6eeedf4..589d0ac 100644 --- a/src-tauri/tauri.conf.json +++ b/src-tauri/tauri.conf.json @@ -22,12 +22,41 @@ } ], "security": { - "csp": null + "csp": { + "default-src": "'self'", + "script-src": "'self'", + "style-src": "'self' 'unsafe-inline'", + "img-src": "'self' data:", + "font-src": "'self'", + "connect-src": "ipc: http://ipc.localhost", + "object-src": "'none'", + "base-uri": "'none'", + "form-action": "'none'", + "frame-ancestors": "'none'" + }, + "devCsp": { + "default-src": "'self'", + "script-src": "'self'", + "style-src": "'self' 'unsafe-inline'", + "img-src": "'self' data:", + "font-src": "'self'", + "connect-src": "ipc: http://ipc.localhost ws://localhost:1420 http://localhost:1420", + "object-src": "'none'", + "base-uri": "'none'", + "form-action": "'none'", + "frame-ancestors": "'none'" + }, + "dangerousDisableAssetCspModification": [ + "style-src" + ] } }, "bundle": { "active": true, - "targets": ["app", "dmg"], + "targets": [ + "app", + "dmg" + ], "icon": [ "icons/icon.icns", "icons/32x32.png", diff --git a/src-tauri/tests/fixtures/chat-db-schema.sql b/src-tauri/tests/fixtures/chat-db-schema.sql new file mode 100644 index 0000000..a2cfe8e --- /dev/null +++ b/src-tauri/tests/fixtures/chat-db-schema.sql @@ -0,0 +1,138 @@ +-- Schema of a real macOS chat.db: tables, indexes and Apple's triggers. +-- Extracted from imessage-exporter's test_data/db/test.db (schema only, no data). +-- Used by src/e2e_tests.rs so tests see the same cascades and triggers as +-- the real database. + +CREATE TABLE _SqliteDatabaseProperties (key TEXT, value TEXT, UNIQUE(key)); + +CREATE TABLE deleted_messages (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, guid TEXT NOT NULL); + +CREATE TABLE chat_handle_join (chat_id INTEGER REFERENCES chat (ROWID) ON DELETE CASCADE, handle_id INTEGER REFERENCES handle (ROWID) ON DELETE CASCADE, UNIQUE(chat_id, handle_id)); + +CREATE TABLE sync_deleted_messages (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, guid TEXT NOT NULL, recordID TEXT ); + +CREATE TABLE message_processing_task (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, guid TEXT NOT NULL, task_flags INTEGER NOT NULL ); + +CREATE TABLE handle (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, id TEXT NOT NULL, country TEXT, service TEXT NOT NULL, uncanonicalized_id TEXT, person_centric_id TEXT, UNIQUE (id, service) ); + +CREATE TABLE sync_deleted_chats (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, guid TEXT NOT NULL, recordID TEXT,timestamp INTEGER); + +CREATE TABLE message_attachment_join (message_id INTEGER REFERENCES message (ROWID) ON DELETE CASCADE, attachment_id INTEGER REFERENCES attachment (ROWID) ON DELETE CASCADE, UNIQUE(message_id, attachment_id)); + +CREATE TABLE sync_deleted_attachments (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, guid TEXT NOT NULL, recordID TEXT ); + +CREATE TABLE kvtable (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, key TEXT UNIQUE NOT NULL, value BLOB NOT NULL); + +CREATE TABLE chat_message_join (chat_id INTEGER REFERENCES chat (ROWID) ON DELETE CASCADE, message_id INTEGER REFERENCES message (ROWID) ON DELETE CASCADE, message_date INTEGER DEFAULT 0, PRIMARY KEY (chat_id, message_id)); + +CREATE TABLE message (ROWID INTEGER PRIMARY KEY AUTOINCREMENT, guid TEXT UNIQUE NOT NULL, text TEXT, replace INTEGER DEFAULT 0, service_center TEXT, handle_id INTEGER DEFAULT 0, subject TEXT, country TEXT, attributedBody BLOB, version INTEGER DEFAULT 0, type INTEGER DEFAULT 0, service TEXT, account TEXT, account_guid TEXT, error INTEGER DEFAULT 0, date INTEGER, date_read INTEGER, date_delivered INTEGER, is_delivered INTEGER DEFAULT 0, is_finished INTEGER DEFAULT 0, is_emote INTEGER DEFAULT 0, is_from_me INTEGER DEFAULT 0, is_empty INTEGER DEFAULT 0, is_delayed INTEGER DEFAULT 0, is_auto_reply INTEGER DEFAULT 0, is_prepared INTEGER DEFAULT 0, is_read INTEGER DEFAULT 0, is_system_message INTEGER DEFAULT 0, is_sent INTEGER DEFAULT 0, has_dd_results INTEGER DEFAULT 0, is_service_message INTEGER DEFAULT 0, is_forward INTEGER DEFAULT 0, was_downgraded INTEGER DEFAULT 0, is_archive INTEGER DEFAULT 0, cache_has_attachments INTEGER DEFAULT 0, cache_roomnames TEXT, was_data_detected INTEGER DEFAULT 0, was_deduplicated INTEGER DEFAULT 0, is_audio_message INTEGER DEFAULT 0, is_played INTEGER DEFAULT 0, date_played INTEGER, item_type INTEGER DEFAULT 0, other_handle INTEGER DEFAULT 0, group_title TEXT, group_action_type INTEGER DEFAULT 0, share_status INTEGER DEFAULT 0, share_direction INTEGER DEFAULT 0, is_expirable INTEGER DEFAULT 0, expire_state INTEGER DEFAULT 0, message_action_type INTEGER DEFAULT 0, message_source INTEGER DEFAULT 0, associated_message_guid TEXT, associated_message_type INTEGER DEFAULT 0, balloon_bundle_id TEXT, payload_data BLOB, expressive_send_style_id TEXT, associated_message_range_location INTEGER DEFAULT 0, associated_message_range_length INTEGER DEFAULT 0, time_expressive_send_played INTEGER, message_summary_info BLOB, ck_sync_state INTEGER DEFAULT 0, ck_record_id TEXT, ck_record_change_tag TEXT, destination_caller_id TEXT, is_corrupt INTEGER DEFAULT 0, reply_to_guid TEXT, sort_id INTEGER, is_spam INTEGER DEFAULT 0, has_unseen_mention INTEGER DEFAULT 0, thread_originator_guid TEXT, thread_originator_part TEXT, syndication_ranges TEXT, synced_syndication_ranges TEXT, was_delivered_quietly INTEGER DEFAULT 0, did_notify_recipient INTEGER DEFAULT 0, date_retracted INTEGER DEFAULT 0, date_edited INTEGER DEFAULT 0, was_detonated INTEGER DEFAULT 0, part_count INTEGER, is_stewie INTEGER DEFAULT 0, is_kt_verified INTEGER DEFAULT 0, is_sos INTEGER DEFAULT 0, is_critical INTEGER DEFAULT 0, bia_reference_id TEXT DEFAULT NULL, fallback_hash TEXT DEFAULT NULL, associated_message_emoji TEXT DEFAULT NULL, is_pending_satellite_send INTEGER DEFAULT 0, needs_relay INTEGER DEFAULT 0, schedule_type INTEGER DEFAULT 0, schedule_state INTEGER DEFAULT 0, sent_or_received_off_grid INTEGER DEFAULT 0); + +CREATE TABLE chat (ROWID INTEGER PRIMARY KEY AUTOINCREMENT, guid TEXT UNIQUE NOT NULL, style INTEGER, state INTEGER, account_id TEXT, properties BLOB, chat_identifier TEXT, service_name TEXT, room_name TEXT, account_login TEXT, is_archived INTEGER DEFAULT 0, last_addressed_handle TEXT, display_name TEXT, group_id TEXT, is_filtered INTEGER DEFAULT 0, successful_query INTEGER, engram_id TEXT, server_change_token TEXT, ck_sync_state INTEGER DEFAULT 0, original_group_id TEXT, last_read_message_timestamp INTEGER DEFAULT 0, cloudkit_record_id TEXT, last_addressed_sim_id TEXT, is_blackholed INTEGER DEFAULT 0, syndication_date INTEGER DEFAULT 0, syndication_type INTEGER DEFAULT 0, is_recovered INTEGER DEFAULT 0, is_deleting_incoming_messages INTEGER DEFAULT 0); + +CREATE TABLE attachment (ROWID INTEGER PRIMARY KEY AUTOINCREMENT, guid TEXT UNIQUE NOT NULL, created_date INTEGER DEFAULT 0, start_date INTEGER DEFAULT 0, filename TEXT, uti TEXT, mime_type TEXT, transfer_state INTEGER DEFAULT 0, is_outgoing INTEGER DEFAULT 0, user_info BLOB, transfer_name TEXT, total_bytes INTEGER DEFAULT 0, is_sticker INTEGER DEFAULT 0, sticker_user_info BLOB, attribution_info BLOB, hide_attachment INTEGER DEFAULT 0, ck_sync_state INTEGER DEFAULT 0, ck_server_change_token_blob BLOB, ck_record_id TEXT, original_guid TEXT UNIQUE NOT NULL, is_commsafety_sensitive INTEGER DEFAULT 0, emoji_image_content_identifier TEXT DEFAULT NULL, emoji_image_short_description TEXT DEFAULT NULL, preview_generation_state INTEGER DEFAULT 0); + +CREATE TABLE chat_recoverable_message_join (chat_id INTEGER REFERENCES chat (ROWID) ON DELETE CASCADE, message_id INTEGER REFERENCES message (ROWID) ON DELETE CASCADE, delete_date INTEGER, ck_sync_state INTEGER DEFAULT 0, PRIMARY KEY (chat_id, message_id), CHECK (delete_date != 0)); + +CREATE TABLE unsynced_removed_recoverable_messages (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, chat_guid TEXT NOT NULL, message_guid TEXT NOT NULL, part_index INTEGER); + +CREATE TABLE recoverable_message_part (chat_id INTEGER REFERENCES chat (ROWID) ON DELETE CASCADE, message_id INTEGER REFERENCES message (ROWID) ON DELETE CASCADE, part_index INTEGER, delete_date INTEGER, part_text BLOB NOT NULL, ck_sync_state INTEGER DEFAULT 0, PRIMARY KEY (chat_id, message_id, part_index), CHECK (delete_date != 0)); + +CREATE TABLE scheduled_messages_pending_cloudkit_delete (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, guid TEXT NOT NULL, recordID TEXT ); + +CREATE INDEX message_attachment_join_idx_message_id ON message_attachment_join(message_id); + +CREATE INDEX chat_idx_chat_identifier_service_name ON chat(chat_identifier, service_name); + +CREATE INDEX chat_handle_join_idx_handle_id ON chat_handle_join(handle_id); + +CREATE INDEX message_attachment_join_idx_attachment_id ON message_attachment_join(attachment_id); + +CREATE INDEX message_idx_date ON message(date); + +CREATE INDEX attachment_idx_purged_attachments_v2 ON attachment(hide_attachment,ck_sync_state,transfer_state) WHERE hide_attachment=0 AND (ck_sync_state=1 OR ck_sync_state=4) AND transfer_state=0; + +CREATE INDEX message_idx_thread_originator_guid ON message(thread_originator_guid); + +CREATE INDEX message_idx_handle ON message(handle_id, date); + +CREATE INDEX message_idx_handle_id ON message(handle_id); + +CREATE INDEX message_idx_is_sent_is_from_me_error ON message(is_sent, is_from_me, error); + +CREATE INDEX chat_message_join_idx_message_id_only ON chat_message_join(message_id); + +CREATE INDEX message_idx_associated_message ON message(associated_message_guid); + +CREATE INDEX chat_idx_chat_identifier ON chat(chat_identifier); + +CREATE INDEX message_processing_task_idx_guid_task_flags ON message_processing_task(guid, task_flags); + +CREATE INDEX chat_message_join_idx_chat_id ON chat_message_join(chat_id); + +CREATE INDEX message_idx_cache_has_attachments ON message(cache_has_attachments); + +CREATE INDEX chat_idx_chat_room_name_service_name ON chat(room_name, service_name); + +CREATE INDEX message_idx_other_handle ON message(other_handle); + +CREATE INDEX message_idx_was_downgraded ON message(was_downgraded); + +CREATE INDEX chat_idx_is_archived ON chat(is_archived); + +CREATE INDEX chat_idx_group_id ON chat(group_id); + +CREATE INDEX message_idx_expire_state ON message(expire_state); + +CREATE INDEX chat_message_join_idx_message_date_id_chat_id ON chat_message_join(chat_id, message_date, message_id); + +CREATE INDEX message_idx_is_read ON message(is_read, is_from_me, is_finished); + +CREATE INDEX message_idx_isRead_isFromMe_itemType ON message(is_read, is_from_me, item_type); + +CREATE INDEX message_idx_failed ON message(is_finished, is_from_me, error); + +CREATE INDEX chat_recoverable_message_join_message_id_idx ON chat_recoverable_message_join(message_id); + +CREATE INDEX attachment_idx_is_sticker ON attachment(is_sticker); + +CREATE INDEX message_idx_schedule_state ON message(schedule_state); + +CREATE INDEX message_idx_undelivered_one_to_one_imessage ON message(cache_roomnames,service,is_sent,is_delivered,was_downgraded,item_type) where cache_roomnames IS NULL AND service IN ('iMessage','RCS') AND is_sent = 1 AND is_delivered = 0 AND was_downgraded = 0 AND item_type == 0 AND schedule_type == 0; + +CREATE INDEX message_idx_is_pending_satellite_message ON message(is_pending_satellite_send) WHERE is_pending_satellite_send=1; + +CREATE INDEX message_idx_is_scheduled_message ON message(schedule_type) WHERE schedule_type=2; + +CREATE TRIGGER after_delete_on_chat_message_join AFTER DELETE ON chat_message_join BEGIN UPDATE message SET cache_roomnames = ( SELECT group_concat(c.room_name) FROM chat c INNER JOIN chat_message_join j ON c.ROWID = j.chat_id WHERE j.message_id = OLD.message_id ) WHERE message.ROWID = OLD.message_id; DELETE FROM message WHERE message.ROWID = OLD.message_id AND OLD.message_id NOT IN (SELECT chat_message_join.message_id from chat_message_join WHERE chat_message_join.message_id = OLD.message_id LIMIT 1) AND OLD.message_id NOT IN (SELECT chat_recoverable_message_join.message_id from chat_recoverable_message_join WHERE chat_recoverable_message_join.message_id = OLD.message_id LIMIT 1); END; + +CREATE TRIGGER after_delete_on_attachment AFTER DELETE ON attachment BEGIN SELECT delete_attachment_path(OLD.filename); END; + +CREATE TRIGGER after_insert_on_message_attachment_join AFTER INSERT ON message_attachment_join BEGIN UPDATE message SET cache_has_attachments = 1 WHERE message.ROWID = NEW.message_id; END; + +CREATE TRIGGER after_delete_on_chat_handle_join AFTER DELETE ON chat_handle_join BEGIN DELETE FROM handle WHERE handle.ROWID = OLD.handle_id AND (SELECT 1 from chat_handle_join WHERE handle_id = OLD.handle_id LIMIT 1) IS NULL AND (SELECT 1 from message WHERE handle_id = OLD.handle_id LIMIT 1) IS NULL AND (SELECT 1 from message WHERE other_handle = OLD.handle_id LIMIT 1) IS NULL; END; + +CREATE TRIGGER after_insert_on_chat_message_join AFTER INSERT ON chat_message_join BEGIN UPDATE message SET cache_roomnames = ( SELECT group_concat(c.room_name) FROM chat c INNER JOIN chat_message_join j ON c.ROWID = j.chat_id WHERE j.message_id = NEW.message_id ) WHERE message.ROWID = NEW.message_id; END; + +CREATE TRIGGER after_delete_on_message AFTER DELETE ON message BEGIN DELETE FROM handle WHERE handle.ROWID = OLD.handle_id AND (SELECT 1 from chat_handle_join WHERE handle_id = OLD.handle_id LIMIT 1) IS NULL AND (SELECT 1 from message WHERE handle_id = OLD.handle_id LIMIT 1) IS NULL AND (SELECT 1 from message WHERE other_handle = OLD.handle_id LIMIT 1) IS NULL; END; + +CREATE TRIGGER update_message_date_after_update_on_message AFTER UPDATE OF date ON message BEGIN UPDATE chat_message_join SET message_date = NEW.date WHERE message_id = NEW.ROWID AND message_date != NEW.date; END; + +CREATE TRIGGER after_delete_on_message_plugin AFTER DELETE ON message WHEN OLD.balloon_bundle_id IS NOT NULL BEGIN SELECT after_delete_message_plugin(OLD.ROWID, OLD.guid); END; + +CREATE TRIGGER add_to_sync_deleted_messages AFTER DELETE ON message BEGIN INSERT INTO sync_deleted_messages (guid, recordID) VALUES (OLD.guid, OLD.ck_record_id); END; + +CREATE TRIGGER after_delete_on_chat_recoverable_message_join AFTER DELETE ON chat_recoverable_message_join BEGIN UPDATE message SET cache_roomnames = ( SELECT group_concat(c.room_name) FROM chat c INNER JOIN chat_message_join j ON c.ROWID = j.chat_id WHERE j.message_id = OLD.message_id ) WHERE message.ROWID = OLD.message_id; DELETE FROM message WHERE message.ROWID = OLD.message_id AND OLD.message_id NOT IN (SELECT chat_message_join.message_id from chat_message_join WHERE chat_message_join.message_id = OLD.message_id LIMIT 1) AND OLD.message_id NOT IN (SELECT chat_recoverable_message_join.message_id from chat_recoverable_message_join WHERE chat_recoverable_message_join.message_id = OLD.message_id LIMIT 1); END; + +CREATE TRIGGER after_delete_on_chat AFTER DELETE ON chat BEGIN DELETE FROM chat_message_join WHERE chat_id = OLD.ROWID; END; + +CREATE TRIGGER before_delete_on_attachment BEFORE DELETE ON attachment BEGIN SELECT before_delete_attachment_path(OLD.ROWID, OLD.guid); END; + +CREATE TRIGGER add_to_sync_deleted_attachments AFTER DELETE ON attachment BEGIN INSERT INTO sync_deleted_attachments (guid, recordID) VALUES (OLD.guid, OLD.ck_record_id); END; + +CREATE TRIGGER delete_associated_messages_after_delete_on_message AFTER DELETE ON message BEGIN DELETE FROM message WHERE (OLD.associated_message_guid IS NULL AND associated_message_guid IS NOT NULL AND guid = OLD.associated_message_guid); END; + +CREATE TRIGGER add_to_deleted_messages AFTER DELETE ON message BEGIN INSERT INTO deleted_messages (guid) VALUES (OLD.guid); END; + +CREATE TRIGGER after_delete_on_message_attachment_join AFTER DELETE ON message_attachment_join BEGIN DELETE FROM attachment WHERE attachment.ROWID = OLD.attachment_id AND (SELECT 1 from message_attachment_join WHERE attachment_id = OLD.attachment_id LIMIT 1) IS NULL; END; + +CREATE TRIGGER update_last_failed_message_date AFTER UPDATE OF error ON message WHEN NEW.error != 0 AND NEW.date > COALESCE((SELECT value FROM kvtable WHERE key = 'lastFailedMessageDate'), 0) BEGIN INSERT OR REPLACE INTO kvtable (key, value) VALUES ('lastFailedMessageDate', NEW.date); INSERT OR REPLACE INTO kvtable (key, value) VALUES ('lastFailedMessageRowID', NEW.rowID); END; diff --git a/src/lib/ipc.ts b/src/lib/ipc.ts index ee0c6e7..c7d42cd 100644 --- a/src/lib/ipc.ts +++ b/src/lib/ipc.ts @@ -69,6 +69,8 @@ export type ChatSummary = { participantCount: number; participantHandles: string[]; messageCount: number; + /** Every chat row in this conversation (e.g. iMessage + SMS); filter by all of them. */ + chatIds: number[]; }; export type ContactSummary = { @@ -79,12 +81,8 @@ export type ContactSummary = { export type FilterSpec = { dateRange?: { start?: string; end?: string }; chatIds?: number[]; - handleIds?: number[]; - attachments?: { - types?: ('image' | 'video' | 'audio' | 'other')[]; - minBytes?: number; - maxBytes?: number; - }; + // Contact (handle) and attachment-type filters aren't applied by the + // backend yet; it rejects them rather than silently widening a delete. }; export type BackupPreview = { @@ -104,6 +102,8 @@ export type RunBackupArgs = { format: ExportFormat; destination: string; copyAttachments?: boolean; + /** Write into a new AES-256 encrypted disk image protected by this password. */ + encryptPassword?: string; }; export type BackupResult = { @@ -114,6 +114,27 @@ export type BackupResult = { manifestPath: string; exportPath: string; format: string; + /** Whether every attachment on this Mac was copied (none failed). */ + includesAttachments: boolean; + /** Per-attachment copy outcomes; null when attachments weren't copied. */ + attachmentCopy: AttachmentCopySummary | null; + /** False for PDF: it can't store non-Latin-1 text, so it isn't a full backup. */ + countsForDelete: boolean; + /** True when exportPath is an encrypted .sparsebundle. */ + encrypted: boolean; +}; + +export type AttachmentCopySummary = { + attempted: number; + copied: number; + bytes: number; + /** File isn't on this Mac (typically offloaded to iCloud). */ + notOnMac: number; + /** Record has no file path at all. */ + unresolvable: number; + failed: number; + /** The first few failures, as "path: reason". */ + failures: string[]; }; export type ProgressPayload = { @@ -128,6 +149,8 @@ export type ICloudState = 'enabled' | 'disabled' | 'unknown'; export type SafetyStatus = { messagesRunning: boolean; icloudMessages: ICloudState; + /** Folder of chat.db copies left by earlier versions, if any remain. */ + legacySnapshotDir: string | null; }; export type DeletePreview = { @@ -135,20 +158,36 @@ export type DeletePreview = { attachmentCount: number; attachmentBytes: number; onDiskFileCount: number; + /** Matched attachments whose file isn't downloaded to this Mac. */ + notOnMacCount: number; + /** Of messageCount: tapbacks outside the filter on messages being deleted. */ + reactionCount: number; }; -export type DeleteScope = 'both' | 'messages_only' | 'attachments_only'; +/** Deleting messages always deletes their attachments (chat.db's triggers do it). */ +export type DeleteScope = 'both' | 'attachments_only'; export type RunDeleteArgs = { filter: FilterSpec; confirmationPhrase: string; - backupVerified?: boolean; - /** Required by the backend when backupVerified is false. */ + /** + * Required unless a backup with this exact filter ran this session (and, + * when attachment files are deleted, copied attachments). The backend + * checks its own record of completed backups. + */ acknowledgeSkipBackup?: boolean; snapshotRoot?: string; deleteScope?: DeleteScope; - /** Must be true when Messages in iCloud is enabled. Required by the backend. */ + /** Must be true unless Messages in iCloud is detected as disabled. */ acknowledgeIcloudSync?: boolean; + /** Required when deleting attachments whose files aren't on this Mac. */ + acknowledgeNotOnMac?: boolean; + /** + * Counts the user confirmed (0 for parts the scope keeps). The backend + * refuses if its plan differs, e.g. because the filter changed. + */ + expectedMessageCount?: number; + expectedAttachmentCount?: number; }; export type DeleteResult = { @@ -161,14 +200,44 @@ export type DeleteResult = { snapshotPath: string; onDiskFilesRemoved: number; onDiskFilesFailed: number; + onDiskFilesSkipped: number; backupVerified: boolean; }; +/** Whether completed backups (from any session) cover what a filter matches now. */ +export type BackupCoverage = { + covered: boolean; + /** …and that backup copied every attachment file. */ + withAttachments: boolean; + exportPath: string | null; +}; + +export type SnapshotKind = 'delete' | 'orphanClean' | 'preRestore'; + +export type SnapshotInfo = { + path: string; + kind: SnapshotKind; + /** When its chat.db was written, Unix seconds. */ + takenAtUnix: number; + bytes: number; +}; + +export type RestoreResult = { + restoredFrom: string; + /** The database just before the restore, so the restore can be undone. */ + preRestoreSnapshot: string; +}; + export type OrphanScan = { dbOrphanCount: number; dbOrphanBytes: number; fsOrphanCount: number; fsOrphanBytes: number; + /** Attachments kept by earlier messages-only deletes (excluded unless asked). */ + retainedCount: number; + retainedBytes: number; + /** Pass back to cleanOrphans; it refuses if the orphans changed since. */ + fingerprint: string; }; export type OrphanCleanResult = { @@ -177,6 +246,8 @@ export type OrphanCleanResult = { dbFilesFailed: number; fsFilesRemoved: number; fsFilesFailed: number; + filesSkipped: number; + snapshotPath: string | null; }; @@ -192,6 +263,15 @@ export const api = { safetyStatus: () => invoke('safety_status'), previewDelete: (filter: FilterSpec) => invoke('preview_delete', { filter }), runDelete: (args: RunDeleteArgs) => invoke('run_delete', { args }), - scanOrphans: () => invoke('scan_orphans'), - cleanOrphans: () => invoke('clean_orphans'), + backupCoverage: (filter: FilterSpec) => + invoke('backup_coverage', { filter }), + listSnapshots: () => invoke('list_snapshots'), + restoreSnapshot: (snapshotPath: string, acknowledgeIcloudSync: boolean) => + invoke('restore_snapshot', { args: { snapshotPath, acknowledgeIcloudSync } }), + scanOrphans: (includeRetained = false) => + invoke('scan_orphans', { includeRetained }), + cleanOrphans: (scanFingerprint: string, includeRetained = false, acknowledgeIcloudSync = false) => + invoke('clean_orphans', { + args: { scanFingerprint, includeRetained, acknowledgeIcloudSync } + }), }; diff --git a/src/routes/+page.svelte b/src/routes/+page.svelte index 41b23f5..7fd5b26 100644 --- a/src/routes/+page.svelte +++ b/src/routes/+page.svelte @@ -15,6 +15,9 @@ type SafetyStatus, type DeletePreview, type DeleteResult, + type BackupCoverage, + type SnapshotInfo, + type RestoreResult, type DeleteScope, type OrphanScan, type OrphanCleanResult, @@ -69,12 +72,14 @@ if (/^\d{4}-\d{2}-\d{2}$/.test(trimmed)) { const [y, m, d] = trimmed.split('-').map(Number); - const date = new Date(trimmed + 'T00:00:00'); + // Build and check in the same (local) calendar: mixing local + // construction with UTC getters rejects every date east of UTC. + const date = new Date(y, m - 1, d); if ( isNaN(date.getTime()) || - date.getUTCFullYear() !== y || - date.getUTCMonth() + 1 !== m || - date.getUTCDate() !== d + date.getFullYear() !== y || + date.getMonth() + 1 !== m || + date.getDate() !== d ) { return { resolved: '', error: 'Invalid date' }; } @@ -96,6 +101,13 @@ : '' ); + // Any date problem blocks preview, backup and delete outright. Never fall + // back to "no date filter": that silently widens the scope to every + // message in the database. + let dateFilterError = $derived( + startDateParsed.error || endDateParsed.error || endDateOrderError + ); + // preview state let preview = $state(null); let previewing = $state(false); @@ -107,6 +119,20 @@ let format = $state('json'); let destination = $state(''); let copyAttachments = $state(true); + // Encrypted backups: an AES-256 disk image opened with this password. + let encryptBackup = $state(false); + let backupPassword = $state(''); + let backupPasswordConfirm = $state(''); + const MIN_PASSWORD_LEN = 8; + const passwordProblem = $derived( + !encryptBackup + ? '' + : backupPassword.length < MIN_PASSWORD_LEN + ? `Use at least ${MIN_PASSWORD_LEN} characters.` + : backupPassword !== backupPasswordConfirm + ? "The passwords don't match." + : '' + ); let running = $state(false); let runError = $state(''); let runResult = $state(null); @@ -119,6 +145,13 @@ let orphanScan = $state(null); let orphanResult = $state(null); let orphanError = $state(''); + // Whether the current scan includes attachments kept by earlier + // messages-only deletes. + let orphanIncludeRetained = $state(false); + // Messages in iCloud state for the cleanup dialog: removing attachment + // records queues them for iCloud deletion too. + let orphanIcloud = $state<'enabled' | 'disabled' | 'unknown'>('unknown'); + let orphanIcloudAck = $state(false); let unlistenMenuClean: UnlistenFn | null = null; let orphanModalCard = $state(null); @@ -177,6 +210,13 @@ let deletePreview = $state(null); let deletePreviewing = $state(false); let deletePreviewSeq = 0; + // Filter the current `deletePreview` was computed for. A preview only + // counts as current when this matches the live filter. + let deletePreviewFilterKey = $state(''); + // Backend's view of whether a backup (from this or an earlier session) + // covers exactly what the filter matches now. Fetched with the preview so + // both describe the same filter. + let coverage = $state(null); let deletePreviewTimer: ReturnType | undefined; let deleteMessages = $state(true); let deleteAttachments = $state(true); @@ -184,7 +224,10 @@ let deleteResult = $state(null); let deleteError = $state(''); let icloudAcknowledged = $state(false); + let notOnMacAcknowledged = $state(false); let skipBackupAcknowledged = $state(false); + // What the user typed to confirm. The backend requires exactly "DELETE". + let typedConfirmation = $state(''); let unlistenFocus: UnlistenFn | null = null; @@ -206,7 +249,7 @@ }).then((fn) => { unlistenProgress = fn; }); - listen('menu:clean-orphans', startOrphanScan).then((fn) => { + listen('menu:clean-orphans', () => startOrphanScan(false)).then((fn) => { unlistenMenuClean = fn; }); }); @@ -308,10 +351,14 @@ } } - function toggleChat(rowid: number) { + function isChatSelected(chat: ChatSummary) { + return chat.chatIds.some((id) => selectedChatIds.has(id)); + } + + function toggleChat(chat: ChatSummary) { const next = new Set(selectedChatIds); - if (next.has(rowid)) next.delete(rowid); - else next.add(rowid); + if (isChatSelected(chat)) for (const id of chat.chatIds) next.delete(id); + else next.add(chat.rowid); selectedChatIds = next; } @@ -327,7 +374,15 @@ filter.dateRange = { start: start || undefined, end: end || undefined }; } if (selectedChatIds.size > 0) { - filter.chatIds = [...selectedChatIds]; + // A listed conversation can span several chat rows (iMessage + SMS…); + // include all of them so nothing is left behind. + // Look up by any member id, since the primary row can change when the + // list is reloaded. + const byMember = new Map(); + for (const c of chats ?? []) for (const cid of c.chatIds) byMember.set(cid, c.chatIds); + const ids = new Set(); + for (const id of selectedChatIds) for (const cid of byMember.get(id) ?? [id]) ids.add(cid); + filter.chatIds = [...ids].sort((a, b) => a - b); } return filter; } @@ -360,7 +415,7 @@ void selectedChatIds; if (status !== 'ok') return; - if (startDateParsed.error || endDateParsed.error || endDateOrderError) return; + if (dateFilterError) return; clearTimeout(previewTimer); previewTimer = setTimeout(runPreview, 400); @@ -379,6 +434,34 @@ void selectedChatIds; runResult = null; skipBackupAcknowledged = false; + // The "not on this Mac" acknowledgement was for a specific set of + // attachments; a different filter needs a fresh decision. + notOnMacAcknowledged = false; + typedConfirmation = ''; + }); + + // Likewise the iCloud acknowledgement is for the state the user saw. A + // derived only notifies when its value changes, so re-probing the same + // state (as runDelete does) doesn't clear it. + const icloudState = $derived(safety?.icloudMessages ?? null); + $effect(() => { + void icloudState; + icloudAcknowledged = false; + }); + + // Backups are plain, unencrypted copies of every message and attachment. + // Point it out more strongly where the folder is likely to leave the Mac + // or be readable by others. + const destinationRisk = $derived.by(() => { + const d = destination; + if (!d) return ''; + if (d.startsWith('/Volumes/')) + return 'This is an external drive. Anyone with the drive can read the backup — many drive formats have no file permissions.'; + if (d.includes('/Library/Mobile Documents/')) + return 'This folder is in iCloud Drive, so the backup will be uploaded to iCloud and may be removed from this Mac to save space.'; + if (/\/Users\/[^/]+\/(Desktop|Documents)(\/|$)/.test(d)) + return 'If "Desktop & Documents" syncing is on in iCloud settings, this backup will be uploaded to iCloud Drive and may be removed from this Mac to save space.'; + return ''; }); async function pickDestination() { @@ -398,19 +481,35 @@ runError = 'Pick a destination folder first.'; return; } + if (dateFilterError) { + runError = `Fix the date filter first: ${dateFilterError}.`; + return; + } + if (passwordProblem) { + runError = passwordProblem; + return; + } running = true; try { runResult = await api.runBackup({ filter: buildFilter(), format, destination, - copyAttachments + copyAttachments, + encryptPassword: encryptBackup ? backupPassword : undefined }); } catch (err) { runError = appErrorMessage(err); } finally { running = false; + // Don't keep the password around in the page longer than needed. + backupPassword = ''; + backupPasswordConfirm = ''; } + // A new backup supersedes any earlier "delete without a backup" choice. + if (runResult) skipBackupAcknowledged = false; + // Refresh what the delete step knows about backups. + if (deleteExpanded && status === 'ok' && !dateFilterError) await runDeletePreview(); } let checkingSafety = $state(false); @@ -429,6 +528,62 @@ } } + // Snapshots that can be restored (undo a delete or cleanup). + let snapshots = $state(null); + let restoring = $state(false); + let restoreError = $state(''); + let restoreResult = $state(null); + + async function loadSnapshots() { + try { + snapshots = await api.listSnapshots(); + } catch (err) { + restoreError = appErrorMessage(err); + } + } + + const snapshotKindLabel: Record = { + delete: 'Before a delete', + orphanClean: 'Before an orphan cleanup', + preRestore: 'Before a restore' + }; + + async function restoreSnapshot(snap: SnapshotInfo) { + restoreError = ''; + restoreResult = null; + const ok = await refreshSafety(); + if (!ok) return; + if (safety?.messagesRunning) { + restoreError = 'Quit the Messages app before restoring.'; + return; + } + const when = new Date(snap.takenAtUnix * 1000).toLocaleString(); + const icloudOn = safety?.icloudMessages !== 'disabled'; + const confirmed = await confirm( + `Restore the Messages database to how it was on ${when}?\n\n` + + 'Messages received on this Mac since then will be removed from it. Attachment files ' + + "deleted since then aren't in the snapshot and stay deleted (restore them from your " + + 'backup). The current database is saved first, so you can undo this.' + + (icloudOn + ? '\n\nMessages in iCloud may be on: how iCloud reconciles a database rolled back ' + + 'underneath it is unpredictable.' + : ''), + { title: 'Restore Snapshot', kind: 'warning' } + ); + if (!confirmed) return; + restoring = true; + try { + // The dialog above named the iCloud risk when it applied, so OK is + // the user's explicit acknowledgement of it. + restoreResult = await api.restoreSnapshot(snap.path, icloudOn); + await Promise.all([loadSnapshots(), runPreview(), runDeletePreview(), loadChats()]); + } catch (err) { + restoreError = appErrorMessage(err); + } finally { + restoring = false; + } + } + async function expandDelete() { deleteError = ''; const ok = await refreshSafety(); @@ -438,19 +593,30 @@ return; } deleteExpanded = true; + void loadSnapshots(); } async function runDeletePreview() { const seq = ++deletePreviewSeq; deletePreviewing = true; deleteError = ''; + const filter = buildFilter(); try { - const result = await api.previewDelete(buildFilter()); - if (seq === deletePreviewSeq) deletePreview = result; + const [result, cov] = await Promise.all([ + api.previewDelete(filter), + api.backupCoverage(filter) + ]); + if (seq === deletePreviewSeq) { + deletePreview = result; + coverage = cov; + deletePreviewFilterKey = JSON.stringify(filter); + } } catch (err) { if (seq === deletePreviewSeq) { deleteError = appErrorMessage(err); deletePreview = null; + coverage = null; + deletePreviewFilterKey = ''; } } finally { if (seq === deletePreviewSeq) deletePreviewing = false; @@ -463,8 +629,19 @@ void endDate; void selectedChatIds; void deleteExpanded; + // The chat list feeds buildFilter() (a conversation can span several + // chat rows), so a reload can change the filter too. + void chats; if (status !== 'ok' || !deleteExpanded) return; + if (dateFilterError) { + // Don't show (or keep) counts for a filter we can't apply. + clearTimeout(deletePreviewTimer); + deletePreview = null; + coverage = null; + deletePreviewFilterKey = ''; + return; + } clearTimeout(deletePreviewTimer); deletePreviewTimer = setTimeout(runDeletePreview, 400); @@ -472,7 +649,10 @@ }); async function runDelete() { - if (!deletePreview) return; + if (!deletePreview || !deletePreviewCurrent || dateFilterError) return; + const filter = buildFilter(); + const expectedMessageCount = deleteMessages ? deletePreview.messageCount : 0; + const expectedAttachmentCount = deleteAttachments ? deletePreview.attachmentCount : 0; const parts: string[] = []; if (deleteMessages && deletePreview.messageCount > 0) @@ -485,7 +665,7 @@ ); const ok = await confirm( - `Permanently delete ${parts.join(' and ')} from chat.db? This cannot be undone.`, + `Permanently delete ${parts.join(' and ')} from ${deleteScopeDescription}? This cannot be undone.`, { title: 'Confirm Delete', kind: 'warning' } ); if (!ok) return; @@ -496,49 +676,100 @@ try { // Final safety probe: Messages.app could have reopened between the // preview and the click. + const icloudBefore = safety?.icloudMessages; const safetyOk = await refreshSafety(); if (!safetyOk) { // refreshSafety already populated deleteError. return; } + if (safety?.icloudMessages !== icloudBefore) { + deleteError = + 'The Messages in iCloud status changed. Review the updated warning above before retrying.'; + return; + } if (safety?.messagesRunning) { deleteError = 'The Messages app is running again — quit it and click Re-check before retrying.'; return; } deleteResult = await api.runDelete({ - filter: buildFilter(), - confirmationPhrase: 'DELETE', - backupVerified: runResult !== null, - acknowledgeSkipBackup: runResult !== null || skipBackupAcknowledged, + filter, + // The backend refuses if what it would delete differs from what the + // dialog above showed. + expectedMessageCount, + expectedAttachmentCount, + confirmationPhrase: typedConfirmation.trim(), + // The backend checks its own record of completed backups; only the + // explicit opt-out is ours to send — and only while it's on screen. + // A tick left over from before a backup ran must not count if that + // backup has since stopped covering the delete (e.g. its drive was + // ejected). + acknowledgeSkipBackup: skipBackupAcknowledged && !backupCoversDelete, deleteScope, - // Surface the iCloud risk to the backend so it can refuse if detection - // flipped to Enabled since the user ticked the acknowledgement box. - acknowledgeIcloudSync: safety?.icloudMessages !== 'enabled' || icloudAcknowledged + // Only the user's own acknowledgement — never auto-ack, or the + // backend gate is bypassed if its detection disagrees with ours. + acknowledgeIcloudSync: icloudAcknowledged, + acknowledgeNotOnMac: notOnMacAcknowledged }); - await Promise.all([runPreview(), runDeletePreview(), loadChats()]); + // One-off decisions: the next delete needs its own. + skipBackupAcknowledged = false; + notOnMacAcknowledged = false; + typedConfirmation = ''; + await Promise.all([runPreview(), runDeletePreview(), loadChats(), loadSnapshots()]); } catch (err) { deleteError = appErrorMessage(err); + // The backend refuses when what it would delete no longer matches the + // preview or the backup (e.g. a message just arrived). Refresh both so + // the user sees the current state instead of retrying stale counts. + void runDeletePreview().then(() => { + if (!deleteError) deleteError = appErrorMessage(err); + }); } finally { deleting = false; } } - const deleteScope = $derived( - deleteMessages && deleteAttachments - ? 'both' - : deleteMessages - ? 'messages_only' - : 'attachments_only' + // Deleting a message always deletes its attachments: chat.db's own + // triggers remove an attachment's record when its last message goes, so + // "keep the attachments" would only leave untracked files behind. + $effect(() => { + if (deleteMessages) deleteAttachments = true; + }); + + const deleteScope = $derived(deleteMessages ? 'both' : 'attachments_only'); + + // Conversations the current filter selects, by name. Chats hidden by the + // search box stay selected, so the delete step must say which ones count. + const selectedChatNames = $derived( + (chats ?? []).filter((c) => isChatSelected(c)).map((c) => displayName(c)) ); - async function startOrphanScan() { + const deleteScopeDescription = $derived.by(() => { + const convs = + selectedChatNames.length === 0 + ? 'all conversations' + : selectedChatNames.length <= 3 + ? selectedChatNames.join(', ') + : `${selectedChatNames.slice(0, 3).join(', ')} and ${selectedChatNames.length - 3} more conversations`; + const s = startDateParsed.resolved; + const e = endDateParsed.resolved; + const dates = s && e ? ` (${s} to ${e})` : s ? ` (from ${s})` : e ? ` (until ${e})` : ''; + return convs + dates; + }); + + async function startOrphanScan(includeRetained = false) { + orphanIncludeRetained = includeRetained; orphanModal = 'scanning'; orphanScan = null; orphanResult = null; orphanError = ''; + orphanIcloudAck = false; try { - orphanScan = await api.scanOrphans(); + orphanIcloud = await api + .safetyStatus() + .then((s) => s.icloudMessages) + .catch(() => 'unknown' as const); + orphanScan = await api.scanOrphans(includeRetained); orphanModal = 'preview'; } catch (err) { orphanError = appErrorMessage(err); @@ -547,10 +778,15 @@ } async function runOrphanClean() { + if (!orphanScan) return; orphanModal = 'cleaning'; orphanError = ''; try { - orphanResult = await api.cleanOrphans(); + orphanResult = await api.cleanOrphans( + orphanScan.fingerprint, + orphanIncludeRetained, + orphanIcloudAck + ); orphanModal = 'done'; } catch (err) { orphanError = appErrorMessage(err); @@ -558,6 +794,10 @@ } } + const orphanNeedsIcloudAck = $derived( + (orphanScan?.dbOrphanCount ?? 0) > 0 && orphanIcloud !== 'disabled' + ); + const totalOrphanItems = $derived( (orphanScan?.dbOrphanCount ?? 0) + (orphanScan?.fsOrphanCount ?? 0) ); @@ -565,15 +805,37 @@ (orphanScan?.dbOrphanBytes ?? 0) + (orphanScan?.fsOrphanBytes ?? 0) ); + // A backup only protects attachment files if it copied them: the safety + // snapshot taken before a delete covers chat.db, not files on disk. + const backupCoversDelete = $derived( + coverage !== null && + coverage.covered && + (!deleteAttachments || coverage.withAttachments) + ); + + // Attachment records whose files only exist in iCloud: no backup can + // include them, and deleting the records makes them unrecoverable. + const deletesNotOnMac = $derived(deleteAttachments && (deletePreview?.notOnMacCount ?? 0) > 0); + + const deletePreviewCurrent = $derived( + deletePreview !== null && + !deletePreviewing && + deletePreviewFilterKey === JSON.stringify(buildFilter()) + ); + const deleteReady = $derived( deletePreview !== null && + deletePreviewCurrent && + !dateFilterError && (deleteMessages || deleteAttachments) && ((deleteMessages && deletePreview.messageCount > 0) || (deleteAttachments && deletePreview.attachmentCount > 0)) && safety !== null && !safety.messagesRunning && - (safety.icloudMessages !== 'enabled' || icloudAcknowledged) && - (runResult !== null || skipBackupAcknowledged) && + (safety.icloudMessages === 'disabled' || icloudAcknowledged) && + (backupCoversDelete || skipBackupAcknowledged) && + typedConfirmation.trim() === 'DELETE' && + (!deletesNotOnMac || notOnMacAcknowledged) && !deleting ); @@ -752,8 +1014,8 @@ @@ -869,8 +1131,55 @@ Copy attachment files into the backup folder + + {#if encryptBackup} +
+ + +
+ {#if passwordProblem && (backupPassword || backupPasswordConfirm)} +

{passwordProblem}

+ {/if} +

+ The backup is saved as a .sparsebundle you open in Finder with this password. + If you forget the password, the backup can't be opened — keep it in your + password manager. +

+ {:else} +

+ This backup won't be encrypted: anyone who can open the folder can read every message and + attachment in it. +

+ {#if destinationRisk} +

{destinationRisk}

+ {/if} + {/if} +
-
@@ -899,26 +1208,77 @@ {#if runResult}
-

Backup complete — {runResult.format.toUpperCase()}.

+

Backup {runResult.attachmentCopy && runResult.attachmentCopy.failed > 0 ? 'finished with errors' : 'complete'} — {runResult.format.toUpperCase()}.

+ {#if runResult.encrypted} +

+ Saved in an encrypted disk image. Double-click it in Finder and enter your password to + open it. +

+ {/if} + {#if !runResult.countsForDelete} +

+ PDF can only store basic Latin text — emoji, smart quotes and non-Latin scripts come out + as “?” — so it doesn't count as a backup for the delete step. Use JSON or HTML for that. +

+ {/if}
  • {runResult.messageCount.toLocaleString()} messages
  • {runResult.conversationCount.toLocaleString()} conversations
    • -
  • - {runResult.attachmentCount.toLocaleString()} attachments - {#if runResult.attachmentBytesCopied > 0} - ({formatBytes(runResult.attachmentBytesCopied)} copied) + {#if runResult.attachmentCopy} + {@const ac = runResult.attachmentCopy} +
  • + {ac.copied.toLocaleString()} of {ac.attempted.toLocaleString()} attachments copied + ({formatBytes(ac.bytes)}) +
    • + {#if ac.notOnMac > 0} +
  • + {ac.notOnMac.toLocaleString()} not downloaded to this Mac (not in this backup) +
    • {/if} - + {#if ac.unresolvable > 0} +
  • {ac.unresolvable.toLocaleString()} with no file to copy
    • + {/if} + {:else} +
  • + {runResult.attachmentCount.toLocaleString()} attachments (referenced, not copied) +
    • + {/if} +
  • Saved to {runResult.exportPath}
    • {#if runResult.manifestPath}
  • Manifest: {runResult.manifestPath}
    • {/if}
+ {#if runResult.attachmentCopy && runResult.attachmentCopy.failed > 0} +

+ {runResult.attachmentCopy.failed.toLocaleString()} attachment{runResult.attachmentCopy + .failed === 1 + ? '' + : 's'} failed to copy: +

+
    + {#each runResult.attachmentCopy.failures as f, i (i)} +
  • {f}
    • + {/each} +
+ {/if}
{/if} + {#snippet reactionNote()} + {#if deletePreview && deletePreview.reactionCount > 0} +

+ Includes {deletePreview.reactionCount.toLocaleString()} tapback{deletePreview.reactionCount === + 1 + ? '' + : 's'} outside the filter on messages being deleted, so the backup preview above shows + fewer messages. +

+ {/if} + {/snippet} +
5 @@ -931,6 +1291,13 @@

{:else} + {#if safety?.legacySnapshotDir} +

+ Earlier versions of Bubble Wrap kept full copies of your messages in + {safety.legacySnapshotDir}, which may sync to iCloud Drive. New snapshots go + to the app's own folder; delete the old ones if you don't need them. +

+ {/if} {#if safety?.messagesRunning}

The Messages app is currently running. Please quit it before deleting messages. @@ -957,20 +1324,26 @@ {/if} {#if safety?.icloudMessages === 'enabled'}

- Messages in iCloud is enabled. Deletes on this Mac will replicate to all - your other devices signed into the same Apple ID. Disable it in - System Settings → [your name] → iCloud → Show More Apps → Messages if you - want local-only deletion. + Messages in iCloud is enabled. The Messages database records every + deletion for iCloud sync, so when Messages next syncs, these messages and attachments are + expected to be deleted from iCloud and from your other devices too — not just this Mac. If + you only want to free up space here, this isn't the right tool while Messages in iCloud is + on (System Settings → [your name] → iCloud → Show More Apps → Messages).

{:else if safety?.icloudMessages === 'unknown'}

- Couldn't confirm Messages-in-iCloud state. If it's enabled, deletes will replicate to your - other devices. + Couldn't confirm Messages-in-iCloud state. If it's enabled, deletions are recorded for + iCloud sync and are expected to remove these messages from iCloud and your other devices + too.

+ {/if}
@@ -979,14 +1352,32 @@ Delete messages + {#if deleteMessages} +

+ Deleting messages always deletes their attachments too — the Messages database does + this itself. To keep conversations and only free up attachment space, untick “Delete + messages”. +

+ {/if}
- {#if deletePreviewing} + {#if dateFilterError} +

Fix the date filter above before deleting: {dateFilterError}.

+ {:else if deletePreviewing || (deletePreview && !deletePreviewCurrent)} +

Counting…

{:else if deletePreview} +

+ Applies to {deleteScopeDescription}{#if selectedChatNames.length > 0 && chatSearch.trim()} + — including selected conversations hidden by the search{/if}. +

{#if deleteMessages && deleteAttachments}

Will delete {deletePreview.messageCount.toLocaleString()} messages and @@ -994,11 +1385,7 @@ ({formatBytes(deletePreview.attachmentBytes)} on disk across {deletePreview.onDiskFileCount.toLocaleString()} files).

- {:else if deleteMessages} -

- Will delete {deletePreview.messageCount.toLocaleString()} messages. - Attachment records and files will be left on disk. -

+ {@render reactionNote()} {:else if deleteAttachments}

Will delete {deletePreview.attachmentCount.toLocaleString()} attachment @@ -1010,10 +1397,42 @@

Select at least one option above.

{/if} - {#if !runResult} + {#if deletesNotOnMac && deletePreview} +

+ {deletePreview.notOnMacCount.toLocaleString()} attachment{deletePreview.notOnMacCount === + 1 + ? '' + : 's'} in this delete {deletePreview.notOnMacCount === 1 ? "isn't" : "aren't"} downloaded + to this Mac, so no backup can include {deletePreview.notOnMacCount === 1 ? 'it' : 'them'}. + Deleting removes the only way to download {deletePreview.notOnMacCount === 1 + ? 'it' + : 'them'} again. Open those conversations in Messages to download first. +

+ + {/if} + + {#if backupCoversDelete && coverage?.exportPath} +

Backed up in {coverage.exportPath}.

+ {/if} + {#if !backupCoversDelete}

- Tip: run a backup first (Step 4). Without a backup you have no way to restore the data - you're about to remove. + {#if coverage?.covered && runResult?.attachmentCopy && runResult.attachmentCopy.failed > 0} + Some attachments failed to copy during your backup, so it doesn't protect them. Fix + the errors listed in Step 4 and re-run the backup, or delete without that protection. + {:else if coverage?.covered} + Your backup of these messages didn't copy every attachment file. Deleted attachments + can't be restored from the safety snapshot — re-run the backup (Step 4) with + attachments copied. + {:else if runResult} + These messages have changed since your backup (new messages may have arrived), so it + no longer covers everything this delete would remove. Re-run the backup (Step 4). + {:else} + Tip: run a backup first (Step 4). Without a backup you have no way to restore the data + you're about to remove. + {/if}

@@ -1098,6 +1578,35 @@ {formatBytes(orphanScan.fsOrphanBytes)} + {#if orphanScan.retainedCount > 0 || orphanIncludeRetained} + + {/if} +

+ Quit Messages before cleaning. Folders that still hold a referenced attachment, files + changed in the last 24 hours and records created in the last 24 hours are left alone. + chat.db is snapshotted before any records are removed, but removed files can't be + restored. +

+ {#if orphanNeedsIcloudAck} +

+ {orphanIcloud === 'enabled' + ? 'Messages in iCloud is enabled.' + : "Couldn't confirm whether Messages in iCloud is enabled."} Removing attachment records + queues them for deletion in iCloud, which can remove them from your other devices too. +

+ + {/if}

{#if totalOrphanItems === 0} No orphaned data found. @@ -1108,7 +1617,11 @@

{#if totalOrphanItems > 0} - {/if} @@ -1135,6 +1648,14 @@ disk{#if filesFailed > 0}, {filesFailed.toLocaleString()} failed{/if} {/if} + {#if orphanResult.filesSkipped > 0} +
  • + {orphanResult.filesSkipped.toLocaleString()} skipped (outside the Messages folders) +
    • + {/if} + {#if orphanResult.snapshotPath} +
  • Snapshot: {orphanResult.snapshotPath}
    • + {/if}
    @@ -1262,6 +1783,7 @@ } .field input[type='search'], .field input[type='text'], + .field input[type='password'], .field select { padding: 7px 10px; font: inherit; @@ -1552,6 +2074,33 @@ letter-spacing: -0.01em; } + .restore { + margin-top: 24px; + padding-top: 16px; + border-top: 1px solid var(--border, rgba(127, 127, 127, 0.25)); + } + + .restore h3 { + margin: 0 0 6px; + font-size: 15px; + } + + .snapshot-list { + list-style: none; + padding: 0; + margin: 8px 0 0; + display: flex; + flex-direction: column; + gap: 6px; + } + + .snapshot-list li { + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; + } + .orphan-rows { display: flex; flex-direction: column;