[schlessera]

Refs #2789

This is a partial delivery of the option 1 ruling, so it uses Refs. It does not meet the issue's fsync-load acceptance criterion; see "Anything left open". The issue is handed back with needs: decision.

What changed

A journal write no longer truncates the SQLite WAL before it begins. Every write used to run wal_checkpoint(TRUNCATE); now a ledger truncates once, when it is constructed at server start. Later writes charge the whole WAL file as it stands, whoever wrote it, on top of the measurement's existing one-commit WAL headroom. A write truncates and retries only when that charge would refuse admission, as every write used to. The capacity measurement now walks only the journal's own b-trees (dbstat WHERE name = ?), not every b-tree in the database. A formerly owned page outside them stays charged while it can still be free, capped at SQLite's unallocated page count. The charge therefore never falls below #2172's page-identity measurement; it can only exceed it, and only when another component reuses a page the journal released. Connection-local TEMP triggers refuse a turn_journal_* row changed outside an accounted write (migration 063 adds the writing flag they read). A store already over its cap still constructs, so recover() can release abandoned reservations. Journal commits stay synchronous=FULL. The server keeps each maintenance temp-budget ledger open instead of opening, committing and closing it per reserve/release. It revalidates the path, inode and R/W access on every use and reopens a replaced file. Those commits use NORMAL: a reservation binds only live processes of one boot (alive() checks the boot id), so a power loss cannot strand a live one.

Why this shape

Three shapes were built and measured; the reasons for each are below.

1. Counts-only ledger, no per-write dbstat. Owned pages moved by Δ(page_count − freelist_count), running totals came from triggers. It measured best: no journal windows in two runs. An independent gpt-6-astra review then reproduced three soundness failures, and I confirmed them in source. F1: free-list deltas discharge journal pages that are still free when the journal reuses another component's pages; one probe lost 244 pages. F2: recordGovernanceEvent() runs inside journal transactions and deletes expired governance rows, which drives owned pages negative. F3: legacy conversion lost freed pages. Counts cannot tell page owners apart, so this shape cannot stay exact. It was dropped.
2. CONTRACT: feat(storage): deliver owner inventory, recovery and capacity guidance #2172's measure() verbatim plus WAL charging. This is exact by construction. Under fsync load it was worse for the journal (4 and 3 windows ≥ 250 ms). The full-database dbstat walk itself waits on I/O when the page cache is under write pressure.
3. This PR: journal-only walk. It is sound against CONTRACT: feat(storage): deliver owner inventory, recovery and capacity guidance #2172, as shown by the oracle test below, and walks only journal pages. The journal still drew 2 windows ≥ 250 ms per run.

A second review round on shape 2 found two more problems: a failed commit's WAL tail could escape tracked debt, and construction could refuse on capacity before recovery. Both are fixed here. Charging the whole current WAL replaced the debt bookkeeping, which removes that whole class: abandoned tails, other processes, resets, journal_size_limit. The cost is that the journal may be charged for non-journal WAL frames. In steady state SQLite's 1000-frame autocheckpoint bounds those to about 4 MiB.

Proof

Before/after, the #2416 method. The #2416 diagnostic patch (20 ms timer-drift probe, bun --cpu-prof) ran on a scratch checkout only; it is not in this PR. The cell was native pi UID 1000, with three dd if=/dev/zero bs=1M count=1024 conv=fsync writers on the app database's filesystem, run under the fixture flock. The table shows windows ≥ 250 ms after ready, assigned by the dominant categorized CPU frames: count, then max.

Head	journal SQLite	maintenance budget	Activity SQLite	other
before: origin/main 6492291	2, max 590 ms	4, max 363 ms	1, max 278 ms	1
shape 1, counts-only (unsound), run 1 / run 2	0 / 0	1* / 0	1, max 385 / 2, max 356	3 / 3
shape 2, verbatim measure, run 1 / run 2	4, max 565 / 3, max 476	0 / 0	4, max 414 / 3, max 410	1 / 1
shape 3 (8b5934bc7, this PR's I/O path), run 1 / run 2	2, max 383 / 2, max 391	1* / 0	3, max 441 / 2, max 458	5 / 3

* In each starred window, one sample out of 9 or 11 sat in the budget's procfs liveness read; native-output publication dominated. The final head differs from 8b5934bc7 only in charging arithmetic (whole WAL instead of tracked debt) and the deferral at construction; neither changes I/O.

The windows that remain hold 2–16 ms of CPU samples in 230–590 ms. The blocked time is unsampled, which is consistent with each synchronous=FULL commit's fsync: journal, Activity, and the native lease DB. A vote picks up the journal when measure() samples land in the same window as its commit.

#2172 accounting tests, unchanged: packages/ui-server/tests/journal-accounting.test.ts passed 10/10 on every shape, including this head.

New tests in packages/ui-server/tests/journal-ledger.test.ts (7) and one in packages/ui-sdk/tests/maintenance-temp-budget.test.ts. Failing-first against origin/main sources: JOURNAL_WRITE_RUNS_NO_CHECKPOINT (main ran 12 wal_checkpoint(TRUNCATE) for 12 writes), JOURNAL_GUARD_turn_journal_objects_INSERT, JOURNAL_COMMITS_ACCUMULATE_WAL, MAINTENANCE_LEDGER_HELD_OPEN (main closed the ledger 6 times).

Mutation receipts. Each row breaks the guard, runs the named suite, and restores the guard; all fail as stated.

Mutation	Failing test / assertion
M1 no escape-guard triggers	refused on every journal table / JOURNAL_GUARD_*
M2 finish() writes outside accounting.write	18 turn-journal.test.ts cases, journal storage write outside capacity accounting (a write escaping accounting fails)
M5 checkpoint on every write	JOURNAL_WRITE_RUNS_NO_CHECKPOINT
M6 admission ignores the WAL	JOURNAL_LARGE_WRITE_TRUNCATES_A_LARGE_WAL
M7 WAL alone refuses without truncating	JournalCapacityError in the "never refuses" test
M8 budget closes per use	MAINTENANCE_LEDGER_HELD_OPEN
M9 budget ignores a replaced file	MAINTENANCE_REPLACED_LEDGER_REOPENED
M10 budget ignores revoked access	MAINTENANCE_REVOKED_LEDGER_NOT_REUSED
M11 released pages dropped	JOURNAL_CHARGE_COVERS_RELEASED_PAGES_RELEASED (page-identity oracle)
M12 construction refuses over cap	JOURNAL_OVER_CAP_STARTUP_RECOVERS

Suites: bun run test packages/ui-server packages/ui-sdk ran 4847 tests: 4842 passed, 5 failed.

• Four were the inbox-runtime.test.ts "mounted internal poke" 30 s timeouts. They fail identically on origin/main 69151ca.
• One was session-start-reliability.test.ts "wire dispatch echoes the correct draft ids", an ordering race. On rerun it failed 1/3 on this branch and passed 6/6 on main. The test touches only WsHost and a fake backend, with no SQLite, journal or budget code.

Typecheck, lint: clean. No package export or entry changed and no bun: import was added, so the pack smoke is not required.

Native hosted proof, real harness (no diagnostic patch), under flock …/native.lock, head f13dccd68 on base dc5d3c732:

• With three dd … conv=fsync writers: passed. All four cells passed, including HOSTED_ADMISSION_HEALTH_WITHIN_1_SECOND with the bound unchanged; the retained-index reproduction failed as expected; exit 0.
• No load, run 3: passed, same shape as above.
• No load, run 1: the health/admission test passed in every cell that ran. The claude uid 0 cell's second test ("schedule Git publication inherits active native peer folder admission") then hit the known intermittent invalid worker mount record refusal (#2701). That code is not touched here.
• No load, run 2: the claude uid 0 cell failed HOSTED_ADMISSION_HEALTH_WITHIN_1_SECOND. One warm-phase probe hit the 1 s timeout and one ping took 1,136 ms: the tests: attribute recurring one-second native hosted health timeouts #2416 symptom, while four other workers' suites shared the host. That run had no instrumentation. Two instrumented runs of that cell on the same head passed (health max 253 and 360 ms); the only window over 250 ms was one Activity SQLite window. Claude's start-turn probe and spawn are still synchronous (#2790). The cause was not attributed and the failed run was not relabelled.

bun run check:pr --base origin/main on f13dccd68: passed. 624 tests across 23 files, "Local preflight passed."

Checklist

• Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
• Acceptance criteria from the issue are all met, or the gap is named below. Gap named below.
• The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
• A changeset is included, or this changes nothing a consumer can see.
• No personal data anywhere in the diff — the leakage gate covers the whole tree.
• The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. The journal cap, its refusal classes and its reporting are unchanged.
• Docs under docs/ reflect the change, or nothing documented moved.
• Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

• The fsync-load criterion is not met for the journal or Activity: 2–3 windows ≥ 250 ms per run remain. Under this load each synchronous=FULL commit is a blocking fsync on the event loop, whatever the accounting does. Options and a recommendation are on #2789, which now carries needs: decision.
• dbstat still runs per write, scoped to the journal's own b-trees. The review showed that exact page accounting with no per-write walk cannot be done from counts.
• Callback replay after a rolled-back capacity refusal (for example observe()'s ++turn.sequence) predates this PR through retain(). This PR adds the same replay for a WAL-forced truncation.
• Follow-up for the uncategorized windows (native lease DB synchronous=FULL, pi transcript writes, native output publication): #2807 (needs scoping).

🤖 Generated with Claude Code