[schlessera]

Closes #2814

What changed

Journal, Activity and native-lease SQLite writes now commit on a write-worker thread instead of the event loop. Each database file gets one worker with its own connection at synchronous=FULL. The worker runs calls one at a time, in the order they were submitted. The event loop awaits each commit's outcome instead of blocking in its fsync.

• Journal. The journal's transaction bodies moved, with their reads, into brain/journal-store.ts. They run through the unchanged createJournalAccounting. The worker attaches the journal's own journal.db (fix(ui-server): give the turn journal its own database file #2843) as journal, on the same connection as the UI database, so the ledger's own attach is a no-op there.
• Activity. The hosted server's Activity store submits its own transactions to the worker. A write made inside the caller's transaction stays inline, so it is still atomic with the caller's rows.
• Native leases. The lease manager commits its durable intents on the lease database's worker.
• Callers. Callers that need a commit's outcome now await it:
  • journal bindSession, writeOperation (which now takes statements), Claude memory persistence, revert reservation, retained-version leases and retained streams;
  • turn revert start;
  • storage removal start and cancel.
• Startup. createTurnJournal exposes ready, which covers startup recovery and the steering-event replay from fix(ui-server): give the turn journal its own database file #2843. The app awaits it before serving.
• Contract. StartTurnRequest.persistClaudeMemory may now return a promise, and the Claude backend awaits it. This is additive, so it carries CONTRACT:.

Why this shape

The ruling (#2789, ruling 1) keeps synchronous=FULL, transaction order and refusal semantics. Each rejected alternative and the reason it lost are in docs/decisions/sqlite-write-worker.md:

• Commit at NORMAL, then fsync the WAL off the loop. That changes the durability level of each commit, which the ruling excludes.
• Replay recorded SQL in the worker. Journal transactions decide from reads inside the same transaction, so a replay could commit a decision made against a different database state.
• Synchronous RPC per statement. Every statement would still block the loop.

Transaction bodies therefore move into the worker as code, with their reads.

Five things the implementation found:

1. App transactions and worker procedures take turns. With a second writer on ui.db, an app BEGIN IMMEDIATE failed with SQLITE_BUSY after about 37 ms despite a 5 s busy timeout, and a deferred read-then-write transaction can lose its snapshot upgrade to a worker commit. The first native run hit this as a governed-turn database is locked. So each outermost app transaction, and each autocommit write, waits until the worker is between procedures (takeTurns). The cost of this is under "Anything left open".
2. Model binding stays synchronous. The model is adopted at once from in-memory turn state, because later governance events read it; the row commits after.
3. A turn waits for its Activity writes. The session slot waits for the recorder's writes before the next turn starts or the session ends. An autonomous turn waits for them before the transaction that commits its terminal, so that terminal still commits inside that transaction.
4. A Bun 1.4.2 test-runner trap. expect(promise).rejects/.resolves waits in a nested event loop that does not deliver worker messages, so an assertion on a promise that needs a worker round trip never settles. Affected tests use expectRejection instead; this is documented in docs/process/testing.md.
5. Two fix(ui-server): give the turn journal its own database file #2843 tests assumed a synchronous start. The journal's steering replay and its startup write now finish behind ready. The replay test asserted before the replay ran. The SIGKILL storage-removal children raced the parent fixture's startup write: the child's first journal write found a reader on the journal's WAL and refused with journal_storage_unavailable. Both tests now await journal.ready; production already awaits it before serving.

Proof

Head ea1758624. It merges origin/main at d2bb3b301 in b92154f43; no force-push. Earlier, d59f57b70 merged c696f995c (#2843).

• Conflicts in that merge:
  • inbox/autonomous-turn.ts: CONTRACT: fix(ui-server): defer and batch the index update after host applications #2851's index flush now runs before recorder.settled(). The flush records its outcome in Activity, so that write commits on the worker before the terminal transaction, like the run's other Activity writes.
  • 13 docs: citation line numbers only. I took main's text and shifted each citation to its anchor in the merged sources. A three-way check found no cited range whose length the branch had changed being given main's length.
• CONTRACT: fix(ui-server): defer and batch the index update after host applications #2851 through the worker. ea1758624 adds two tests:
  • A hosted session's turn-end flush runs on a file database with the worker-backed store, and the rows are read after the session returns (ACTIVITY_INDEX_OUTCOME_COMMITTED_OFF_LOOP).
  • An autonomous turn's flush holds its Activity write, and the terminal transaction must wait for it (AUTONOMOUS_TERMINAL_WAITS_FOR_INDEX_OUTCOME).
• Main has moved again since then, to 1f032f95b (2 commits). A trial merge is textually clean, and tsc and tests/decision-citations.test.ts (44/44) pass on it. I did not push it, because it would void the proof below.

Local gates on ea1758624

Gate	Result
bun run check:pr --base origin/main	passed (972/972)
bun scripts/check-packages.ts (pack smoke)	passed: every publishable package packed, installed and passed its probes
tsc --noEmit, bun run lint	clean
bun run test packages/ui-server packages/ui-sdk	5104 pass, 5 fail out of 5109. The 5 are in ui-sdk maintenance-temp-budget (3), ui-sdk mask-scratch (1) and ui-server http-supported-mounts (1). They ran while native proofs, dd writers and a mutation batch loaded the host. Rerun alone, those three files pass 135/135 twice on this head and twice on main d2bb3b301.
bun run test packages/ui-backend-claude	771/771, including provider-history.test.ts
bun run test ./tests	1953 tests across 177 files; 7 fail, in native-diagram-inputs, integration-command and unit-workers/changed-unit-workers. The same 7 fail on main d2bb3b301, and did on c696f995c, so this branch did not introduce them.
#2172 tests, packages/ui-server/tests/journal-accounting.test.ts	unchanged from origin/main; 10/10

Native hosted proof on ea1758624. prove-native-host.ts ran all four cells under the exclusive fixture lock. Its dd if=/dev/zero bs=1M count=1024 conv=fsync writers ran outside the lock.

Load	Exit	Max /api/health per cell (first / warm), ms
none	0	claude 1000: 161 / 46 · pi 1000: 56 / 35 · claude 0: 183 / 77 · pi 0: 53 / 120
3 fsync writers	0	claude 1000: 437 / 404 · pi 1000: 53 / 32 · claude 0: 166 / 259 · pi 0: 156 / 354

HOSTED_ADMISSION_HEALTH_WITHIN_1_SECOND passed in both, with the bound unchanged. The only error: line in either is the expected, verified automatic-index failure reproduction. The previous head, 633663dab, had passed the same proof both ways, with a highest probe of 786 ms under load. Earlier single-cell runs on the first merged head had timed out a probe in 2 of 6.

Lag attribution (the #2416 method). The native pi UID 1000 cell ran under three fsync writers, measured with the 20 ms timer-drift probe and bun --cpu-prof at 500 µs. Windows ≥ 250 ms after boot, categorised by sampled frames:

Run	Journal	Activity	Native lease	App-connection writers	Boot
before r1 (main d2bb3b301)	0	2 (max 436)	2 (max 339)	0	1
before r2 (main d2bb3b301)	0	0	0	1 (398)	0
after r1 (ea1758624)	0	0	0	1 (372)	0
after r2 (ea1758624)	0	0	0	3 (max 326)	0
before, on c696f995c (2 runs)	0–1	3–5	3–6	1–5	—
after, on 633663dab (2 runs)	0	0	0	2–6	—

The host was less loaded this time, so these before runs show fewer windows than the earlier pair, which is kept above for comparison. The after runs also record each app-connection write's wait for its turn and each worker call's duration. That diagnostic is not committed. Per the ruling on #2814, waits of this kind belong to #2850. Two of the four after windows are such waits:

• an auth-session write waited 384 ms for a worker bindModel, which is a journal commit;
• a turn-budget write waited 285 ms for a worker writeLifecycle.

Mutation receipts. M1–M20 were re-run on ea1758624, each against its named suite, then restored; each fails:

#	Mutation	Failing test
M1	worker opens below FULL, guard removed	"every write-worker commit runs at synchronous=FULL and a lowered commit is refused"
M2	no pre-commit FULL check	same, and "an attached database file is owned by the same write connection, at FULL"
M3	odd calls posted late	"writes reach the database in submission order across channels, and refusals keep their fields"
M4	file databases use the inline writer	"a file database commits on the write worker while the event loop keeps running"
M5	journal calls skip the recovery gate	"startup recovery commits before a write submitted during construction, and writes keep their order"
M6	journal refusals lose their class	"journal refusals keep their class across the worker and roll the whole write back"
M7	app connection keeps no journal guard	"the app connection refuses journal rows while the worker connection commits them"
M8	in-transaction Activity write goes to the worker	"a write inside the caller's transaction stays atomic with it"
M9	Activity clock defaults taken by the worker	"clock defaults are taken at submission, not when the worker reaches the write"
M10	worker never reloads pricing	"worker rollups price like the app's store and follow a republished table"
M11	retained stream ends before its release commits	"a retained raw stream pins its object until the reader ends"
M12	idle writer stays referenced	"an idle write worker does not keep its process alive"
M13	attached files skip the FULL check	"an attached database file is owned by the same write connection, at FULL"
M14	autonomous terminal before Activity writes settle	"killed inside staging compensation: the restart repeats removal and acknowledges once"
M15	retained copy returns before its lease is released	"a genuinely full brain volume preserves the source or retained Undo original"
M16	app transactions skip their turn	"an app-connection transaction that reads, then writes, waits for its turn instead of failing"
M17	every app transaction made immediate	"a read-only app snapshot keeps its deferred mode and never holds the write lock"
M18	autocommit statements skip their turn	"app writes, autocommit or not, never land while a worker procedure runs"
M19	bindModel refuses the active turn's binding	"runtime main identity preserves earlier unknown events through completion and resume"
M20	session slot ends before its recorder's writes commit	"a session's next turn waits until the previous turn's Activity writes commit" (ACTIVITY_SLOT_WAITS_FOR_RECORDER_WRITES)

Merge-specific mutations:

• Hosted session: dropping await recorder?.settled() fails "turn-end index outcomes commit on the write worker before the turn's terminal". Dropping the turn-end settleIndex() fails the same test on ACTIVITY_INDEX_OUTCOME_COMMITTED_OFF_LOOP.
• Autonomous turn: swapping the index barrier and recorder.settled(), or dropping settled(), fails "an autonomous turn's index outcome commits before the transaction that writes its terminal" on AUTONOMOUS_TERMINAL_WAITS_FOR_INDEX_OUTCOME. Dropping the barrier fails the same test because the flush never runs.

The #2843 test fixes, without await journal.ready:

• the replay test fails 3 of 3 runs on JOURNAL_STARTUP_REPLAYS_STEERING;
• the SIGKILL recovery cases fail 2–4 of 4 per run on STORAGE_REMOVAL_REAL_CHILD_BOUNDARY, and the child's stderr shows its journal_storage_unavailable refusal.

Checklist

• Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
• Acceptance criteria from the issue are all met, or the gap is named below.
• The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
• A changeset is included, or this changes nothing a consumer can see. (.changeset/sqlite-commits-off-event-loop.md)
• No personal data anywhere in the diff — the leakage gate covers the whole tree.
• The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. (docs/integration-contract/package-api.md, persistClaudeMemory)
• Docs under docs/ reflect the change, or nothing documented moved.
• Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

• Ruling on ui-server: journal and Activity SQLite commits block the hosted event loop on fsync #2814 (option 1). This PR closes ui-server: journal and Activity SQLite commits block the hosted event loop on fsync #2814 for the direct journal, Activity and native-lease commits. Neither run had a window in those classes. Under the ruling, a window where an app-connection write waits for a worker commit belongs to #2850, which moves the remaining app-connection writers onto the worker incrementally.
• Journal commits hold back ui.db writes. The ledger's BEGIN IMMEDIATE takes the write lock on every attached file, and the JS turn covers every worker procedure. So a journal commit to journal.db also holds back app writes to ui.db; after r1 measured one such wait at 384 ms. Taking only the journal file's lock would need the ledger's fresh-checkpoint check and its governance-event writes reworked. It is recorded on Epic: nothing on the hosted server's event loop waits for a ui.db fsync #2850.
• origin/main has moved 2 commits since the merge (see Proof). The trial merge is clean.

🤖 Generated with Claude Code