Skip to content

fix(ui-server): move storage scans to the write worker and guard against app-connection writes - #3029

Merged
schlessera merged 10 commits into
mainfrom
fix/3004-app-write-guard-044
Oct 26, 2026
Merged

schlessera merged 10 commits into
mainfrom
fix/3004-app-write-guard-044

Conversation

@schlessera

Copy link
Copy Markdown
Owner

Closes #3004
Closes #3028
Refs #2850

This is the final slice of epic #2850. It moves the last hosted-path ui.db writers to the write worker (#3028), proves by inventory that none remain on the app connection, and adds a guard that fails the hosted test suites if one returns.

What changed

The last writers (#3028). The inventory found one group no sub-issue owned: shared storage scan leases and storage maintenance operations. #2997 had left them out. Their transactions now run verbatim on the worker (packages/ui-server/src/storage/shared-scan-writes.ts, packages/ui-server/src/storage/maintenance-writes.ts):

  • a scan's collector starts only after its lease commits, and its pid bookkeeping settles before the collector wrapper returns;
  • a finish commits before its result is served;
  • a maintenance route answers once its receipt is durable;
  • a cancellation is durable before it aborts its owner;
  • staging cleanup awaits each item's intent;
  • refusals keep their class and status.

gpt-6-astra drafted this part; I reviewed it and added the mutation receipts.

Observing the app connection (observeAppWrites, markAppPhase, packages/ui-server/src/db/write-worker.ts). takeTurns already wraps every outermost app-connection transaction and autocommit write. When an observer is registered, it compares SQLite's total_changes() before and after each one, and reports any turn that changed rows, with its stack and the app's phase. The phase is boot while createApp builds the app, serving once it returns, and standalone for a component built outside it. With no observer registered, nothing is measured. The API is not exported from any package entry, so the API report is unchanged.

The inventory (scripts/app-write-audit-preload.ts, scripts/app-write-inventory.ts). The preload records every observed write while a suite runs, and the script groups them by the nearest server frame. A stack that holds a test frame counts as a fixture: a test's own write, or a product call a test made directly on the app's database to seed it. The method, the result and the named exceptions are in docs/decisions/sqlite-write-worker.md ("The inventory and its guard").

The guard (guardAppWrites, packages/ui-server/tests/helpers/app-write-guard.ts). httpContractApp registers it on every hosted test app once the app serves. It records each app-connection write reached from server code alone, and the helper's close() fails the test with APP_CONNECTION_WRITE_WHILE_SERVING, naming the site. The 28 route and mount suites built on that helper are now the guard.

Inventory result

Run over the ui-server suite and ./tests on a1dcc134e (2,452 observed writes; full table under Proof). The serving phase holds only three kinds of site, none of them a hosted path:

  • Inbox dispatch fixture. The triage, dispatcher and autonomous-turn path runs only through INBOX_DISPATCH_FIXTURE. createApp supplies no production dispatcher (packages/ui-server/src/inbox/runtime.ts:29).
  • Test-built budgets. ws-turn-budgets.test.ts:77 builds budgets on the app's database without its writer.
  • The guard test's own probe.

Boot holds only the schedule root's first insert (rootInfo). Standalone components (the writer gate, turn scope and reverts built by tests outside createApp) take the app's writer when hosted.

Named exceptions:

  • Boot writes: migrations, recoveries, rootInfo.
  • Operational restore: writes inside its caller's transaction.
  • The cron runner: writes on its own connection.
  • The inbox dispatch fixture.
  • Components a test builds without the app's writer.

Proof

Head 791624100. It contains main 2f10e0407 (#3024): the #3028 slice, merged into this branch, then the observer, guard, inventory and docs. Every heavy run went through the host slot wrapper. The suites, native proof and attribution ran on a1dcc134e, which is this head minus the docs commit that records the inventory. check:pr ran again on this head.

Gate Result
bun run check:pr --base origin/main on 791624100 (and on a1dcc134e) passed (local preflight)
bun run test packages/ui-server, with the guard active in every httpContractApp suite and the inventory preload 4187 pass, 0 fail
bun run test ./tests, with the inventory preload 1966 pass, 0 fail
tsc --noEmit, bun run lint, decision-citations, api-surface clean; 96/96
native hosted proof (prove-native-host.ts, all four cells, under flock on the fixture lock, dd … conv=fsync writers outside the lock) no load: passed; 3 fsync writers: passed (both with the deliberate retained-index-namespace failure verified)
#3028 alone on 7e3ab6a1b: native proof, no load and 3 fsync writers passed

Failing first. The guard first ran over the ui-server suite on main 2f10e0407, before #3028 was merged into this branch. 110 hosted-suite tests failed with APP_CONNECTION_WRITE_WHILE_SERVING, naming storage/shared-scan.ts (register, unregister, reap), storage/coordinator.ts (start, finish) and storage/maintenance.ts (write, start). That run also showed two classification problems, both fixed before this head:

  • product calls a test made directly on app.db (roadblock seeding, rollupRun, revokePrincipal) had been counted as hosted;
  • shallow stacks hid the test frame.

Mutations (each applied, its named test run, restored):

# Mutation Fails
G1 app writes are never reported APP_WRITE_GUARD_REFUSES_SERVING_WRITE
G2 the guard ignores serving writes APP_WRITE_GUARD_REFUSES_SERVING_WRITE
G3 the hosted helper does not assert the guard APP_WRITE_GUARD_REFUSES_SERVING_WRITE
G4 the app never marks itself serving APP_WRITE_GUARD_REFUSES_SERVING_WRITE
G5 a fixture's own write counts as hosted APP_WRITE_GUARD_FIXTURE_AND_BOOT_WRITES_ALLOWED
K1 scan leases on the app connection APP_CONNECTION_WRITE storage_scan_leases
K2 the collector starts before its lease commits STORAGE_SCAN_START_WAITS_FOR_LEASE
K4 maintenance on the app connection STORAGE_MAINTENANCE_ROUTE_WAITS_FOR_RECEIPT: the inline insert is refused on the app connection, so no receipt is answered

K3 (collector registration not awaited) survived as an equivalent mutant. The collector process runs from spawn either way, and the worker FIFO keeps the awaited unregister behind register, so no observable state differs.

Native lag, after every slice (#2416 method with the turn wait joined; diagnostic not committed). Native pi UID 1000 cell, 3 dd … conv=fsync writers, 3 runs on a1dcc134e:

Checklist

  • Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
  • Acceptance criteria from the issue are all met, or the gap is named below.
  • The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
  • A changeset is included, or this changes nothing a consumer can see. (.changeset/storage-scans-off-event-loop.md for ui-server: storage scan leases and maintenance operations commit on the write worker #3028; the observer, guard and scripts change nothing a consumer sees.)
  • No personal data anywhere in the diff — the leakage gate covers the whole tree.
  • The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. (Nothing moved: api-report/ui-server.txt is unchanged.)
  • Docs under docs/ reflect the change, or nothing documented moved. (docs/decisions/sqlite-write-worker.md; shifted citations.)
  • Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

Nothing for #3004 or #3028. Epic #2850's definition of done is for the coordinator to check after this merges. All of the epic's writer sub-issues are closed or close here. The inventory and the native receipts are recorded above and in the decision doc, and that doc no longer lists any "other app-connection writers".

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant