Skip to content

fix(ui-server): commit reference sources and managed worktrees on the write worker - #3026

Merged
schlessera merged 4 commits into
mainfrom
fix/2999-reference-writes-offloop-044
Oct 26, 2026
Merged

schlessera merged 4 commits into
mainfrom
fix/2999-reference-writes-offloop-044

Conversation

@schlessera

Copy link
Copy Markdown
Owner

Closes #2999
Refs #2850

Reference sources and managed worktrees commit their ui.db writes on the write worker at synchronous=FULL. That includes the 15-second checkpoint observation.

What changed

Every DB transaction in these services now runs verbatim in a worker procedure. They share one reference channel (packages/ui-server/src/brain/reference-writes.ts), split into three modules:

  • packages/ui-server/src/brain/reference-worktree-writes.ts
  • packages/ui-server/src/brain/reference-source-writes.ts
  • packages/ui-server/src/brain/reference-worktree-hygiene-writes.ts

Git and filesystem work stays on the app thread. No transaction was split. Without a registered writer (tests, the cron runner) the same procedures commit inline.

What moved:

  • Checkpoint observation. The journal's checkpoint ledger awaits the record commit before the pool creates the checkpoint, and the forget commit after release (referenceWorktreeJournalManager). Both are reached from attachWorktree and the 15 s scope tick.
  • Worktree leases and generations. Allocate, grant (access), renew, release, discard, open and close generation, expiry, reconcile, settle, removal receipts and reassessment.
  • Reference sources. Request with its lease, operation, create, acquire/publish, withdraw and revalidate writes; bind-time views; revocation. Boot-time recover stays inline.
  • Hygiene findings. Reconciliation, disposition and removal receipts.

What callers wait for:

  • Tools: each answers only once its lease, grant or release has committed.
  • Generations: a turn's workers bind a generation only after it has committed.
  • Turn lifetime: renew and release still read their turn's live() immediately before writing, inside the worktree's serial section. They await the worker commit inside that same section. A turn's end on that worktree (release, generation close, removal) runs through the same serial queue and the same worker FIFO, so a renewal whose live() passed is ordered before the end's effects, as it was inline. A renewal that finds its turn ended still changes nothing. removeWorktree's authorize() works the same way.

Why this shape

gpt-6-astra drafted it and stopped on the fence question above. Its proposal was worker-readable lifetime state. That isn't needed: holding the existing serial section through the commit keeps the same order. I made that ruling and added a test for it.

Proof

Head 153f9ccd0. The change (drafted by gpt-6-astra, reviewed here; see "Why this shape"), a test fix (renewal test fails fast when the turn lifetime is never consulted) and merges of origin/main up to 64af9b5a0 (#3023), with doc conflicts only, resolved. Every heavy run went through the host slot wrapper, on this head.

Gate Result
bun run check:pr --base origin/main passed (local preflight)
bun run test ./tests 1966 pass, 0 fail
bun run test packages/ui-server 4151 pass, 2 fail. Neither is in this change's path. close-cancels-shared-runs timed out waiting for its fixture's stats read to start (#2866, also seen on #3022). reference-transport's acquisition case was refused worker_state_busy by the host-wide maintenance temp budget, which another process held at the time (#2859). That file passes 3/3 alone on this head.
focused: reference-writes-offloop, every reference-*, *worktree*, turn-journal*, *hygiene*, schedules, app-writes-offloop 336/336
tsc --noEmit, bun run lint, decision-citations, api-surface clean; 96/96
native hosted proof (prove-native-host.ts, all four cells, under flock on the fixture lock, dd … conv=fsync writers outside the lock) no load: passed; 3 fsync writers: passed (both with the deliberate retained-index-namespace failure verified)

An earlier round on 505138907 hit the known policy-write-boundary Pi shell-write flake (#2988) in ./tests and the ws-turn-budgets window (#3009) in the ui-server suite. Neither recurred here.

Focused tests (packages/ui-server/tests/reference-writes-offloop.test.ts, new; a real file-backed ui.db and write worker). The app connection refuses writes to the reference tables, and the worker refuses any commit below FULL. The cases:

  • the checkpoint observation and release await FULL commits without blocking the loop;
  • source requests, allocation, grants, renewal, release and generation boundaries commit on the worker;
  • an admitted renewal commits before the ended turn's release and generation close, and a queued late renewal stays fenced;
  • hygiene reconciliation, disposition and removal receipts;
  • source refresh, failed-refresh bookkeeping, bind-time revalidation and superseded-root release;
  • a revocation sees a private publication submitted before it, and withdraws it after its commit;
  • the journal's attached worktree and scope observation share the journal worker's order;
  • failed allocation, expiry, reassessment and authorized removal keep their receipts.

Mutations (each applied, its named test run, restored):

# Mutation Fails
W1 reference writes on the app connection APP_CONNECTION_WRITE reference_views
W2 checkpoint record not awaited before the checkpoint exists REFERENCE_CHECKPOINT_LOOP_RUNS_WHILE_WORKER_HELD
W3 renewal not awaited inside its serial section REFERENCE_ADMITTED_RENEWAL_COMMITS
W4 renewal ignores its turn's end REFERENCE_RENEW_CONSULTS_TURN_LIFETIME
W5 generation open not awaited REFERENCE_GENERATION_NOT_BOUND_BEFORE_COMMIT

W4 first hung the test, which waited for a live() call that never came. The wait is now bounded, and the mutant fails on its own label.

Lag. None of the before/after attribution runs on this epic shows an app-connection turn over 20 ms from a reference module, in the native pi UID 1000 cell under three fsync writers. The cell's turns don't call the reference tools, and its scope ticks recorded no checkpoint-ledger turn of that size. So no window ≥ 250 ms was attributable to this group before or after, and the focused tests and mutations carry the evidence.

Checklist

  • Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
  • Acceptance criteria from the issue are all met, or the gap is named below.
  • The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
  • A changeset is included, or this changes nothing a consumer can see. (.changeset/reference-writes-off-event-loop.md)
  • No personal data anywhere in the diff — the leakage gate covers the whole tree.
  • The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. (Nothing moved: api-report/ui-server.txt is unchanged.)
  • Docs under docs/ reflect the change, or nothing documented moved. (docs/decisions/sqlite-write-worker.md, docs/decisions/managed-worktrees.md and shifted citations.)
  • Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

The remaining work is #3002 (account and security administration, another worker) and #3004 (the final inventory and guard) under epic #2850.

🤖 Generated with Claude Code

…ites-offloop-044

# Conflicts:
#	docs/decisions/sqlite-write-worker.md
…ites-offloop-044

# Conflicts:
#	docs/decisions/http-api-boundary.md
#	docs/decisions/session-principals.md
#	docs/decisions/sqlite-write-worker.md
#	docs/plans/async-collaboration-requirements.md
#	docs/plans/async-collaboration.md
#	docs/plans/navigation-recovery.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ui-server: reference sources and managed worktrees commit on the write worker

1 participant