Skip to content

fix(ui-server): commit governance events and turn scope on the write worker - #2884

Merged
schlessera merged 4 commits into
mainfrom
fix/2850-governance-scope-offloop-044
Oct 25, 2026
Merged

schlessera merged 4 commits into
mainfrom
fix/2850-governance-scope-offloop-044

Conversation

@schlessera

@schlessera schlessera commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Refs #2850

First slice of #2850 (option 3 from #2814): the governance-event and turn-scope writers.

What changed

The hosted server no longer commits a governance event or a turn-scope row on its app connection. Every governance event raised outside a journal transaction (the Activity runtime's events, egress records, scope caps, bridge and catalog events, policy changes, revert conflicts, scratch overage), the session's first taint, roadblock classification, the journal's startup steering replay and every turn_scope insert, save, end and restart sweep now commit on the ui.db write worker from #2830, at synchronous=FULL, in submission order with the journal's and Activity's writes. Each event is validated, and its clock taken, where it is raised, so an invalid event still throws at its call site. The transaction bodies are the ones that ran inline (insertGovernanceEvent, insertFirstTaint, recordTurnRoadblocks, the scope statements); events the journal records inside its own transactions stay there.

Four reads used to follow those writes on the same connection, so each now waits for what it read:

  • Taint (G6). The egress proxy awaits a promise returned from admitted, so a connection's first upstream byte still waits for its taint's commit. Admission counts a taint from the moment it is submitted (GovernanceWrites.taint), so a second connection during that commit is narrowed as before, and the command policy's live taint read uses the same view.
  • Scope decisions. TurnScope.decide is now async. It waits until the turn's earlier scope writes have committed, decides against the committed row with nothing awaited between that check and the decision, and settles after its own writes, so the summary dispatch.ts sends next reads them.
  • Scope broadcasts. changed() runs once the saved row commits, because the journal summary it is broadcast with reads that row.
  • Egress records. A turn's outcome, and the session's next turn, wait for prepareTurnEgress(...).settled(), so a failed record still fails the turn and contacted hosts are visible to the next turn.

A process without a write worker (the cron runner, tests on :memory:) runs the same procedures on its own connection behind the same promises (procedureCaller).

Why this shape

It follows #2830's model: writers move as procedure modules (governance/event-writes.ts, brain/turn-scope-writes.ts), not as replayed SQL, so reads inside each transaction still happen against the state it commits. The alternative for scope decisions was reading the live in-memory summary instead of the row; turn-scope-headroom.test.ts ("refuses a tampered ceiling") showed the row read is deliberate, so the decision waits for the commit instead. Governance and scope ship together because scope caps record scope_cap events and restart roadblocks, and the ordering test covers both.

The new #2850 finding (an auth-session write in principals.ts waiting 384 ms behind a worker bindModel) is in scope as its own group: once that writer is on the worker queue, it waits there instead of on the loop, without reworking the journal ledger's locks. The decision record now says so.

Proof

Refresh onto main. Merged origin/main three times while proof ran (no force-push): e03c3c68c, 5eb26d1dc and then 6e3d41723 (onto 6814f2ce5). Conflicts were doc citations, plus imports in inbox/autonomous-turn.ts (this PR's governance writes beside main's new providerDeniedTypes). Citations were re-pointed after each merge. On the final head, tsc --noEmit, bun run lint and decision-citations 44/44 pass. Per the stack rule, the full re-proof ran on #2896's top, which contains this branch: see #2896.

Head 365e82960, one commit on b192ca0c0 (#2830 merged). Receipts are local; draft CI covers the rest.

Local gates

Gate Result
bun run check:pr --base origin/main passed (948/948)
bun scripts/check-packages.ts (pack smoke; new worker-loaded modules resolve from dist/) passed
tsc --noEmit, bun run lint, tests/decision-citations.test.ts clean; 44/44
bun run test packages/ui-server packages/ui-sdk 5119 pass, 2 fail of 5121. The 2 are http-supported-mounts ("a dirty failed mount discards its resources…") and governance-policy ("a pre-existing unpinned peer…"), the load-sensitive pair #2830 also recorded. Rerun alone: 131/131 on this head and 131/131 on origin/main.
bun run test packages/ui-backend-claude packages/ui-backend-pi 1254/1254, including provider-history.test.ts
bun run test ./tests 1944 pass, 9 fail, 1 error of 1953. integration-command (1 + the posix_spawn 'git' error) and native-diagram-inputs (4) fail identically on origin/main. The 4 policy-write-boundary cases pass when rerun; that file fails 3 other cases on both origin/main and this head in the same rerun, so it is environmental here.

Focused tests (packages/ui-server/tests/app-writes-offloop.test.ts, new; turn-egress.test.ts, two new cases). Temporary triggers make the app connection refuse every write to the moved tables (APP_CONNECTION_WRITE), and a probe on the worker connection refuses any write below FULL and logs commit order. They cover: governance events, first taint, binding, roadblocks and the Activity runtime's events commit on the worker; a pending taint counts for admission; scope rows and their cap event commit in submission order after an Activity write; changed() fires after the commit; a connection's first byte waits behind a held worker commit for its taint while a second host is narrowed; settled() surfaces a late record failure; the recorder's settled() waits for its roadblock commit.

Mutations (each applied on this head, its named tests run, restored; logs kept locally)

# Mutation Fails
M1 governance writes run on the app connection "governance events commit on the write worker…" on APP_CONNECTION_WRITE governance_events
M2 governance procedure commits below FULL same test, SQLITE_WRITE_BELOW_FULL
M3 scope rows written on the app connection "turn scope rows and their cap events…" on APP_CONNECTION_WRITE turn_scope
M4 scope procedure commits below FULL same test (SCOPE_CAP_EVENT_BEFORE_ITS_SUMMARY: the refused save is missing from the commit log)
M5 scope summary announced before its commit SCOPE_CHANGED_AFTER_COMMIT
M6 scope cap event submitted after its summary SCOPE_CAP_EVENT_BEFORE_ITS_SUMMARY
M7 a pending taint does not count for admission GOVERNANCE_PENDING_TAINT_IS_IN_FORCE; the egress case fails too
M8 egress proxy does not await the admission record TURN_EGRESS_BYTES_AFTER_TAINT_COMMIT
M9 settled() does not wait for egress records TURN_EGRESS_LATE_RECORD_FAILURE_IS_FINAL_AFTER_SETTLED, TURN_EGRESS_RECORDS_COMMITTED_BY_SETTLED
M10 a scope decision does not wait for earlier scope writes ws-scope-caps.test.ts, both adapters: SCOPE_ADAPTER_FIRST_GRANT_AND_DUPLICATE_ACCEPTED
M11 recorder settled() does not wait for its roadblocks RECORDER_SETTLED_WAITS_FOR_ROADBLOCKS

Test changes that follow the async writes. ws-main-model.test.ts waits for the governance row to commit before reading it; the row's model is still taken when the event is raised, so MODEL_LIVE_EARLY_UNKNOWN asserts the same thing. ws-scope-caps.test.ts waits for the pause summary that counts every call seen so far: each summary now reaches the client once its row commits, so the backlogged one arrives after the first rather than in the same tick; the assertions are unchanged. interactive-share.test.ts moves its bridge-promise assertions from .resolves/.rejects to plain await and expectRejection. A pending worker write inside Bun 1.4.2's nested expect loop never settled (the trap in docs/process/testing.md). The turn-scope* tests await the now-async decide. command-taint.test.ts is unchanged. Two ui-sdk tests that copy scratch-prune.ts rewrite its new import path.

Native hosted proof (prove-native-host.ts, all four cells, under flock on the fixture lock, dd … conv=fsync writers outside the lock), on a tree identical to this head:

Load Result
none passed (all four cells; the deliberate retained-index-namespace failure verified)
3 fsync writers, run 1 claude UID 0 failed schedule Git publication inherits active native peer folder admission: verifyWorkerFolderMountsSteps read "invalid worker mount record" (ui-sdk launcher, not changed here)
3 fsync writers, run 2 passed, all four cells
origin/main, 3 fsync writers failed claude UID 1000 on HOSTED_ADMISSION_HEALTH_WITHIN_1_SECOND

Lag attribution (the #2416 method with the turn wait joined, as on #2814: 20 ms timer-drift probe, bun --cpu-prof at 500 µs, every app-connection turn > 20 ms logged with its call site and every worker call > 20 ms; diagnostic not committed). Native pi UID 1000 cell, 3 fsync writers.

  • Before (origin/main b192ca0c0, 2 runs, plus 3 runs on CONTRACT: fix(ui-server): commit journal, Activity and native-lease SQLite off the event loop #2830's head 633663dab). Windows ≥ 250 ms after ready attributed to this PR's writers: the Activity runtime's governance event (451 ms, own commit), turn-scope save (404 ms, own commit) and the first taint (317 ms, waiting 331 ms behind a worker appendEvent). Over the two origin/main runs, 26 app-connection turns > 20 ms were these writers (turn-scope.ts ×22, session-taint.ts ×2, activity/runtime.ts ×4).
  • After (this head, 3 runs): no app-connection turn from governance/events.ts, session-taint.ts, activity/runtime.ts or turn-scope.ts at all. The windows ≥ 250 ms left after ready are session-work.ts via the session catalog (299 ms) and the session catalog's own commit (254 ms), which are later groups of this issue.

Changes that are not writes

TurnScope.decide and begin are async, prepareTurnEgress gained settled(), and the egress proxy's admitted hook may return a promise. All of these are internal, so docs/integration-contract/ is unchanged.

Checklist

  • Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
  • Acceptance criteria from the issue are all met, or the gap is named below. (This slice meets them for the writers it moves; the remaining groups are listed below.)
  • The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
  • A changeset is included, or this changes nothing a consumer can see. (.changeset/governance-scope-off-event-loop.md)
  • No personal data anywhere in the diff — the leakage gate covers the whole tree.
  • The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. (Nothing moved: every changed signature is @internal or under /internal.)
  • Docs under docs/ reflect the change, or nothing documented moved. (docs/decisions/sqlite-write-worker.md, taint citations, shifted code citations.)
  • Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

Still on the app connection, in the order the attribution suggests: turn budgets (observe at run-session.ts:676, up to 366 ms before), session catalog with message sources and session work, the writer gate, auth sessions (principals.ts, the #2850 finding), the Activity notifier and push ticks, and the rest the inventory found (storage/history-store.ts, governed-manifest.ts, session-profiles.ts, schedules, inbox, settings). Each is a following PR on this issue.

🤖 Generated with Claude Code

…worker

Governance events raised outside a journal transaction, the session's
first taint, roadblock classification, the journal's steering replay and
every turn_scope row now commit on the ui.db write worker at
synchronous=FULL, in submission order with journal and Activity writes,
instead of on the event loop (#2850). Events are validated and their
clock taken where they are raised.

Reads that followed those writes wait for them: a connection's first byte
waits for its taint's commit while admission counts a pending taint; a
scope decision runs against the committed row and settles after its own
writes; a turn's outcome waits for its egress records; the scope summary
is broadcast once its row commits.

Refs #2850

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…cope-offloop-044

# Conflicts:
#	docs/decisions/governed-writable-workers.md
#	docs/decisions/session-principals.md
#	docs/decisions/voice-permission.md
#	docs/plans/async-collaboration-requirements.md
#	docs/plans/async-collaboration.md
#	docs/plans/navigation-recovery.md
…cope-offloop-044

# Conflicts:
#	docs/decisions/governed-writable-workers.md
#	docs/decisions/managed-worktrees.md
#	docs/decisions/scheduled-tasks.md
#	docs/decisions/session-principals.md
#	docs/decisions/voice-permission.md
#	docs/plans/async-collaboration-requirements.md
#	docs/plans/async-collaboration.md
#	docs/plans/navigation-recovery.md
#	packages/ui-server/src/inbox/autonomous-turn.ts
…cope-offloop-044

# Conflicts:
#	docs/decisions/governed-writable-workers.md
@schlessera
schlessera marked this pull request as ready for review October 25, 2026 08:09
@schlessera
schlessera merged commit 8351c98 into main Oct 25, 2026
13 of 14 checks passed
schlessera added a commit that referenced this pull request Oct 25, 2026
Layer 2 of #2850, re-parented onto main after #2884's squash merge.
The tree is byte-identical to the proven head 50354cf.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
schlessera added a commit that referenced this pull request Oct 25, 2026
Layer 2 of #2850, re-parented onto main after #2884's squash merge.
The tree is byte-identical to the proven head 50354cf.

Co-authored-by: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant