Skip to content

fix(ui-server): commit retries, policy proposals, drafts, transcription claims and settings on the write worker - #3023

Merged
schlessera merged 17 commits into
mainfrom
fix/2998-chat-writers-offloop-044
Oct 26, 2026
Merged

schlessera merged 17 commits into
mainfrom
fix/2998-chat-writers-offloop-044

Conversation

@schlessera

Copy link
Copy Markdown
Owner

Closes #2998
Refs #2850

The remaining foreground chat writers commit on the ui.db write worker at synchronous=FULL: retained retries, policy proposals, composer drafts, voice transcription claims and settings.

What changed

Each moved write is the statement or transaction that ran inline, run in a worker procedure. Without a writer (tests, the cron runner) the same code runs inline.

Retained retries (packages/ui-server/src/ws/session-catalog-writes.ts, packages/ui-server/src/ws/retry-requests.ts):

  • Save: still advertises Retry at once. It is stored after its pending session row.
  • Invalidation: takes effect when submitted. peekRetry and replayed history stop showing the retry immediately. If the invalidation's delete fails, the worker keeps refusing a reservation of that obsolete request, and the failure is reported.
  • Restart: that in-memory refusal does not survive a restart. So a reservation also refuses once the session's accepted work (session_work) names a later request than the retained turn. A test simulates a failed delete followed by a restart.
  • Reservation: keeps its receipt and its consumption of eligibility in one immediate transaction. Dispatch awaits it before sending the receipt or the prompt, and a duplicate never dispatches twice.

Policy proposals (packages/ui-server/src/brain/policy-proposal-writes.ts):

  • Insert: commits before approval is asked.
  • "Writing" review: commits before any policy bytes are written. A crash then reads back as "unknown" and never repeats the write.
  • Emission: reviews are emitted after their save commits, and history readers wait on settled().

Composer drafts (packages/ui-server/src/drafts/writes.ts):

  • Routes: save, upload, remove and bind answer the committed revision. DraftError keeps its class, status and body.
  • Acceptance: an accepted message's draft tombstone is submitted without blocking the turn. acceptDraft already ignored the outcome, and a failure is logged by identity only. All draft writes share one worker channel, so a later save follows the tombstone.
  • Reads: the list and get routes wait for every draft write submitted before them.

Voice transcription (packages/ui-server/src/voice/transcription-writes.ts):

  • Claim: the provider is dispatched only after the claim commits, so a duplicate never dispatches twice.
  • Complete and remove: commit on the worker too. Refusals keep their class and receipt.
  • Authorization: runs there as well, including the lazy account-partition seed insert.
  • Boot: the startup recovery sweep stays inline.

Settings (packages/ui-server/src/db/settings-writes.ts): one settings writer owns these, and each consumer awaits the commit before its dependent read or answer:

  • model preferences;
  • remembered tool grants;
  • digest visits, dismissals and generation;
  • subscription-proof retention (the hourly check, status reads and successful-turn recording).

Why this shape

The question in each case was what a caller relied on when the write committed inline.

  • Retries: must stay fail-closed. An invalidation that does not commit must never let an obsolete prompt run, which is why there is a durable check besides the in-memory one.
  • Draft acceptance: runs inside the bridge's synchronous session_info handler. Making it awaited would let later backend events overtake it. Its outcome was never used, so it is submitted, and the reads wait instead.

gpt-6-astra drafted both slices; I reviewed them and added fixes:

  • the restart-durable retry check;
  • tests that record a failure inside a WebSocket send or a route answer, instead of throwing it into dispatch's catch;
  • saved-transcription.test.ts now waits for the provider call instead of a fixed 20 ms.

Proof

Head b6a5a0ff5. The two slices (each drafted by gpt-6-astra, reviewed and fixed here; see "Why this shape"), one merged into the other, then merges of origin/main up to fd51cadf4 (#3015), resolving doc conflicts and the session-catalog-writes.ts overlap with #3011's local exchanges. The full suites and native proof ran on 57356800e, which contains main b99489b3c (#3014). The last merge brought in #3015 (schedules: app.ts wiring and this decision doc); on this head I reran check:pr, typecheck, lint, citations and the focused tests. Every heavy run went through the host slot wrapper.

Gate Result
bun run check:pr --base origin/main on b6a5a0ff5 (and on 57356800e) passed (local preflight)
bun run test packages/ui-server on 57356800e 4126 pass, 0 fail
bun run test ./tests on 57356800e 1965 pass, 1 fail. The failure is policy-write-boundary's Claude interactive case, AREAS_WRITABLE (worker scratch unwritable), already open as #3010, #2987, #2923 and #2988. Interleaved runs alone failed one case in 15 on main from a worktree beside this one, and 5 in 15 here, always in a scratch, shell-write or Git-lock case. #2999's branch hit it too, and none of these cases runs the writers this PR moves.
focused on b6a5a0ff5: retry-proposal-offloop, drafts-settings-offloop, drafts, transcription, settings, models, tool permissions, digest, subscription, ws-activity, the allowlist and approval channel, retry, policy proposals, app-writes-offloop, dispatch, local exchanges, classification, schedules 407/407
tsc --noEmit, bun run lint, decision-citations, api-surface on b6a5a0ff5 clean; 96/96
native hosted proof (prove-native-host.ts, all four cells, under flock on the fixture lock, dd … conv=fsync writers outside the lock) on 57356800e no load: passed; 3 fsync writers: passed (both with the deliberate retained-index-namespace failure verified)

Focused tests (retry-proposal-offloop.test.ts and drafts-settings-offloop.test.ts, both new). The app connection refuses writes to the watched tables, and the worker refuses any commit below FULL.

Mutations (each applied, its named test run, restored):

# Mutation Fails
R1 retry writes on the app connection APP_CONNECTION_WRITE retry_requests
R2 a pending clear does not hide the retry RETRY_PENDING_CLEAR_HIDDEN
R3 a failed clear does not keep the invalidation RETRY_FAILED_CLEAR_FAILS_CLOSED
R4 the receipt is sent before the reservation commits RETRY_PENDING_RESERVATION_NO_ACK
R5 no durable superseded check RETRY_RESTART_SUPERSEDED_FAILS_CLOSED
P1 the writing review is not awaited before publication POLICY_REVIEW_EMITTED_AFTER_COMMIT
P2 the insert is not awaited before approval POLICY_PENDING_INSERT_DUPLICATE_APPROVES_ONCE
P3 proposal writes on the app connection APP_CONNECTION_WRITE policy_proposals
D1 draft writes on the app connection APP_CONNECTION_WRITE session_draft_attachments
D2 a draft refusal loses its class DRAFT_CONFLICT_REVIVED_CLASS
D3 draft reads do not wait for submitted writes DRAFT_GET_WAITS_FOR_ACCEPT_COMMIT
T1 transcription writes on the app connection TRANSCRIPTION_ROUTE_ANSWERED_BEFORE_CLAIM: the route answers 500 because its inline write hit the app-connection refusal
T2 dispatch does not wait for the claim's commit TRANSCRIPTION_NO_DISPATCH_BEFORE_CLAIM_COMMIT
T3 a transcription refusal loses its class TRANSCRIPTION_DUPLICATE_REFUSED
G1 settings on the app connection APP_CONNECTION_WRITE settings

Several first passes failed for the wrong reason. These now fail on their named guards:

  • R4 tripped an expect thrown inside a WebSocket send, which dispatch's catch swallowed.
  • R3 surfaced only as a thrown reservation.
  • T1 hung until the test timed out.

Lag. The native pi UID 1000 cell under three fsync writers drives none of these writers. The before-runs recorded for #2996 and #2997 showed no app-connection turn over 20 ms from any of them, so the focused tests and mutations carry the evidence.

Checklist

  • Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
  • Acceptance criteria from the issue are all met, or the gap is named below.
  • The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
  • A changeset is included, or this changes nothing a consumer can see. (.changeset/retries-proposals-off-event-loop.md, .changeset/drafts-settings-off-event-loop.md)
  • No personal data anywhere in the diff — the leakage gate covers the whole tree.
  • The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved. (Nothing moved: api-report/ui-server.txt is unchanged.)
  • Docs under docs/ reflect the change, or nothing documented moved. (docs/decisions/sqlite-write-worker.md; shifted and re-pointed citations.)
  • Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

The remaining groups are the open sub-issues of epic #2850.

🤖 Generated with Claude Code

…t-writers-offloop-044

# Conflicts:
#	docs/decisions/sqlite-write-worker.md
…-offloop-044

# Conflicts:
#	docs/decisions/http-api-boundary.md
#	docs/decisions/session-principals.md
#	docs/decisions/sqlite-write-worker.md
#	docs/plans/async-collaboration-requirements.md
#	docs/plans/async-collaboration.md
#	docs/plans/navigation-recovery.md
#	packages/ui-server/src/ws/session-catalog-writes.ts
…-offloop-044

# Conflicts:
#	docs/decisions/sqlite-write-worker.md
#	docs/plans/async-collaboration-requirements.md
#	docs/plans/async-collaboration.md
…-offloop-044

# Conflicts:
#	docs/decisions/http-api-boundary.md
#	docs/decisions/session-principals.md
#	docs/decisions/sqlite-write-worker.md
#	docs/plans/async-collaboration-requirements.md
#	docs/plans/async-collaboration.md
#	docs/plans/navigation-recovery.md
…-offloop-044

# Conflicts:
#	docs/decisions/sqlite-write-worker.md
@schlessera schlessera added this to the 0.44.0 milestone Oct 26, 2026
# Conflicts:
#	docs/decisions/sqlite-write-worker.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ui-server: retained retries, drafts, policy proposals, transcription claims and settings commit on the write worker

1 participant