Skip to content

fix(ui-server): record a Claude memory save in the journal's observation order - #2888

Merged
schlessera merged 5 commits into
mainfrom
fix/main-red-memory-observation-044
Oct 25, 2026
Merged

schlessera merged 5 commits into
mainfrom
fix/main-red-memory-observation-044

Conversation

@schlessera

@schlessera schlessera commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Closes #2893
Closes #2889

Fixes the main-red regression from #2830. The failing cell is turn-journal-runtime.test.ts "persisted external memory uses the existing reader and authenticated recovery".

What changed

This PR fixes two things.

  1. The test. The journal runtime fixture now awaits the memory copy-out before it ends the memory turn. Since CONTRACT: fix(ui-server): commit journal, Activity and native-lease SQLite off the event loop #2830, persistClaudeMemory may return a promise. The fixture still called memoryState.persist(...) as if it were synchronous and then called finish at once. In production, the Claude backend awaits copy-out before its turn ends: releaseTerminal/emitTerminal → finishWorker → worker.finish() → completions, which include state.persist (packages/ui-backend-claude/src/turn-runner.ts, worker-spawn.ts). The ui-react: the journal runtime test finishes its memory turn before the awaited memory record, failing two cells #2889 hypothesis is therefore correct about the trigger. This is a test-ordering bug, and production's Claude path does not strand a save at finish. Awaiting does not weaken the test. It now models the production order exactly.
  2. The journal. CONTRACT: fix(ui-server): commit journal, Activity and native-lease SQLite off the event loop #2830 introduced a product defect that production can reach. persistClaudeMemory and recordClaudeMemory now run in the journal's serialize order, beside observations and finish (packages/ui-server/src/brain/turn-journal.ts:514-544, plus the internal recordClaudeMemory).

The product defect: memory records reuse an observed sequence

observe() reads turn.sequence before its recordObservation commit and assigns the result after it. Before #2830 those steps were synchronous. Now a commit sits between them. A recordClaudeMemory that lands in that window takes ++turn.sequence from the stale base. It reuses an observed change's sequence and the ${turnId}:${sequence} steering_change dedupe key, so the journal UI and version(…, sequence, side) can resolve the wrong change.

Production reaches this window. A Claude memory save is recorded at worker exit, while the 15 s scope timer or another active turn's toolEnded observation can be committing. Revert recovery is a second caller (turn-revert.ts, const target).

Journal-API hardening (same change, not reachable through the Claude backend)

The same ordering also makes the journal API safe for a caller that does not await:

  • A save already in flight when finish runs is recorded before the turn ends.
  • A save requested after finish is refused before save() writes the external note. Before, it passed the up-front active check, wrote the note, and was refused only at the record step.

The refusal for an inactive turn and its message are unchanged.

Why this shape

  • Test-only fix. Awaiting in the test alone makes the main-red pass (receipt below), but it leaves the sequence-reuse defect.
  • Letting recordClaudeMemory accept inactive turns. This would hide genuinely late observations, so I did not do it.
  • Why serialize. It is the order that begin, finish and observations already share. Memory persistence never waits on a held operation: its writes go through call/retain, not tail. So it cannot deadlock. The public recordClaudeMemory serializes the internal function. Persistence, which is already serialized, calls the internal function directly.
  • Other CONTRACT: fix(ui-server): commit journal, Activity and native-lease SQLite off the event loop #2830 await sites checked in turn-journal.ts. bindSession, bindModel, toolStarted/toolEnded, coverPaths, finishReserved, retainBefore, observeUserPolicy, attachWorktree and observe are either synchronous before their first await or already ordered. The Activity recorder has its own orderedWrites order.
  • Docs. The ordering is recorded in docs/decisions/sqlite-write-worker.md. Citations moved by the edit are refreshed. The public signatures are unchanged: the API report is identical, and no CONTRACT: change was needed.

Proof

Started from origin/main d9f4405. Merged origin/main 5322d30 (citation-only conflicts, re-pointed). Final head a10edb3.

Failing-first. The three new tests in packages/ui-server/tests/claude-memory-journal.test.ts were run on unfixed origin/main, and again on b192ca0c0 itself in a temporary detached worktree. All three fail for the claimed reason in both runs:

  • a memory record made while an observation of its turn commits takes the next free sequence fails with MEMORY_RECORD_NEVER_REUSES_AN_OBSERVED_SEQUENCE. It received [["claude-memory://counsel.md",1],["harbour.md",1]]. This is the product defect.
  • a memory save in flight when its turn finishes is recorded before the turn ends fails with … memory observation requires an active turn at recordClaudeMemory ← persistClaudeMemory. This is the main-red stack.
  • a memory save requested after its turn's finish is refused before it saves fails with Received message: "… memory observation requires an active turn". The late save had passed the guard and written the note.

The runtime cell. Before either change, -t 'persisted external memory' failed 2/2 locally. With only the test await, on the unfixed origin/main journal, it passes 2/2. With only the journal fix, on the unawaited test, it passes 2/2. With both, the full file passes 24/24.

Mutations (restored after each run):

  • M1: persistClaudeMemory unserialized, as (async …)(). The in-flight test fails with memory observation requires an active turn, and the late-save test fails with toThrow(expected).
  • M2: the public recordClaudeMemory not serialized. The sequence test fails with MEMORY_RECORD_NEVER_REUSES_AN_OBSERVED_SEQUENCE.

Final head a10edb3 (after merging main):

  • bun run test packages/ui-server: 3955 tests in 331 files, exit 0.
  • bun run test ./tests: 1954 tests in 177 files, exit 0.
  • turn-journal-runtime.test.ts: 24/24.
  • bun run check:pr --base origin/main: exit 0.
  • Draft CI: CI and contract both success.
  • tsc --noEmit, bun run lint, and the citation, api-surface and memory tests (105/105) are clean.

Before the merge:

  • bun run test packages/ui-server: 3939 pass, 0 fail (on ebe204d; later commits change only the inline type, docs, the changeset and the ui-react test).
  • claude-memory-journal.test.ts: 9/9. tests/api-surface.test.ts plus memory tests: 61/61. decision-citations: 44/44.
  • ui-backend-claude memory tests (provider-denied-memory, worker-save-refused) plus the ui-react runtime file: 33/33.
  • bun run test ./tests on c8e7c17: exit 0, 1954 tests in 177 files. Penelope/Odysseus … admission appear only as output from an inner planned-failure fixture, and they appear the same way on main.
  • tsc --noEmit and bun run lint are clean. bun run check:pr --base origin/main: exit 0 on c8e7c17 and on the final head 8de2ab3.

Checklist

  • Focused local preflight/behavioral receipts and complete selected hosted proof passed; actual head/base/job checkout evidence is in Proof.
  • Acceptance criteria from the issue are all met, or the gap is named below.
  • The PR and completed/advanced issues have verified release milestones under docs/process/github.md.
  • A changeset is included, or this changes nothing a consumer can see.
  • No personal data anywhere in the diff — the leakage gate covers the whole tree.
  • The integration contract index or relevant component is updated in this same commit, or nothing in the machine surface moved.
  • Docs under docs/ reflect the change, or nothing documented moved.
  • Provider history stays append-only (docs/decisions/provider-history.md): the provider-history guard passes, or nothing on a turn's path changed.

Anything left open

🤖 Generated with Claude Code

…ion order

Since journal commits moved to the write worker (#2814, #2830),
persistClaudeMemory awaits several commits before it records the saved
note, and nothing ordered it against the turn's finish or a committing
observation:

- finish could remove the turn from `active` while a save was still
  committing, so recordClaudeMemory refused the already-saved note with
  "memory observation requires an active turn" and the turn ended
  partial with the note unconfirmed;
- a save requested after finish passed the up-front `active` check,
  wrote the external note, and only then was refused;
- an observation reads a turn's sequence before its commit and stores
  the result after, so a memory record in between reused an observed
  sequence (and its steering_change dedupe key).

persistClaudeMemory and recordClaudeMemory now run through the journal's
serialize order beside observations and finish. A save requested before
finish is recorded before the turn ends; one requested after is refused
before it saves; a record takes the next free sequence. The refusal for
an inactive turn is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@schlessera schlessera added this to the 0.44.0 milestone Oct 25, 2026
… changeset

The memory target type alias leaked into the ui-server API report;
spell the target inline as before. Citations follow the moved lines.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… turn

The Claude backend awaits copy-out before its turn ends (releaseTerminal ->
finishWorker -> worker.finish), and persistClaudeMemory may return a
promise since #2830. The journal runtime fixture still called persist as
if it were synchronous and ended the turn at once (#2889).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@schlessera schlessera added type: fix Something is broken and must behave the way it is documented. area: ui-server Hono app factory: turn coordinator, auth, session catalog, routes, cron. area: ui-react React chat/files/voice components, stores, WebSocket transport. labels Oct 25, 2026
schlessera and others added 2 commits October 25, 2026 07:04
…started no runs)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…bservation-044

# Conflicts:
#	docs/decisions/governed-writable-workers.md
#	docs/decisions/managed-worktrees.md
#	docs/plans/storage-lifecycle.md
@schlessera
schlessera marked this pull request as ready for review October 25, 2026 07:20
@schlessera
schlessera merged commit 6814f2c into main Oct 25, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ui-react React chat/files/voice components, stores, WebSocket transport. area: ui-server Hono app factory: turn coordinator, auth, session catalog, routes, cron. type: fix Something is broken and must behave the way it is documented.

Projects

None yet

1 participant