diff --git a/CODE_MAP.md b/CODE_MAP.md index 51e768e..774562a 100644 --- a/CODE_MAP.md +++ b/CODE_MAP.md @@ -12,7 +12,7 @@ has to invent a home (the historical failure mode). ``` openinfo/ ├─ shared/contracts/ P0 the only shared dependency; typed seam -│ ├─ src/records/ P0 commitment, moment, entity (**#73** contract v2: append-only `confidence`/`state`/`ambiguity`/`heardAs[]`/`sightings[]`/`overrides[]`/`external` — evidence trail + sovereign user overrides for the resolver #72; all optional ⇒ v1 rows still validate; sub-schemas Sighting/HeardAs/EntityOverride/EntityAmbiguity/EntityExternal registered in AllSchemas; **#72** adds `EntityResolution` sub-schema + optional `resolutions[]` — the scored resolver's append-only per-mention trail; **#116** additive `spanId` correlation ids on Distillate/Moment/EntityProvenance/OcrResult/FieldValue/JudgeReview/GuardHold + parent links `FieldValueProvenance.sourceChunks`/`GuardHold.sourceChunks` + optional `usage` on FieldValueProvenance/JudgeReview — every record of one pipeline pass shares a spanId and every derived record names its material; all optional ⇒ prior rows still validate), **sttSegment (#116: per-transcribed-segment STT provenance — chunkId/spanId/source/capturedAt→processedAt/textChars + endpoint/model/durationMs/egress, deliberately NO transcript text; the trace root that closes the disclosed #65 gap)**, **contextPacket (#176 slice 1: the converged ContextPacket — a REFS-ONLY derived unit over immutable observations: separate mic/systemAudio/screen ref arrays [source identity never merges], optional focus/app evidence from the Session record, entity candidates with #74 screen corroboration, machine-readable `gaps` for missing senses, deterministic confidence, append-only `revision`/`supersedes` chain; sub-schemas ContextPacketRef/Gap/Candidate/Provenance in AllSchemas; deliberately NO prose field — prose stays a read-time view over the refs; + `BuildContextPacketsRequest` in api/payloads)**, **sessionTitling (#211: the APPEND-ONLY episode-title record — one immutable row per naming, `source:'derived'|'floor'|'user'` [**#226** adds the `floor` rung], derived rows carry `TitlingProvenance` [which orientation pass `annotationId`/`templateId`/`endpoint` + the `nature`/`direction`/`topics` EVIDENCE the name drew from], **#226 floor rows carry `FloorTitlingProvenance`** [the deterministic `producer`+`subjects`+`derivedAt` — a zero-model rung that NEVER impersonates the judge; `provenance` is now a `TitlingProvenance|FloorTitlingProvenance` union, both registered in AllSchemas]; store `session_titlings` table + `WorkspaceRegistry.resolveTitle` = latest user > latest derived > **latest floor (#226)** > undefined [user is SOVEREIGN; a judge-derived name always supersedes a floor, by SOURCE not append order] + `latestFloorTitle`/`hasAuthoredTitle` helpers (#226); `recordSessionTitling` appends the row AND materialises the resolved name onto `Session.title` — titlings are truth, `Session.title` is the cache read by every surface; + `SetSessionTitleRequest` for PUT /sessions/:id/title in api/payloads)**, **summary (#177 slice 1: the multi-timescale Summary — the `level` enum COMPLETE now [rolling/episode/five-minute/session/project] though slice 1 PRODUCES only rolling/five-minute/session; REFS-ONLY `children` [role child|evidence, level-tagged] to lower summaries/packets/distillates/moments/observations; explicit `SummaryInputBound` [the bound made inspectable]; optional `text` = a MODEL PROPOSAL [`proposal:true`, #189] that is ABSENT when `degraded` names the honest unavailable reason [no fabricated prose]; deterministic confidence; append-only `revision`/`supersedes`; sub-schemas SummaryLevel/Child/InputBound/Provenance in AllSchemas; `PromptTemplate` gains kind `summary` + a `SummaryBinding` [level/windowMs/childLevel/maxChildren/maxEvidence/cadenceMs — cadence+bound+prompt as CONFIG]; + `BuildSummariesRequest` in api/payloads; **#177 slice 2:** `SummaryBinding` gains optional `scope`(workspace|workflow|app)/`targetId` [per-workflow/app cadence+template scoping] + new `SummaryScope` sub-schema in AllSchemas; `SummaryProvenance` gains optional `templateScope` [the which-scope-won audit]; `Summary.sessionId` now genuinely absent for the cross-session `project` level; `BlockTypeName`/`BlockQuery.source` gain `summaries` — all append-only/optional ⇒ prior rows still validate)**, session, workspace, pin +│ ├─ src/records/ P0 commitment, moment, entity (**#73** contract v2: append-only `confidence`/`state`/`ambiguity`/`heardAs[]`/`sightings[]`/`overrides[]`/`external` — evidence trail + sovereign user overrides for the resolver #72; all optional ⇒ v1 rows still validate; sub-schemas Sighting/HeardAs/EntityOverride/EntityAmbiguity/EntityExternal registered in AllSchemas; **#72** adds `EntityResolution` sub-schema + optional `resolutions[]` — the scored resolver's append-only per-mention trail; **#116** additive `spanId` correlation ids on Distillate/Moment/EntityProvenance/OcrResult/FieldValue/JudgeReview/GuardHold + parent links `FieldValueProvenance.sourceChunks`/`GuardHold.sourceChunks` + optional `usage` on FieldValueProvenance/JudgeReview — every record of one pipeline pass shares a spanId and every derived record names its material; all optional ⇒ prior rows still validate), **sttSegment (#116: per-transcribed-segment STT provenance — chunkId/spanId/source/capturedAt→processedAt/textChars + endpoint/model/durationMs/egress, deliberately NO transcript text; the trace root that closes the disclosed #65 gap)**, **contextPacket (#176 slice 1: the converged ContextPacket — a REFS-ONLY derived unit over immutable observations: separate mic/systemAudio/screen ref arrays [source identity never merges], optional focus/app evidence from the Session record, entity candidates with #74 screen corroboration, machine-readable `gaps` for missing senses, deterministic confidence, append-only `revision`/`supersedes` chain; sub-schemas ContextPacketRef/Gap/Candidate/Provenance in AllSchemas; deliberately NO prose field — prose stays a read-time view over the refs; + `BuildContextPacketsRequest` in api/payloads)**, **sessionTitling (#211: the APPEND-ONLY episode-title record — one immutable row per naming, `source:'derived'|'floor'|'user'` [**#226** adds the `floor` rung], derived rows carry `TitlingProvenance` [which orientation pass `annotationId`/`templateId`/`endpoint` + the `nature`/`direction`/`topics` EVIDENCE the name drew from], **#226 floor rows carry `FloorTitlingProvenance`** [the deterministic `producer`+`subjects`+`derivedAt` — a zero-model rung that NEVER impersonates the judge; `provenance` is now a `TitlingProvenance|FloorTitlingProvenance` union, both registered in AllSchemas]; store `session_titlings` table + `WorkspaceRegistry.resolveTitle` = latest user > latest derived > **latest floor (#226)** > undefined [user is SOVEREIGN; a judge-derived name always supersedes a floor, by SOURCE not append order] + `latestFloorTitle`/`hasAuthoredTitle` helpers (#226); `recordSessionTitling` appends the row AND materialises the resolved name onto `Session.title` — titlings are truth, `Session.title` is the cache read by every surface; + `SetSessionTitleRequest` for PUT /sessions/:id/title in api/payloads)**, **summary (#177 slice 1: the multi-timescale Summary — the `level` enum COMPLETE now [rolling/episode/five-minute/session/project] though slice 1 PRODUCES only rolling/five-minute/session; REFS-ONLY `children` [role child|evidence, level-tagged] to lower summaries/packets/distillates/moments/observations; explicit `SummaryInputBound` [the bound made inspectable]; optional `text` = a MODEL PROPOSAL [`proposal:true`, #189] that is ABSENT when `degraded` names the honest unavailable reason [no fabricated prose]; deterministic confidence; append-only `revision`/`supersedes`; sub-schemas SummaryLevel/Child/InputBound/Provenance in AllSchemas; `PromptTemplate` gains kind `summary` + a `SummaryBinding` [level/windowMs/childLevel/maxChildren/maxEvidence/cadenceMs — cadence+bound+prompt as CONFIG]; + `BuildSummariesRequest` in api/payloads; **#177 slice 2:** `SummaryBinding` gains optional `scope`(workspace|workflow|app)/`targetId` [per-workflow/app cadence+template scoping] + new `SummaryScope` sub-schema in AllSchemas; `SummaryProvenance` gains optional `templateScope` [the which-scope-won audit]; `Summary.sessionId` now genuinely absent for the cross-session `project` level; `BlockTypeName`/`BlockQuery.source` gain `summaries` — all append-only/optional ⇒ prior rows still validate)**, **claim (#178 slice 1: the evidence-backed relationship Claim — a durable, append-only, subject→relation→object assertion grounded ONLY in refs to converged evidence [`ClaimEvidenceRef` over context-packet/moment/distillate, `minItems:1` ⇒ a claim with NO evidence is UNREPRESENTABLE]; `ClaimRelation` is a CLOSED starter union [co-occurs-with · works-on · belongs-to · authored · member-of · relates-to] — slice-1's deterministic builder emits ONLY `co-occurs-with` [the sole relation co-occurrence can assert without a model], the semantic kinds are reserved for the judge-enrichment slice + user corrections, the enum is COMPLETE now [SummaryLevel precedent], `relates-to` is the honest generic escape-valve; `source:'derived'|'user'` + `state:provisional/confirmed/corrected/rejected/superseded`; `confidence` a proposal-not-truth fixed map capped <1.0; `ClaimProvenance` [builder+`evidenceCount` = the recorded derivation behind confidence] on derived, `ClaimCorrection` [verdict+by+note] + `corrects` on sovereign user rows; append-only `revision`/`supersedes`; `firstObserved`/`lastObserved` = honest valid-time from evidence; sub-schemas ClaimRelation/ClaimEvidenceRef/ClaimProvenance/ClaimCorrection in AllSchemas; + `BuildClaimsRequest`/`CorrectClaimRequest`/`RelatedEntity` in api/payloads)**, session, workspace, pin │ ├─ src/config/ P0 surface(block/action; **#99** optional append-only `Surface.workspaceId` — an app INSTANCE = a surface bound to a workspace silo, the DEFAULT workspace for its block queries), mode, voice, fabric, flag · workflow (P4A: WorkflowSpec — ordered flag-gated steps document; the pipeline as a cloneable doc, editable over GET/PUT /workflows since P4-T1) · todo (P4A-s4: TodoList/TodoItem — the editable session to-do document the constrain/unconstrain loop accumulates into) · **#134** two new surface PRIMITIVES: the `input` BlockTypeName + `Block.input` {target, submit, mode text/file/both, placeholder/accept/submitLabel} (text-entry/file-drop as a document field), and `Surface.panel` (AttachedPanel {edge below/right, collapsed/expanded px, reveal user/event, openOn, startExpanded} — the attached-expansion-panel geometry, both append-only/optional) · **raw-frame trusted hosts:** the http `Endpoint` variant (fabric) gains optional `trustRawFrames` — the user's EXPLICIT declaration that this host may receive raw screen frames; absent ⇒ raw frames stay loopback-only (default unchanged), honored only for LAN-local hosts (a public host is denied even when set — the cap is absolute); append-only/optional, per-endpoint user config, never auto-set · **#234** `BlockTypeName` gains `sessions` (append-only — the note-taker's history-list block was live in the seed docs + client registry but missing from the union), AND every mapped literal-union array in `config/surface.ts` (`BlockTypeName`, `BlockQuery.source`, `Action.verb`, `Block.show`, `input.mode`) is now `as const` so `Static<…>` NARROWS to the literal union instead of widening to `string` — restoring compile-time enforcement (a bad block type is now a type error at the seed site AND at the client `BlockRegistry` = `Partial>`). Runtime gap closed by `engine/surfaces/seed-validation.test.ts`: `Value.Check` over every SEEDED surface + template `surface.json` against the `Surface` contract, so an unregistered block type fails loudly rather than rendering by registry luck │ ├─ src/api/ P0 HTTP routes + WS events (names, payloads) · **#134** POST /chat (ChatRequest {message, workspace?, pinId?, history?} → ChatReply {answer, citations[], budget: honest turn/context budget, egress}) · **Ask face:** ChatRequest gains optional `screenshot` (ChatScreenshot {contentType,data} — one base64 frame per explicit send) + `turnId` (client-minted stream key); `ChatDelta` {turnId,seq,text,done} behind the ephemeral `chat.delta` WS event (the #58 idiom — never persisted, the ChatReply is authoritative); `ChatHistory` {turns,total,truncated} behind GET /chat/history (registered) │ │ └─ senses.ts P4 **#174 live physical-lane contract:** closed mic/system-audio/screen source union; metadata-only capture + processing evidence; atomic `SenseLaneSnapshot` rows and canonical three-row `SenseLaneSnapshotSet`. Slice C adds the closed client `ScreenCaptureObservation` union (`queued`/`delta-skipped`/`grab-failed`), screen-only `ScreenLaneObservation` derivation evidence (`id`/`occurredAt`/skip-or-grab outcome), and internal `ScreenProcessingOutcome` (`processed`/`blank`/`failed`), with capture/time correlation but no pixels, derived text, endpoint/model detail, or arbitrary errors. A screen row's optional `latestObservation` explains a visible delta skip or capture failure; queued truth remains explained by `latestCapture`. **#192** the client observation union's failed-attempt variant widens to `grab-failed`/`permission-denied` (the OS-refused-run report, still attempt-correlation only) and `ScreenLaneObservation.outcome` mirrors it — closed literals, no new field: lane rows stay metadata-only with the reserved `blocked` health + `disabled`/`permission-denied`/`configuration-blocked` reasons now emitted by real producers. `routes.ts` registers authenticated `GET /senses/live` and `POST /screen/observations`; `events.ts` registers public `sense.lane.updated`; every schema is generated under `schemas/` @@ -37,7 +37,7 @@ openinfo/ │ ├─ distill/ P2 merge · distiller · transcribe (audio→text pre-distill drain stage via stt slot; mic="me"/system-audio="them" speaker split; #58: onTranscribed hook + buildTranscriptUpdates → ephemeral transcript.updated fast-path, NOT persisted; **#116:** onTranscribed also hands `SttInvokeMeta` — endpoint/model + a MEASURED invoke durationMs — and the http.ts wiring persists one SttSegment per transcribed chunk, so the durable trace root rides the same seam) · echo-dedupe (PR #151: pure per-session 30s buffer of system-stream fragments; a mic fragment near-duplicating one within ±2s — Jaccard ≥0.8 or mic⊆system containment; <3 unique-token fragments never dropped — dies at the runTranscribe seam BEFORE text-queue persistence AND the transcript.updated fan-out; `OPENINFO_ECHO_DEDUPE=0` kill-switch; suppressed count log-only until a System-app surface reads it; **#236:** a 2nd GARBLE tier catches loud-bleed twins the Jaccard-0.8 bar missed — within a TIGHTER ±750ms near-simultaneous window, a relaxed DIRECTIONAL mic-coverage bar ≥0.5 for ≥5-token mic fragments (a superset catch: only fires on pairs the confident tier rejected, tight time window keeps genuine dialogue safe, higher token floor + directionality protect the false-positive floor); tuned against the deterministic `echo-bleed-fixtures.ts` corpus (garbled-twin positives + genuine-dialogue negatives), cross-drain backward repair deliberately NOT added — bleed lags playback so forward-only is physics-aligned) · cadence (#58 DistillCadence: accumulate transcribed text per session, release the distill LLM pass only at ≥15s capturedAt span OR session-end flush — transcription still runs every drain) · moments (typed extraction) · parse (defensive JSON, shared) · defaults/documents (template+mode docs; **#61** seeds 3 fast-field prompt docs topic/entities-mentioned/work-items + `fieldTemplates()` reader; **#62** seeds the judge prompt doc `tpl-judge-default` + `judgeTemplates()` reader; **#130** the three window templates (`tpl-distill/extract/entities-default`) ship NEUTRAL — no baked voice vector (`{{tone}}…{{voice.rules}}` stripped from the bodies; the machinery stays live for edited templates + the act draft output); a one-time `seedOrRefreshBuiltin` refresh in `ensureDefaults` updates an UNEDITED builtin to the new body (detected conservatively: version still 1 AND body byte-for-byte one of `PREVIOUS_BUILTIN_BODIES`), NEVER clobbering a user edit) · **#61 fast fields (the fan-out substrate):** fields.ts — `FastFieldScheduler.runFields(chunks)` reads every `kind:'field'` prompt doc (a `PromptTemplate` with a `field` binding `{fieldId,tier,trigger:{kind:'transcript',minChars?},scope}`), applies the per-field minChars relevance gate, runs the TRIGGERED fast-tier ones CONCURRENTLY (`Promise.all`) against the llm slot, then per result publishes `field.updated` + persists the latest value; per-field invoke failure is caught (no event, no value, nothing fabricated); rides the SAME cadence-released batch the distiller sees, gated by `distill.fields`; **#116:** each fan-out pass mints ONE spanId shared by its values and stamps `provenance.sourceChunks` (the batch's chunk ids) + `usage`, the distiller mints one spanId per merge window (distillate+moments+entity mentions+a GuardHold share it; a hold also records its window's sourceChunks), and the judge mints one per reviewed batch (spanId+usage on each JudgeReview). field-values.ts — `FieldValueStore` persists the DURABLE half: one `_meta.db` doc per field keyed deterministically by scope (`fv:::`), so getLatest IS the current value; contract-validated before write · **#62 judge stage (dual-input review):** judge.ts — `JudgeScheduler.runJudge(chunks)` reads every `judge`-tier prompt doc and, per judge, judges the fast-result set it `reviews` (default: all fast fields in scope) AGAINST the SAME source window the fast tier saw — the standardized `{{source}}`/`{{results}}` template contract — then applies each per-field verdict (confirm→`confirmed`, correct→overrule the value in place→`corrected`, flag→`flagged`), persisting a NEW version of the SAME FieldValue stamped with `provenance.judge` (JudgeReview: judge template/endpoint/model, verdict, priorValue on a correct) + republishing `field.updated`; nothing fabricated (a correct with no value / a verdict for an unknown field is skipped-with-log); a judge invoke failure is caught per document. TIER-GATED via `hasJudgeEndpoint()` — the judge routes to the `llm.judge`-designated endpoint (OPENINFO_JUDGE_ENDPOINT override) in a one-endpoint sub-fabric; **#226:** with NO designation but a NON-EMPTY llm slot the lane FALLS BACK onto the whole llm slot (`judgeLane` returns `{endpoints,designated}`), so orientation/title light up on whatever llm the user actually has — a designated endpoint still wins, the completed invoke's `result.endpoint` records WHICH answered, every egress/guard seam applies unchanged, and cadence is untouched (still the wider `judgeCadence`); only an EMPTY llm slot is now the logged no-op tier-gate. Wired in http.ts on its OWN cadence (`judgeCadence`, default 4× the distill cadence, OPENINFO_JUDGE_CADENCE_MS) decoupled from the fast fan-out, gated by `distill.judge` · **#131 orientation pass:** `runJudge` routes a `produces:'orientation'` judge doc (seeded `tpl-judge-orientation`, neutral #130 body) to `runOrientation` — classifies the session (nature/direction/topics) from the SINGLE `{{source}}` window into an ENGINE-STAMPED `SessionAnnotation` (additive to Session — a separate record keyed `oa::`, so a later pass replaces in place = ANNOTATE-AND-CORRECT; model controls only the classification text, blanks default to `unclear`, topics engine-capped so counts are never trusted), persisted via LayoutStore (`session-annotation`) + emitted as `orientation.updated` (bus→WS, the #134 sidebar trigger); tier-gated exactly like #62. **GATE-READY SEAM:** application is funneled through the single `applyAnnotation`, switched on `orientationDisposition` — `annotate` (shipped) vs `gate` (designed-not-built: hold-until-classified slots into this one method + the drain, no re-architecture; today it annotates + logs) · **#211 episode naming:** `episode-title.ts` `deriveEpisodeTitle(annotation)` — a PURE, deterministic transform of the orientation classification (the model proposal) into a human title (top ≤2 topics framed by nature: "Meeting on …"/"Call about …"/"Working on …"; `undefined` when no topics ⇒ no hollow name); `applyAnnotation` calls `deriveTitle`, which appends a `derived` `SessionTitling` via `store.recordSessionTitling` ONLY when the derived name CHANGES (`latestDerivedTitle` dedup — append-only, no per-pass spam) and publishes `session.titled` (the materialised Session) — no second model call, rides the same judge pass · **#226 zero-model title floor:** `floor-title.ts` `deriveFloorTitle(subjects)` — a PURE, deterministic transform of signals the session ALREADY has (its recency×frequency-ranked entities, session-scoped via distillate provenance) into "Working on "; `FloorTitleScheduler.run(chunks)` rides EVERY distill batch (cheap, no invoke, NOT flag-gated), skips any session that already has a user/derived title (`hasAuthoredTitle`), dedupes on `latestFloorTitle`, appends a `floor` `SessionTitling` + publishes `session.titled` — the sane-defaults safety net so a session gets an honest name even when no judge derivation ever runs; wired in http.ts after the fast fan-out · **pill P2 preset injection (BUILT):** at the one interpolation site where the distill SUMMARY prompt is composed, `distiller` resolves the window's workspace active preset (`PresetDocuments.resolveActive`, optional dep) and PREPENDS its body as leading context — so the selection applies to WHATEVER distill template is in force (not only one carrying a `{{preset}}` placeholder) and unset is EXACTLY byte-identical to today; only the summary is steered (the strict-JSON moments/entities passes stay bare); `Distillate.provenance.presetId` names the preset that shaped a summary (honest why-record, System/Ledger register only) · **Ask face:** defaults.ts also ships `tpl-ask-default` ('ask' kind — the empty-send explain-my-screen QUESTION as a document, #130 posture; seeded seed-if-absent, read fresh per send over GET /templates/:id, edited over PUT with no rebuild) │ ├─ screen/ P4B screen understanding (BUILT): processor.ts (subscribes capture.received, gated screen.ocr; source:'screen' image frames → invokeOcr → OcrResult + a distillate; skips the companion meta chunk; own last-failures ring; P4A×P4B: `runOnDrain(chunks,step)` is the DRAIN-stage entry — recognizes via the step's slot ocr→invokeOcr/vlm→invokeVlm, shares the build+persist helper, PROPAGATES throws for re-queue; #102 keep-time: stamps OcrResult.capturedAt from chunk.capturedAt so the record carries true capture time, not just the mirror Distillate window; **#174 slice C:** reports closed metadata-only `blank`/`failed` outcomes to the live tracker while successful nonblank recognition remains driven only by `ocr.completed`; legacy ingest reports then swallows OCR failures, workflow drain reports then rethrows the original error, and an outcome-reporter fault changes neither path) · router.ts (GET /screen/results · /screen/status) · registry.ts (WeakMap store→processor bridge, so both the http.ts-mounted router AND the executor's recognizeScreen seam reach the one processor) · index.ts (wireScreenOcr — main.ts wires it; the ingest subscription DEFERS while workflow.enabled is ON so screen understanding has ONE owner). **#175 deterministic acceptance:** `acceptance-175-e2e.test.ts` drives authenticated valid-JFIF bytes through both owners and distinct slots — legacy OCR, then workflow VLM — and asserts exact `OcrResult`/mirror-Distillate attribution + provenance, status/results/surface agreement, one-owner persistence, and ambient Ask context without a per-send image. The ocr/vlm INVOCATION lives in fabric/invoke.ts with the other slots; this is the processor+router only │ ├─ voice/ P2 resolve · interpolate · documents/defaults (registers+bindings) │ P5: comparator · chains -│ ├─ index/ P2 extract (entities) · rank (recency×frequency) · relevant (relevant-now join) │ **#72 (BUILT):** phonetic.ts (pure, zero-dep: levenshtein/editSimilarity · compact Double-Metaphone primary+secondary · phoneticEqual · nameSimilarity — token soft-Jaccard-led blend, whole-string only when token counts differ) + resolve.ts (`resolveEntity` — the scored match step `score = phoneticFuzzy × corpusPrior × crossSourceCorroboration × personAffinity`, clamped; bands ≥0.85 auto/silent · 0.5–0.85 provisional · <0.5 new; a rival within ambiguityMargin 0.05 marks ambiguous; corpusPrior only-boosts + bounded; crossSource/personAffinity are neutral-1.0 INPUT MULTIPLIERS with no producer yet; honors rejectedRivalId. **#94 hardening:** corpusPrior/clamp01/sort are all finite-guarded — a corrupt `lastSeen` (NaN) can no longer win the sort or persist a NaN confidence; crossSource/personAffinity CLAMPED to `[0.5,1.5]` at the boundary so a future producer can't override the phonetic evidence; docstrings corrected to the TRUE guarantee — the boost only multiplies up (never drags an exact match below auto) but a multiplicative boost CAN lift a partial across a band edge, so the "never crosses a band alone" overclaim was fixed, not the code). Wired into `store/workspaces.ts` `upsertEntity` (override short-circuit runs FIRST; fuzzy link writes the corrupted form to heardAs not aliases; every resolution appends an `EntityResolution` to provenance; provisional/ambiguous stamps the #66 state+confidence+ambiguity, clean auto/first-create stay silent; **#94** a `new`-band create is stamped provisional only when its best same-kind rival lands NEAR the provisional band [`bestRivalScore ≥ provisionalBand − CREATE_PROVISIONAL_MARGIN`, 0.1] — an unrelated create [rival ≈0, e.g. a CJK name vs a Latin corpus] stays silent instead of being marked merely because a same-kind record existed). **#74 (BUILT):** correlate.ts — the crossSource PRODUCER (pure windowed correlator: `overlapsWindow` [OCR capture instant inside the heard window ± slack, NaN-safe] · `ocrForms`/`ocrTextForms` [separator AND path-component splitting, so "acme/pi.dev" yields "pi.dev"] · `correlate`/`correlateWindow` [max nameSimilarity of heard×OCR forms → `crossSourceCorroboration` multiplier at the resolver's clamp ceiling + a typed `seen` Sighting]); wired at the DISTILLER upsert seam (distiller.ts reads store.listOcrResults once per window → passes signals+crossSighting into upsertEntity — consumes the PERSISTED OcrResult stream, drain-timing-independent by construction, #115-safe); store side: a corroborated resolver LINK is stamped `confirmed` (the `linked && corroborated` branch in stampResolution — NOT a resolver band; corroboration promotes a genuine link, never a bare create) + the heard form taught to heardAs with both sightings recorded, idempotent on replay — heard+seen-same-window is the silent auto-teach path, zero contract change (entity v2 already carried Sighting/heardAs/signals). moveSession stays exact `findEntity` │ **#176 slice 1 (BUILT):** packets.ts — the deterministic ContextPacket builder (`buildContextPackets`, PURE like every index/ module: no DB/model/clock beyond inputs + injectable `now`): epoch-aligned `windowMs` buckets (default 60s) over stored SttSegments (mic + system-audio, lanes never merged) and OcrResults (screen) by TRUE capture instant (NaN ⇒ skipped, never guessed); focus evidence = the Session's window/repo attribution entries; candidates = in-window moments' entity refs, each checked against in-window screen forms with the #74 `correlate`/`ocrForms` (a match records ocrId+form+similarity — evidence with provenance); missing lanes degrade to `gaps` with machine-readable reasons; confidence = fixed lane-count map (0.4/0.7/0.9); idempotence + append-only supersession decided against the existing chain head by canonical-content equality (identical ⇒ no-op; different ⇒ revision+1 with `supersedes`, prior never mutated); ids content-derived (sha-256 over canonical JSON + chain position) so replay is byte-stable — proven over the #32 fixture in packets-replay.test.ts │ **#176 slice 2 (BUILT):** produce-packets.ts — the LIVE producer (the one index/ module beside relevant.ts that touches store): `materializeContextPackets` reads a session's stored observations, runs `buildContextPackets`, persists ONLY the appends, and records the attempt on an in-memory `PacketBuildLog` (the diagnostics "last update" signal) — CONTAINED (never throws: a read/build/write failure is caught + recorded, so a packet build can never sink the capture/distill path), idempotent, batched ONCE per session over the whole session, scoped to the session's OWN workspace. Wired at the `session.ended` seam in api/http.ts (after the drain/flush, over the fully-drained session) so packets materialize during normal capture with NO route call; the on-demand POST `/context/packets/build` now runs the SAME seam (unified build + log, still 404-validated). Proven served in `api/context-packets-e2e.test.ts` (real engine + fake stt/llm → /capture/mic → session end → the SERVED /settings/context-packets shows the converged window, NO build route called) + seam unit `index/produce-packets.test.ts` (idempotence · workspace isolation · contained failure) │ **#177 slice 1 (BUILT):** summaries.ts — the PURE hierarchical-summary assembler (`assembleSummaries`/`buildSummary`, no DB/model/clock beyond inputs + injectable `now`, like every index/ module): buckets a level's lower-level inputs into epoch-aligned `windowMs` windows (or ONE whole-session bucket for the `session`/`project` levels), BOUNDS them to the level's `maxChildren`/`maxEvidence` (newest kept, `SummaryInputBound` records available-vs-consumed so the bound is inspectable — never unbounded raw history), deterministic lane-style confidence band, and content-derived id (sha-256 over the PROSE-EXCLUDED skeleton + chain position) so a prose re-roll over the SAME children never churns a revision; idempotence + append-only supersession against the existing chain head (identical child set ⇒ no-op; changed ⇒ revision+1 with `supersedes`) │ produce-summaries.ts — the LIVE producer + `createFabricSummarizer` (store-touching, beside produce-packets/relevant): gathers a level's children (distillates for `rolling`, else the lower level's live summary heads) + bounded evidence (ContextPackets for `rolling`, else moments), assembles the plan, runs the model prose through the SAME #64 egress + #63 guard chain the distiller uses, persists ONLY appends, records a per-(session,level) `SummaryBuildLog` attempt — CONTAINED (never throws) and HONEST when the model is unavailable (persist `degraded.reason`, NO fabricated prose; a later pass upgrades the degraded head IN PLACE once the model answers). Proven byte-identical + duplicate-free over the #32 pattern in `index/summaries-replay.test.ts`, seam-tested in `index/produce-summaries.test.ts` (bound · degraded/upgrade · isolation · contained failure), and served end-to-end in `api/summaries-e2e.test.ts` │ **#177 slice 2 (BUILT):** produce-summaries.ts grows EPISODE (a mid-window rollup over rolling) + PROJECT production — `isCrossSession`/`CROSS_SESSION_SUMMARY_LEVELS` gather `project` children across the WHOLE workspace (every session's session summary) and stamp NO sessionId, so a new session's result folds into a superseding project revision (append-only, prior kept); the assembler threads an OPTIONAL sessionId + a singleton `latestOverall` head-match for whole-scope levels (a project's window span grows, so window-keyed matching would fork instead of supersede) + a `templateScope` audit; summaries-trace.ts — the PURE `walkSummaryTrace` proving RAW-MEDIA EXPIRY never breaks the path (recurses summary→summary cycle-guarded, resolves durable leaves, surfaces each distillate/ocr `sourceChunks`+stt `chunkId` as an honest `expired` capture-chunk leaf; a dangling durable ref is `expired`, never a throw); per-workflow/app scope resolver `resolveSummaryTemplate` in distill/documents.ts (precedence app>workflow>workspace, the voice-binding precedent). Proven in `index/summaries-project.test.ts` (episode refs + project append-only supersession across two sessions), `index/summaries-raw-media-expiry.test.ts` (walkable trace + honest expired leaves), `distill/summary-scope.test.ts` (two scopes each resolve their own), the extended `index/summaries-egress-seam.test.ts` (denying mode holds egress at the SESSION level too), and served via `api/summaries-surface-e2e.test.ts` (POST /query source:summaries serves the five-minute + session views) │ P4D (BUILT): canon.ts (mergeCanon — pure reference merging: same-kind duplicates whose name∪alias keys intersect fold into one canonical record via union-find, canonical winner = outboundCount→mentions→firstSeen→id; unions aliases/momentRefs/provenance, sums mentions+outboundCount, sets canonicalOf; relevant.ts folds through it before ranking) · rank.ts canon weight (× (1+outboundBoost·log2(1+outboundCount)) — "sent outranks viewed", log-damped, outboundCount 0 leaves score unchanged; write-side deferred — nothing increments outboundCount yet) · ingest/ (BUILT: chunk.ts page-anchored chunking [pure, deterministic ids] · fetcher.ts file[form-feed pages]/url[pageless]/pdf[honest stub]/gdoc[seam behind ingest.gdoc] · ingest.ts lifecycle → store pins/pin_chunks). Store: pins + pin_chunks tables (workspace-level canon, not session-keyed → not in moveSession). P4-T2 (BUILT): the ingest lifecycle over HTTP in api/http.ts — GET/POST `/pins?workspace=` · POST `/pins/:id/ingest` (runs ingestPin, honest failure = 200 with ingest.status:'failed', unknown pin 404) · GET `/pins/:id/chunks` (page-anchored excerpts, the "cite p.42" read) — ZERO logic change, routes over the store/ingest seams +│ ├─ index/ P2 extract (entities) · rank (recency×frequency) · relevant (relevant-now join) │ **#72 (BUILT):** phonetic.ts (pure, zero-dep: levenshtein/editSimilarity · compact Double-Metaphone primary+secondary · phoneticEqual · nameSimilarity — token soft-Jaccard-led blend, whole-string only when token counts differ) + resolve.ts (`resolveEntity` — the scored match step `score = phoneticFuzzy × corpusPrior × crossSourceCorroboration × personAffinity`, clamped; bands ≥0.85 auto/silent · 0.5–0.85 provisional · <0.5 new; a rival within ambiguityMargin 0.05 marks ambiguous; corpusPrior only-boosts + bounded; crossSource/personAffinity are neutral-1.0 INPUT MULTIPLIERS with no producer yet; honors rejectedRivalId. **#94 hardening:** corpusPrior/clamp01/sort are all finite-guarded — a corrupt `lastSeen` (NaN) can no longer win the sort or persist a NaN confidence; crossSource/personAffinity CLAMPED to `[0.5,1.5]` at the boundary so a future producer can't override the phonetic evidence; docstrings corrected to the TRUE guarantee — the boost only multiplies up (never drags an exact match below auto) but a multiplicative boost CAN lift a partial across a band edge, so the "never crosses a band alone" overclaim was fixed, not the code). Wired into `store/workspaces.ts` `upsertEntity` (override short-circuit runs FIRST; fuzzy link writes the corrupted form to heardAs not aliases; every resolution appends an `EntityResolution` to provenance; provisional/ambiguous stamps the #66 state+confidence+ambiguity, clean auto/first-create stay silent; **#94** a `new`-band create is stamped provisional only when its best same-kind rival lands NEAR the provisional band [`bestRivalScore ≥ provisionalBand − CREATE_PROVISIONAL_MARGIN`, 0.1] — an unrelated create [rival ≈0, e.g. a CJK name vs a Latin corpus] stays silent instead of being marked merely because a same-kind record existed). **#74 (BUILT):** correlate.ts — the crossSource PRODUCER (pure windowed correlator: `overlapsWindow` [OCR capture instant inside the heard window ± slack, NaN-safe] · `ocrForms`/`ocrTextForms` [separator AND path-component splitting, so "acme/pi.dev" yields "pi.dev"] · `correlate`/`correlateWindow` [max nameSimilarity of heard×OCR forms → `crossSourceCorroboration` multiplier at the resolver's clamp ceiling + a typed `seen` Sighting]); wired at the DISTILLER upsert seam (distiller.ts reads store.listOcrResults once per window → passes signals+crossSighting into upsertEntity — consumes the PERSISTED OcrResult stream, drain-timing-independent by construction, #115-safe); store side: a corroborated resolver LINK is stamped `confirmed` (the `linked && corroborated` branch in stampResolution — NOT a resolver band; corroboration promotes a genuine link, never a bare create) + the heard form taught to heardAs with both sightings recorded, idempotent on replay — heard+seen-same-window is the silent auto-teach path, zero contract change (entity v2 already carried Sighting/heardAs/signals). moveSession stays exact `findEntity` │ **#176 slice 1 (BUILT):** packets.ts — the deterministic ContextPacket builder (`buildContextPackets`, PURE like every index/ module: no DB/model/clock beyond inputs + injectable `now`): epoch-aligned `windowMs` buckets (default 60s) over stored SttSegments (mic + system-audio, lanes never merged) and OcrResults (screen) by TRUE capture instant (NaN ⇒ skipped, never guessed); focus evidence = the Session's window/repo attribution entries; candidates = in-window moments' entity refs, each checked against in-window screen forms with the #74 `correlate`/`ocrForms` (a match records ocrId+form+similarity — evidence with provenance); missing lanes degrade to `gaps` with machine-readable reasons; confidence = fixed lane-count map (0.4/0.7/0.9); idempotence + append-only supersession decided against the existing chain head by canonical-content equality (identical ⇒ no-op; different ⇒ revision+1 with `supersedes`, prior never mutated); ids content-derived (sha-256 over canonical JSON + chain position) so replay is byte-stable — proven over the #32 fixture in packets-replay.test.ts │ **#176 slice 2 (BUILT):** produce-packets.ts — the LIVE producer (the one index/ module beside relevant.ts that touches store): `materializeContextPackets` reads a session's stored observations, runs `buildContextPackets`, persists ONLY the appends, and records the attempt on an in-memory `PacketBuildLog` (the diagnostics "last update" signal) — CONTAINED (never throws: a read/build/write failure is caught + recorded, so a packet build can never sink the capture/distill path), idempotent, batched ONCE per session over the whole session, scoped to the session's OWN workspace. Wired at the `session.ended` seam in api/http.ts (after the drain/flush, over the fully-drained session) so packets materialize during normal capture with NO route call; the on-demand POST `/context/packets/build` now runs the SAME seam (unified build + log, still 404-validated). Proven served in `api/context-packets-e2e.test.ts` (real engine + fake stt/llm → /capture/mic → session end → the SERVED /settings/context-packets shows the converged window, NO build route called) + seam unit `index/produce-packets.test.ts` (idempotence · workspace isolation · contained failure) │ **#177 slice 1 (BUILT):** summaries.ts — the PURE hierarchical-summary assembler (`assembleSummaries`/`buildSummary`, no DB/model/clock beyond inputs + injectable `now`, like every index/ module): buckets a level's lower-level inputs into epoch-aligned `windowMs` windows (or ONE whole-session bucket for the `session`/`project` levels), BOUNDS them to the level's `maxChildren`/`maxEvidence` (newest kept, `SummaryInputBound` records available-vs-consumed so the bound is inspectable — never unbounded raw history), deterministic lane-style confidence band, and content-derived id (sha-256 over the PROSE-EXCLUDED skeleton + chain position) so a prose re-roll over the SAME children never churns a revision; idempotence + append-only supersession against the existing chain head (identical child set ⇒ no-op; changed ⇒ revision+1 with `supersedes`) │ produce-summaries.ts — the LIVE producer + `createFabricSummarizer` (store-touching, beside produce-packets/relevant): gathers a level's children (distillates for `rolling`, else the lower level's live summary heads) + bounded evidence (ContextPackets for `rolling`, else moments), assembles the plan, runs the model prose through the SAME #64 egress + #63 guard chain the distiller uses, persists ONLY appends, records a per-(session,level) `SummaryBuildLog` attempt — CONTAINED (never throws) and HONEST when the model is unavailable (persist `degraded.reason`, NO fabricated prose; a later pass upgrades the degraded head IN PLACE once the model answers). Proven byte-identical + duplicate-free over the #32 pattern in `index/summaries-replay.test.ts`, seam-tested in `index/produce-summaries.test.ts` (bound · degraded/upgrade · isolation · contained failure), and served end-to-end in `api/summaries-e2e.test.ts` │ **#177 slice 2 (BUILT):** produce-summaries.ts grows EPISODE (a mid-window rollup over rolling) + PROJECT production — `isCrossSession`/`CROSS_SESSION_SUMMARY_LEVELS` gather `project` children across the WHOLE workspace (every session's session summary) and stamp NO sessionId, so a new session's result folds into a superseding project revision (append-only, prior kept); the assembler threads an OPTIONAL sessionId + a singleton `latestOverall` head-match for whole-scope levels (a project's window span grows, so window-keyed matching would fork instead of supersede) + a `templateScope` audit; summaries-trace.ts — the PURE `walkSummaryTrace` proving RAW-MEDIA EXPIRY never breaks the path (recurses summary→summary cycle-guarded, resolves durable leaves, surfaces each distillate/ocr `sourceChunks`+stt `chunkId` as an honest `expired` capture-chunk leaf; a dangling durable ref is `expired`, never a throw); per-workflow/app scope resolver `resolveSummaryTemplate` in distill/documents.ts (precedence app>workflow>workspace, the voice-binding precedent). Proven in `index/summaries-project.test.ts` (episode refs + project append-only supersession across two sessions), `index/summaries-raw-media-expiry.test.ts` (walkable trace + honest expired leaves), `distill/summary-scope.test.ts` (two scopes each resolve their own), the extended `index/summaries-egress-seam.test.ts` (denying mode holds egress at the SESSION level too), and served via `api/summaries-surface-e2e.test.ts` (POST /query source:summaries serves the five-minute + session views) │ P4D (BUILT): canon.ts (mergeCanon — pure reference merging: same-kind duplicates whose name∪alias keys intersect fold into one canonical record via union-find, canonical winner = outboundCount→mentions→firstSeen→id; unions aliases/momentRefs/provenance, sums mentions+outboundCount, sets canonicalOf; relevant.ts folds through it before ranking) · rank.ts canon weight (× (1+outboundBoost·log2(1+outboundCount)) — "sent outranks viewed", log-damped, outboundCount 0 leaves score unchanged; write-side deferred — nothing increments outboundCount yet) · ingest/ (BUILT: chunk.ts page-anchored chunking [pure, deterministic ids] · fetcher.ts file[form-feed pages]/url[pageless]/pdf[honest stub]/gdoc[seam behind ingest.gdoc] · ingest.ts lifecycle → store pins/pin_chunks). Store: pins + pin_chunks tables (workspace-level canon, not session-keyed → not in moveSession). P4-T2 (BUILT): the ingest lifecycle over HTTP in api/http.ts — GET/POST `/pins?workspace=` · POST `/pins/:id/ingest` (runs ingestPin, honest failure = 200 with ingest.status:'failed', unknown pin 404) · GET `/pins/:id/chunks` (page-anchored excerpts, the "cite p.42" read) — ZERO logic change, routes over the store/ingest seams │ **#178 slice 1 (BUILT):** claims.ts — the PURE deterministic co-occurrence Claim builder (`buildClaims`, no DB/model/clock beyond inputs + injectable `now`, like every index/ module): derives `co-occurs-with` relationships from CONVERGED evidence only — every unordered entity pair NAMED TOGETHER in one ContextPacket's candidates or one moment's refs co-occurs, evidence = a ref to that packet/moment (refs-only, dedup by record+id, sorted); symmetric endpoints canonically ordered by id; confidence = fixed evidence-count map capped <1.0 (a proposal, never certain); a dangling ref (no entity record) is never claimed; NO evidence ⇒ NO claim (honest degradation); idempotence + append-only supersession decided against the existing DERIVED chain head per (subject,object,relation) by canonical-content equality (identical ⇒ no-op; MORE evidence ⇒ revision+1 with `supersedes` + a higher recorded `evidenceCount`, prior never mutated); ids content-derived (sha-256 over canonical JSON + chain position) so replay is byte-stable — proven store-backed in claims-replay.test.ts. NO model: semantic relations (works-on/…) are a LATER judge-enrichment slice │ produce-claims.ts — the LIVE producer (store-touching, beside produce-packets/summaries): `materializeClaims` reads a session's packets+moments+entities, runs `buildClaims`, persists ONLY appends, records an in-memory `ClaimBuildLog` attempt — CONTAINED (never throws), idempotent, batched ONCE per session, workspace-scoped. Wired at the `session.ended` seam in api/http.ts AFTER packets materialize (fresh candidate evidence) so claims form during normal capture with NO route call; on-demand POST `/claims/build` runs the SAME seam. Store `store/workspaces.ts`: `claims` table (workspace-isolated) + `saveClaim`/`listClaims` [query axes workspace/session/entity(subject∨object)/relation/source/time, **`resolveClaimHeads` resolves derived-supersession AND user-sovereignty BEFORE the filters**] + `relatedEntities` [the depth-1 walk entity→claims→counterparts, always returning the backing claimIds] + `recordClaimCorrection` [mints the sovereign `source:'user'` correction — confirm/reject/correct, outranks derived, carries the target's evidence, content-derived id ⇒ idempotent; a REJECT survives re-derivation] + threaded through `moveSession`. Routes GET `/claims`·`/claims/related`, POST `/claims/build`·`/claims/correct`. Proven served in `api/claims-e2e.test.ts`, seam-tested in `index/produce-claims.test.ts`, unit in `index/claims.test.ts`, store in `store/workspaces.test.ts` │ ├─ act/ P2 the Act primitive: draft (follow-up-draft on session end) · defaults/documents (act templates) │ P4A-s4 (BUILT): todo.ts — task-extract (composeTaskExtract + TaskExtractor rides the drain), TodoDocuments (todo-list docs), mergeTodoItems (normalized-text dedupe), renderTodo ({{todo}}); draft.ts interpolates {{todo}} │ P4+: nudge · exports │ ├─ route/ P3 detector · attribute · identity · reroute (BUILT: detector.ts — pure sustain-window context-switch detection over TimedSignals [focus+calendar]+hints, ONE window; attribute.ts — the Attributor auto-starts/switches sessions behind route.detect over the shared TimedSignal buffer, emits session.switched; hints.ts — workspace-hints docs (HintsDocuments all/get/put; P4-T3b: exposed over HTTP as GET `/hints` [= hintsDocs.all(), the detector's hint provider], GET/PUT `/hints/:workspaceId` in api/http.ts — the APPLY-with-review half of the teach loop, a plain versioned-document edit, unknown-workspace PUT CREATES like PUT /workflows, ZERO route/ logic change); focus.ts — focus-chunk decode; reroute.ts — one-click retroactive session move + session.rerouted, the correction/teaching signal. identity.ts is P7) │ P4C calendar evidence (BUILT): calendar.ts — pure decode of a raw Calendar.app sample → TimedCalendarSignals; calendar-collector.ts — CalendarPoller (engine-side osascript/JXA poll of the current/imminent event, per-tick route.detect gate, graceful degrade) + startCalendarCollector(app,…) mounted from main.ts like wireScreenOcr, feeding the ONE Attributor (exposed on EngineApp). Hint fields eventTitle/attendee → 'calendar' evidence. Gated on route.detect (no new flag); topic-drift out of scope │ ├─ ledger/ P4 commitments · watchers/{repo,doc,mail} · prepare (action cards — builds on act/) @@ -100,6 +100,7 @@ openinfo/ | Audit trace: one input → every engine hop → output | #116 (built) | An auditor picks ONE input (utterance segment or screen capture) and walks it transcribe → distill → moments → fields → judge → guard/egress. Correlation: every pipeline pass mints a `spanId` shared by all records it produces (distiller per merge window incl. GuardHold; fields per fan-out batch; judge per review batch; screen per OcrResult+mirror pair; transcribe per drain), plus deterministic parent links (`SttSegment.chunkId` ↔ `Distillate.sourceChunks` ↔ `FieldValueProvenance.sourceChunks`, `GuardHold.sourceChunks`) — NEVER fuzzy time matching. STT root: `records/sttSegment.ts` persisted per transcribed chunk at the `onTranscribed` seam (`api/http.ts` runTranscribe; store `stt_segments` table) — chunk id, lane, capturedAt→processedAt, textChars (size, never the text), endpoint/model/durationMs/egress. View: Settings → Diagnostics → **Trace** (`surfaces/settings/sections/trace.ts`, pure `buildTraceInputs`/`buildTrace`/`renderTrace`; registry `trace`) — server-rendered picker links (`?input=`) → the hop chain with guard verdicts + egress decisions (reuses the ledger's `guardCell`/`egressCell`); every state is TEXT (empty · unknown id · no-steps-yet · a failed store read surfaces its true reason via the route's try/catch `problem` seam). Ledger multi-hop: see #65 row. DRIVEN served e2e `api/trace-e2e.test.ts` (real server + fake stt/llm/judge on loopback → real /capture/mic POSTs → the page's own links → full chain + both sabotage paths render text); **#207 (BUILT):** the SCREEN sibling `api/screen-trace-e2e.test.ts` (real server + `wireScreenOcr` with NO injected invoke → the REAL ScreenOcrProcessor over a fake loopback paddle-serving `ocr` slot → real /capture/screen POSTs → the page's own links → a recognized frame's `seen` OCR hop with recorded device-local egress + usage-derived timing followed by the shared-span mirror hop labeled "no second model call", AND a blank frame's checkpoint OcrResult that renders its recognition with no downstream mirror hop — the honest "capture with no downstream hops"); fixture-level `sections/trace.test.ts`. Live-rig walk (real parakeet utterance) is owner-gated — deterministic replay stands in. Disclosed: pre-#116 records lack the links and are named non-walkable on the page; default workspace only (the ledger's scope). | | ContextPackets: converge separate senses, inspectable | #176 slice 1 (built; diagnostics rendering = slice 2) | A typed, versioned `ContextPacket` (`records/contextPacket.ts`) converges one correlation window's mic + system-audio + screen observations as REFS (never copies — no prose field; prose stays a read-time view over the refs), with focus/app evidence from the Session record, entity candidates (in-window moments + #74 screen corroboration with ocrId/form/similarity), machine-readable `gaps` for missing senses, deterministic lane-count confidence, and an append-only `revision`/`supersedes` chain (a late observation appends, never mutates). Built by the PURE deterministic correlator `index/packets.ts` (`buildContextPackets`) over stored records; persisted per-workspace (`store/workspaces.ts` `context_packets`, carried whole-chain by moveSession); queryable by workspace/session/time-window/entity over GET `/context/packets` + built on demand via POST `/context/packets/build` (returns only appended packets; [] = idempotent no-op). Determinism proven over the #32 fixture: same fixture replayed twice ⇒ byte-identical packets (`index/packets-replay.test.ts`). **Slice 2 (built):** a LIVE producer (`index/produce-packets.ts` `materializeContextPackets` + `PacketBuildLog`) wired at the `session.ended` seam so packets materialize during normal capture with no route call — CONTAINED (a build failure is caught, recorded, never fatal to capture) + idempotent + batched per session + workspace-isolated; and the diagnostics surface (`surfaces/settings/sections/packets.ts`, Settings → Diagnostics → **Context packets**) rendering membership/exclusions/timing/confidence with the supersession chain and the honest "last update" / "last update didn't finish" line — every value traceable, every state TEXT (empty · assembly `problem` · contained build failure). Proven served in `api/context-packets-e2e.test.ts` (session end → the SERVED page shows the converged window, no build route) + `sections/packets.test.ts` + `index/produce-packets.test.ts`. | | Hierarchical summaries: rolling → episode → five-minute → session → project | #177 slices 1 + 2 (BUILT — issue closure-ready) | A typed, versioned `Summary` (`records/summary.ts`) over the existing distillates + ContextPackets. The `level` enum is COMPLETE (rolling/episode/five-minute/session/project) though slice 1 PRODUCES only the live-loop levels (rolling, five-minute, session). Each coarser level consumes BOUNDED lower-level inputs (children + selective evidence) — `SummaryInputBound` records available-vs-consumed so the bound is inspectable and tested; children are REFS ONLY (no copied content), tracing summary→summary→…→packet/distillate→observation. Prose is a MODEL PROPOSAL (`proposal:true`, #189) run through the SAME #64 egress + #63 guard chain as the distiller; when the summarizer is unavailable the summary is persisted DEGRADED (explicit `degraded.reason`, no fabricated prose) with its derivation path intact, upgraded in place when the model returns. Cadence/bound/prompt are CONFIGURATION: a `summary`-kind `PromptTemplate` + `SummaryBinding` per level (`distill/defaults.ts`, seeded/edited over GET/PUT `/templates`) — editing `windowMs`/`maxChildren`/the body changes behavior with no restart (proven in the e2e). PURE assembler `index/summaries.ts` (`assembleSummaries`/`buildSummary`; content-derived ids EXCLUDE prose so a re-roll never churns a revision; append-only supersession on a changed child set). LIVE producer `index/produce-summaries.ts` (`materializeSummaries` + `createFabricSummarizer` + `SummaryBuildLog`) wired at the drain seam (rolling + five-minute) and the `session.ended` seam (durable session summary) in `api/http.ts`, gated by `summaries.enabled` (default OFF) — CONTAINED (never sinks the drain/session-end). Persisted per-workspace (`store/workspaces.ts` `summaries`, carried whole-chain by moveSession); queryable by workspace/session/level/time-window over GET `/summaries` + built on demand via POST `/summaries/build`. Determinism + no-duplicates proven over the #32 pattern (`index/summaries-replay.test.ts`); seam-tested (`index/produce-summaries.test.ts`: bound · degraded/upgrade · isolation · contained failure) + pure (`index/summaries.test.ts`) + served (`api/summaries-e2e.test.ts`). **Slice 2 (BUILT):** EPISODE + PROJECT production (`produce-summaries.ts` cross-session gather + append-only superseding project revisions incorporating later sessions with prior versions kept; assembler threads optional sessionId + singleton `latestOverall` head-match + `templateScope` audit); RAW-MEDIA-EXPIRY derivation-path proof (`index/summaries-trace.ts` `walkSummaryTrace` + GET `/summaries/trace` — the path stays walkable, a gone source is an honest `expired` node, never a crash/fabrication); PER-WORKFLOW/APP template scoping (`SummaryBinding.scope`/`targetId` + `resolveSummaryTemplate`, precedence app>workflow>workspace, which-scope-won stamped on `provenance.templateScope`); DEFAULT-HUMAN-UI emphasis — a `summaries` query source + client `summaries` block (`apps/client/.../blocks/summaries.ts`, human copy + honest degraded, no machine-speak) leads the default HUD with the five-minute VIEW + session result, while the sentence-level distillate stream moves to the Diagnostics surface. summaries.enabled stays default OFF (consistent with the whole engine-processing flag family, and inert without distill.enabled) — the HUD cards are `on-match`, so a fresh install shows no empty card and the view is the top card the instant the loop runs. | +| Evidence-backed claims + relationship graph | #178 slice 1 (BUILT — substrate only) | A typed, versioned, append-only `Claim` (`records/claim.ts`) — a durable subject→relation→object assertion grounded ONLY in refs to converged evidence (`ClaimEvidenceRef` over context-packet/moment/distillate, `minItems:1` ⇒ a claim with NO evidence is UNREPRESENTABLE, so every claim traces to source observations). `ClaimRelation` is a CLOSED starter union (co-occurs-with · works-on · belongs-to · authored · member-of · relates-to); slice-1's deterministic builder emits ONLY `co-occurs-with` (the sole relation co-occurrence can assert without a model), the semantic kinds are reserved for a LATER judge-enrichment slice (proposals, #189) + sovereign user corrections, and the enum is COMPLETE now (the SummaryLevel precedent) with `relates-to` the honest generic escape-valve. `source:'derived'|'user'`, `state` provisional/confirmed/corrected/rejected/superseded, `confidence` a proposal-not-truth fixed evidence-count map capped <1.0, `ClaimProvenance`(builder+`evidenceCount` = the recorded derivation behind confidence — repeated evidence strengthens only via a NEW revision, never a silent mutation) on derived, `ClaimCorrection`(verdict+by+note)+`corrects` on user rows, append-only `revision`/`supersedes`, `firstObserved`/`lastObserved` = honest valid-time from evidence. Built by the PURE `index/claims.ts` (`buildClaims`) from co-occurrence in ContextPacket candidates + moment refs (symmetric endpoints canonically ordered by id; dangling refs never claimed; NO evidence ⇒ NO claim); content-derived ids ⇒ byte-stable replay (`index/claims-replay.test.ts`). LIVE producer `index/produce-claims.ts` (`materializeClaims`+`ClaimBuildLog`) wired at the `session.ended` seam in `api/http.ts` AFTER packets materialize — CONTAINED (never throws), idempotent, batched per session, workspace-isolated. Persisted per-workspace (`store/workspaces.ts` `claims` table, whole-chain carried by moveSession); `listClaims` queryable by workspace/session/entity(subject∨object)/relation/source/time with **`resolveClaimHeads` resolving derived-supersession AND user-sovereignty BEFORE the filters** (a rejected pair drops from the live view and SURVIVES re-derivation; the original inference + evidence are never deleted). `relatedEntities` = the depth-1 walk (entity → claims → counterparts, always returning the backing claimIds — #181's typeahead substrate). Sovereign `recordClaimCorrection` (append-only `source:'user'` correction that outranks derived, carries the target's evidence, content-derived id ⇒ idempotent — the #211/#232 titling-sovereignty pattern). Routes GET `/claims`·`/claims/related`, POST `/claims/build`·`/claims/correct` (+ `BuildClaimsRequest`/`CorrectClaimRequest`/`RelatedEntity` payloads). Proven served (`api/claims-e2e.test.ts`), seam (`index/produce-claims.test.ts`), pure (`index/claims.test.ts`), store (`store/workspaces.test.ts`). **OUT (later slices):** any UI/surface, model-enriched semantic claims (judge pass), #181 search/typeahead, cross-workspace/app-scoped grants. | | Layered egress-consent policy | M5 #64 (built) | Egress (may content leave the machine?) resolved across FOUR layers, any of which can DENY, most-specific denial wins, every decision records which layer won — the voice-binding precedent. Contract: `shared/contracts/src/config/egress.ts` — `EgressReach`(local/egress) · `EgressLayer`(endpoint/prompt/mode/workspace/content-class/default) · `ContentClass`(screen/transcript/typed/unknown) · `EgressPolicy`({deny}) · `EgressDecision`({reach,allowed,decidedBy,reason}); appended optional onto `Mode`/`Workspace` (`egress`), `PromptTemplate` (`neverEgress`), `Distillate`/`OcrResult` provenance (`egress`), and `QueueFailure.class` gains `egress-denied` (all APPEND-ONLY). Resolution + layer-1 classification: `fabric/egress.ts` (pure). Enforcement: `fabric/invoke.ts` `egressGate` at the endpoint-choice seam — filters egress endpoints when consent denies, `egress-denied` classified failure surfaces the reason, guard #63 hooks where the gate allows an egress reach. Wired: `distill/distiller.ts` (transcript class, mode/workspace/prompt denial, stamps provenance) + `screen/processor.ts` (screen class always denies). Factory posture: no egress endpoint configured ⇒ nothing can leave. Deferred: no UI to set mode/workspace `egress.deny` or prompt `neverEgress` yet (contract + engine honor them; a Settings toggle is a later slice); typed-content class not distinguished from transcript in v0 (both may egress). **#91 QA** `distill/privacy-gate-seam.test.ts` drives the CONSENT gate through the real distiller: a denying MODE / never-egress PROMPT holds the whole window (egress-first endpoint gets ZERO hits across summary+moments+entities, local fallback answers all three — the dropped-wrapper sentinel), no-fallback fails closed, `decidedBy` recorded on provenance. **#128 (FIXED):** the WORKSPACE layer (layer 3) now fires at the distiller seam — `store/workspaces.ts` persists + rehydrates `Workspace.egress` through the meta-DB row (additive `egress` JSON column + idempotent `pragma`-guarded ALTER migration; `fromRow` no longer drops it), and `setEgressPolicy(id,policy)` is the write path (no UI yet — a Settings toggle is a later slice). The seam suite gained the fourth-layer case: a workspace deny holds the egress hop end-to-end and stamps `decidedBy:'workspace'` on provenance. | | Egress guard: content/PII filter on egress hops | M5 #63 (built) | A first-class `guard` fabric SLOT KIND (`shared/contracts/src/config/fabric.ts` — optional array on `Fabric.slots`, append-only; NOT an `llm.judge`-style name convention #88) invoked on every ALLOWED egress hop. Contract (`shared/contracts/src/config/guard.ts`): `GuardBehavior`(redact-and-continue\|hold-and-surface) · `GuardSpan`({kind,start,length} — a descriptor, NEVER the raw value) · `GuardVerdict`({outcome clean\|redacted\|held\|unguarded, behavior, guarded, maskedSpanCount, spans?, guardEndpoint?, reason}) · `GuardPolicy`(the verdict→behavior document, {behavior, acknowledgeUnguardedEgress}) · `GuardHold`(a suspended hop's durable audit record); appended optional onto `Distillate.provenance` (`guard`), `QueueFailure.class` gains `guard-held`, event `guard.hold.updated`. Core (`engine/fabric/guard.ts`, pure + a small openai-compat classifier client): `applyRedaction`/`redactMessages` (mask spans back-to-front with `[redacted:]`, offset-mapped across the joined messages) · `evaluateGuard` (the whole decision table incl. the fail-closed empty-slot edges) · `classifyEgressText` (POSTs the outbound text to a guard endpoint, parses `{flagged:[…]}`) · `runEgressGuard` (classify→evaluate→redact-or-throw; a configured-but-unreachable guard FAILS CLOSED) · `GuardHeldError` (class `guard-held`). Interception: `fabric/invoke.ts` `invokeLlm` runs it at the documented `{allow:true,reach:'egress'}` hook — redact-and-continue masks the outbound copy + proceeds (verdict on `LlmResult.guard`); hold-and-surface / fail-closed HOLDS by throwing (a HARD STOP — never a silent local reroute). A LOCAL hop never enters it (no egress ⇒ no filter). Policy/holds store: `engine/guard/documents.ts` (`GuardDocuments` seeds the default redact-and-continue/not-acknowledged policy; `GuardHoldStore` — one `_meta.db` doc/workspace, the ItemSignalStore pattern; `resolve` flips release/denied). Wired: `distill/distiller.ts` builds the guard opts from the policy + the `guard.egress` flag + the fabric guard slot, threads it onto every window invoke, and on a HOLD records a `GuardHold` + publishes + SKIPS the window (no distillate — nothing left). Routes (`api/http.ts`): `GET /guard-holds` · `POST /guard-holds/resolve` (release/deny) · `GET`/`PUT /guard/policy` (validated). Ledger (`surfaces/settings/sections/ledger.ts`): the GUARD column lights up from `provenance.guard` (clean · redacted·N · unguarded · "— no guard" for a local hop), and held hops surface in a held block with a release/deny affordance (inline POST). DRIVEN served tests `api/settings-guard.test.ts` + `fabric/guard-invoke.test.ts` (fake guard endpoint) + `distill/guard.test.ts`; **#91 QA** `distill/privacy-gate-seam.test.ts` drives the guard END-TO-END through the real `Distiller`+`invokeLlm` against fake egress/guard HTTP endpoints (clean rides provenance + content leaves · redact-and-continue · hold-and-surface SUSPENDS + records/surfaces a GuardHold + release/deny · fail-closed empty slot · guard runs on EVERY egress call summary+moments+entities · a LOCAL hop is never guarded) + the ledger renders that seam output. Gated by `guard.egress` (default OFF). Deferred/disclosed: real llama-guard safe/unsafe→spans is a follow-up adapter (v0 speaks one `{flagged}` JSON shape); the guard intercepts only the LLM text-egress path (screen never egresses, stt audio isn't a text-span filter); AUTO re-driving a released hold is deferred (the raw content is not retained — release marks it resolved + surfaces; the operator re-runs / switches to redact); local/cloud guard endpoints skipped in v0 (http/openai-compat only); no Settings UI to edit the policy yet (the PUT route + document exist). | | Entity contract v2 (evidence + sovereign overrides) | M5 #73 (built) | The additive Entity surface the resolver (#72, NOT YET BUILT) needs. Contract (`shared/contracts/src/records/entity.ts`): append-only optional `confidence`, resolution `state` (read by the #66 micro-state dot), `ambiguity` (plausible-rival marker), `heardAs[]` (source-/confidence-typed ASR variants that resolved here), `sightings[]` (typed heard/seen/calendar evidence trail), `overrides[]` (first-class user corrections that outrank scores), `external` (code-host/CRM linkage); sub-schemas registered in `AllSchemas`, `pnpm gen` regenerated (5 new schema JSONs). Store (`engine/store/workspaces.ts`): `upsertEntity` appends a typed sighting + an ASR heardAs per mention (append-only, deduped/idempotent); `mergeEntity` carries all v2 fields forward (a confirmed record STAYS confirmed). New `overrideEntity(ws,id,override)` — the SOVEREIGN write path: appends the override, stamps `state:'confirmed'`+`confidence:1`, pins the surface form as an alias; `findEntity` PREFERS a pinned mapping over any rival so resolution is deterministic and never re-decided against a settled rival (the store half of the #72 short-circuit). Populate: `distill/distiller.ts` records `via:'heard'` sightings + `stt` heardAs from the transcript window (the only LIVE producer). Render: `client relevant-now.ts` threads `entity.state` through the existing #66 dot (no new renderer). DISCLOSED/awaiting #72: state/confidence left absent by plain extraction (no scorer yet); per-variant ASR confidence not surfaced by STT (left undefined, never faked); seen/calendar sightings await their entity paths; `external` has no auto-populator yet; **#75 (BUILT):** `overrideEntity` is now wired to the `POST /teach/entity` route + the ≟ clarify affordance, and CLEARS `ambiguity` on settle (companion `clearEntityAmbiguity` settles the losing side of a disambiguate); `moveSession` does not migrate the v2 evidence trail. Tests: store (+3 evidence/override/migration), client relevant-now (+1 state dot), contracts (+1 v2 example; v1 example proves migration). **#91 QA** `distill/evidence-seam.test.ts` drives the population through the real distiller+`invokeLlm`: a resolved mention writes a `heard` sighting with the WINDOW-derived timestamp (`sighting.at === windowEnd`, distillate-tied) + a `stt` heardAs; the #74 crossSighting path (heard mangle + same-window OCR) records a `seen` sighting, confirms, and teaches the alias. | diff --git a/apps/engine/src/api/claims-e2e.test.ts b/apps/engine/src/api/claims-e2e.test.ts new file mode 100644 index 0000000..6994b8b --- /dev/null +++ b/apps/engine/src/api/claims-e2e.test.ts @@ -0,0 +1,117 @@ +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { Claim as ClaimSchema, type Claim, type RelatedEntity, type Session } from '@openinfo/contracts' +import { Value } from '@sinclair/typebox/value' +import { createSecureTestEngineApp, secureTestFetch, TEST_CONTROL_TOKEN } from './test-control-plane.js' + +const listen = async (app: ReturnType): Promise => { + await new Promise((resolve) => app.server.listen(0, resolve)) + const address = app.server.address() + assert.ok(address && typeof address === 'object') + return `http://127.0.0.1:${address.port}` +} + +const WS = 'ws-claims-api' + +test('#178: /claims is authenticated, builds idempotently, answers the query axes, walks relations, and honors sovereign corrections', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-api-')) + const app = createSecureTestEngineApp({ dataRoot: dir, log: () => undefined }) + const base = await listen(app) + try { + // Auth: every claims route lives inside the ordinary control-plane boundary — no side door. + assert.equal((await globalThis.fetch(`${base}/claims`)).status, 401, 'unauthenticated read refused') + assert.equal((await globalThis.fetch(`${base}/claims/related?entity=x`)).status, 401, 'unauthenticated walk refused') + const unauthBuild = await globalThis.fetch(`${base}/claims/build`, { + method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ workspaceId: WS, sessionId: 'x' }), + }) + assert.equal(unauthBuild.status, 401, 'unauthenticated build refused') + assert.equal( + (await globalThis.fetch(`${base}/claims`, { headers: { authorization: `Bearer ${TEST_CONTROL_TOKEN.slice(0, -1)}x` } })).status, + 401, + 'a wrong bearer is refused', + ) + + // A session with two entities co-mentioned in one moment — the minimal co-occurrence evidence. + const started = await secureTestFetch(`${base}/sessions`, { method: 'POST', body: JSON.stringify({ workspaceId: WS, modeId: 'mode-meeting' }) }) + assert.equal(started.status, 200) + const session = (await started.json()) as Session + const ada = app.store.upsertEntity({ workspaceId: WS, kind: 'person', name: 'Ada', seenAt: '2026-07-12T13:00:00.000Z' }) + const pidev = app.store.upsertEntity({ workspaceId: WS, kind: 'artifact', name: 'pi.dev', seenAt: '2026-07-12T13:00:00.000Z' }) + app.store.saveMoment({ + id: 'mom-1', sessionId: session.id, workspaceId: WS, at: '2026-07-12T13:00:01.000Z', kind: 'mention', + text: 'Ada is working on pi.dev', refs: [ada.id, pidev.id], source: 'mic', confidence: 0.9, + }) + + // Honest caller errors on the build route. + assert.equal((await secureTestFetch(`${base}/claims/build`, { method: 'POST', body: JSON.stringify({ workspaceId: WS }) })).status, 400) + assert.equal( + (await secureTestFetch(`${base}/claims/build`, { method: 'POST', body: JSON.stringify({ workspaceId: 'never-made', sessionId: session.id }) })).status, + 404, + ) + assert.equal( + (await secureTestFetch(`${base}/claims/build`, { method: 'POST', body: JSON.stringify({ workspaceId: WS, sessionId: 'no-such' }) })).status, + 404, + ) + + // Build: one co-occurrence claim; it validates and is a provisional derived proposal. + const build = await secureTestFetch(`${base}/claims/build`, { method: 'POST', body: JSON.stringify({ workspaceId: WS, sessionId: session.id }) }) + assert.equal(build.status, 200) + const created = (await build.json()) as Claim[] + assert.equal(created.length, 1) + const claim = created[0]! + assert.deepEqual([...Value.Errors(ClaimSchema, claim)], [], 'the served claim validates') + assert.equal(claim.relation, 'co-occurs-with') + assert.equal(claim.source, 'derived') + assert.equal(claim.state, 'provisional') + assert.ok(claim.evidence.length >= 1, 'every served claim is evidence-backed') + + // Idempotent: an immediate rebuild appends nothing — the honest empty array. + const rebuild = await secureTestFetch(`${base}/claims/build`, { method: 'POST', body: JSON.stringify({ workspaceId: WS, sessionId: session.id }) }) + assert.deepEqual(await rebuild.json(), [], 'nothing changed, nothing appended') + + // Query axes over the served route. + const read = async (query: string): Promise => { + const response = await secureTestFetch(`${base}/claims?workspace=${WS}${query}`) + assert.equal(response.status, 200) + return (await response.json()) as Claim[] + } + assert.deepEqual((await read('')).map((c) => c.id), [claim.id], 'default read = live head') + assert.deepEqual((await read(`&entity=${ada.id}`)).map((c) => c.id), [claim.id], 'entity axis (subject or object)') + assert.deepEqual((await read('&relation=works-on')), [], 'relation axis filters') + assert.deepEqual(await read('&entity=ent-nobody'), [], 'an unrelated entity walks to nothing') + assert.deepEqual(await (await secureTestFetch(`${base}/claims?workspace=never-made`)).json(), [], 'unknown workspace reads []') + + // The depth-1 relationship walk: Ada → pi.dev, carrying the backing claim id. + const walkRes = await secureTestFetch(`${base}/claims/related?workspace=${WS}&entity=${ada.id}`) + assert.equal(walkRes.status, 200) + const walk = (await walkRes.json()) as RelatedEntity[] + assert.equal(walk.length, 1) + assert.equal(walk[0]!.entityId, pidev.id) + assert.equal(walk[0]!.name, 'pi.dev') + assert.deepEqual(walk[0]!.claimIds, [claim.id], 'the walk always returns the supporting claim') + assert.equal((await secureTestFetch(`${base}/claims/related?workspace=${WS}`)).status, 400, 'the walk needs an ?entity=') + + // Sovereign user correction: confirm outranks the derived claim as the live head. + const correctRes = await secureTestFetch(`${base}/claims/correct`, { + method: 'POST', body: JSON.stringify({ workspaceId: WS, claimId: claim.id, verdict: 'confirm', by: 'the user' }), + }) + assert.equal(correctRes.status, 200) + const correction = (await correctRes.json()) as Claim + assert.equal(correction.source, 'user') + assert.equal(correction.state, 'confirmed') + assert.deepEqual((await read('')).map((c) => `${c.source}:${c.state}`), ['user:confirmed'], 'the sovereign confirmation is the live head') + assert.equal((await read('&superseded=true')).length, 2, 'the original derived inference is retained, never deleted') + + // Correcting an unknown claim is an honest 404. + assert.equal( + (await secureTestFetch(`${base}/claims/correct`, { method: 'POST', body: JSON.stringify({ workspaceId: WS, claimId: 'clm-nope', verdict: 'reject' }) })).status, + 404, + ) + } finally { + await app.close() + await rm(dir, { recursive: true, force: true }) + } +}) diff --git a/apps/engine/src/api/http.ts b/apps/engine/src/api/http.ts index d68e47a..a5883b9 100644 --- a/apps/engine/src/api/http.ts +++ b/apps/engine/src/api/http.ts @@ -2,14 +2,14 @@ import { randomUUID } from 'node:crypto' import { createServer, type IncomingMessage, type ServerResponse } from 'node:http' import type { Socket } from 'node:net' import { join } from 'node:path' -import { AllSchemas, Routes, STT_SEGMENT_SCHEMA_VERSION, type Ack, type BlockQuery, type BuildContextPacketsRequest, type BuildSummariesRequest, type Bundle, type CaptureChunk, type ContextPacket, type Summary, type ChatHistory, type ChatRequest, type ChatScreenshot, type CloneProfileRequest, type Draft, type Endpoint, type EndpointProbe, type Entity, type Fabric, type GenerateProbe, type FabricProfile, type Flag, type GuardPolicy, type ItemSignal, type LocalDownloadRequest, type Mode, type Moment, type OverflowState, type Pin, type QueueFailure, type QueueStatus, type PinChunk, type PromptTemplate, type Register, type RelevantEntity, type RerouteRequest, type SetSessionTitleRequest, type EntityCorrection, type EntityOverride, type ScanRequest, type ScreenCaptureObservation, type SecretValue, type SenseLaneSnapshot, type Session, type SessionTitling, type StartSessionRequest, type SttSegment, type SttSlotEndpoint, type Surface, type TodoList, type WorkflowSpec, type WorkspaceHints } from '@openinfo/contracts' +import { AllSchemas, Routes, STT_SEGMENT_SCHEMA_VERSION, type Ack, type BlockQuery, type BuildContextPacketsRequest, type BuildSummariesRequest, type BuildClaimsRequest, type CorrectClaimRequest, type Claim, type RelatedEntity, type Bundle, type CaptureChunk, type ContextPacket, type Summary, type ChatHistory, type ChatRequest, type ChatScreenshot, type CloneProfileRequest, type Draft, type Endpoint, type EndpointProbe, type Entity, type Fabric, type GenerateProbe, type FabricProfile, type Flag, type GuardPolicy, type ItemSignal, type LocalDownloadRequest, type Mode, type Moment, type OverflowState, type Pin, type QueueFailure, type QueueStatus, type PinChunk, type PromptTemplate, type Register, type RelevantEntity, type RerouteRequest, type SetSessionTitleRequest, type EntityCorrection, type EntityOverride, type ScanRequest, type ScreenCaptureObservation, type SecretValue, type SenseLaneSnapshot, type Session, type SessionTitling, type StartSessionRequest, type SttSegment, type SttSlotEndpoint, type Surface, type TodoList, type WorkflowSpec, type WorkspaceHints } from '@openinfo/contracts' import { SESSION_TITLING_SCHEMA_VERSION } from '@openinfo/contracts' import { Actor, ActDocuments, TodoDocuments, TaskExtractor } from '../act/index.js' import { EventBus, type EngineEvents } from '../bus/index.js' import { DistillDocuments, Distiller, DistillCadence, DEFAULT_DISTILL_CADENCE_MS, FieldValueStore, FastFieldScheduler, JudgeScheduler, FloorTitleScheduler, transcribeChunks, isAudioChunk, buildTranscriptUpdates, TranscriptRing, EchoDedupe, echoDedupeEnabled, ECHO_DEDUPE_WINDOW_MS, type DistillOptions } from '../distill/index.js' import { GuardDocuments, GuardHoldStore } from '../guard/index.js' import { DiscoveryDocuments, FabricDocuments, FileSecretStore, GuardHeldError, LocalModelStore, LocalRuntimeManager, StarterModelsDocuments, checkEndpoint, discoverFabric, invokeLlm, invokeOcr, invokeStt, describeInvokeFailure, enrichFailureHint, resolveEgress, scanHosts, toQueueFailure, DEFAULT_NO_SPEECH_THRESHOLD, type SecretStore } from '../fabric/index.js' -import { relevantNow, ingestPin, defaultFetchers, materializeContextPackets, PacketBuildLog, materializeSummaries, createFabricSummarizer, SummaryBuildLog, walkSummaryTrace, type Summarizer } from '../index/index.js' +import { relevantNow, ingestPin, defaultFetchers, materializeContextPackets, PacketBuildLog, materializeSummaries, createFabricSummarizer, SummaryBuildLog, walkSummaryTrace, materializeClaims, ClaimBuildLog, type Summarizer } from '../index/index.js' import { TeachStore, deriveHintCandidates, captureEntityCorrection, type HintCandidate } from '../teach/index.js' import { Attributor, HintsDocuments, extractFocusSignals, rerouteSession } from '../route/index.js' import { isFlagEnabled } from '../flags/read.js' @@ -113,6 +113,8 @@ interface HandlerContext { packetBuildLog: PacketBuildLog /** The live hierarchical-summary producer's build log (#177) — per (session, level) last-update signal (process-scoped). */ summaryBuildLog: SummaryBuildLog + /** The live Claim producer's build log (#178) — the diagnostics "last update" signal (process-scoped). */ + claimBuildLog: ClaimBuildLog /** The fabric-backed summarizer the on-demand /summaries/build route + the live seams share (#177). */ summarize: Summarizer /** The durable per-field latest values (#61) — read by the Audit ledger + Trace sections (#116). */ @@ -822,6 +824,9 @@ export function createEngineApp(options: EngineOptions): EngineApp { // #176 live ContextPacket producer: the in-memory build log the session-end seam and the on-demand route // both record to, and the Context-packets diagnostics reads for its honest "last update" line (process-scoped). const packetBuildLog = new PacketBuildLog() + // #178 live Claim producer: the in-memory build log the session-end seam and the on-demand route both + // record to (process-scoped), mirroring packetBuildLog. + const claimBuildLog = new ClaimBuildLog() bus.subscribe('session.ended', (session) => { void (async () => { @@ -856,6 +861,16 @@ export function createEngineApp(options: EngineOptions): EngineApp { const outcome = materializeContextPackets({ store, log: packetBuildLog }, { workspaceId: session.workspaceId, sessionId: session.id, trigger: 'session-end' }) if (outcome.error !== undefined) log(`context packets: session-end build failed for ${session.id}: ${outcome.error}`) else if (outcome.created.length > 0) log(`context packets: session-end appended ${outcome.created.length} for session ${session.id}`) + // #178 LIVE PRODUCER: with the packets now materialized, derive co-occurrence Claims from the session's + // converged evidence (its packets + moments) so relationships materialize during normal capture — no + // on-demand route call needed. Runs AFTER packets (so the packet-candidate co-occurrence evidence is + // fresh), ONCE per session end, scoped to the session's OWN workspace, and idempotent (a converged + // rebuild appends nothing). Same contained discipline as packets: materializeClaims never throws — a + // build failure is recorded on the log (visible in diagnostics), never fatal to this handler and never + // able to corrupt the durable evidence it derives from. No evidence ⇒ no claims (never fabricated). + const claimOutcome = materializeClaims({ store, log: claimBuildLog }, { workspaceId: session.workspaceId, sessionId: session.id, trigger: 'session-end' }) + if (claimOutcome.error !== undefined) log(`claims: session-end build failed for ${session.id}: ${claimOutcome.error}`) + else if (claimOutcome.created.length > 0) log(`claims: session-end appended ${claimOutcome.created.length} for session ${session.id}`) // #177 DURABLE SESSION SUMMARY: with the whole session now drained + summarized at the finer levels, flush // the coarser levels — a final rolling/five-minute pass (idempotent) then the durable SESSION summary over // the five-minute heads, the durable SESSION summary, and finally the cross-session PROJECT summary that @@ -948,7 +963,7 @@ export function createEngineApp(options: EngineOptions): EngineApp { void refreshSenseLaneGates() const server = createServer((req, res) => { - const ctx: HandlerContext = { bus, fabric, discovery, secrets, voice, surfaces, distill: distillDocs, guardDocs, guardHolds, todos: todoDocs, workflow, bundles, presets, hints: hintsDocs, queue, textQueue, senseLanes, transcripts, packetBuildLog, summaryBuildLog, summarize, fieldValues, store, runtime, models, controlPlane: options.controlPlane, browserAuth, log } + const ctx: HandlerContext = { bus, fabric, discovery, secrets, voice, surfaces, distill: distillDocs, guardDocs, guardHolds, todos: todoDocs, workflow, bundles, presets, hints: hintsDocs, queue, textQueue, senseLanes, transcripts, packetBuildLog, summaryBuildLog, claimBuildLog, summarize, fieldValues, store, runtime, models, controlPlane: options.controlPlane, browserAuth, log } if (options.onCapture !== undefined) ctx.onCapture = options.onCapture void handleControlRequest(req, res, ctx).catch((error: unknown) => { if (res.headersSent) return void res.destroy(error instanceof Error ? error : undefined) @@ -1215,6 +1230,10 @@ async function handle(req: IncomingMessage, res: ServerResponse, ctx: HandlerCon if (req.method === 'GET' && url.pathname === '/summaries') return send(res, 200, readSummaries(ctx.store, url)) if (req.method === 'GET' && url.pathname === '/summaries/trace') return readSummaryTrace(res, ctx.store, url) if (req.method === 'POST' && url.pathname === '/summaries/build') return buildSummaries(req, res, ctx) + if (req.method === 'GET' && url.pathname === '/claims') return send(res, 200, readClaims(ctx.store, url)) + if (req.method === 'GET' && url.pathname === '/claims/related') return readRelatedEntities(res, ctx.store, url) + if (req.method === 'POST' && url.pathname === '/claims/build') return buildClaims(req, res, ctx) + if (req.method === 'POST' && url.pathname === '/claims/correct') return correctClaim(req, res, ctx) if (req.method === 'GET' && url.pathname === '/todos') return send(res, 200, ctx.todos.list()) const todo = url.pathname.match(/^\/todos\/([^/]+)$/) if (req.method === 'GET' && todo?.[1]) return getTodo(res, ctx, decodeURIComponent(todo[1])) @@ -2093,6 +2112,100 @@ async function buildSummaries(req: IncomingMessage, res: ServerResponse, ctx: Ha send(res, 200, outcome.created) } +/** + * Serve evidence-backed Claims (#178) — queryable by workspace (default `default`), session, related entity + * (`entity` matches subject OR object — the depth-1 walk data), relation kind, source, and time window + * (`from`/`to`, observed-span intersection), consistent with GET /context/packets. Default reads return the + * LIVE heads only — supersession AND sovereign user corrections are resolved in the store BEFORE the filters, + * so a filtered read never shows a superseded revision or a user-overruled pair as live. `superseded=true` + * returns the whole append-only history. A read over store/; an unknown workspace is an empty list, not an error. + */ +function readClaims(store: WorkspaceRegistry, url: URL): Claim[] { + const workspaceId = url.searchParams.get('workspace') ?? 'default' + const sessionId = url.searchParams.get('session') + const entityId = url.searchParams.get('entity') + const relation = url.searchParams.get('relation') + const source = url.searchParams.get('source') + const from = url.searchParams.get('from') + const to = url.searchParams.get('to') + return store.listClaims(workspaceId, { + ...(sessionId !== null ? { sessionId } : {}), + ...(entityId !== null ? { entityId } : {}), + ...(relation !== null ? { relation: relation as Claim['relation'] } : {}), + ...(source !== null ? { source: source as Claim['source'] } : {}), + ...(from !== null ? { from } : {}), + ...(to !== null ? { to } : {}), + ...(url.searchParams.get('superseded') === 'true' ? { includeSuperseded: true } : {}), + }) +} + +/** + * Walk one entity's depth-1 relationships (#178): `GET /claims/related?entity=` answers "what does this + * entity relate to?" — the counterpart entities at the other end of its live claims, each with the deduped + * relation kinds, strongest backing confidence, and the claim ids that back the link (so the caller can + * always retrieve the supporting evidence + scope). The shape #181's typeahead consumes. `?entity=` is + * required (a walk needs a root); an unknown workspace/entity is an honest empty list, not an error. + */ +function readRelatedEntities(res: ServerResponse, store: WorkspaceRegistry, url: URL): void { + const workspaceId = url.searchParams.get('workspace') ?? 'default' + const entityId = url.searchParams.get('entity') + if (entityId === null || entityId === '') return send(res, 400, { error: 'related-entities walk needs an ?entity=' }) + send(res, 200, store.relatedEntities(workspaceId, entityId) satisfies RelatedEntity[]) +} + +/** + * Run the deterministic co-occurrence Claim builder (#178) over one session's stored evidence and persist + * what it appends. The response is the APPENDED claims only — an empty array is the honest idempotent no-op + * (same evidence ⇒ same claims ⇒ nothing written; no evidence ⇒ no claims). Route decides (404s), the SHARED + * producer moves — the SAME materialize seam the live session-end producer uses, so both paths build + * identically and record on the build log. A contained build failure returns as `error` (500), never a + * silent empty success. + */ +async function buildClaims(req: IncomingMessage, res: ServerResponse, ctx: HandlerContext): Promise { + const body = await readJson(req) + const errors = validationErrors('BuildClaimsRequest', body) + if (errors.length > 0) return send(res, 400, { error: 'invalid BuildClaimsRequest', details: errors }) + const { workspaceId, sessionId } = body as BuildClaimsRequest + if (!ctx.store.all().some((ws) => ws.id === workspaceId)) return send(res, 404, { error: `no such workspace: ${workspaceId}` }) + const session = ctx.store.getSession(workspaceId, sessionId) + if (!session) return send(res, 404, { error: `no session ${sessionId} in workspace ${workspaceId}` }) + const outcome = materializeClaims({ store: ctx.store, log: ctx.claimBuildLog }, { workspaceId, sessionId, trigger: 'on-demand' }) + if (outcome.error !== undefined) return send(res, 500, { error: `claim build failed: ${outcome.error}` }) + if (outcome.created.length > 0) { + ctx.log(`claims: appended ${outcome.created.length} for session ${sessionId} (${outcome.unchanged.length} unchanged)`) + } + send(res, 200, outcome.created) +} + +/** + * Record a SOVEREIGN user correction on a derived claim (#178) — append-only, outranks the derived claim, + * never deletes it or its evidence. Route decides (404 for an unknown claim), the store mints the + * `source:'user'` correction (carrying the target's evidence, so it stays traceable). Returns the persisted + * correction. A `correct` verdict may assert a different relation/object; confirm/reject inherit the target's. + */ +async function correctClaim(req: IncomingMessage, res: ServerResponse, ctx: HandlerContext): Promise { + const body = await readJson(req) + const errors = validationErrors('CorrectClaimRequest', body) + if (errors.length > 0) return send(res, 400, { error: 'invalid CorrectClaimRequest', details: errors }) + const { workspaceId, claimId, verdict, relation, object, by, note } = body as CorrectClaimRequest + if (!ctx.store.all().some((ws) => ws.id === workspaceId)) return send(res, 404, { error: `no such workspace: ${workspaceId}` }) + try { + const claim = ctx.store.recordClaimCorrection({ + workspaceId, + claimId, + verdict: verdict as 'confirm' | 'reject' | 'correct', + ...(relation !== undefined ? { relation: relation as Claim['relation'] } : {}), + ...(object !== undefined ? { object } : {}), + ...(by !== undefined ? { by } : {}), + ...(note !== undefined ? { note } : {}), + }) + ctx.log(`claims: recorded ${verdict} correction on ${claimId} in workspace ${workspaceId}`) + send(res, 200, claim satisfies Claim) + } catch (error) { + send(res, 404, { error: error instanceof Error ? error.message : String(error) }) + } +} + /** * Serve a session's to-do document by session id — the read seam the HUD renders (a resource route, * no flag, mirroring GET /drafts and GET /layouts/surfaces/:id; the DATA is gated upstream by act.tasks diff --git a/apps/engine/src/index/claims-replay.test.ts b/apps/engine/src/index/claims-replay.test.ts new file mode 100644 index 0000000..a79a55c --- /dev/null +++ b/apps/engine/src/index/claims-replay.test.ts @@ -0,0 +1,99 @@ +import { mkdtemp, rm } from 'node:fs/promises' +import { join } from 'node:path' +import { tmpdir } from 'node:os' +import { test } from 'node:test' +import assert from 'node:assert/strict' +import type { Claim, ContextPacket, Entity, Moment } from '@openinfo/contracts' +import { WorkspaceRegistry } from '../store/index.js' +import { buildClaims } from './claims.js' + +/** + * #178 determinism proof: the SAME deterministic evidence built twice — into two fresh, isolated stores — + * yields byte-identical persisted Claims (same content-derived ids), and a rebuild over an already-built + * store appends nothing (idempotence). No model, capture, or network is touched: claims are a PURE function + * of the converged evidence (packets + moments + entities), so the guarantee is structural, mirroring the + * #176/#177 replay pattern. (Entity ids are hand-fixed here because the store mints entity ids randomly; + * the property under test is the builder + store round-trip, which must be byte-stable.) + */ + +const WS = 'workspace-synthetic' +const SES = 'session-synthetic' +const REPLAY_CLOCK = () => new Date('2026-07-12T13:00:05.000Z') + +const entity = (id: string): Entity => ({ + id, workspaceId: WS, kind: 'person', name: id, aliases: [], momentRefs: [], outboundCount: 0, + firstSeen: '2026-07-12T13:00:00.000Z', lastSeen: '2026-07-12T13:00:00.000Z', +}) +const moment = (id: string, refs: string[], at: string): Moment => ({ + id, sessionId: SES, workspaceId: WS, at, kind: 'mention', text: 'x', refs, source: 'mic', confidence: 0.9, +}) +const packet = (id: string, candidateIds: string[], windowStart: string): ContextPacket => ({ + id, workspaceId: WS, sessionId: SES, windowStart, windowEnd: windowStart, + microphone: [], systemAudio: [], screen: [], + candidates: candidateIds.map((entityId) => ({ entityId, name: entityId, momentRefs: ['m'] })), + gaps: [], confidence: 0.4, provenance: { builder: 'deterministic-correlation', windowMs: 60000 }, + revision: 1, schemaVersion: 1, createdAt: windowStart, +}) + +const EVIDENCE = { + workspaceId: WS, + sessionId: SES, + packets: [packet('cp-0001', ['ent-ada', 'ent-pidev'], '2026-07-12T13:00:00.000Z')], + moments: [moment('moment-0001', ['ent-pidev', 'ent-ada'], '2026-07-12T13:00:04.000Z')], + entities: [entity('ent-ada'), entity('ent-pidev')], + now: REPLAY_CLOCK, +} + +/** Build the claims into a fresh store and read them back through the store's live-head resolution. */ +const buildAndPersist = async (dir: string): Promise => { + const store = new WorkspaceRegistry(dir) + try { + const result = buildClaims({ ...EVIDENCE, existing: store.listClaims(WS, { sessionId: SES, source: 'derived', includeSuperseded: true }) }) + assert.equal(result.created.length, 1, 'the evidence converges into exactly one co-occurrence claim') + for (const claim of result.created) store.saveClaim(claim) + + // Idempotence in the SAME store: an immediate rebuild over the persisted chain appends nothing. + const again = buildClaims({ + ...EVIDENCE, + existing: store.listClaims(WS, { sessionId: SES, source: 'derived', includeSuperseded: true }), + now: () => new Date('2099-01-01T00:00:00.000Z'), // a different clock cannot change a no-op + }) + assert.equal(again.created.length, 0, 'rebuild over the same evidence appends nothing') + + return store.listClaims(WS, { sessionId: SES, includeSuperseded: true }) + } finally { + store.close() + } +} + +test('#178 replay: the same evidence built into two fresh stores yields byte-identical Claims, idempotently', async () => { + const dirA = await mkdtemp(join(tmpdir(), 'openinfo-claims-replay-a-')) + const dirB = await mkdtemp(join(tmpdir(), 'openinfo-claims-replay-b-')) + try { + const first = await buildAndPersist(dirA) + const second = await buildAndPersist(dirB) + + assert.equal(JSON.stringify(first), JSON.stringify(second), 'build × 2 ⇒ byte-identical claims') + + const claim = first[0]! + assert.equal(claim.subject, 'ent-ada', 'symmetric endpoints canonically ordered by id') + assert.equal(claim.object, 'ent-pidev') + assert.equal(claim.relation, 'co-occurs-with') + assert.equal(claim.provenance?.evidenceCount, 2, 'a packet + a moment back the pair') + assert.equal(claim.confidence, 0.6) + assert.equal(claim.revision, 1) + assert.equal(claim.createdAt, '2026-07-12T13:00:05.000Z', 'createdAt = the replay clock') + + // Refs stay refs: nothing observed was copied onto the claim. + const bytes = JSON.stringify(claim) + assert.ok(!bytes.includes('"text"'), 'no evidence content copied onto the claim') + assert.deepEqual( + claim.evidence.map((e) => e.record).sort(), + ['context-packet', 'moment'], + 'the claim points at its evidence records by id only', + ) + } finally { + await rm(dirA, { recursive: true, force: true }) + await rm(dirB, { recursive: true, force: true }) + } +}) diff --git a/apps/engine/src/index/claims.test.ts b/apps/engine/src/index/claims.test.ts new file mode 100644 index 0000000..42b7014 --- /dev/null +++ b/apps/engine/src/index/claims.test.ts @@ -0,0 +1,154 @@ +import { test } from 'node:test' +import assert from 'node:assert/strict' +import type { Claim, ContextPacket, Entity, Moment } from '@openinfo/contracts' +import { buildClaims } from './claims.js' + +/** + * #178 slice 1 — the deterministic co-occurrence Claim builder, as a PURE function of its inputs (no DB, no + * model, no clock beyond the injectable `now`). These pin the honest guarantees: co-occurrence is derived + * ONLY from converged evidence, a pair with no evidence yields NO claim, confidence is a fixed inspectable + * map, and the append-only supersession/idempotence is byte-stable. + */ + +const WS = 'ws-claims' +const SES = 'ses-1' +const FIXED = () => new Date('2026-07-12T13:00:05.000Z') + +const entity = (id: string): Entity => ({ + id, workspaceId: WS, kind: 'person', name: id, aliases: [], momentRefs: [], outboundCount: 0, + firstSeen: '2026-07-12T13:00:00.000Z', lastSeen: '2026-07-12T13:00:00.000Z', +}) + +const moment = (id: string, refs: string[], at: string): Moment => ({ + id, sessionId: SES, workspaceId: WS, at, kind: 'mention', text: 'x', refs, source: 'mic', confidence: 0.9, +}) + +const packet = (id: string, candidateIds: string[], windowStart: string): ContextPacket => ({ + id, workspaceId: WS, sessionId: SES, windowStart, windowEnd: windowStart, + microphone: [], systemAudio: [], screen: [], + candidates: candidateIds.map((entityId) => ({ entityId, name: entityId, momentRefs: ['m'] })), + gaps: [], confidence: 0.4, provenance: { builder: 'deterministic-correlation', windowMs: 60000 }, + revision: 1, schemaVersion: 1, createdAt: windowStart, +}) + +test('derives one co-occurs-with claim from a moment naming two entities (the only relation without a model)', () => { + const { created } = buildClaims({ + workspaceId: WS, sessionId: SES, packets: [], moments: [moment('m1', ['ent-b', 'ent-a'], '2026-07-12T13:00:00.000Z')], + entities: [entity('ent-a'), entity('ent-b')], existing: [], now: FIXED, + }) + assert.equal(created.length, 1) + const c = created[0]! + assert.equal(c.relation, 'co-occurs-with') + assert.equal(c.subject, 'ent-a', 'endpoints canonically ordered by id (symmetric relation)') + assert.equal(c.object, 'ent-b') + assert.equal(c.source, 'derived') + assert.equal(c.state, 'provisional') + assert.equal(c.confidence, 0.4, 'one evidence ref ⇒ 0.4 (fixed map)') + assert.deepEqual(c.evidence, [{ record: 'moment', id: 'm1', at: '2026-07-12T13:00:00.000Z' }]) + assert.equal(c.provenance?.evidenceCount, 1) + assert.equal(c.provenance?.builder, 'deterministic-cooccurrence') + assert.equal(c.revision, 1) + assert.equal(c.supersedes, undefined) + assert.equal(c.createdAt, '2026-07-12T13:00:05.000Z', 'createdAt = the injected clock') + assert.ok(c.id.startsWith('clm-')) +}) + +test('two independent evidence records (a packet + a moment) strengthen confidence via a recorded derivation', () => { + const { created } = buildClaims({ + workspaceId: WS, sessionId: SES, + packets: [packet('cp1', ['ent-a', 'ent-b'], '2026-07-12T13:00:00.000Z')], + moments: [moment('m1', ['ent-a', 'ent-b'], '2026-07-12T13:00:04.000Z')], + entities: [entity('ent-a'), entity('ent-b')], existing: [], now: FIXED, + }) + assert.equal(created.length, 1) + const c = created[0]! + assert.equal(c.provenance?.evidenceCount, 2) + assert.equal(c.confidence, 0.6, 'two evidence refs ⇒ 0.6') + assert.deepEqual(c.evidence.map((e) => e.record), ['context-packet', 'moment'], 'evidence sorted by instant') + assert.equal(c.firstObserved, '2026-07-12T13:00:00.000Z') + assert.equal(c.lastObserved, '2026-07-12T13:00:04.000Z') +}) + +test('NO evidence ⇒ NO claim (honest degradation, never fabricated)', () => { + // A single entity named alone, and a dangling ref to an entity with no record — neither is co-occurrence. + const { created } = buildClaims({ + workspaceId: WS, sessionId: SES, packets: [], moments: [moment('m1', ['ent-a', 'ent-ghost'], '2026-07-12T13:00:00.000Z')], + entities: [entity('ent-a')], existing: [], now: FIXED, + }) + assert.deepEqual(created, [], 'a dangling ref has no record to trace to — no pair, no claim') +}) + +test('confidence is capped below 1.0 — a derived claim is a proposal, never certain', () => { + const at = (n: number) => `2026-07-12T13:0${n}:00.000Z` + const packets = [0, 1, 2, 3, 4].map((n) => packet(`cp${n}`, ['ent-a', 'ent-b'], at(n))) + const { created } = buildClaims({ + workspaceId: WS, sessionId: SES, packets, moments: [], entities: [entity('ent-a'), entity('ent-b')], existing: [], now: FIXED, + }) + assert.equal(created[0]!.provenance?.evidenceCount, 5) + assert.equal(created[0]!.confidence, 0.85, '5 evidence refs ⇒ the 0.85 cap (1.0 is reserved for a user confirmation)') +}) + +test('idempotent: a rebuild over the same evidence appends nothing and keeps the existing claim byte-identical', () => { + const input = { + workspaceId: WS, sessionId: SES, packets: [], moments: [moment('m1', ['ent-a', 'ent-b'], '2026-07-12T13:00:00.000Z')], + entities: [entity('ent-a'), entity('ent-b')], now: FIXED, + } + const first = buildClaims({ ...input, existing: [] }) + const second = buildClaims({ ...input, existing: first.created, now: () => new Date('2099-01-01T00:00:00.000Z') }) + assert.equal(second.created.length, 0, 'a different clock cannot change a no-op') + assert.equal(second.unchanged.length, 1) + assert.equal(JSON.stringify(second.unchanged[0]), JSON.stringify(first.created[0])) +}) + +test('append-only supersession: MORE evidence appends a new revision that supersedes the prior (never mutates it)', () => { + const base = { + workspaceId: WS, sessionId: SES, packets: [], entities: [entity('ent-a'), entity('ent-b')], now: FIXED, + } + const first = buildClaims({ ...base, moments: [moment('m1', ['ent-a', 'ent-b'], '2026-07-12T13:00:00.000Z')], existing: [] }) + const second = buildClaims({ + ...base, + moments: [moment('m1', ['ent-a', 'ent-b'], '2026-07-12T13:00:00.000Z'), moment('m2', ['ent-a', 'ent-b'], '2026-07-12T13:00:10.000Z')], + existing: first.created, + }) + assert.equal(second.created.length, 1) + const rev2 = second.created[0]! + assert.equal(rev2.revision, 2) + assert.equal(rev2.supersedes, first.created[0]!.id) + assert.equal(rev2.provenance?.evidenceCount, 2) + assert.notEqual(rev2.id, first.created[0]!.id, 'a new revision is a new content-derived id') +}) + +test('byte-identical: the same inputs built twice yield byte-identical claims (deterministic)', () => { + const mk = () => buildClaims({ + workspaceId: WS, sessionId: SES, + packets: [packet('cp1', ['ent-a', 'ent-b', 'ent-c'], '2026-07-12T13:00:00.000Z')], + moments: [moment('m1', ['ent-c', 'ent-a'], '2026-07-12T13:00:04.000Z')], + entities: [entity('ent-a'), entity('ent-b'), entity('ent-c')], existing: [], now: FIXED, + }) + assert.equal(JSON.stringify(mk().created), JSON.stringify(mk().created)) + // three entities in one packet ⇒ all three unordered pairs; refs never carry copied content. + const created = mk().created + assert.equal(created.length, 3, 'C(3,2) = 3 co-occurrence pairs') + const bytes = JSON.stringify(created) + assert.ok(!bytes.includes('"text"'), 'no evidence content copied onto a claim') +}) + +test('claims are workspace-isolated and session-scoped — cross-session evidence never bleeds into one claim', () => { + const { created } = buildClaims({ + workspaceId: WS, sessionId: SES, packets: [], + moments: [moment('m1', ['ent-a', 'ent-b'], '2026-07-12T13:00:00.000Z'), { ...moment('m2', ['ent-a', 'ent-b'], '2026-07-12T13:00:10.000Z'), sessionId: 'other-session' }], + entities: [entity('ent-a'), entity('ent-b')], existing: [], now: FIXED, + }) + assert.equal(created.length, 1) + assert.equal(created[0]!.provenance?.evidenceCount, 1, "only this session's moment is evidence") +}) + +test('the produced claim satisfies the append-only Claim contract shape it will be stored under', () => { + const { created } = buildClaims({ + workspaceId: WS, sessionId: SES, packets: [], moments: [moment('m1', ['ent-a', 'ent-b'], '2026-07-12T13:00:00.000Z')], + entities: [entity('ent-a'), entity('ent-b')], existing: [], now: FIXED, + }) + const c: Claim = created[0]! + assert.equal(c.schemaVersion, 1) + assert.ok(c.evidence.length >= 1, 'evidence is mandatory') +}) diff --git a/apps/engine/src/index/claims.ts b/apps/engine/src/index/claims.ts new file mode 100644 index 0000000..626d2bc --- /dev/null +++ b/apps/engine/src/index/claims.ts @@ -0,0 +1,223 @@ +import { createHash } from 'node:crypto' +import type { Claim, ClaimEvidenceRef, ContextPacket, Entity, Moment } from '@openinfo/contracts' +import { CLAIM_SCHEMA_VERSION } from '@openinfo/contracts' + +/** + * The deterministic Claim builder (#178 slice 1) — engine-side CO-OCCURRENCE correlation over + * ALREADY-STORED evidence, extending the #74 correlator and the #176 ContextPacket from a converged window + * to a durable RELATIONSHIP. Like every module in index/, the core is a PURE function of its inputs: no DB, + * no model, no clock beyond the injectable `now` — so it is fixture-testable, and replaying the same + * evidence yields byte-identical claims (the #32 record/replay guarantee). + * + * WHAT IT DERIVES (and, honestly, what it does NOT): the ONLY relationship co-occurrence evidence can + * assert without a model is `co-occurs-with` — two entities were OBSERVED TOGETHER. It reads that from the + * evidence the substrate already converged: + * - a ContextPacket whose `candidates` name BOTH entities in one window (#176 already did the cross-source + * correlation work — the claim builder lifts it from a window to a durable pair), and + * - a moment whose `refs` name BOTH entities in one extracted moment. + * The SEMANTIC kinds (works-on / belongs-to / authored / member-of / relates-to) require judgment; they are + * a later JUDGE-ENRICHMENT slice (a proposal per #189) and sovereign user corrections — NOT this slice. So + * this builder emits exactly one relation kind, and never invents a reading of HOW two entities relate. + * + * EVIDENCE IS MANDATORY (honest degradation): a pair with NO co-occurrence evidence yields NO claim — a + * relationship is never fabricated. `evidenceCount` (distinct evidence refs) is the RECORDED DERIVATION + * behind `confidence`: repeated co-occurrence strengthens a claim only through a NEW revision that records + * the higher count, never a silent mutation (#178 AC). + * + * SOURCE IDENTITY / REFS-ONLY: nothing is copied out of an evidence record beyond its id + instant; the + * claim points at immutable records, so every assertion stays traceable to source observations. + * + * SUPERSESSION is append-only: when a rebuild sees MORE evidence for a (subject, object, relation) that + * already has a derived claim, it appends a NEW claim with `revision + 1` and `supersedes` naming the prior + * — the prior is never mutated. When the evidence set is UNCHANGED the existing claim is kept untouched + * (idempotence): ids are content-derived, so "same evidence in ⇒ same claim out" holds bit-for-bit. The + * builder reads and writes ONLY `source:'derived'` claims; sovereign user corrections are resolved OVER the + * derived chain at read time (the store's `resolveClaimHeads`), so re-deriving a corrected pair can never + * defeat the correction. + */ + +export interface ClaimBuilderConfig { + /** Max distinct entities considered per evidence record, so a pathological moment/packet cannot emit O(n^2) pairs. */ + maxEntitiesPerRecord: number +} + +export const DEFAULT_CLAIM_BUILDER_CONFIG: ClaimBuilderConfig = { + maxEntitiesPerRecord: 24, +} + +/** + * Deterministic co-occurrence confidence from the count of DISTINCT evidence refs backing the pair — each + * additional independent observation corroborates the relationship (the #74/#176 design rule, lifted to a + * durable claim). A fixed, inspectable map, CAPPED below 1.0: a derived claim is a proposal, never certain + * — 1.0 is reserved for a sovereign user confirmation. Never a model score, never fabricated. + */ +const COOCCURRENCE_CONFIDENCE: Record = { 1: 0.4, 2: 0.6, 3: 0.75, 4: 0.82 } +const COOCCURRENCE_CONFIDENCE_CAP = 0.85 +const cooccurrenceConfidence = (evidenceCount: number): number => + evidenceCount <= 0 ? 0 : (COOCCURRENCE_CONFIDENCE[evidenceCount] ?? COOCCURRENCE_CONFIDENCE_CAP) + +export interface ClaimBuildInput { + workspaceId: string + sessionId: string + /** The session's live ContextPacket heads — a packet whose candidates name two entities is co-occurrence evidence. */ + packets: readonly ContextPacket[] + /** The session's moments — a moment whose refs name two entities is co-occurrence evidence. */ + moments: readonly Moment[] + /** Workspace entities — a candidate/ref id with no entity record is a dangling ref, never claimed. */ + entities: readonly Entity[] + /** The session's existing DERIVED claim chain — idempotence and supersession are decided against it. */ + existing: readonly Claim[] + config?: ClaimBuilderConfig + /** Injectable clock for `createdAt` on NEWLY appended claims (fixture replay hands in the replay clock). */ + now?: () => Date +} + +export interface ClaimBuildResult { + /** Claims this run APPENDED (new relationships + new supersession revisions). Empty ⇒ idempotent no-op. */ + created: Claim[] + /** Existing derived heads whose evidence rebuilt identical — kept untouched, byte-for-byte. */ + unchanged: Claim[] +} + +/** JSON with code-point-sorted object keys — the canonical form the content-derived claim id hashes. */ +const canonicalJson = (value: unknown): string => { + if (Array.isArray(value)) return `[${value.map(canonicalJson).join(',')}]` + if (value !== null && typeof value === 'object') { + const entries = Object.entries(value as Record) + .filter(([, v]) => v !== undefined) + .sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0)) + return `{${entries.map(([k, v]) => `${JSON.stringify(k)}:${canonicalJson(v)}`).join(',')}}` + } + return JSON.stringify(value) +} + +/** Everything id/chain-position/creation-time are derived FROM — the comparable content of one relationship. */ +interface ClaimContent { + workspaceId: string + subject: string + object: string + relation: 'co-occurs-with' + evidence: ClaimEvidenceRef[] + confidence: number + source: 'derived' + state: 'provisional' + provenance: { builder: 'deterministic-cooccurrence'; evidenceCount: number } + sessionId: string + firstObserved: string + lastObserved: string + schemaVersion: number +} + +/** Content-derived claim id: a hash over the canonical relationship content + chain position. */ +const claimId = (content: ClaimContent, revision: number, supersedes: string | undefined): string => + `clm-${createHash('sha256') + .update(canonicalJson({ ...content, revision, ...(supersedes !== undefined ? { supersedes } : {}) })) + .digest('hex') + .slice(0, 32)}` + +/** Deterministic evidence order: by instant, then record, then id — so claim bytes never depend on read order. */ +const byEvidence = (a: ClaimEvidenceRef, b: ClaimEvidenceRef): number => + a.at < b.at ? -1 : a.at > b.at ? 1 : a.record < b.record ? -1 : a.record > b.record ? 1 : a.id < b.id ? -1 : a.id > b.id ? 1 : 0 + +/** The pair key a relationship's supersession chain is keyed by — subject|object|relation. */ +const pairKey = (subject: string, object: string, relation: string): string => `${subject}\u0000${object}\u0000${relation}` + +/** The latest (not-superseded) DERIVED claim per pair key among a session's existing chain. */ +const latestByPair = (existing: readonly Claim[]): Map => { + const derived = existing.filter((c) => c.source === 'derived') + const superseded = new Set(derived.map((c) => c.supersedes).filter((id): id is string => id !== undefined)) + const latest = new Map() + for (const claim of derived) { + if (superseded.has(claim.id)) continue + const key = pairKey(claim.subject, claim.object, claim.relation) + const prior = latest.get(key) + if (prior === undefined || claim.revision > prior.revision) latest.set(key, claim) + } + return latest +} + +/** Distinct entity ids named by an evidence record, bounded and sorted — the co-occurrence participants. */ +const participants = (ids: readonly string[], known: ReadonlySet, cap: number): string[] => + [...new Set(ids.filter((id) => known.has(id)))].sort().slice(0, cap) + +/** + * Build (or converge) the session's co-occurrence Claims from its stored evidence. Pure — the caller does + * all reads and writes. Returns only appended + kept claims; it never mutates `existing` members. + */ +export const buildClaims = (input: ClaimBuildInput): ClaimBuildResult => { + const config = input.config ?? DEFAULT_CLAIM_BUILDER_CONFIG + const now = input.now ?? (() => new Date()) + const known = new Set(input.entities.map((e) => e.id)) + + // 1) Gather co-occurrence evidence per unordered entity pair, keyed by subject|object (subject = min id). + const evidenceByPair = new Map>() + const addPair = (a: string, b: string, ref: ClaimEvidenceRef): void => { + const [subject, object] = a < b ? [a, b] : [b, a] + const key = pairKey(subject, object, 'co-occurs-with') + const refs = evidenceByPair.get(key) ?? new Map() + refs.set(`${ref.record}\u0000${ref.id}`, ref) // dedup by (record, id): one evidence record backs a pair once + evidenceByPair.set(key, refs) + } + const emitPairs = (ids: string[], ref: ClaimEvidenceRef): void => { + for (let i = 0; i < ids.length; i++) for (let j = i + 1; j < ids.length; j++) addPair(ids[i]!, ids[j]!, ref) + } + + for (const moment of input.moments) { + if (moment.sessionId !== input.sessionId) continue + emitPairs(participants(moment.refs, known, config.maxEntitiesPerRecord), { record: 'moment', id: moment.id, at: moment.at }) + } + for (const packet of input.packets) { + if (packet.sessionId !== input.sessionId) continue + emitPairs( + participants(packet.candidates.map((c) => c.entityId), known, config.maxEntitiesPerRecord), + { record: 'context-packet', id: packet.id, at: packet.windowStart }, + ) + } + + // 2) Build a claim per pair with evidence; decide idempotence / append-only supersession vs the chain head. + const latest = latestByPair(input.existing) + const created: Claim[] = [] + const unchanged: Claim[] = [] + + for (const key of [...evidenceByPair.keys()].sort()) { + const [subject, object] = key.split('\u0000') as [string, string] + const evidence = [...evidenceByPair.get(key)!.values()].sort(byEvidence) + const firstObserved = evidence[0]!.at + const lastObserved = evidence[evidence.length - 1]!.at + const content: ClaimContent = { + workspaceId: input.workspaceId, + subject, + object, + relation: 'co-occurs-with', + evidence, + confidence: cooccurrenceConfidence(evidence.length), + source: 'derived', + state: 'provisional', + provenance: { builder: 'deterministic-cooccurrence', evidenceCount: evidence.length }, + sessionId: input.sessionId, + firstObserved, + lastObserved, + schemaVersion: CLAIM_SCHEMA_VERSION, + } + + const head = latest.get(pairKey(subject, object, 'co-occurs-with')) + if (head !== undefined) { + const { id: _i, revision: _r, supersedes: _s, createdAt: _c, ...headContent } = head + if (canonicalJson(headContent) === canonicalJson(content)) { + unchanged.push(head) + continue + } + } + const revision = head === undefined ? 1 : head.revision + 1 + const supersedes = head?.id + created.push({ + id: claimId(content, revision, supersedes), + ...content, + revision, + ...(supersedes !== undefined ? { supersedes } : {}), + createdAt: now().toISOString(), + }) + } + + return { created, unchanged } +} diff --git a/apps/engine/src/index/index.ts b/apps/engine/src/index/index.ts index 90af38d..954091c 100644 --- a/apps/engine/src/index/index.ts +++ b/apps/engine/src/index/index.ts @@ -93,6 +93,22 @@ export { type TraceRef, type TraceSourceStatus, } from './summaries-trace.js' +export { + buildClaims, + DEFAULT_CLAIM_BUILDER_CONFIG, + type ClaimBuilderConfig, + type ClaimBuildInput, + type ClaimBuildResult, +} from './claims.js' +export { + materializeClaims, + ClaimBuildLog, + type ClaimBuildAttempt, + type ClaimBuildTrigger, + type MaterializeClaimsScope, + type MaterializeClaimsDeps, + type MaterializeClaimsOutcome, +} from './produce-claims.js' export { mergeCanon, type CanonGroup, type CanonResult } from './canon.js' export { relevantNow, type RelevantNowOptions } from './relevant.js' export * from './ingest/index.js' diff --git a/apps/engine/src/index/produce-claims.test.ts b/apps/engine/src/index/produce-claims.test.ts new file mode 100644 index 0000000..c923cce --- /dev/null +++ b/apps/engine/src/index/produce-claims.test.ts @@ -0,0 +1,110 @@ +import { mkdtemp, rm } from 'node:fs/promises' +import { join } from 'node:path' +import { tmpdir } from 'node:os' +import { test } from 'node:test' +import assert from 'node:assert/strict' +import type { Moment } from '@openinfo/contracts' +import { WorkspaceRegistry } from '../store/index.js' +import { materializeClaims, ClaimBuildLog } from './produce-claims.js' + +/** + * #178 slice-1 live seam — the store-touching producer that wires the pure builder to the store at session + * end. These pin the non-negotiables: idempotence (re-running is a no-op), workspace isolation (a build + * reads/writes only its own workspace), CONTAINED FAILURE (a store error is recorded, never thrown), and the + * build log surfaces the attempt. + */ + +const CLOCK = () => new Date('2026-07-12T13:00:05.000Z') + +/** Seed a session with two entities co-mentioned in one moment — the minimal co-occurrence evidence. */ +const seedCoOccurrence = (store: WorkspaceRegistry, workspaceId: string, sessionId: string): void => { + const a = store.upsertEntity({ workspaceId, kind: 'person', name: 'Ada', seenAt: '2026-07-12T13:00:00.000Z' }) + const b = store.upsertEntity({ workspaceId, kind: 'artifact', name: 'pi.dev', seenAt: '2026-07-12T13:00:00.000Z' }) + const moment: Moment = { + id: `mom-${sessionId}`, sessionId, workspaceId, at: '2026-07-12T13:00:01.000Z', kind: 'mention', + text: 'Ada is working on pi.dev', refs: [a.id, b.id], source: 'mic', confidence: 0.9, + } + store.saveMoment(moment) +} + +test('the session-end seam materializes co-occurrence claims from stored evidence, idempotently', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-produce-claims-')) + try { + const store = new WorkspaceRegistry(dir) + const log = new ClaimBuildLog() + seedCoOccurrence(store, 'default', 'ses-1') + + const first = materializeClaims({ store, log, now: CLOCK }, { workspaceId: 'default', sessionId: 'ses-1', trigger: 'session-end' }) + assert.equal(first.error, undefined) + assert.equal(first.created.length, 1, 'the co-mention yields one claim') + assert.equal(first.created[0]!.relation, 'co-occurs-with') + + // Idempotent: a second run over the converged session appends nothing. + const second = materializeClaims({ store, log, now: CLOCK }, { workspaceId: 'default', sessionId: 'ses-1', trigger: 'session-end' }) + assert.equal(second.created.length, 0, 'rebuild over converged evidence is a no-op') + assert.equal(second.unchanged.length, 1) + + // The build log surfaces the latest attempt (the diagnostics "last update" read). + const attempt = log.latestFor('default', 'ses-1') + assert.equal(attempt?.created, 0) + assert.equal(attempt?.unchanged, 1) + assert.equal(attempt?.error, undefined) + + assert.equal(store.listClaims('default', { sessionId: 'ses-1' }).length, 1, 'exactly one live claim persisted') + store.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('a build reads and writes ONLY its own workspace (isolation)', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-produce-claims-iso-')) + try { + const store = new WorkspaceRegistry(dir) + store.ensureWorkspace({ id: 'ws-a', name: 'A' }) + store.ensureWorkspace({ id: 'ws-b', name: 'B' }) + seedCoOccurrence(store, 'ws-a', 'ses-a') + seedCoOccurrence(store, 'ws-b', 'ses-b') + + const a = materializeClaims({ store }, { workspaceId: 'ws-a', sessionId: 'ses-a', trigger: 'session-end' }) + assert.equal(a.created.length, 1) + assert.equal(store.listClaims('ws-a').length, 1, 'ws-a has its claim') + assert.equal(store.listClaims('ws-b').length, 0, 'ws-b untouched by the ws-a build') + store.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('CONTAINED FAILURE: a store error is caught, recorded on the log, and returned — never thrown', () => { + const log = new ClaimBuildLog() + const brokenStore = { + listContextPackets() { throw new Error('disk gone') }, + listMoments() { return [] }, + listEntities() { return [] }, + listClaims() { return [] }, + saveClaim(c: unknown) { return c }, + } as unknown as WorkspaceRegistry + + const outcome = materializeClaims({ store: brokenStore, log }, { workspaceId: 'default', sessionId: 'ses-1', trigger: 'session-end' }) + assert.equal(outcome.error, 'disk gone', 'the failure is returned, not thrown') + assert.deepEqual(outcome.created, []) + const attempt = log.latestFor('default', 'ses-1') + assert.equal(attempt?.error, 'disk gone', 'the reason is on the log for the diagnostics surface') +}) + +test('a session with no co-occurrence evidence produces no claims (honest degradation)', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-produce-claims-empty-')) + try { + const store = new WorkspaceRegistry(dir) + // One entity mentioned alone — never a relationship. + const a = store.upsertEntity({ workspaceId: 'default', kind: 'person', name: 'Solo', seenAt: '2026-07-12T13:00:00.000Z' }) + store.saveMoment({ id: 'mom-solo', sessionId: 'ses-1', workspaceId: 'default', at: '2026-07-12T13:00:01.000Z', kind: 'mention', text: 'Solo', refs: [a.id], source: 'mic', confidence: 0.9 }) + + const outcome = materializeClaims({ store }, { workspaceId: 'default', sessionId: 'ses-1', trigger: 'session-end' }) + assert.deepEqual(outcome.created, [], 'no pair ⇒ no claim, never fabricated') + store.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) diff --git a/apps/engine/src/index/produce-claims.ts b/apps/engine/src/index/produce-claims.ts new file mode 100644 index 0000000..a101c67 --- /dev/null +++ b/apps/engine/src/index/produce-claims.ts @@ -0,0 +1,124 @@ +import type { WorkspaceRegistry } from '../store/index.js' +import { buildClaims, type ClaimBuildResult } from './claims.js' + +/** + * The LIVE Claim producer (#178 slice 1) — the impure runtime seam that wires the pure builder + * (`claims.ts`) to the store, so co-occurrence claims materialize during normal capture WITHOUT anyone + * calling the on-demand POST route. Like `produce-packets.ts`, this is a store-touching index/ seam: it + * reads a session's already-converged evidence (its ContextPackets + moments), runs the deterministic + * builder, persists ONLY what the builder appends, and records the attempt's outcome so a failure is + * VISIBLE, never silently swallowed. + * + * CONTAINED FAILURE (non-negotiable): building a claim is derived, best-effort convenience — it must never + * block or fail the capture/distill/session-end path that produced the evidence. So `materializeClaims` + * NEVER throws: any error is caught, recorded on the build log with its reason, and returned in the + * outcome. The session-end seam calls this AFTER the packets are materialized, so the packet-candidate + * co-occurrence evidence is fresh; a build failure loses nothing but the (rebuildable) derived claim. + * + * IDEMPOTENT + BATCHED: the builder is idempotent (same evidence in ⇒ nothing appended), so re-running over + * a converged session is a safe no-op. It runs ONCE per session end over the whole session — not per + * observation — so it adds no per-record hot-loop cost. WORKSPACE ISOLATION: every read and the write are + * scoped to the session's own workspace, never a shared table. NO MODEL: slice 1 is deterministic + * co-occurrence only; a judge-enrichment pass (semantic relations) is a LATER slice. + */ + +/** What triggered a build attempt — the live session-end seam, or the on-demand route. */ +export type ClaimBuildTrigger = 'session-end' | 'on-demand' + +/** + * One recorded build attempt — the diagnostics "last update" signal. `error` present ⇒ the build did not + * finish (created/unchanged are 0) and its reason is carried for display; absent ⇒ the build converged and + * `created`/`unchanged` say what it did. Process-scoped (cleared on restart), mirroring PacketBuildLog — + * the durable truth is the claims themselves. + */ +export interface ClaimBuildAttempt { + workspaceId: string + sessionId: string + trigger: ClaimBuildTrigger + /** ISO instant the attempt ran. */ + at: string + /** claims appended this attempt (new relationships + supersession revisions). */ + created: number + /** existing derived heads that rebuilt identical — kept untouched. */ + unchanged: number + /** present ⇒ the attempt failed; the contained reason, for the diagnostics "last update didn't finish" line. */ + error?: string +} + +/** + * The latest build attempt per (workspace, session), in memory — bounded, latest-only (mirrors + * PacketBuildLog). Diagnostics reads the most recent attempt so a live-seam failure surfaces as text. + */ +export class ClaimBuildLog { + private readonly latest = new Map() + + private key(workspaceId: string, sessionId: string): string { + return `${workspaceId}\u0000${sessionId}` + } + + /** Record (replacing any prior) the latest attempt for its session. */ + record(attempt: ClaimBuildAttempt): void { + this.latest.set(this.key(attempt.workspaceId, attempt.sessionId), attempt) + } + + /** The latest attempt for one session, or undefined when no build has run for it this process. */ + latestFor(workspaceId: string, sessionId: string): ClaimBuildAttempt | undefined { + return this.latest.get(this.key(workspaceId, sessionId)) + } + + /** Every session's latest attempt in a workspace, newest attempt first — the diagnostics read. */ + recentForWorkspace(workspaceId: string): ClaimBuildAttempt[] { + return [...this.latest.values()] + .filter((attempt) => attempt.workspaceId === workspaceId) + .sort((a, b) => (a.at < b.at ? 1 : a.at > b.at ? -1 : 0)) + } +} + +export interface MaterializeClaimsScope { + workspaceId: string + sessionId: string + trigger: ClaimBuildTrigger +} + +export interface MaterializeClaimsDeps { + store: WorkspaceRegistry + /** the diagnostics build log to record the attempt on (optional so the seam is testable without one). */ + log?: ClaimBuildLog + /** injectable clock for the recorded attempt time and appended claims' createdAt. */ + now?: () => Date +} + +/** The builder result plus a contained failure reason (present ⇒ nothing was built or persisted). */ +export interface MaterializeClaimsOutcome extends ClaimBuildResult { + error?: string +} + +/** + * Build (or converge) one session's co-occurrence Claims from its stored evidence and persist what the + * builder appends. Reads and writes are scoped to `scope.workspaceId`. NEVER throws: a read/build/write + * failure is caught, recorded on the log with its reason, and returned as `outcome.error`. It reads the + * session's DERIVED chain (including superseded revisions) so supersession is decided correctly, and never + * reads or writes sovereign user corrections — those are resolved over the chain at read time. + */ +export const materializeClaims = (deps: MaterializeClaimsDeps, scope: MaterializeClaimsScope): MaterializeClaimsOutcome => { + const now = deps.now ?? (() => new Date()) + const { workspaceId, sessionId, trigger } = scope + try { + const result = buildClaims({ + workspaceId, + sessionId, + packets: deps.store.listContextPackets(workspaceId, { sessionId }), + moments: deps.store.listMoments(workspaceId, sessionId), + entities: deps.store.listEntities(workspaceId), + existing: deps.store.listClaims(workspaceId, { sessionId, source: 'derived', includeSuperseded: true }), + now, + }) + for (const claim of result.created) deps.store.saveClaim(claim) + deps.log?.record({ workspaceId, sessionId, trigger, at: now().toISOString(), created: result.created.length, unchanged: result.unchanged.length }) + return result + } catch (error) { + const message = error instanceof Error ? error.message : String(error) + deps.log?.record({ workspaceId, sessionId, trigger, at: now().toISOString(), created: 0, unchanged: 0, error: message }) + return { created: [], unchanged: [], error: message } + } +} diff --git a/apps/engine/src/store/workspaces.test.ts b/apps/engine/src/store/workspaces.test.ts index 1573eb0..7dc3628 100644 --- a/apps/engine/src/store/workspaces.test.ts +++ b/apps/engine/src/store/workspaces.test.ts @@ -861,3 +861,155 @@ test('#176: moveSession carries the whole append-only packet chain with the sess await rm(dir, { recursive: true, force: true }) } }) + +// ── #178 evidence-backed claims ────────────────────────────────────────────────────────────────────── +import type { Claim } from '@openinfo/contracts' + +const derivedClaim = (o: Partial & { subject: string; object: string }): Claim => ({ + id: o.id ?? `clm-${o.subject}-${o.object}-r${o.revision ?? 1}`, + workspaceId: o.workspaceId ?? 'default', + subject: o.subject, + object: o.object, + relation: o.relation ?? 'co-occurs-with', + evidence: o.evidence ?? [{ record: 'moment', id: 'm1', at: '2026-07-12T13:00:00.000Z' }], + confidence: o.confidence ?? 0.4, + source: 'derived', + state: 'provisional', + provenance: { builder: 'deterministic-cooccurrence', evidenceCount: o.evidence?.length ?? 1 }, + sessionId: o.sessionId ?? 'ses-1', + firstObserved: o.firstObserved ?? '2026-07-12T13:00:00.000Z', + lastObserved: o.lastObserved ?? '2026-07-12T13:00:00.000Z', + revision: o.revision ?? 1, + ...(o.supersedes !== undefined ? { supersedes: o.supersedes } : {}), + schemaVersion: 1, + createdAt: o.createdAt ?? '2026-07-12T13:00:05.000Z', +}) + +test('#178 listClaims resolves supersession + user sovereignty BEFORE the filters', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-store-')) + try { + const registry = new WorkspaceRegistry(dir) + const r1 = derivedClaim({ subject: 'ent-a', object: 'ent-b', id: 'clm-r1', revision: 1 }) + const r2 = derivedClaim({ subject: 'ent-a', object: 'ent-b', id: 'clm-r2', revision: 2, supersedes: 'clm-r1', confidence: 0.6 }) + registry.saveClaim(r1) + registry.saveClaim(r2) + + // Default read: only the live head; the superseded revision is hidden even under an entity filter. + assert.deepEqual(registry.listClaims('default', { entityId: 'ent-a' }).map((c) => c.id), ['clm-r2'], 'live head only') + assert.deepEqual( + registry.listClaims('default', { includeSuperseded: true }).map((c) => c.id).sort(), + ['clm-r1', 'clm-r2'], + 'the whole append-only chain is retrievable', + ) + // Relation / source filters compose over the resolved heads. + assert.equal(registry.listClaims('default', { relation: 'works-on' }).length, 0) + assert.equal(registry.listClaims('default', { source: 'derived' }).length, 1) + assert.equal(registry.listClaims('unknown-ws').length, 0, 'unknown workspace reads as []') + registry.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('#178 a user CONFIRM outranks the derived claim as the live head, without deleting it', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-confirm-')) + try { + const registry = new WorkspaceRegistry(dir) + const derived = derivedClaim({ subject: 'ent-a', object: 'ent-b', id: 'clm-d' }) + registry.saveClaim(derived) + const correction = registry.recordClaimCorrection({ workspaceId: 'default', claimId: 'clm-d', verdict: 'confirm', by: 'the user' }) + assert.equal(correction.source, 'user') + assert.equal(correction.state, 'confirmed') + assert.equal(correction.confidence, 1) + assert.deepEqual(correction.evidence, derived.evidence, 'the correction carries the target evidence (stays traceable)') + + const live = registry.listClaims('default') + assert.deepEqual(live.map((c) => c.id), [correction.id], 'the sovereign confirmation is the live head; the derived claim is dropped') + assert.equal(registry.listClaims('default', { includeSuperseded: true }).length, 2, 'the original inference is never deleted') + registry.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('#178 a user REJECT removes the pair from the live view AND survives re-derivation (sovereignty)', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-reject-')) + try { + const registry = new WorkspaceRegistry(dir) + registry.saveClaim(derivedClaim({ subject: 'ent-a', object: 'ent-b', id: 'clm-d1', revision: 1 })) + registry.recordClaimCorrection({ workspaceId: 'default', claimId: 'clm-d1', verdict: 'reject', note: 'coincidence' }) + assert.equal(registry.listClaims('default').length, 0, 'a rejected pair is NOT a live relationship') + + // The builder later RE-DERIVES the pair with more evidence (a new revision) — sovereignty must hold. + registry.saveClaim(derivedClaim({ subject: 'ent-a', object: 'ent-b', id: 'clm-d2', revision: 2, supersedes: 'clm-d1', confidence: 0.6 })) + assert.equal(registry.listClaims('default').length, 0, 're-deriving a rejected pair does not resurrect it') + assert.ok(registry.listClaims('default', { includeSuperseded: true }).length >= 3, 'the derived revisions + the reject are retained as history') + registry.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('#178 a user CORRECT asserts a different relationship that becomes the live head', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-correct-')) + try { + const registry = new WorkspaceRegistry(dir) + registry.saveClaim(derivedClaim({ subject: 'ent-a', object: 'ent-b', id: 'clm-d' })) + const corrected = registry.recordClaimCorrection({ workspaceId: 'default', claimId: 'clm-d', verdict: 'correct', relation: 'works-on', note: 'Ada maintains it' }) + assert.equal(corrected.state, 'corrected') + assert.equal(corrected.relation, 'works-on') + + const live = registry.listClaims('default') + assert.deepEqual(live.map((c) => `${c.relation}:${c.source}`), ['works-on:user'], 'the corrected relationship replaces the derived co-occurrence') + + // A re-issued identical correction is idempotent (content-derived id ⇒ insert-or-replace, no duplicate). + registry.recordClaimCorrection({ workspaceId: 'default', claimId: 'clm-d', verdict: 'correct', relation: 'works-on', note: 'Ada maintains it' }) + assert.equal(registry.listClaims('default').length, 1, 're-issuing the same correction does not duplicate it') + registry.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('#178 relatedEntities walks depth-1 and always returns the backing claim ids', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-walk-')) + try { + const registry = new WorkspaceRegistry(dir) + const a = registry.upsertEntity({ workspaceId: 'default', kind: 'person', name: 'Ada', seenAt: '2026-07-12T13:00:00.000Z' }) + const b = registry.upsertEntity({ workspaceId: 'default', kind: 'artifact', name: 'pi.dev', seenAt: '2026-07-12T13:00:00.000Z' }) + const c = registry.upsertEntity({ workspaceId: 'default', kind: 'person', name: 'Cyd', seenAt: '2026-07-12T13:00:00.000Z' }) + registry.saveClaim(derivedClaim({ subject: a.id, object: b.id, id: 'clm-ab', confidence: 0.6 })) + registry.saveClaim(derivedClaim({ subject: c.id, object: a.id, id: 'clm-ca', confidence: 0.4 })) + + const related = registry.relatedEntities('default', a.id) + assert.deepEqual(related.map((r) => r.name), ['pi.dev', 'Cyd'], 'both counterparts, strongest confidence first') + const pidev = related.find((r) => r.entityId === b.id)! + assert.deepEqual(pidev.relations, ['co-occurs-with']) + assert.deepEqual(pidev.claimIds, ['clm-ab'], 'the walk names the backing claim so evidence is retrievable') + assert.equal(pidev.confidence, 0.6) + registry.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) + +test('#178 moveSession carries a session\'s claims (derived + corrections) to the destination workspace', async () => { + const dir = await mkdtemp(join(tmpdir(), 'openinfo-claims-move-')) + try { + const registry = new WorkspaceRegistry(dir) + registry.ensureWorkspace({ id: 'src', name: 'Src' }) + registry.ensureWorkspace({ id: 'dst', name: 'Dst' }) + registry.saveSession({ id: 'ses-m', workspaceId: 'src', modeId: 'mode-x', startedAt: '2026-07-12T13:00:00.000Z', attribution: { evidence: [], confidence: 1 } }) + registry.saveClaim(derivedClaim({ subject: 'ent-a', object: 'ent-b', workspaceId: 'src', sessionId: 'ses-m', id: 'clm-move' })) + registry.recordClaimCorrection({ workspaceId: 'src', claimId: 'clm-move', verdict: 'confirm' }) + + registry.moveSession('ses-m', 'src', 'dst') + assert.equal(registry.listClaims('src', { includeSuperseded: true }).length, 0, 'source no longer holds the session claims') + const moved = registry.listClaims('dst', { includeSuperseded: true }) + assert.equal(moved.length, 2, 'both the derived claim and the sovereign correction moved') + assert.ok(moved.every((c) => c.workspaceId === 'dst'), 'workspace scope rewritten on the moved rows') + registry.close() + } finally { + await rm(dir, { recursive: true, force: true }) + } +}) diff --git a/apps/engine/src/store/workspaces.ts b/apps/engine/src/store/workspaces.ts index 5d011bd..f31b18c 100644 --- a/apps/engine/src/store/workspaces.ts +++ b/apps/engine/src/store/workspaces.ts @@ -1,9 +1,9 @@ -import { randomUUID } from 'node:crypto' +import { createHash, randomUUID } from 'node:crypto' import { mkdirSync } from 'node:fs' import { join } from 'node:path' import Database from 'better-sqlite3' -import type { ChatTurn, ContextPacket, Distillate, Draft, Entity, EntityProvenance, EntityOverride, EntityResolution, EgressPolicy, FieldValue, GuardHold, HeardAs, Moment, OcrResult, Pin, PinChunk, Session, SessionAnnotation, SessionTitling, Sighting, SttSegment, Summary, SummaryLevel, TodoList, Workspace } from '@openinfo/contracts' -import { ChatTurn as ChatTurnSchema, ContextPacket as ContextPacketSchema, Entity as EntitySchema, Pin as PinSchema, PinChunk as PinChunkSchema, Summary as SummarySchema } from '@openinfo/contracts' +import type { ChatTurn, Claim, ClaimRelation, ContextPacket, Distillate, Draft, Entity, EntityProvenance, EntityOverride, EntityResolution, EgressPolicy, FieldValue, GuardHold, HeardAs, Moment, OcrResult, Pin, PinChunk, RelatedEntity, Session, SessionAnnotation, SessionTitling, Sighting, SttSegment, Summary, SummaryLevel, TodoList, Workspace } from '@openinfo/contracts' +import { ChatTurn as ChatTurnSchema, Claim as ClaimSchema, CLAIM_SCHEMA_VERSION, ContextPacket as ContextPacketSchema, Entity as EntitySchema, Pin as PinSchema, PinChunk as PinChunkSchema, Summary as SummarySchema } from '@openinfo/contracts' import { Value } from '@sinclair/typebox/value' import { DEFAULT_RESOLVER_CONFIG, resolveEntity, type Resolution, type ResolutionSignals, type ResolverConfig } from '../index/resolve.js' import { DEFAULT_GAZETTEER, GAZETTEER_KEY, GAZETTEER_KIND, gazetteerRivals, type GazetteerDocument } from '../index/gazetteer.js' @@ -389,6 +389,194 @@ export class WorkspaceRegistry { return summaries } + /** + * Append one Claim (#178) to its workspace's OWN sqlite file. Claims are APPEND-ONLY derived assertions: + * a revision never mutates or deletes its predecessor — it is a new row whose `supersedes` links the + * chain, and a sovereign user correction is a new `source:'user'` row that OUTRANKS (never overwrites) + * the derived claim it `corrects`. `insert or replace` keyed on the (content-derived) id makes a replayed + * build byte-stable (same content ⇒ same id ⇒ replace-in-place), mirroring saveContextPacket. Contract- + * validated before write (the last-line-of-defense idiom) — so the "a claim with no evidence is + * unrepresentable" invariant holds at the store boundary too. Only this path writes claims. + */ + saveClaim(claim: Claim): Claim { + this.ensureWorkspace({ id: claim.workspaceId, name: claim.workspaceId }) + const db = this.openWorkspace(claim.workspaceId) + const { subject, relation, object } = claim + if (!Value.Check(ClaimSchema, claim)) { + throw new Error(`claim failed contract validation: ${subject} ${relation} ${object}`) + } + db.prepare( + 'insert or replace into claims (id, session_id, subject, object, relation, source, first_observed, created_at, body) values (?, ?, ?, ?, ?, ?, ?, ?, ?)', + ).run(claim.id, claim.sessionId ?? null, claim.subject, claim.object, claim.relation, claim.source, claim.firstObserved, claim.createdAt, JSON.stringify(claim)) + return claim + } + + /** + * Resolve a claim set to its LIVE heads (#178) — supersession AND sovereignty, in one pure pass, so a + * reader never sees a superseded revision or a user-overruled pair as live: + * 1. DERIVED SUPERSESSION: a derived revision named by another's `supersedes` is dropped. + * 2. USER SOVEREIGNTY: for each (subject, object, relation) pair a user has ruled on, the LATEST user + * correction wins (a `reject` asserts the pair is NOT a relationship, so it drops out of the live + * view entirely; a `confirm`/`correct` becomes the live head). A DERIVED claim on a pair the user + * ruled on is dropped — so RE-DERIVING a corrected pair can never defeat the correction (the builder + * keeps appending derived revisions, but they stay behind the sovereign head at read time). + * The original inference and its evidence are never deleted — `includeSuperseded` returns the whole + * append-only history (revisions + overruled derived claims + reject markers) for audit. Pure over the + * list; the source of truth is the append-only table. + */ + static resolveClaimHeads(claims: readonly Claim[], includeSuperseded = false): Claim[] { + if (includeSuperseded) return [...claims] + const pk = (c: Claim): string => `${c.subject}\u0000${c.object}\u0000${c.relation}` + const byId = new Map(claims.map((c) => [c.id, c])) + const superseded = new Set(claims.map((c) => c.supersedes).filter((id): id is string => id !== undefined)) + // The pair each user correction ruled on is the pair of the claim it CORRECTS (a `correct` may assert a + // different relation/object than the target, so the target's pair is the one being overruled). + const rulingPairKey = (u: Claim): string => { + const target = u.corrects !== undefined ? byId.get(u.corrects) : undefined + return target !== undefined ? pk(target) : pk(u) + } + const latestUser = new Map() + for (const u of claims) { + if (u.source !== 'user') continue + const key = rulingPairKey(u) + const prior = latestUser.get(key) + if (prior === undefined || u.createdAt > prior.createdAt || (u.createdAt === prior.createdAt && u.id > prior.id)) latestUser.set(key, u) + } + const live: Claim[] = [] + for (const c of claims) { + if (c.source !== 'derived' || superseded.has(c.id)) continue + if (latestUser.has(pk(c))) continue // a sovereign correction sits over this pair + live.push(c) + } + for (const u of latestUser.values()) { + if (u.correction?.verdict === 'reject') continue // a rejected pair is NOT a live relationship + live.push(u) + } + return live.sort((a, b) => (a.firstObserved < b.firstObserved ? -1 : a.firstObserved > b.firstObserved ? 1 : a.id < b.id ? -1 : a.id > b.id ? 1 : 0)) + } + + /** + * Query a workspace's Claims (#178) — the axes the issue names: workspace (the DB), session, related + * entity (subject OR object), relation kind, source, and time window (`from`/`to` keep claims whose + * observed span INTERSECTS the range). SUPERSESSION AND SOVEREIGNTY ARE RESOLVED FIRST (resolveClaimHeads) + * so a filtered read can never present a superseded revision or a user-overruled pair as live merely + * because its successor/correction fell outside the filter. `includeSuperseded` returns the whole + * append-only history. Unknown workspace reads as [] (mirrors listContextPackets) — asking is never an error. + */ + listClaims( + workspaceId: string, + opts: { + sessionId?: string + subject?: string + object?: string + entityId?: string + relation?: ClaimRelation + source?: Claim['source'] + from?: string + to?: string + includeSuperseded?: boolean + } = {}, + ): Claim[] { + if (!this.all().some((ws) => ws.id === workspaceId)) return [] + const db = this.openWorkspace(workspaceId) + const rows = opts.sessionId + ? (db.prepare('select body from claims where session_id = ? order by first_observed, created_at, id').all(opts.sessionId) as { body: string }[]) + : (db.prepare('select body from claims order by first_observed, created_at, id').all() as { body: string }[]) + let claims = WorkspaceRegistry.resolveClaimHeads(rows.map((row) => JSON.parse(row.body) as Claim), opts.includeSuperseded) + if (opts.source !== undefined) claims = claims.filter((c) => c.source === opts.source) + if (opts.relation !== undefined) claims = claims.filter((c) => c.relation === opts.relation) + if (opts.subject !== undefined) claims = claims.filter((c) => c.subject === opts.subject) + if (opts.object !== undefined) claims = claims.filter((c) => c.object === opts.object) + if (opts.entityId !== undefined) claims = claims.filter((c) => c.subject === opts.entityId || c.object === opts.entityId) + if (opts.from !== undefined) claims = claims.filter((c) => c.lastObserved > opts.from!) + if (opts.to !== undefined) claims = claims.filter((c) => c.firstObserved < opts.to!) + return claims + } + + /** + * The depth-1 relationship walk (#178): entity → its live claims → the counterpart entity at the other + * end of each. Answers "what does this entity relate to?" — the shape #181's typeahead and the recall + * surfaces consume. Each counterpart carries the deduped relation kinds, the strongest backing + * confidence, and the `claimIds` so a consumer can always retrieve the supporting evidence + scope (the + * #178 invariant: a walk result always resolves to its supporting claims). A read-time PROJECTION over the + * resolved claim heads — never a second truth store. A counterpart with no entity record is skipped (a + * dangling id has nothing to name — never fabricated). Sorted strongest-first, then by id (deterministic). + */ + relatedEntities(workspaceId: string, entityId: string): RelatedEntity[] { + const claims = this.listClaims(workspaceId, { entityId }) + const entities = new Map(this.listEntities(workspaceId).map((e) => [e.id, e])) + const byCounterpart = new Map; confidence: number; claimIds: string[] }>() + for (const c of claims) { + const counterpart = c.subject === entityId ? c.object : c.subject + if (counterpart === entityId) continue // a self-loop has no counterpart to walk to + const entry = byCounterpart.get(counterpart) ?? { relations: new Set(), confidence: 0, claimIds: [] } + entry.relations.add(c.relation) + entry.confidence = Math.max(entry.confidence, c.confidence) + entry.claimIds.push(c.id) + byCounterpart.set(counterpart, entry) + } + const out: RelatedEntity[] = [] + for (const [id, entry] of byCounterpart) { + const entity = entities.get(id) + if (entity === undefined) continue + out.push({ entityId: id, name: entity.name, relations: [...entry.relations].sort(), confidence: entry.confidence, claimIds: [...entry.claimIds].sort() }) + } + return out.sort((a, b) => b.confidence - a.confidence || (a.entityId < b.entityId ? -1 : a.entityId > b.entityId ? 1 : 0)) + } + + /** + * Record a SOVEREIGN user correction on a derived claim (#178) — the durable, append-only assertion that + * OUTRANKS the derived claim without deleting it or its evidence (the roadmap invariant; the titling- + * sovereignty pattern). It mints a `source:'user'` Claim that `corrects` the target: a `confirm` affirms + * it (state `confirmed`), a `reject` denies the relationship (state `rejected` — the pair drops out of the + * live view), a `correct` asserts a different relation/object (state `corrected`). The correction CARRIES + * THE TARGET'S EVIDENCE REFS, so it stays traceable to the same source observations it reinterprets (the + * evidence-is-mandatory invariant holds). The id is content-derived over (target, verdict, relation, + * object, note) EXCLUDING the timestamp, so re-issuing the same correction is idempotent (insert-or- + * replace, no duplicate). Contract-validated on write via saveClaim. Throws only for an unknown claim id. + */ + recordClaimCorrection(input: { + workspaceId: string + claimId: string + verdict: 'confirm' | 'reject' | 'correct' + relation?: ClaimRelation + object?: string + by?: string + note?: string + }): Claim { + const target = this.listClaims(input.workspaceId, { includeSuperseded: true }).find((c) => c.id === input.claimId) + if (target === undefined) throw new Error(`no claim ${input.claimId} in workspace ${input.workspaceId}`) + // confirm/reject inherit the target's relation/object; only a `correct` may assert a different one. + const relation: ClaimRelation = input.verdict === 'correct' && input.relation !== undefined ? input.relation : target.relation + const object = input.verdict === 'correct' && input.object !== undefined ? input.object : target.object + const state: Claim['state'] = input.verdict === 'confirm' ? 'confirmed' : input.verdict === 'reject' ? 'rejected' : 'corrected' + const at = new Date().toISOString() + const id = `clm-user-${createHash('sha256') + .update(`${target.id}\u0000${input.verdict}\u0000${relation}\u0000${object}\u0000${input.note ?? ''}`) + .digest('hex') + .slice(0, 24)}` + const claim: Claim = { + id, + workspaceId: input.workspaceId, + subject: target.subject, + object, + relation, + evidence: target.evidence, // carry the target's evidence → traceable to the same observations + confidence: 1, // a sovereign human decision, not a proposal + source: 'user', + state, + correction: { verdict: input.verdict, at, ...(input.by !== undefined ? { by: input.by } : {}), ...(input.note !== undefined ? { note: input.note } : {}) }, + corrects: target.id, + ...(target.sessionId !== undefined ? { sessionId: target.sessionId } : {}), + firstObserved: target.firstObserved, + lastObserved: target.lastObserved, + revision: 1, + schemaVersion: CLAIM_SCHEMA_VERSION, + createdAt: at, + } + return this.saveClaim(claim) + } + /** List a workspace's OcrResults (default all), oldest first; `sessionId` narrows to one session. */ listOcrResults(workspaceId: string, sessionId?: string): OcrResult[] { const db = this.openWorkspace(workspaceId) @@ -887,8 +1075,8 @@ export class WorkspaceRegistry { * This is the ONLY module that opens DB handles, so route/ asks store to move a session (dep rule 2). * * WHAT MOVES: the session record (with the authoritative manual correction appended atomically), its - * distillates, moments, drafts, OcrResults, SttSegments, and ContextPackets (the whole append-only - * chain, superseded revisions included), plus + * distillates, moments, drafts, OcrResults, SttSegments, ContextPackets, Summaries, and Claims (the whole + * append-only chains, superseded revisions and sovereign user corrections included), plus * the session-scoped document state in _meta.db: complete FieldValue and SessionAnnotation histories, * GuardHolds, and the current TodoList. ENTITIES are workspace-level aggregates, not session-keyed, * so they are re-aggregated (see below), never blindly copied. @@ -941,6 +1129,9 @@ export class WorkspaceRegistry { const contextPackets = this.listContextPackets(fromWorkspaceId, { sessionId, includeSuperseded: true }) // #177: the WHOLE append-only summary chain moves too (includeSuperseded), for the same reason as packets. const summaries = this.listSummaries(fromWorkspaceId, { sessionId, includeSuperseded: true }) + // #178: the WHOLE append-only claim history moves (includeSuperseded) — derived revisions AND sovereign + // user corrections — so a rerouted session keeps its relationship graph and its human corrections intact. + const claims = this.listClaims(fromWorkspaceId, { sessionId, includeSuperseded: true }) const movedDistillateIds = new Set(distillates.map((d) => d.id)) const movedMomentIds = new Set(moments.map((m) => m.id)) @@ -1024,6 +1215,12 @@ export class WorkspaceRegistry { const moved: Summary = { ...summary, workspaceId: toWorkspaceId } toDb.prepare('insert or replace into summaries (id, session_id, level, window_start, window_end, created_at, body) values (?, ?, ?, ?, ?, ?, ?)').run(moved.id, moved.sessionId ?? null, moved.level, moved.windowStart, moved.windowEnd, moved.createdAt, JSON.stringify(moved)) } + for (const claim of claims) { + // The id stays stable across the move (opaque once minted); only the workspace scope is rewritten, so a + // destination rebuild converges on the moved chain instead of forking a new one (as packets/summaries do). + const moved: Claim = { ...claim, workspaceId: toWorkspaceId } + toDb.prepare('insert or replace into claims (id, session_id, subject, object, relation, source, first_observed, created_at, body) values (?, ?, ?, ?, ?, ?, ?, ?, ?)').run(moved.id, moved.sessionId ?? null, moved.subject, moved.object, moved.relation, moved.source, moved.firstObserved, moved.createdAt, JSON.stringify(moved)) + } toDb.prepare('insert or replace into sessions (id, started_at, ended_at, body) values (?, ?, ?, ?)').run(movedSession.id, movedSession.startedAt, movedSession.endedAt ?? null, JSON.stringify(movedSession)) })() @@ -1047,6 +1244,7 @@ export class WorkspaceRegistry { fromDb.prepare('delete from context_packets where session_id = ?').run(sessionId) fromDb.prepare('delete from session_titlings where session_id = ?').run(sessionId) fromDb.prepare('delete from summaries where session_id = ?').run(sessionId) + fromDb.prepare('delete from claims where session_id = ?').run(sessionId) fromDb.prepare('delete from sessions where id = ?').run(sessionId) })() @@ -1580,6 +1778,13 @@ export class WorkspaceRegistry { db.prepare( 'create table if not exists summaries (id text primary key, session_id text, level text not null, window_start text not null, window_end text not null, created_at text not null, body text not null)', ).run() + // #178: evidence-backed relationship claims — subject-relation-object assertions grounded in refs to the + // converged evidence above (context packets / moments), append-only with derived-supersession + sovereign + // user corrections. `session_id` is the derivation scope (nullable, mirroring summaries). Additive `create + // table if not exists` in the per-workspace open path ⇒ existing DBs gain it on next open (no migration step). + db.prepare( + 'create table if not exists claims (id text primary key, session_id text, subject text not null, object text not null, relation text not null, source text not null, first_observed text not null, created_at text not null, body text not null)', + ).run() // The Ask face's persistent app-scoped chat thread — one thread per workspace (owner canon 2026-07-11; // upstream glass left ask-history vestigial). `seq` (autoincrement) is the stable turn order: turns in // one exchange share a created_at millisecond, so time alone cannot order them. Additive `create table diff --git a/docs/PHASE4-NOTES.md b/docs/PHASE4-NOTES.md index 740ecff..2b276f8 100644 --- a/docs/PHASE4-NOTES.md +++ b/docs/PHASE4-NOTES.md @@ -6618,3 +6618,80 @@ until these measurements demand it. engine **1082** (1069 after #234 + 13: 9 in `echo-bleed-fixtures.test.ts`, 4 garble-tier cases in `echo-dedupe.test.ts`). One known parallel-load flake `api/summaries-surface-e2e.test.ts` (unrelated to this slice) failed under the parallel run and passed 1/1 on isolated rerun. No schema changes (no contracts touched). System-only; no PII. +## Slice: evidence-backed claims + relationship graph — substrate only *(#178 slice 1, 2026-07-17)* + +The durable representation for LEARNED RELATIONSHIPS the system lacked — a person works on a repo, a repo +belongs to an app, an entity relates to an earlier author. Slice 1 is SUBSTRATE ONLY: contract + deterministic +builder + store/queries + honest correction path. No surface, no model-enriched semantic claims, no #181 +typeahead, no cross-workspace — all named OUT below. + +**Contract (append-only, schemas regenerated zero-drift).** New `records/claim.ts`: a typed, versioned, +append-only `Claim` — `subject → relation → object`, grounded ONLY in `evidence` refs (`ClaimEvidenceRef` over +the closed set context-packet/moment/distillate) with `minItems:1`, so **a claim with no evidence is +UNREPRESENTABLE** — every claim traces to source observations at the schema boundary. Framed as proposal- +not-truth: `source:'derived'|'user'`, `state` provisional/confirmed/corrected/rejected/superseded, `confidence` +deterministic (never a model score), `ClaimProvenance`(builder + `evidenceCount` = the recorded derivation +behind confidence) on derived rows, `ClaimCorrection`(verdict/by/note) + `corrects` on sovereign user rows, +append-only `revision`/`supersedes`, and `firstObserved`/`lastObserved` = the honest valid-time span derived +from the evidence. Sub-schemas ClaimRelation/ClaimEvidenceRef/ClaimProvenance/ClaimCorrection registered in +AllSchemas; `BuildClaimsRequest`/`CorrectClaimRequest`/`RelatedEntity` in api/payloads; routes in api/routes.ts. + +**The relation-kind decision (the load-bearing choice).** `ClaimRelation` is a CLOSED starter union +[co-occurs-with · works-on · belongs-to · authored · member-of · relates-to], not an open string. Rationale: +relation is a QUERY axis (and later a UI/semantic axis), so an open string invites un-queryable drift — a +typo'd/free-text kind nothing can index (the recurring un-sane-default failure). A closed union keeps every +kind inspectable and routable; `relates-to` is the deliberate generic escape-valve so the union never forces a +wrong-fit kind; extending beyond these is a reviewed contract change (new literal + `CLAIM_SCHEMA_VERSION` bump ++ regen) — deliberate, not accidental (the MomentKind/EntityKind/SummaryLevel precedent). The enum is COMPLETE +now even though **slice 1's deterministic builder emits ONLY `co-occurs-with`** — the sole relation that is +TRUE BY CONSTRUCTION from co-occurrence evidence with no model in the loop. The semantic kinds are reserved for +a LATER judge-enrichment slice (their output is a proposal, #189) and for sovereign user corrections. + +**Deterministic builder (no model this slice).** `index/claims.ts` `buildClaims` — a PURE function (no +DB/model/clock beyond an injectable `now`, like every index/ module). Co-occurrence = two entities NAMED +TOGETHER in one ContextPacket's candidates (reusing #176's converged correlation) or one moment's refs; +evidence = a ref to that packet/moment. Symmetric endpoints canonically ordered by id (so (A,B)/(B,A) collapse +to one claim); dangling refs (no entity record) are never claimed; **NO evidence ⇒ NO claim** (honest +degradation, never fabricated). Confidence = a fixed evidence-count map capped <1.0 (a derived claim is a +proposal; 1.0 is reserved for a user confirmation), and **repeated evidence strengthens a claim only through a +NEW revision** that records the higher `evidenceCount` — never a silent mutation. Idempotence + append-only +supersession are decided against the existing derived chain head per (subject,object,relation) by +canonical-content equality; content-derived ids (sha-256 over canonical JSON + chain position) make replay +byte-stable — proven store-backed in `index/claims-replay.test.ts` (same evidence into two fresh stores ⇒ +byte-identical claims; rebuild appends nothing). + +**Wired at the session-end seam.** `index/produce-claims.ts` `materializeClaims` + `ClaimBuildLog` — the +store-touching live producer, wired in `api/http.ts` at the `session.ended` seam AFTER packets materialize (so +the packet-candidate co-occurrence evidence is fresh), with the SAME contained-failure discipline as packets: +it NEVER throws (a read/build/write failure is caught, recorded on the log, returned), is idempotent, batched +ONCE per session, and workspace-isolated. + +**Store + relationship queries.** `store/workspaces.ts`: a workspace-isolated `claims` table; `saveClaim` +(contract-validated last-line-of-defense); `listClaims` queryable by workspace/session/entity(subject∨object)/ +relation/source/time, with **`resolveClaimHeads` resolving derived-supersession AND user-sovereignty BEFORE the +filters** (so a filtered read can never present a superseded revision or a user-overruled pair as live); +`relatedEntities` = the depth-1 walk (entity → its live claims → the counterpart entities, each carrying the +deduped relations, strongest confidence, and the backing `claimIds` so a walk result always resolves to its +supporting evidence + scope — #181's typeahead substrate); threaded through `moveSession` so a rerouted session +keeps its relationship graph + corrections. Routes GET `/claims`·`/claims/related`, POST `/claims/build`· +`/claims/correct`. + +**Honest degradation + sovereign corrections.** No evidence ⇒ no claims (above). `recordClaimCorrection` mints +an append-only `source:'user'` correction that OUTRANKS the derived claim without deleting it or its evidence +(the #211/#232 titling-sovereignty pattern): confirm (state confirmed, becomes the live head), reject (the pair +drops from the live view and **survives re-derivation** — resolution is by pair key, so the builder re-deriving +a rejected pair can never resurrect it), correct (asserts a different relation/object). The correction carries +the target's evidence (stays traceable) and a content-derived id (idempotent re-issue). The original inference ++ evidence are always retrievable via `includeSuperseded`. + +**OUT (named, not gold-plated) — later slices.** Any UI/surface (a Claims diagnostics view is a later slice, +the #176/#177 slice-2 precedent); MODEL-ENRICHED semantic claims (the judge-enrichment pass that promotes +co-occurrence into works-on/belongs-to/… — a proposal per #189); #181 local entity candidate search/typeahead +(this slice ships the `relatedEntities` walk it will consume); cross-workspace / app-scoped memory grants +(depends on #182 AppRuntime — the `appId` scope field is deliberately NOT added as a dead field). + +**Gates.** contracts 114 (111 + a Claim refs-only/closed-union test + claim.coOccurrence/claim.userConfirm +example-validate tests) · fixtures 15 · client 606 · engine 1077 (1056 + 21: claims builder 10, replay 1, +produce-claims seam 4, store 6 — but counted as claims-e2e 1 folded in the engine total). Schemas regenerated +zero-drift (Claim/ClaimRelation/ClaimEvidenceRef/ClaimProvenance/ClaimCorrection/BuildClaimsRequest/ +CorrectClaimRequest/RelatedEntity). All green under Node 22. diff --git a/shared/contracts/examples/claim.coOccurrence.json b/shared/contracts/examples/claim.coOccurrence.json new file mode 100644 index 0000000..8e3aaa5 --- /dev/null +++ b/shared/contracts/examples/claim.coOccurrence.json @@ -0,0 +1,21 @@ +{ + "id": "clm-1a2b3c4d5e6f708192a3b4c5d6e7f809", + "workspaceId": "workspace-synthetic", + "subject": "ent-ada", + "object": "ent-pidev", + "relation": "co-occurs-with", + "evidence": [ + { "record": "context-packet", "id": "cp-9c1f4a2b7d3e5f60718293a4b5c6d7e8", "at": "2026-07-12T13:00:00.000Z" }, + { "record": "moment", "id": "moment-0001", "at": "2026-07-12T13:00:04.000Z" } + ], + "confidence": 0.6, + "source": "derived", + "state": "provisional", + "provenance": { "builder": "deterministic-cooccurrence", "evidenceCount": 2 }, + "sessionId": "session-synthetic", + "firstObserved": "2026-07-12T13:00:00.000Z", + "lastObserved": "2026-07-12T13:00:04.000Z", + "revision": 1, + "schemaVersion": 1, + "createdAt": "2026-07-12T13:00:05.000Z" +} diff --git a/shared/contracts/examples/claim.userConfirm.json b/shared/contracts/examples/claim.userConfirm.json new file mode 100644 index 0000000..cfbca97 --- /dev/null +++ b/shared/contracts/examples/claim.userConfirm.json @@ -0,0 +1,21 @@ +{ + "id": "clm-99887766554433221100ffeeddccbbaa", + "workspaceId": "workspace-synthetic", + "subject": "ent-ada", + "object": "ent-pidev", + "relation": "works-on", + "evidence": [ + { "record": "context-packet", "id": "cp-9c1f4a2b7d3e5f60718293a4b5c6d7e8", "at": "2026-07-12T13:00:00.000Z" } + ], + "confidence": 1, + "source": "user", + "state": "corrected", + "correction": { "verdict": "correct", "at": "2026-07-12T14:00:00.000Z", "by": "the user", "note": "Ada is the maintainer of pi.dev" }, + "corrects": "clm-1a2b3c4d5e6f708192a3b4c5d6e7f809", + "sessionId": "session-synthetic", + "firstObserved": "2026-07-12T13:00:00.000Z", + "lastObserved": "2026-07-12T13:00:04.000Z", + "revision": 1, + "schemaVersion": 1, + "createdAt": "2026-07-12T14:00:00.000Z" +} diff --git a/shared/contracts/schemas/BuildClaimsRequest.json b/shared/contracts/schemas/BuildClaimsRequest.json new file mode 100644 index 0000000..7c5ad44 --- /dev/null +++ b/shared/contracts/schemas/BuildClaimsRequest.json @@ -0,0 +1,21 @@ +{ + "$id": "BuildClaimsRequest", + "additionalProperties": false, + "type": "object", + "required": [ + "workspaceId", + "sessionId" + ], + "properties": { + "workspaceId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "sessionId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + } + } +} diff --git a/shared/contracts/schemas/Claim.json b/shared/contracts/schemas/Claim.json new file mode 100644 index 0000000..1c0dd1c --- /dev/null +++ b/shared/contracts/schemas/Claim.json @@ -0,0 +1,260 @@ +{ + "$id": "Claim", + "additionalProperties": false, + "type": "object", + "required": [ + "id", + "workspaceId", + "subject", + "object", + "relation", + "evidence", + "confidence", + "source", + "state", + "firstObserved", + "lastObserved", + "revision", + "schemaVersion", + "createdAt" + ], + "properties": { + "id": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "workspaceId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "subject": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "object": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "relation": { + "$id": "ClaimRelation", + "description": "the relationship kind: co-occurs-with (symmetric; the only kind the slice-1 deterministic builder emits) · works-on/belongs-to/authored/member-of (directional semantic kinds — judge-enrichment slice + user corrections) · relates-to (generic fallback)", + "anyOf": [ + { + "const": "co-occurs-with", + "type": "string" + }, + { + "const": "works-on", + "type": "string" + }, + { + "const": "belongs-to", + "type": "string" + }, + { + "const": "authored", + "type": "string" + }, + { + "const": "member-of", + "type": "string" + }, + { + "const": "relates-to", + "type": "string" + } + ] + }, + "evidence": { + "minItems": 1, + "description": "refs to the evidence records backing this claim — a claim with no evidence is unrepresentable", + "type": "array", + "items": { + "$id": "ClaimEvidenceRef", + "additionalProperties": false, + "type": "object", + "required": [ + "record", + "id", + "at" + ], + "properties": { + "record": { + "description": "which record table the id names — the closed set of claim evidence sources", + "anyOf": [ + { + "const": "context-packet", + "type": "string" + }, + { + "const": "moment", + "type": "string" + }, + { + "const": "distillate", + "type": "string" + } + ] + }, + "id": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "at": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + } + } + } + }, + "confidence": { + "minimum": 0, + "maximum": 1, + "type": "number" + }, + "source": { + "description": "derived (deterministic builder proposal) · user (sovereign human correction, outranks derived)", + "anyOf": [ + { + "const": "derived", + "type": "string" + }, + { + "const": "user", + "type": "string" + } + ] + }, + "state": { + "description": "provisional (derived proposal) · confirmed/corrected/rejected (user verdicts) · superseded (an older revision a newer one replaced)", + "anyOf": [ + { + "const": "provisional", + "type": "string" + }, + { + "const": "confirmed", + "type": "string" + }, + { + "const": "corrected", + "type": "string" + }, + { + "const": "rejected", + "type": "string" + }, + { + "const": "superseded", + "type": "string" + } + ] + }, + "provenance": { + "$id": "ClaimProvenance", + "additionalProperties": false, + "type": "object", + "required": [ + "builder", + "evidenceCount" + ], + "properties": { + "builder": { + "const": "deterministic-cooccurrence", + "type": "string" + }, + "evidenceCount": { + "minimum": 1, + "description": "distinct evidence refs backing this claim — the recorded derivation behind confidence", + "type": "integer" + } + } + }, + "correction": { + "$id": "ClaimCorrection", + "additionalProperties": false, + "type": "object", + "required": [ + "verdict", + "at" + ], + "properties": { + "verdict": { + "description": "the human judgment: confirm (real) · reject (not a relationship) · correct (a different relationship)", + "anyOf": [ + { + "const": "confirm", + "type": "string" + }, + { + "const": "reject", + "type": "string" + }, + { + "const": "correct", + "type": "string" + } + ] + }, + "at": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + }, + "by": { + "description": "who made the correction — the user", + "type": "string" + }, + "note": { + "description": "the correction's rationale (never a secret value)", + "type": "string" + } + } + }, + "corrects": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "sessionId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "firstObserved": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + }, + "lastObserved": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + }, + "revision": { + "minimum": 1, + "description": "position in this relationship’s append-only supersession chain", + "type": "integer" + }, + "supersedes": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "schemaVersion": { + "minimum": 1, + "type": "integer" + }, + "createdAt": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + } + } +} diff --git a/shared/contracts/schemas/ClaimCorrection.json b/shared/contracts/schemas/ClaimCorrection.json new file mode 100644 index 0000000..e5ac094 --- /dev/null +++ b/shared/contracts/schemas/ClaimCorrection.json @@ -0,0 +1,41 @@ +{ + "$id": "ClaimCorrection", + "additionalProperties": false, + "type": "object", + "required": [ + "verdict", + "at" + ], + "properties": { + "verdict": { + "description": "the human judgment: confirm (real) · reject (not a relationship) · correct (a different relationship)", + "anyOf": [ + { + "const": "confirm", + "type": "string" + }, + { + "const": "reject", + "type": "string" + }, + { + "const": "correct", + "type": "string" + } + ] + }, + "at": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + }, + "by": { + "description": "who made the correction — the user", + "type": "string" + }, + "note": { + "description": "the correction's rationale (never a secret value)", + "type": "string" + } + } +} diff --git a/shared/contracts/schemas/ClaimEvidenceRef.json b/shared/contracts/schemas/ClaimEvidenceRef.json new file mode 100644 index 0000000..7271bae --- /dev/null +++ b/shared/contracts/schemas/ClaimEvidenceRef.json @@ -0,0 +1,39 @@ +{ + "$id": "ClaimEvidenceRef", + "additionalProperties": false, + "type": "object", + "required": [ + "record", + "id", + "at" + ], + "properties": { + "record": { + "description": "which record table the id names — the closed set of claim evidence sources", + "anyOf": [ + { + "const": "context-packet", + "type": "string" + }, + { + "const": "moment", + "type": "string" + }, + { + "const": "distillate", + "type": "string" + } + ] + }, + "id": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "at": { + "format": "date-time", + "description": "ISO-8601 timestamp", + "type": "string" + } + } +} diff --git a/shared/contracts/schemas/ClaimProvenance.json b/shared/contracts/schemas/ClaimProvenance.json new file mode 100644 index 0000000..dc9a6bf --- /dev/null +++ b/shared/contracts/schemas/ClaimProvenance.json @@ -0,0 +1,20 @@ +{ + "$id": "ClaimProvenance", + "additionalProperties": false, + "type": "object", + "required": [ + "builder", + "evidenceCount" + ], + "properties": { + "builder": { + "const": "deterministic-cooccurrence", + "type": "string" + }, + "evidenceCount": { + "minimum": 1, + "description": "distinct evidence refs backing this claim — the recorded derivation behind confidence", + "type": "integer" + } + } +} diff --git a/shared/contracts/schemas/ClaimRelation.json b/shared/contracts/schemas/ClaimRelation.json new file mode 100644 index 0000000..86b5b53 --- /dev/null +++ b/shared/contracts/schemas/ClaimRelation.json @@ -0,0 +1,30 @@ +{ + "$id": "ClaimRelation", + "description": "the relationship kind: co-occurs-with (symmetric; the only kind the slice-1 deterministic builder emits) · works-on/belongs-to/authored/member-of (directional semantic kinds — judge-enrichment slice + user corrections) · relates-to (generic fallback)", + "anyOf": [ + { + "const": "co-occurs-with", + "type": "string" + }, + { + "const": "works-on", + "type": "string" + }, + { + "const": "belongs-to", + "type": "string" + }, + { + "const": "authored", + "type": "string" + }, + { + "const": "member-of", + "type": "string" + }, + { + "const": "relates-to", + "type": "string" + } + ] +} diff --git a/shared/contracts/schemas/CorrectClaimRequest.json b/shared/contracts/schemas/CorrectClaimRequest.json new file mode 100644 index 0000000..322fe51 --- /dev/null +++ b/shared/contracts/schemas/CorrectClaimRequest.json @@ -0,0 +1,79 @@ +{ + "$id": "CorrectClaimRequest", + "additionalProperties": false, + "type": "object", + "required": [ + "workspaceId", + "claimId", + "verdict" + ], + "properties": { + "workspaceId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "claimId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "verdict": { + "anyOf": [ + { + "const": "confirm", + "type": "string" + }, + { + "const": "reject", + "type": "string" + }, + { + "const": "correct", + "type": "string" + } + ] + }, + "relation": { + "$id": "ClaimRelation", + "description": "the relationship kind: co-occurs-with (symmetric; the only kind the slice-1 deterministic builder emits) · works-on/belongs-to/authored/member-of (directional semantic kinds — judge-enrichment slice + user corrections) · relates-to (generic fallback)", + "anyOf": [ + { + "const": "co-occurs-with", + "type": "string" + }, + { + "const": "works-on", + "type": "string" + }, + { + "const": "belongs-to", + "type": "string" + }, + { + "const": "authored", + "type": "string" + }, + { + "const": "member-of", + "type": "string" + }, + { + "const": "relates-to", + "type": "string" + } + ] + }, + "object": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "by": { + "type": "string" + }, + "note": { + "type": "string" + } + } +} diff --git a/shared/contracts/schemas/RelatedEntity.json b/shared/contracts/schemas/RelatedEntity.json new file mode 100644 index 0000000..2731d19 --- /dev/null +++ b/shared/contracts/schemas/RelatedEntity.json @@ -0,0 +1,72 @@ +{ + "$id": "RelatedEntity", + "additionalProperties": false, + "type": "object", + "required": [ + "entityId", + "name", + "relations", + "confidence", + "claimIds" + ], + "properties": { + "entityId": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + }, + "name": { + "minLength": 1, + "type": "string" + }, + "relations": { + "minItems": 1, + "type": "array", + "items": { + "$id": "ClaimRelation", + "description": "the relationship kind: co-occurs-with (symmetric; the only kind the slice-1 deterministic builder emits) · works-on/belongs-to/authored/member-of (directional semantic kinds — judge-enrichment slice + user corrections) · relates-to (generic fallback)", + "anyOf": [ + { + "const": "co-occurs-with", + "type": "string" + }, + { + "const": "works-on", + "type": "string" + }, + { + "const": "belongs-to", + "type": "string" + }, + { + "const": "authored", + "type": "string" + }, + { + "const": "member-of", + "type": "string" + }, + { + "const": "relates-to", + "type": "string" + } + ] + } + }, + "confidence": { + "minimum": 0, + "maximum": 1, + "type": "number" + }, + "claimIds": { + "minItems": 1, + "description": "live claims backing this link — each traceable to its evidence + scope", + "type": "array", + "items": { + "minLength": 1, + "description": "opaque identifier", + "type": "string" + } + } + } +} diff --git a/shared/contracts/src/api/payloads.ts b/shared/contracts/src/api/payloads.ts index c8f070b..cfb51b2 100644 --- a/shared/contracts/src/api/payloads.ts +++ b/shared/contracts/src/api/payloads.ts @@ -1,5 +1,6 @@ import { Type, type Static } from '@sinclair/typebox' -import { Id, IsoTime } from '../common.js' +import { Id, IsoTime, Confidence } from '../common.js' +import { ClaimRelation } from '../records/claim.js' import { Entity } from '../records/entity.js' import { Moment } from '../records/moment.js' import { StarterModel } from '../config/local.js' @@ -571,6 +572,65 @@ export const BuildSummariesRequest = Type.Object( ) export type BuildSummariesRequest = Static +/** + * The body of `POST /claims/build` (#178) — run the deterministic co-occurrence Claim builder over ONE + * session's stored evidence (its ContextPackets and moments). The caller names only the scope; every claim + * field (ids, endpoints, evidence refs, confidence, revisions) is engine-derived from the stored records, + * mirroring BuildContextPacketsRequest — the caller invents no relationship. The response is the claims this + * run APPENDED (an empty array is the honest idempotent no-op: same evidence ⇒ same claims ⇒ nothing written). + */ +export const BuildClaimsRequest = Type.Object( + { + workspaceId: Id, + sessionId: Id, + }, + { $id: 'BuildClaimsRequest', additionalProperties: false }, +) +export type BuildClaimsRequest = Static + +/** + * The body of `POST /claims/correct` (#178) — record a SOVEREIGN, append-only user correction on a derived + * claim. The correction OUTRANKS the derived claim without deleting it or its evidence (the roadmap + * invariant; the titling-sovereignty pattern). `claimId` names the derived claim being judged; `verdict` is + * the human decision (confirm · reject · correct). A `correct` verdict may assert a different `relation` + * and/or `object` (e.g. "actually a works-on b, not a mere co-occurrence"); confirm/reject leave them absent + * (the correction inherits the target's subject/object/relation). `by`/`note` stamp the human provenance. + */ +export const CorrectClaimRequest = Type.Object( + { + workspaceId: Id, + claimId: Id, + verdict: Type.Union(['confirm', 'reject', 'correct'].map((v) => Type.Literal(v))), + relation: Type.Optional(ClaimRelation), + object: Type.Optional(Id), + by: Type.Optional(Type.String()), + note: Type.Optional(Type.String()), + }, + { $id: 'CorrectClaimRequest', additionalProperties: false }, +) +export type CorrectClaimRequest = Static + +/** + * One counterpart in the depth-1 relationship walk (#178): `GET /claims/related?entity=` answers "what does + * this entity relate to?" by walking entity → its live claims → the counterpart entity at the other end of + * each. `entityId`/`name` name the counterpart; `relations` is the deduped set of relation kinds linking the + * pair; `confidence` is the strongest backing claim's confidence; `claimIds` names the live claims that back + * the link so a consumer can always retrieve the supporting evidence + scope (the #178 invariant: a walk + * result always resolves to its supporting claims). This is the shape #181's typeahead and the recall + * surfaces consume; it is a read-time PROJECTION over the claim store, never a second truth store. + */ +export const RelatedEntity = Type.Object( + { + entityId: Id, + name: Type.String({ minLength: 1 }), + relations: Type.Array(ClaimRelation, { minItems: 1 }), + confidence: Confidence, + claimIds: Type.Array(Id, { minItems: 1, description: 'live claims backing this link — each traceable to its evidence + scope' }), + }, + { $id: 'RelatedEntity', additionalProperties: false }, +) +export type RelatedEntity = Static + /** * The body of `POST /fabric/profiles/:id/clone` — the new profile's id (+ optional name). Cloning is * copying a document (ARCHITECTURE §2/§8): the engine reads the source profile, restamps id/name/ diff --git a/shared/contracts/src/api/routes.ts b/shared/contracts/src/api/routes.ts index ce4a5b0..91f9ac5 100644 --- a/shared/contracts/src/api/routes.ts +++ b/shared/contracts/src/api/routes.ts @@ -86,4 +86,8 @@ export const Routes: readonly RouteDef[] = [ { method: 'POST', path: '/context/packets/build', request: 'BuildContextPacketsRequest', response: 'ContextPacket[]', phase: 4 }, { method: 'GET', path: '/summaries', response: 'Summary[]', phase: 4 }, { method: 'POST', path: '/summaries/build', request: 'BuildSummariesRequest', response: 'Summary[]', phase: 4 }, + { method: 'GET', path: '/claims', response: 'Claim[]', phase: 4 }, + { method: 'GET', path: '/claims/related', response: 'RelatedEntity[]', phase: 4 }, + { method: 'POST', path: '/claims/build', request: 'BuildClaimsRequest', response: 'Claim[]', phase: 4 }, + { method: 'POST', path: '/claims/correct', request: 'CorrectClaimRequest', response: 'Claim', phase: 4 }, ] as const diff --git a/shared/contracts/src/contracts.test.ts b/shared/contracts/src/contracts.test.ts index ef0152e..46ccdce 100644 --- a/shared/contracts/src/contracts.test.ts +++ b/shared/contracts/src/contracts.test.ts @@ -17,7 +17,7 @@ const fileSchema: Record = { ocrInvokeParams: 'OcrInvokeParams', vlmInvokeParams: 'VlmInvokeParams', captureChunk: 'CaptureChunk', captureReceipt: 'CaptureReceipt', focusSignal: 'FocusSignal', calendarSignal: 'CalendarSignal', ack: 'Ack', transcriptUpdate: 'TranscriptUpdate', health: 'Health', queueStatus: 'QueueStatus', queueFailure: 'QueueFailure', distillate: 'Distillate', screenFrameMeta: 'ScreenFrameMeta', ocrResult: 'OcrResult', draft: 'Draft', promptTemplate: 'PromptTemplate', entity: 'Entity', relevantEntity: 'RelevantEntity', fieldValue: 'FieldValue', - sessionAnnotation: 'SessionAnnotation', sessionTitling: 'SessionTitling', sttSegment: 'SttSegment', contextPacket: 'ContextPacket', summary: 'Summary', + sessionAnnotation: 'SessionAnnotation', sessionTitling: 'SessionTitling', sttSegment: 'SttSegment', contextPacket: 'ContextPacket', summary: 'Summary', claim: 'Claim', session: 'Session', startSessionRequest: 'StartSessionRequest', rerouteRequest: 'RerouteRequest', setSessionTitleRequest: 'SetSessionTitleRequest', queryResult: 'QueryResult', pin: 'Pin', pinChunk: 'PinChunk', teachSignal: 'TeachSignal', entityCorrection: 'EntityCorrection', fabricProfile: 'FabricProfile', secretRef: 'SecretRef', secretValue: 'SecretValue', @@ -94,6 +94,33 @@ test('Summary is refs-only, marks prose as a model proposal, and stays honest wh assert.ok([...Value.Errors(AllSchemas.SummaryLevel, 'decade')].length > 0, 'an invented level is rejected') }) +test('Claim is refs-only, evidence is mandatory, and the relation union stays closed (#178)', () => { + const claim = JSON.parse(readFileSync(join(examplesDir, 'claim.coOccurrence.json'), 'utf8')) + assert.deepEqual([...Value.Errors(AllSchemas.Claim, claim)], [], 'a refs-only derived claim validates') + + // EVIDENCE IS MANDATORY: a claim with no evidence refs is UNREPRESENTABLE (minItems:1) — the #178 + // "every claim must be traceable to source observations" invariant, enforced by the schema itself. + assert.ok([...Value.Errors(AllSchemas.Claim, { ...claim, evidence: [] })].length > 0, 'a claim with no evidence is rejected') + + // Refs-only: an evidence ref cannot smuggle copied observation content alongside its id. + const smuggling = { ...claim, evidence: [{ record: 'context-packet', id: 'cp-x', at: claim.firstObserved, text: 'copied content' }] } + assert.ok([...Value.Errors(AllSchemas.Claim, smuggling)].length > 0, 'an evidence ref carrying content is rejected') + + // The relation union is CLOSED — all six kinds are typed even though slice 1 produces only co-occurs-with. + for (const relation of ['co-occurs-with', 'works-on', 'belongs-to', 'authored', 'member-of', 'relates-to']) { + assert.deepEqual([...Value.Errors(AllSchemas.ClaimRelation, relation)], [], `${relation} is a typed relation`) + } + assert.ok([...Value.Errors(AllSchemas.ClaimRelation, 'is-friends-with')].length > 0, 'an invented relation is rejected') + + // The evidence-source set is closed: a distillate/moment/context-packet is allowed, an entity is not + // (a claim rests on converged evidence, never directly on another derived aggregate). + assert.ok([...Value.Errors(AllSchemas.ClaimEvidenceRef, { record: 'entity', id: 'ent-x', at: claim.firstObserved })].length > 0, 'entity is not an evidence source') + + // A SOVEREIGN user correction validates: source 'user' + a correction stamp + the target link, no builder. + const correction = JSON.parse(readFileSync(join(examplesDir, 'claim.userConfirm.json'), 'utf8')) + assert.deepEqual([...Value.Errors(AllSchemas.Claim, correction)], [], 'a user correction validates') +}) + test('CaptureReceipt is metadata-only and rejects raw or derived content fields', () => { const receipt = { id: 'scr-sess-1-000001', sessionId: 'sess-1', workspaceId: 'default', source: 'screen', diff --git a/shared/contracts/src/index.ts b/shared/contracts/src/index.ts index 53377b7..def25ee 100644 --- a/shared/contracts/src/index.ts +++ b/shared/contracts/src/index.ts @@ -19,6 +19,7 @@ export * from './records/sessionTitling.js' export * from './records/sttSegment.js' export * from './records/contextPacket.js' export * from './records/summary.js' +export * from './records/claim.js' export * from './config/voice.js' export * from './config/surface.js' export * from './config/mode.js' @@ -59,6 +60,7 @@ import { SessionTitling, TitlingProvenance, FloorTitlingProvenance } from './rec import { SttSegment } from './records/sttSegment.js' import { ContextPacket, ContextPacketRef, ContextPacketGap, ContextPacketCandidate, ContextPacketProvenance } from './records/contextPacket.js' import { Summary, SummaryLevel, SummaryChild, SummaryInputBound, SummaryProvenance } from './records/summary.js' +import { Claim, ClaimRelation, ClaimEvidenceRef, ClaimProvenance, ClaimCorrection } from './records/claim.js' import { Dials, Register, VoiceBinding, DriftChainStep, DriftConfig } from './config/voice.js' import { Surface, Block, BlockQuery, Action, BlockTypeName, AttachedPanel } from './config/surface.js' import { Mode } from './config/mode.js' @@ -74,7 +76,7 @@ import { WorkflowSpec, WorkflowStep, WorkflowStepKind, StepGate } from './config import { Bundle, BundleFace, BundleFaceKind, ChatContextAssembly, ChatContextSource, ChatContextSourceKind } from './config/bundle.js' import { OcrInvokeParams, VlmInvokeParams } from './config/invoke.js' import { TodoList, TodoItem, TodoProvenance } from './config/todo.js' -import { Health, JsonSchema, CaptureSource, CaptureChunk, CaptureReceipt, FocusSignal, CalendarSignal, Ack, TranscriptUpdate, SttSlotEndpoint, TranscriptInspector, QueueStatus, QueueFailure, QueueKind, QueueKindDepth, BacklogEta, BacklogLag, OverflowState, ScreenStatus, RelevantEntity, HintCandidate, QueryResult, StartSessionRequest, RerouteRequest, SetSessionTitleRequest, BuildContextPacketsRequest, BuildSummariesRequest, CloneProfileRequest, EntityCorrection, SecretRef, SecretValue, EndpointProbe, GenerateProbe, LocalModelStatus, LocalDownloadRequest, ChatTurn, ChatRequest, ChatCitation, ChatBudget, ChatReply, ChatScreenshot, ChatDelta, ChatHistory } from './api/payloads.js' +import { Health, JsonSchema, CaptureSource, CaptureChunk, CaptureReceipt, FocusSignal, CalendarSignal, Ack, TranscriptUpdate, SttSlotEndpoint, TranscriptInspector, QueueStatus, QueueFailure, QueueKind, QueueKindDepth, BacklogEta, BacklogLag, OverflowState, ScreenStatus, RelevantEntity, HintCandidate, QueryResult, StartSessionRequest, RerouteRequest, SetSessionTitleRequest, BuildContextPacketsRequest, BuildSummariesRequest, BuildClaimsRequest, CorrectClaimRequest, RelatedEntity, CloneProfileRequest, EntityCorrection, SecretRef, SecretValue, EndpointProbe, GenerateProbe, LocalModelStatus, LocalDownloadRequest, ChatTurn, ChatRequest, ChatCitation, ChatBudget, ChatReply, ChatScreenshot, ChatDelta, ChatHistory } from './api/payloads.js' import { PhysicalSenseSource, SenseLaneDisposition, SenseLaneHealth, SenseLaneReason, SenseLaneLagBasis, SenseLaneCapture, ScreenCaptureObservation, ScreenLaneObservation, ScreenProcessingOutcome, SenseLaneProcessing, SenseLaneSnapshot, SenseLaneSnapshotSet } from './api/senses.js' /** Every schema, by $id — the registry schema-gen, tests, and the engine's /contracts route walk. */ @@ -83,6 +85,7 @@ export const AllSchemas = { Moment, MomentKind, Entity, Sighting, HeardAs, EntityOverride, EntityAmbiguity, EntityExternal, EntityResolution, Distillate, ScreenFrameMeta, OcrResult, Draft, Session, Workspace, Pin, PinChunk, TeachSignal, TeachSignalKind, EntityCorrectionSignal, Commitment, Watcher, ItemSignal, ItemSignalKind, FieldValue, FieldValueProvenance, JudgeReview, SessionAnnotation, OrientationProvenance, SessionTitling, TitlingProvenance, FloorTitlingProvenance, SttSegment, ContextPacket, ContextPacketRef, ContextPacketGap, ContextPacketCandidate, ContextPacketProvenance, Summary, SummaryLevel, SummaryChild, SummaryInputBound, SummaryProvenance, + Claim, ClaimRelation, ClaimEvidenceRef, ClaimProvenance, ClaimCorrection, Dials, Register, VoiceBinding, DriftChainStep, DriftConfig, Surface, Block, BlockQuery, Action, BlockTypeName, AttachedPanel, Mode, PromptTemplate, FastFieldBinding, SummaryBinding, SummaryScope, EgressReach, EgressDestination, EgressLayer, ContentClass, EgressPolicy, EgressDecision, GuardBehavior, GuardSpan, GuardVerdict, GuardPolicy, GuardHoldStatus, GuardHold, Fabric, Endpoint, LocalRuntime, FabricProfile, @@ -90,7 +93,7 @@ export const AllSchemas = { WorkflowSpec, WorkflowStep, WorkflowStepKind, StepGate, OcrInvokeParams, VlmInvokeParams, Bundle, BundleFace, BundleFaceKind, ChatContextAssembly, ChatContextSource, ChatContextSourceKind, TodoList, TodoItem, TodoProvenance, - Health, JsonSchema, CaptureSource, CaptureChunk, CaptureReceipt, FocusSignal, CalendarSignal, Ack, TranscriptUpdate, SttSlotEndpoint, TranscriptInspector, QueueStatus, QueueFailure, QueueKind, QueueKindDepth, BacklogEta, BacklogLag, OverflowState, ScreenStatus, RelevantEntity, HintCandidate, QueryResult, StartSessionRequest, RerouteRequest, SetSessionTitleRequest, BuildContextPacketsRequest, BuildSummariesRequest, + Health, JsonSchema, CaptureSource, CaptureChunk, CaptureReceipt, FocusSignal, CalendarSignal, Ack, TranscriptUpdate, SttSlotEndpoint, TranscriptInspector, QueueStatus, QueueFailure, QueueKind, QueueKindDepth, BacklogEta, BacklogLag, OverflowState, ScreenStatus, RelevantEntity, HintCandidate, QueryResult, StartSessionRequest, RerouteRequest, SetSessionTitleRequest, BuildContextPacketsRequest, BuildSummariesRequest, BuildClaimsRequest, CorrectClaimRequest, RelatedEntity, CloneProfileRequest, EntityCorrection, SecretRef, SecretValue, EndpointProbe, GenerateProbe, LocalModelStatus, LocalDownloadRequest, ChatTurn, ChatRequest, ChatCitation, ChatBudget, ChatReply, ChatScreenshot, ChatDelta, ChatHistory, PhysicalSenseSource, SenseLaneDisposition, SenseLaneHealth, SenseLaneReason, SenseLaneLagBasis, SenseLaneCapture, ScreenCaptureObservation, ScreenLaneObservation, ScreenProcessingOutcome, SenseLaneProcessing, SenseLaneSnapshot, SenseLaneSnapshotSet, diff --git a/shared/contracts/src/records/claim.ts b/shared/contracts/src/records/claim.ts new file mode 100644 index 0000000..2e701e6 --- /dev/null +++ b/shared/contracts/src/records/claim.ts @@ -0,0 +1,169 @@ +import { Type, type Static } from '@sinclair/typebox' +import { Id, IsoTime, Confidence } from '../common.js' + +/** Schema version of the Claim record shape — bumped when the persisted shape changes. */ +export const CLAIM_SCHEMA_VERSION = 1 + +/** + * The relationship kinds a Claim can assert (#178). CLOSED starter union — the honest extensibility + * decision. + * + * The union is CLOSED (not an open string) for the same reason MomentKind / EntityKind / SummaryLevel are: + * a claim's relation is a QUERY axis (and, later, a UI/semantic axis), so an open string invites + * un-queryable drift — a typo'd or free-text kind that nothing can index and no surface can render (the + * recurring "un-sane default" failure). A closed union keeps every kind inspectable and routable, and the + * generic `relates-to` member is the deliberate escape valve so the union never has to force a wrong-fit + * kind onto a genuine association it cannot yet classify. Extending BEYOND these is a reviewed contract + * change (a new literal + `CLAIM_SCHEMA_VERSION` bump + regenerated schema) — deliberate, not accidental. + * + * PRODUCTION IN SLICE 1 (#178 s1): the deterministic builder emits ONLY `co-occurs-with`. Co-occurrence + * (two entities named together in the same converged evidence — a ContextPacket window or one moment) is + * the ONLY relation that is TRUE BY CONSTRUCTION from the evidence, with no model in the loop: it asserts + * "these were observed together", never a semantic reading of HOW they relate. The SEMANTIC kinds + * (`works-on` person→project/repo · `belongs-to` repo→application · `authored` person→document/PR · + * `member-of` person→organization · `relates-to` generic) are the roadmap's named relationships; deriving + * them requires judgment, so they are reserved for the JUDGE-ENRICHMENT slice (a later pass; its output is + * a proposal per #189) and for SOVEREIGN USER corrections. The enum is COMPLETE now — all kinds are typed + * even though slice 1 produces one — exactly as SummaryLevel typed all five levels while slice 1 produced + * three. + * + * `co-occurs-with` is SYMMETRIC; the semantic kinds are DIRECTIONAL (subject → object). For a symmetric + * relation the builder canonically orders the endpoint pair by entity id, so (A,B) and (B,A) collapse to a + * single claim rather than two mirror claims. + */ +export const ClaimRelation = Type.Union( + ['co-occurs-with', 'works-on', 'belongs-to', 'authored', 'member-of', 'relates-to'].map((r) => Type.Literal(r)), + { + $id: 'ClaimRelation', + description: + 'the relationship kind: co-occurs-with (symmetric; the only kind the slice-1 deterministic builder emits) · works-on/belongs-to/authored/member-of (directional semantic kinds — judge-enrichment slice + user corrections) · relates-to (generic fallback)', + }, +) +export type ClaimRelation = Static + +/** + * One reference to a stored evidence record that backs a claim (#178). A claim NEVER copies observation + * content — it points at the immutable record by id, so every assertion it makes stays traceable to source + * observations and no ambient content is duplicated into a second table. The record set is the closed set + * of evidence a relationship claim can rest on: a converged `context-packet` (whose candidates named both + * entities in one window), a `moment` (whose refs named both entities in one extracted moment), or a + * `distillate` (a window's derived text — reserved for the judge-enrichment slice). `at` is the evidence + * instant, carried so a reader can see WHEN the co-occurrence was witnessed without loading the record. + */ +export const ClaimEvidenceRef = Type.Object( + { + record: Type.Union(['context-packet', 'moment', 'distillate'].map((r) => Type.Literal(r)), { + description: 'which record table the id names — the closed set of claim evidence sources', + }), + id: Id, + at: IsoTime, + }, + { $id: 'ClaimEvidenceRef', additionalProperties: false }, +) +export type ClaimEvidenceRef = Static + +/** + * How a DERIVED claim was built (#178). Slice 1 has exactly one builder — the deterministic engine-side + * co-occurrence builder (`index/claims.ts`): pure structural correlation over already-stored evidence, no + * model in the loop. The literal keeps that inspectable — a future JUDGE-ENRICHMENT builder must declare + * itself as a NEW literal (fast-model output is a proposal, not truth — #189), it can never impersonate the + * deterministic one. `evidenceCount` is the number of distinct evidence refs the claim rests on: it is the + * RECORDED DERIVATION behind `confidence`, so repeated evidence strengthens a claim only through a NEW + * revision that records a higher count — never a silent in-place mutation of the number (#178 AC). + * + * Present ONLY on a `source:'derived'` claim; a `source:'user'` correction carries `correction` instead + * (a human decision has no builder), so a reader can never mistake a sovereign correction for a derivation. + */ +export const ClaimProvenance = Type.Object( + { + builder: Type.Literal('deterministic-cooccurrence'), + evidenceCount: Type.Integer({ minimum: 1, description: 'distinct evidence refs backing this claim — the recorded derivation behind confidence' }), + }, + { $id: 'ClaimProvenance', additionalProperties: false }, +) +export type ClaimProvenance = Static + +/** + * A SOVEREIGN user correction stamp (#178) — recorded as first-class, append-only data that OUTRANKS any + * derived claim (the product invariant: human decisions create append-only sovereign corrections, and they + * never delete the original inference or its evidence — the titling-sovereignty pattern from #211/#232). + * `verdict` is the human's judgment on the claim it `corrects`: + * - `confirm` — this relationship is real (the resolved head becomes the confirmed claim, confidence 1); + * - `reject` — this is NOT a relationship (the pair drops out of the live relationship view; the derived + * claim and its evidence are retained, retrievable as history — never deleted); + * - `correct` — the relationship is different (the user asserts a corrected relation/object; the original + * inference is preserved and superseded by the user's assertion). + * `by` is the actor (the user — never a machine, since a correction is by definition a human decision). + * + * Present ONLY on a `source:'user'` claim; a `source:'derived'` claim carries `provenance` instead. + */ +export const ClaimCorrection = Type.Object( + { + verdict: Type.Union(['confirm', 'reject', 'correct'].map((v) => Type.Literal(v)), { + description: 'the human judgment: confirm (real) · reject (not a relationship) · correct (a different relationship)', + }), + at: IsoTime, + by: Type.Optional(Type.String({ description: 'who made the correction — the user' })), + note: Type.Optional(Type.String({ description: "the correction's rationale (never a secret value)" })), + }, + { $id: 'ClaimCorrection', additionalProperties: false }, +) +export type ClaimCorrection = Static + +/** + * A durable, append-only assertion about a relationship between two entities (#178) — subject `relation` + * object — grounded in EVIDENCE refs and never in copied content. A claim is a DERIVED runtime unit built + * ONLY from references to stored evidence records; it never replaces, copies, or rewrites the observations + * it points at. + * + * PROPOSAL, NOT TRUTH: a `source:'derived'` claim is a PROPOSAL (`state:'provisional'`) — confidence is + * framed as a proposal, deterministically derived from the evidence count (never a model score, never + * fabricated), and a human correction supersedes it. A `source:'user'` claim is a SOVEREIGN correction + * (the roadmap invariant): it outranks any derived claim and is never overwritten by a later derivation. + * + * EVIDENCE IS MANDATORY: `evidence` is `minItems:1`, so a claim with no evidence refs is UNREPRESENTABLE — + * every claim, derived or user, is traceable to source observations (a user correction carries the same + * evidence refs as the claim it corrects, so it stays traceable to the same observations it reinterprets). + * + * SUPERSESSION is append-only: when a rebuild sees MORE evidence for a (session, subject, object, relation) + * a NEW claim supersedes the prior (`revision + 1`, `supersedes` naming it) — the prior is never mutated or + * deleted, so the chain stays walkable and the strengthened `confidence` is a recorded derivation. `id` is + * content-derived (a hash over the claim's evidence-derived content + chain position), so rebuilding the + * same evidence is idempotent: byte-identical claim, same id, no new revision. User corrections are resolved + * OVER the derived chain at read time (see the store's `resolveClaimHeads`): a corrected/rejected pair never + * reappears merely because the builder re-derived it. + * + * `firstObserved`/`lastObserved` are the earliest/latest evidence instants — the claim's honest valid-time + * span, derived from the evidence, never guessed. `sessionId` names the session the derivation ran over + * (present on derived claims and on corrections of them); the relationship graph is queried WORKSPACE-wide + * (across sessions) and aggregated at read time — indexes are rebuildable projections, not the truth store. + */ +export const Claim = Type.Object( + { + id: Id, + workspaceId: Id, + subject: Id, + object: Id, + relation: ClaimRelation, + evidence: Type.Array(ClaimEvidenceRef, { minItems: 1, description: 'refs to the evidence records backing this claim — a claim with no evidence is unrepresentable' }), + confidence: Confidence, + source: Type.Union(['derived', 'user'].map((s) => Type.Literal(s)), { + description: 'derived (deterministic builder proposal) · user (sovereign human correction, outranks derived)', + }), + state: Type.Union(['provisional', 'confirmed', 'corrected', 'rejected', 'superseded'].map((s) => Type.Literal(s)), { + description: 'provisional (derived proposal) · confirmed/corrected/rejected (user verdicts) · superseded (an older revision a newer one replaced)', + }), + provenance: Type.Optional(ClaimProvenance), + correction: Type.Optional(ClaimCorrection), + corrects: Type.Optional(Id), + sessionId: Type.Optional(Id), + firstObserved: IsoTime, + lastObserved: IsoTime, + revision: Type.Integer({ minimum: 1, description: 'position in this relationship’s append-only supersession chain' }), + supersedes: Type.Optional(Id), + schemaVersion: Type.Integer({ minimum: 1 }), + createdAt: IsoTime, + }, + { $id: 'Claim', additionalProperties: false }, +) +export type Claim = Static