Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/airtightness-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ action. No keyword/regex bank interprets user intent in the enforcement path.
| Constraint | File(s) | Mechanism |
| --- | --- | --- |
| Reflections and recaps use the user's words | `validator.ts`, `stage1-loop.ts` | Every distinct substantive content word in each claim must be supported by cited eligible evidence; ordinary function-word glue is ignored. Visible text is reconstructed from validated claims rather than trusted from a model-authored wrapper. |
| Reflections and recaps keep the user's negation | `validator.ts`, `stage1-loop.ts` | Negation words (`not`/`no`/`never`/n't/`without`…) are taken out of the free-glue exemption: a claim must use exactly the negation words of the `userPhrase`s and `relationSpan` it cites. Inside this check n't, `not` and apostrophe-less spellings compare equal; lexical grounding still requires the user's own spelling, so end to end the mirror keeps it. This is pointer verification, not meaning detection: no clause parsing or idiom list, so negation elsewhere in the user's sentence never blocks a mirror, but a pointer wider than the claim that carries a negation the claim omits is rejected, and the repair note asks for the narrower pointer. Known gap: a negation dropped by citing the words on either side of it is not caught; the user's per-chunk confirmation covers it. English only. |
| Relationships/hierarchy are stated by the user in one breath | `validator.ts` | Each cited span must ground `>= 0.75` of its phrase within a *single* utterance. For hierarchy/connection claims, the model declares the literal `relationSpan` connective; code verifies it is a verbatim substring of both the claim text and one cited utterance, and that most claim content words ground in that same utterance. No relation-word bank participates. |
| A reflection is only ever a proposal | `stage1-loop.ts`, `proposal-store.ts`, `App.tsx` | A validated reflection becomes a `shown` proposal with per-claim confirm/decline; only confirmed chunks reach the gateway and become cards. |
| Reflections never carry novel L2 prose | `stage1-loop.ts` | Reflections remain source-backed at every level. L0 is limited to one recorded user moment; L1/L2 cross-turn or draft+chat synthesis is accepted as `ai_connected`. Novel L2 language must use an attributed suggestion or map proposal. |
Expand Down
15 changes: 14 additions & 1 deletion src/api.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,18 @@ describe("typed assistant response parser", () => {
expect(first[0].content).toContain("a reflection may draw from only one recorded user moment");
expect(first[0].content).toContain("explicitly instructs you to nest one referenced card in another");
expect(first[0].content).toContain("Use direct quotation only when it makes the referent clearer");
expect(first[0].content).toContain("For a large or abstract turn");
expect(first[0].content).toContain("For a large or abstract turn that does not state its structure explicitly");
// Stated structure is mirrored, not questioned; size alone never justifies a question.
expect(first[0].content).toContain("STATED STRUCTURE:");
expect(first[0].content).toContain("mirror it in that turn rather than asking about it");
expect(first[0].content).toContain("each [id] in the Source Bank is one sentence");
expect(first[0].content).toContain("Turn length is never by itself a reason to ask instead of mirror.");
expect(first[0].content).toContain("Negation words are not glue");
expect(first[0].content).toContain("of the userPhrases and relationSpan it cites");
expect(first[0].content).toContain("spelled as the user wrote it");
expect(first[0].content).toContain("cite only the words the claim reproduces");
expect(first[0].content).toContain("takes precedence over the turn-shape, map-pacing, and reflection-rhythm facts");
expect(first[0].content).toContain("one recorded user moment (one user turn, which may contain several sentences and several claims)");
expect(fetchMock).toHaveBeenCalledTimes(2);
});

Expand Down Expand Up @@ -237,6 +248,8 @@ describe("typed assistant response parser", () => {
expect(messages[0].content).toContain("never invent a dangling candidateId");
expect(messages[0].content).toContain("one exact contiguous substring from DRAFT");
expect(messages[0].content).not.toContain("L0 NON-DIRECTIVE OBJECTIVE");
// The stated-structure rule is level-independent.
expect(messages[0].content).toContain("STATED STRUCTURE:");
});

it("uses a bounded Responses tool repair with the matching call id", async () => {
Expand Down
7 changes: 5 additions & 2 deletions src/api.ts

Large diffs are not rendered by default.

42 changes: 42 additions & 0 deletions src/stage1-loop.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { buildContext, createConversationState, deriveClaimAttribution, MAX_MODE
import { cardRef, resetIdCounter } from "./store";
import { ASSISTANCE_CONTRACTS } from "./assistance-contract";
import { ModelResponseValidationError } from "./assistant-response";
import { POLARITY_REPAIR_NOTE } from "./validator";

beforeEach(() => resetIdCounter());

Expand Down Expand Up @@ -154,6 +155,47 @@ describe("typed Stage 1 controller", () => {
expect(result.response).toMatchObject({ kind: "question" });
});

it("repairs a polarity rejection from a broad pointer without reaching the forced question", async () => {
const state = createConversationState();
const store = new ThoughtUnitStore();
const model = vi.fn(async (context: import("./llm-contract").LLMContext, rejection?: import("./assistant-response").StructuredRejection) => {
const sourceId = context.bank[0]!.id;
const id = rejection ? "c2" : "c1";
// First attempt cites a pointer wider than the claim, carrying a "not" the claim leaves out.
const userPhrase = rejection ? "money matters" : "Not only money matters";
return {
response: { kind: "reflection" as const, text: "money matters", reflection: { claims: [{
id, text: "money matters", candidateId: id, target: "idea" as const,
sourceSpans: [{ claimText: "money matters", userPhrase, utteranceIds: [sourceId] }],
}] } },
advisory: { candidateUpserts: [{ id, target: "idea" as const, gist: "money", addEvidenceIds: [sourceId], status: "active" as const }] },
};
});

const result = await processTurn(state, "Not only money matters, but time matters too.", model, defaultConfig, store.toLLMContext(), { mapRevision: 0, requireConnectionLabel: true, store });
expect(model).toHaveBeenCalledTimes(2);
expect(model.mock.calls[1]?.[1]).toMatchObject({ code: "reflection_validation_failed", reflectionRecovery: { stage: "informed_repair" } });
expect(model.mock.calls[1]?.[1]?.detail).toContain(POLARITY_REPAIR_NOTE);
expect(result.response).toMatchObject({ kind: "reflection", text: "money matters" });
});

it("forwards the polarity repair note when a grounded recap's negation does not match its pointer", async () => {
const state = createConversationState();
const store = new ThoughtUnitStore();
const model = vi.fn(async (context: import("./llm-contract").LLMContext, rejection?: import("./assistant-response").StructuredRejection) => {
if (rejection) return { response: { kind: "question" as const, text: "What matters most here?" } };
const sourceId = context.bank[0]!.id;
return { response: { kind: "grounded_recap" as const, text: "recap", recap: { claims: [{
id: "r1", text: "money is the main problem", target: "idea" as const,
sourceSpans: [{ claimText: "money is the main problem", userPhrase: "Money is not the main problem", utteranceIds: [sourceId] }],
}] } } };
});

await processTurn(state, "Money is not the main problem.", model, defaultConfig, store.toLLMContext(), { mapRevision: 0, requireConnectionLabel: true, store });
expect(model.mock.calls[1]?.[1]).toMatchObject({ code: "grounded_recap_validation_failed" });
expect(model.mock.calls[1]?.[1]?.detail).toContain(POLARITY_REPAIR_NOTE);
});

it("repairs a question whose draft anchor is not an exact current-draft substring", async () => {
const state = createConversationState();
state.draft = "Those classes probably did matter.";
Expand Down
5 changes: 3 additions & 2 deletions src/stage1-loop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import { CandidateStore, SourceBank } from "./store";
import { detectTurnShape } from "./turn-shape";
import { detectLatestUserLanguagePattern, type LanguageContext } from "./language-context";
import type { CandidateThought, CandidateTarget, GroundedClaim, SourceUtterance } from "./types";
import { validateGroundedClaims, validateMirror } from "./validator";
import { POLARITY_REPAIR_NOTE, validateGroundedClaims, validateMirror } from "./validator";

export interface ProcessTurnOptions {
mapRevision: number;
Expand Down Expand Up @@ -240,7 +240,8 @@ function createProposal(envelope: AssistantResponseEnvelope, state: Conversation
}
const validation = validateGroundedClaims(claims, state.bank.getAll(), config);
if (!validation.ok || !claims.every((claim) => deriveClaimAttribution(claim, state.bank.getAll()) === "asserted")) {
return { rejection: { code: "grounded_recap_validation_failed", detail: "The recap must use exact, source-backed user wording." }, diagnostics: [diagnostic("validation", "rejected", "grounded_recap_validation_failed", "Grounded recap evidence pointers did not validate.")] };
const polarityFailed = validation.claims.some((claim) => claim.checks.some((check) => check.check === "polarity" && !check.ok));
return { rejection: { code: "grounded_recap_validation_failed", detail: `The recap must use exact, source-backed user wording.${polarityFailed ? ` ${POLARITY_REPAIR_NOTE}` : ""}` }, diagnostics: [diagnostic("validation", "rejected", "grounded_recap_validation_failed", "Grounded recap evidence pointers did not validate.")] };
}
return { diagnostics: [diagnostic("validation", "accepted", "grounded_recap_valid", "Grounded recap evidence pointers validated; no map proposal was created.")] };
}
Expand Down
2 changes: 1 addition & 1 deletion src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ export interface GroundedRecap {
claims: GroundedClaim[];
}

export type MirrorCheckName = "lexical_grounding" | "span_grounding";
export type MirrorCheckName = "lexical_grounding" | "span_grounding" | "polarity";

/** A sub-part of a check, reported for transparency/calibration. */
export interface MirrorCheckPart {
Expand Down
98 changes: 98 additions & 0 deletions src/validator.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -505,3 +505,101 @@ describe("mirror validator — 2 grounding checks", () => {
expect(result.ok).toBe(true);
});
});

describe("mirror validator — polarity", () => {
const span = (text: string, utterance: SourceUtterance, phrase = text) => ({ claimText: text, utteranceIds: [utterance.id], userPhrase: phrase });
const polarityOf = (text: string, spans: ReturnType<typeof span>[], bank: SourceUtterance[]) =>
checkOf(validateMirror({ claims: [claim(text, spans)] }, bank, defaultConfig).claims[0], "polarity")?.ok;

it("rejects a mirror that adds a negation the user never said", () => {
const said = u("Money is the main problem in my essay.");
for (const text of ["Money is not the main problem", "Money is never the main problem", "Money isn't the main problem", "Money is no main problem"]) {
const result = validateMirror({ claims: [claim(text, [span(text, said, "Money is the main problem")])] }, [said], defaultConfig);
expect(result.ok, text).toBe(false);
expect(checkOf(result.claims[0], "polarity")?.ok, text).toBe(false);
expect(result.claims[0].message).toContain("negation words do not match");
}
});

it("rejects a mirror that drops a negation inside the cited phrase", () => {
const said = u("Money isn't the main problem in my essay.");
const result = validateMirror({ claims: [claim("Money is the main problem", [span("Money is the main problem", said, "Money isn't the main problem")])] }, [said], defaultConfig);
expect(checkOf(result.claims[0], "polarity")?.ok).toBe(false);
expect(result.ok).toBe(false);
});

it("rejects swapping one negation word for another", () => {
const said = u("Money is not the main problem, and it is never easy.");
const result = validateMirror({ claims: [claim("Money is never the main problem", [span("Money is never the main problem", said, "Money is not the main problem")])] }, [said], defaultConfig);
expect(checkOf(result.claims[0], "polarity")?.ok).toBe(false);
expect(result.ok).toBe(false);
});

it("compares n't, apostrophe-less and spelled-out 'not' as the same negation word", () => {
const said = u("Money is not the main problem, and I can’t ignore time.");
expect(polarityOf("Money isn't the main problem", [span("Money isn't the main problem", said, "Money is not the main problem")], [said])).toBe(true);
expect(polarityOf("Money isnt the main problem", [span("Money isnt the main problem", said, "Money is not the main problem")], [said])).toBe(true);
expect(polarityOf("I cannot ignore time", [span("I cannot ignore time", said, "I can’t ignore time")], [said])).toBe(true);
});

// Spelling equivalence is polarity-only: lexical grounding still requires the
// user's own spelling, exactly as on main. The prompt asks the model to keep it.
it("still requires the user's own negation spelling end to end", () => {
const said = u("Money is not the main problem, and I can’t ignore time.");
expect(validateMirror({ claims: [claim("Money isn't the main problem", [span("Money isn't the main problem", said, "Money is not the main problem")])] }, [said], defaultConfig).ok).toBe(false);
expect(validateMirror({ claims: [claim("Money is not the main problem", [span("Money is not the main problem", said)])] }, [said], defaultConfig).ok).toBe(true);
expect(validateMirror({ claims: [claim("I can't ignore time", [span("I can't ignore time", said, "I can’t ignore time")])] }, [said], defaultConfig).ok).toBe(true);
});

it("rejects a pointer that carries a negation the claim leaves out, and accepts the narrower pointer", () => {
const said = u("Not only money matters, but time matters too.");
const broad = validateMirror({ claims: [claim("money matters", [span("money matters", said, "Not only money matters")])] }, [said], defaultConfig);
expect(checkOf(broad.claims[0], "polarity")?.ok).toBe(false);
expect(broad.claims[0].message).toContain("cite only the words the claim reproduces");
expect(validateMirror({ claims: [claim("money matters", [span("money matters", said)])] }, [said], defaultConfig).ok).toBe(true);
});

it("does not read a standalone letter T as a negation", () => {
const said = u("Section T: safety matters.");
const result = validateMirror({ claims: [claim("Section T safety matters", [span("Section T safety matters", said, "Section T: safety matters")])] }, [said], defaultConfig);
expect(checkOf(result.claims[0], "polarity")?.ok).toBe(true);
expect(result.ok).toBe(true);
});

// The check compares a claim with its cited pointers, not the surrounding
// sentence, so negation elsewhere in the user's sentence never blocks a mirror.
it.each([
["Not only money matters, but time matters too.", "money matters"],
["Without delay, submit the draft.", "submit the draft"],
["No matter what, clarity matters.", "clarity matters"],
["Money matters, not time.", "money matters"],
["I do not want a long intro; the argument comes first.", "the argument comes first"],
])("keeps a faithful unnegated mirror from %s passing", (sentence, phrase) => {
const said = u(sentence);
const result = validateMirror({ claims: [claim(phrase, [span(phrase, said)])] }, [said], defaultConfig);
expect(checkOf(result.claims[0], "polarity")?.ok).toBe(true);
expect(result.ok).toBe(true);
});

it("keeps faithful negated mirrors passing", () => {
const negated = u("Money doesn't matter as much as time.");
const result = validateMirror({ claims: [claim("money doesn't matter as much as time", [span("money doesn't matter as much as time", negated, "Money doesn't matter as much as time")])] }, [negated], defaultConfig);
expect(result.ok).toBe(true);
});

it("counts a negation carried only by the relationSpan", () => {
const said = u("Speed does not support quality.");
const spans = [span("speed", said), span("quality", said)];
const negated = validateMirror({ claims: [claim("speed does not support quality", spans, "connection", { utteranceId: said.id, text: "does not support" })] }, [said], defaultConfig);
expect(checkOf(negated.claims[0], "polarity")?.ok).toBe(true);
const dropped = validateMirror({ claims: [claim("speed does support quality", spans, "connection", { utteranceId: said.id, text: "does not support" })] }, [said], defaultConfig);
expect(checkOf(dropped.claims[0], "polarity")?.ok).toBe(false);
});

it("does not treat Chinese text as negated", () => {
const said = u("写作自由很重要");
const result = validateMirror({ claims: [claim("写作自由很重要", [span("写作自由很重要", said)])] }, [said], defaultConfig);
expect(checkOf(result.claims[0], "polarity")?.ok).toBe(true);
expect(result.ok).toBe(true);
});
});
Loading
Loading