diff --git a/docs/adr/0028-host-owned-llm-writing-diagnostics.md b/docs/adr/0028-host-owned-llm-writing-diagnostics.md new file mode 100644 index 00000000..3f903133 --- /dev/null +++ b/docs/adr/0028-host-owned-llm-writing-diagnostics.md @@ -0,0 +1,165 @@ +# ADR 0028: Host-owned, revision-bound writing diagnostics + +Status: Proposed + +## Context + +Inkspan already provides deterministic authoring, revision evidence, revision-scoped W3C `TextPositionSelector` evidence, guarded restore, and a host-owned model-assistance boundary. Host applications need a Grammarly-like writing-guidance surface that can mark a passage, explain a concern, propose a replacement, let the author act explicitly, and keep every proposal bound to the exact document revision reviewed by the host. + +Spelling, grammar, clarity, concision, discourse structure, workplace pragmatics, audience appropriateness, technical precision, actionability, and preservation of author intent are contextual language judgments. Fixed keyword lists, regular-expression phrase detectors, domain-suffix lists, hand-written phrase tables, opaque category names, and positional repair rules cannot establish those meanings across paraphrases, quotations, languages, and recipient contexts. + +Inkspan must remain a modular editor. It must not gain email semantics, tenant policy, model credentials, network transport, an LLM provider dependency, persistence authority, or authority to decide whether prose is appropriate. Hosts likewise should not have to scrape ProseMirror DOM nodes or maintain private editor forks to display revision-safe guidance. + +## Alternatives considered + +- **Host-specific DOM overlays.** Rejected because DOM offsets are not a supported document contract, become stale after transactions, fragment keyboard/screen-reader behavior, and force each host to rebuild navigation, application, conflict handling, and undo semantics. +- **A deterministic keyword or regex checker inside Inkspan.** Rejected because lexical triggers are not evidence of grammar, intent, tone, pragmatics, or technical correctness. Deterministic code may validate shape and coordinates but may not manufacture semantic judgments. +- **Inkspan invokes an LLM directly.** Rejected because the editor would acquire provider, credential, network, privacy, retention, availability, routing, and tenant-policy responsibilities that belong to the host. +- **The host sends whole-document rewrites.** Rejected because replacement obscures individual reasons, weakens author control, destroys revision-local evidence, and makes unintended meaning changes harder to detect. +- **Preserve diagnostics by mapping ranges across edits.** Rejected for v1 because structural position continuity does not prove that a host semantic judgment remains current after content changes. +- **Host-owned judgments through a generic Inkspan diagnostic contract.** Selected because it preserves Inkspan's provider-neutral deterministic core while giving every host one revision-safe, accessible presentation and action surface. + +## Decision + +Inkspan will expose an additive writing-diagnostic contract. The host supplies already-produced proposals. Inkspan validates, revision-binds, resolves, renders, navigates, applies one selected replacement, ignores, dismisses, requests an explanation, and reports privacy-minimized action evidence. Inkspan does not decide whether prose is correct or appropriate. + +A v1 diagnostic includes: + +- a bounded opaque `diagnosticId`; +- an exact `CwlEditorDocumentRevision` for the reviewed immutable snapshot; +- an exact `CwlEditorTextProjectionIdentity`; +- a W3C `TextPositionSelector` using inclusive `start` and exclusive `end` Unicode-code-point offsets; +- an opaque bounded `categoryCode`; +- one of `advisory`, `important`, or `critical` presentation priority; +- a plain-text title and explanation; +- an optional plain-text replacement; +- optional finite confidence in `[0, 1]`; and +- privacy-minimized workflow/judge-policy provenance identifiers with no source text, credential, or raw model output. + +Selector values are non-negative safe integers with `start <= end`. Collapsed selectors are valid evidence and remain navigable, but create no inline range decoration. + +Inkspan performs only deterministic validation and editor operations: + +- exact schema/type/enum/length/count/duplicate/resource validation; +- rejection of accessors, prototypes, symbols, extra fields, sparse arrays, and hostile reflection; +- projection-version and strong-revision checks; +- Unicode-code-point and grapheme-boundary checks; +- exact selector resolution against one immutable projected snapshot; +- safe plain-text rendering and replacement handling; +- stale-generation rejection; +- one-action-at-a-time replacement, ordinary undo, and privacy-minimized action callbacks. + +Every local or collaborative transaction with docChanged === true invalidates the complete active diagnostic generation. + +No diagnostic is preserved, mapped, remapped, repaired, or re-admitted through ProseMirror mapping, Yjs relative positions, nearest-text search, quote search, keyword search, or semantic guessing. A host that wants current guidance after an edit supplies a new set bound to a newly derived exact revision. + +Version 1 applies exactly one explicitly selected diagnostic at a time. Immediately before mutation, Inkspan derives and compares the exact current revision again. A stale, missing, invalid, or ambiguous proposal never mutates the document. Successful application uses one ordinary ProseMirror transaction, normal undo history, a resulting revision derived from the post-transaction document, and complete invalidation of the remaining generation. + +Inkspan does not derive aria-invalid or any other semantic accessibility state from opaque host strings. Inline decorations contain static Inkspan classes, priority styling, and an opaque identifier only. Category, priority, title, explanation, and actions are available as text and native controls in the named diagnostics panel. + +Diagnostics are advisory. Inkspan does not block form submission, sending, persistence, export, or collaboration merely because diagnostics remain. A host may implement separate product policy outside the editor package. + +## Ownership boundary + +Inkspan owns: + +- deterministic document state and serialization; +- revision/projection/selector validation; +- strict diagnostic schema and resource validation; +- semantic-neutral decorations and accessible diagnostic navigation; +- explicit Focus, Apply, Ignore, Dismiss, and Explain actions; +- strict stale-generation invalidation; +- ordinary document transactions, undo, and predictable focus behavior; +- privacy-minimized action events and redacted errors. + +The host owns: + +- model selection, orchestration, prompts, rubrics, examples, and language policy; +- source email/thread/recipient/role/organization context; +- semantic categories, confidence calibration, acceptance policy, and abstention; +- provider credentials and data-processing approval; +- diagnostic persistence, retention, evaluation, monitoring, and escalation; +- any send, save, compliance, or submission gate. + +## Consequences + +A host can provide inline writing guidance without forking the editor. Inkspan remains usable offline and when every model provider is unavailable. The public contract becomes broader and requires packed-package, standalone, collaborative, SSR, accessibility, concurrency, hostile-input, and cross-engine evidence. + +The design separates semantic authority from deterministic integrity. A host model may be wrong about prose; Inkspan can still prove that the proposal was admitted for one exact snapshot, was not moved onto changed content, and was not applied after the document changed. + +Strict invalidation may require more host refreshes, including after unrelated edits. This cost is accepted because v1 cannot prove semantic continuity through structural mapping. + +## Failure and recovery + +- Missing, malformed, oversized, duplicate, unsupported-projection, or invalid-selector input is rejected without document mutation. +- Provider timeout, quota failure, malformed model result, or policy abstention remains host-owned absence/status; Inkspan authoring stays available. +- Any local or remote document change invalidates the complete generation before further action authority exists. +- Async digest/selector work from an invalidated generation is discarded. +- Apply/Ignore/Dismiss/Explain against a missing or stale generation returns typed non-mutating evidence. +- Host callback exceptions are contained and cannot corrupt editor state. +- Recovery is a newly supplied diagnostic set for the current exact revision. Inkspan never fabricates a fallback judgment. + +## Security and privacy impact + +Diagnostics and replacements are untrusted host-controlled input. The exact data-property boundary rejects executable or ambiguous object shapes. Host strings render as text; v1 accepts no HTML, JavaScript, command, arbitrary TipTap JSON, arbitrary transaction, or embedded host callback. + +The contract does not require copying selected source text. Revision and selector evidence is privacy-minimized. Hosts transmitting documents to a model own authorization, encryption, provider data-use terms, retention, audit, regional processing, and consent. + +Default action events may contain only opaque identifier, exact revision evidence, opaque category, generation, action, and bounded reason code. They do not contain authored source text, replacement text, explanation, prompt, raw model output, document envelope, credential, email recipient, or tenant identifier. + +Opaque values that happen to contain words such as `spelling`, `grammar`, `mechanics`, `rude`, `incorrect`, or multilingual equivalents gain no semantic behavior or ARIA authority. + +## Accessibility + +Diagnostics are available without relying on color, hover, pointer input, animation, or generated CSS content. The built-in panel provides: + +- a named region, count, and ordered list; +- category, priority, title, and explanation as text; +- previous/next navigation and explicit affected-range focus; +- native Apply, Ignore, Dismiss, and Explain actions; +- disabled Apply when no replacement exists; +- polite status after ordinary actions; +- an assertive alert only for an actual application conflict; +- stable focus when async diagnostics arrive; and +- standalone/collaborative parity. + +Underlines are a visual supplement, not the semantic accessibility surface. + +## Compatibility and migration + +The feature is additive. Hosts that omit diagnostics retain identical editor, serialization, form, persistence, export, and collaboration behavior. The diagnostic contract remains optional and provider-neutral. + +No canonical document-envelope or database migration is required because diagnostics are noncanonical review state. A future explicit semantic accessibility field or cross-edit continuity model requires a new versioned type, compatibility plan, and ADR; v1 opaque fields gain no retrospective meaning. + +The feature may be released only when root and framework-neutral subpaths, ESM/CommonJS/types/CSS, React peer compatibility, package consumers, SSR, browser evidence, and rollback contracts agree. + +## Verification + +Acceptance requires tests proving: + +- strict exact-field/resource validation, duplicate rejection, and hostile-reflection containment; +- Unicode astral, Korean/CJK, combining, emoji, bidirectional, grapheme, empty, and collapsed selector behavior; +- exact revision/projection binding and one-snapshot selector resolution; +- local and Yjs remote `docChanged` invalidation; +- no transaction mapping, nearest-text, quote, keyword, or semantic repair; +- no semantic ARIA derivation from opaque host fields; +- one selected replacement only, exact pre-mutation recheck, ordinary undo, and resulting revision; +- safe text rendering of hostile titles, explanations, categories, identifiers, and replacements; +- keyboard/focus/status/alert/non-color accessibility; +- standalone/collaborative parity and no awareness publication; +- SSR/hydration and packed ESM/CommonJS/types/CSS compatibility; +- no model SDK, credential, environment, database, filesystem, or network dependency in the package; +- no authored/model text in default action evidence or telemetry; +- exact 100% owned production statement, branch, function, and line coverage plus complete public documentation. + +Contrast fixtures with identical keywords but different meanings, and paraphrases with different words but the same issue, must prove that Inkspan produces zero diagnostics unless the host supplies them. + +## Research and standards traceability + +The decision uses W3C Web Annotation Data Model Unicode-code-point `TextPositionSelector` semantics together with stronger exact revision binding. It uses ProseMirror/TipTap immutable state and transaction contracts, RFC 9110 strong entity-tag semantics, and the existing Inkspan deterministic-versus-model-assisted authoring boundary. LLM judgments remain fallible host evidence rather than editor truth; the companion doctoring record contains APA 7 references and calibration implications. + +## Rollback or supersession + +Rollback removes optional diagnostic props, controller state, decorations, panel, styles, and action APIs while preserving canonical documents, revisions, persistence, and collaboration data. + +Supersession requires a new ADR if Inkspan is proposed to own model invocation, semantic classification, persistence authority, submission policy, semantic ARIA inference, cross-edit diagnostic continuity, or batch application. Such a change must provide provider neutrality, privacy, offline/degraded operation, accessibility, compatibility, migration, recovery, and rollback evidence. diff --git a/docs/adr/0029-writing-diagnostics-v1-strict-invalidation.md b/docs/adr/0029-writing-diagnostics-v1-strict-invalidation.md new file mode 100644 index 00000000..cd1eb573 --- /dev/null +++ b/docs/adr/0029-writing-diagnostics-v1-strict-invalidation.md @@ -0,0 +1,104 @@ +# ADR 0029: Strict invalidation and semantic-neutral accessibility for writing diagnostics v1 + +Status: Proposed + +## Context + +ADR 0028, its design specification, and its implementation plan established the correct high-level boundary: hosts own every semantic writing judgment while Inkspan owns deterministic validation, revision/selector integrity, presentation, accessibility, and ordinary editor transactions. + +Two lower-level clauses nevertheless permitted incompatible first-release interpretations: + +1. some prose allowed a diagnostic to survive a local transaction when ProseMirror mapping appeared valid, while the implementation plan's global contract required every local or collaborative document change to invalidate every active diagnostic; and +2. the decoration plan proposed `aria-invalid="spelling"` when a host category “maps to mechanics,” although `categoryCode` is deliberately opaque and the v1 contract contains no explicit semantic accessibility field. + +Leaving those ambiguities unresolved would force implementers either to invent semantic mapping from an opaque string or to maintain two competing stale-diagnostic lifecycles. + +## Alternatives considered + +### Preserve diagnostics through transaction mapping + +Rejected for v1. ProseMirror can map structural positions, but position continuity does not prove that a host model's semantic judgment still applies to changed prose. This also conflicts with the already implemented controller/decorations direction and complicates collaborative edits, asynchronous digest races, application revalidation, testing, and host refresh ownership. + +### Derive ARIA validity from `categoryCode` + +Rejected. `categoryCode` is host-defined opaque metadata. Comparing it with strings such as `spelling`, `grammar`, `mechanics`, or language-specific equivalents would be the semantic keyword fallback that ADR 0028 explicitly prohibits. + +### Add an explicit semantic ARIA enum to v1 + +Deferred. A future version may add a bounded, explicitly declared accessibility semantic if evidence shows that decoration-level `aria-invalid` is interoperable and useful across supported browser/assistive-technology combinations. That addition requires its own versioned contract, tests, compatibility review, and ADR. + +### Strictly invalidate on every document change and keep decorations semantically neutral + +Selected for v1 because it is deterministic, explainable, provider-neutral, privacy-minimized, compatible with the existing implementation stack, and safe under both standalone and collaborative editing. + +## Decision + +For writing diagnostics v1: + +- every transaction with `docChanged === true`, whether local or collaborative, invalidates the complete active diagnostic generation before any further display or mutation authority can be used; +- no diagnostic range is preserved, remapped, repaired, or re-admitted through ProseMirror mapping, nearest-text search, quote search, keyword search, or semantic guessing; +- a host that wants current guidance after a document change must submit a new diagnostic set bound to a newly derived exact document revision; +- final application still performs exact current-state verification under the implementation plan, but mapping is not an alternate admission path; +- inline decorations contain static Inkspan classes, priority styling, and an opaque diagnostic identifier only; +- Inkspan does not derive `aria-invalid`, spelling/grammar state, or any other semantic accessibility assertion from `categoryCode`, title, explanation, replacement, confidence, provenance, or source text; +- category, priority, title, explanation, and actions remain available as plain text in the named diagnostics panel, while underlines are a visual supplement rather than the sole information channel. + +This ADR narrows and supersedes only the conflicting lifecycle and decoration-accessibility clauses in ADR 0028, the 2026-08-12 design specification, and the original implementation plan. Their product/host/model authority, security, privacy, revision, packaging, and release decisions remain in force. + +## Consequences + +### Positive + +- one lifecycle applies to local edits, remote edits, digest races, focus, action callbacks, and replacement application; +- no hidden semantic classifier is introduced into Inkspan; +- stale diagnostics fail closed without pretending position continuity proves meaning continuity; +- hosts receive a clear refresh responsibility; +- accessibility remains complete through explicit panel text and actions without unsupported semantic ARIA claims. + +### Trade-offs + +- even an unrelated document edit invalidates all active diagnostics in v1; +- hosts may perform more review refreshes; +- Inkspan does not preserve diagnostic continuity across edits until a future, separately governed evidence model exists. + +These costs are accepted because deterministic invalidation is safer and easier to validate than a partially semantic remapping policy. + +## Failure and recovery + +- A document-changing transaction clears active decorations and marks the generation stale. +- A digest or selector result completing for an invalidated generation is discarded. +- Apply/ignore/dismiss/explain operations against a missing or stale generation return typed non-mutating results. +- The host may recover only by supplying a newly admitted diagnostic set for the current exact revision. +- No offline or model-unavailable fallback fabricates a diagnostic. + +## Security and privacy impact + +The decision prevents opaque host category strings from gaining semantic execution or accessibility authority. It also prevents stale model output from being moved onto changed content. No additional authored text, replacement text, prompt, model output, tenant identifier, provider credential, or document envelope enters telemetry or DOM attributes. + +## Accessibility impact + +The panel must expose a named region, count, ordered list, category, priority, title, explanation, affected-range navigation, and explicit Apply/Ignore/Dismiss/Explain controls. Information must remain available without color, hover, pointer input, animation, or generated CSS content. Asynchronous arrival must not steal focus. An assertive alert is reserved for an actual application conflict; ordinary actions use polite status messaging. + +## Compatibility and migration + +The decision is additive to hosts that do not enable writing diagnostics. Diagnostics remain noncanonical review state, so no document-envelope, persistence, collaboration, or database migration is required. Existing Task 1–4 implementation work already follows strict invalidation and, after the corresponding runtime correction, semantic-neutral decoration behavior. Downstream tasks must consume the same exact contract; predecessor checks and reviews do not transfer after any stack refresh. + +A future public contract may add explicit accessibility semantics or evidence-backed diagnostic continuity only through a new versioned type and compatibility plan. Opaque v1 category strings do not acquire new meaning retrospectively. + +## Verification + +Acceptance requires deterministic tests proving: + +- local and Yjs remote `docChanged` transactions invalidate the complete generation; +- no transaction mapping preserves diagnostics in v1; +- stale asynchronous work cannot install decorations or emit actions; +- decoration attributes contain no title, explanation, replacement, category text, model output, HTML, or derived semantic ARIA state; +- diagnostics with category codes such as `spelling`, `grammar`, `mechanics`, multilingual equivalents, or attacker-controlled lookalikes receive identical semantic-neutral decoration handling; +- the accessible panel exposes host strings as text and remains usable without visual-only cues; +- Inkspan produces no diagnostic when the host supplies none. + +## Rollback or supersession + +Rollback removes the optional diagnostic surface without altering canonical document envelopes, revisions, persistence records, collaboration data, or host storage. The host simply stops supplying diagnostics and the editor retains its ordinary deterministic behavior. + +A future diagnostic-continuity design may supersede strict invalidation only with a versioned evidence model that proves target identity across changes, defines standalone/collaborative parity, contains privacy and accessibility semantics, and passes cross-engine and realistic semantic-integrity validation. A future semantic ARIA field likewise requires an explicit typed contract rather than inference from opaque strings. diff --git a/docs/adr/README.md b/docs/adr/README.md index df8b7b80..3b5a5919 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -2,7 +2,7 @@ Status values: Proposed, Accepted, Superseded. -This index records durable architectural decisions. Protected-main implementation is required before a feature-specific decision may be treated as shipped behavior. Detailed ADR files are authoritative for their decision; this index is authoritative for discovery and status. +This index records durable architectural decisions. An ADR on a branch or protected `main` is not shipped behavior by status alone. `Accepted` requires protected-main implementation or process authority plus applicable verification evidence. | ADR | Status | Decision | |---|---|---| @@ -32,29 +32,28 @@ This index records durable architectural decisions. Protected-main implementatio | [0024](0024-bounded-docx-paragraph-alignment.md) | Accepted | Bounded paragraph alignment in deterministic DOCX output | | [0025](0025-bounded-docx-heading-alignment.md) | Accepted | Bounded heading alignment in deterministic DOCX output | | [0026](0026-bounded-docx-external-hyperlinks.md) | Accepted | Bounded external hyperlinks in deterministic DOCX rich text | +| [0028](0028-host-owned-llm-writing-diagnostics.md) | Proposed | Host-owned, revision-bound LLM writing diagnostics | +| [0029](0029-writing-diagnostics-v1-strict-invalidation.md) | Proposed | Strict writing-diagnostic invalidation and semantic-neutral accessibility | ## Decision discipline -- **Proposed**: documented decision whose acceptance evidence is incomplete or whose governing implementation/operational boundary is still being validated; not protected-main implementation authority by status alone. -- **Accepted**: decision has protected-main implementation or process authority with the applicable verification evidence and its canonical ADR status has been reconciled to that authority. -- **Superseded**: retained for historical traceability but explicitly replaced by a later ADR. +- **Proposed:** the decision is documented, but implementation or operational acceptance evidence is incomplete. +- **Accepted:** protected `main` contains the governing implementation or process and its verification evidence is current. +- **Superseded:** the ADR remains for history but a later ADR replaces it. -ADR decision status and implementation maturity are related but distinct. An ADR file may be present on protected `main` while its decision remains Proposed; conversely, an implemented capability can expose stale documentation until the ADR is reconciled. Canonical fitness and traceability records state implementation maturity explicitly, and an ADR cannot promote an unmerged feature to shipped behavior. - -When a decision changes materially, add or supersede an ADR rather than silently rewriting history. +Material changes add or supersede an ADR rather than silently rewriting history. ## ADR quality requirements -Every material ADR records explicit sections for: +Every material ADR documents the following evidence explicitly: -1. context and the problem boundary; -2. materially distinct alternatives considered; -3. the selected decision; -4. consequences and ownership trade-offs; -5. failure and recovery semantics; -6. security and privacy impact; -7. compatibility and migration behavior; -8. verification/acceptance evidence; and -9. rollback or explicit supersession conditions. +- context and the problem boundary; +- materially distinct alternatives considered; +- the selected decision and its consequences; +- failure and recovery semantics; +- security and privacy impact; +- compatibility and migration behavior; +- verification/acceptance evidence; +- rollback or explicit supersession conditions. -Accessibility, operability, research/standards traceability, and release implications are included wherever the decision affects them. Canonical PRD, TRD, Architecture, contracts, UML, data/evidence model, threat model, test strategy, operability, and traceability documents must remain synchronized with Accepted decisions. +Accessibility, operability, standards/research traceability, release impact, and synchronized PRD/TRD/Architecture/contracts/UML/data model/threat model/test strategy are also required where affected. diff --git a/docs/doctoring/llm-writing-diagnostics-measurement-boundary.md b/docs/doctoring/llm-writing-diagnostics-measurement-boundary.md new file mode 100644 index 00000000..09de0e31 --- /dev/null +++ b/docs/doctoring/llm-writing-diagnostics-measurement-boundary.md @@ -0,0 +1,111 @@ +# LLM writing diagnostics: measurement and editor-boundary doctoring + +**Status:** Proposed design evidence; no shipped Inkspan diagnostic feature is claimed. +**Date:** 2026-08-12 + +## Decision supported by this record + +Inkspan may display and safely apply host-supplied writing diagnostics, but it must not infer grammar, tone, pragmatics, or technical quality from source text. Semantic judgment belongs to a host-owned LLM review and calibration system. Inkspan owns deterministic revision, selector, validation, accessibility, and mutation integrity. + +This division is not merely packaging preference. LLM evaluators are useful but behave like fallible measurement instruments. Their scores can vary with model, rubric, language, answer order, verbosity, formatting, self-provenance, and irrelevant surface artifacts. A deterministic editor cannot turn those outputs into truth; it can ensure only that a proposal is displayed and applied to the correct document state. + +## Evidence synthesis + +### LLMs can produce useful structured evaluations + +G-Eval demonstrated that rubric-driven, form-filling LLM evaluation can align more closely with human judgments than traditional automatic NLG metrics in the studied summarization setting. MT-Bench and Chatbot Arena likewise established LLM-as-a-Judge as a scalable evaluation method while explicitly documenting position, verbosity, self-enhancement, and reasoning limitations. + +**Product implication:** a host can use an LLM to generate criterion-level writing diagnostics, but the output must remain structured, versioned, and reviewable rather than becoming an opaque whole-document rewrite. + +### Evaluator output is biased and context-dependent + +Subsequent work has shown that LLM evaluators are not uniformly fair or stable. Reported problems include position bias, self-preference, disagreement across evaluation dimensions, sensitivity to apologetic or verbose artifacts, persuasion, auxiliary guidance, and multilingual inconsistency. Larger models do not automatically remove these problems. + +**Product implication:** Inkspan must not interpret a model confidence value as editor authority. The host must own judge calibration, multi-model or independent verification when warranted, abstention, monitoring, and human review. + +### Multilingual writing guidance requires language-specific evidence + +Multilingual judge studies report substantial inconsistency across languages and poorer behavior in some lower-resource settings. A feature that works in English cannot claim equivalent Korean, Japanese, Chinese, Vietnamese, or code-switched reliability merely because the underlying model accepts those languages. + +**Product implication:** Inkspan's selector, grapheme, accessibility, and rendering contracts must be language-neutral, while the host's semantic quality claims remain language-profile-specific and empirically validated. + +### Psychometric treatment is appropriate + +Recent psychometric work argues that a judge should be characterized as a measuring device rather than reported only by scalar agreement. Relevant properties include baseline response under null inputs, response to controlled quality ladders, surface cross-sensitivity, positional false preference, criterion movement, reliability, and calibration. IRT-based evaluation research likewise shows why item difficulty and discrimination matter and why fixed average scores can hide poor items or unstable rankings. + +**Product implication:** the editor API carries policy and provenance versions but does not perform calibration. A host such as Naruon may use fast-mlsirm to turn criterion-level polytomous judge responses into calibrated evaluation evidence. + +### Position selectors need revision state + +The W3C Web Annotation Data Model defines `TextPositionSelector` with Unicode-code-point offsets, inclusive start, exclusive end, logical text order, and a recommendation not to split grapheme clusters. It also warns that position selectors are brittle when a resource changes and recommends additional state. + +**Product implication:** Inkspan binds every writing diagnostic to its own strong document revision and declared projection. It does not recover a stale suggestion by keyword search, nearest-text matching, or positional guessing. + +## Deterministic validation is not semantic classification + +The design permits deterministic code for: + +- identifier and strong-revision syntax; +- exact JSON/schema validation; +- resource bounds and duplicate rejection; +- Unicode, selector, and grapheme boundaries; +- safe-link, inline-image, clipboard, and document-schema enforcement; +- current-revision comparison and transaction mapping; +- overlap detection and ordinary undo. + +The design prohibits deterministic code from asserting that prose is impolite, unclear, grammatically wrong, technically imprecise, or non-actionable based on lexical triggers. Regex validation of an identifier is materially different from a keyword-based language judgment. + +## Governance implications + +The host's LLM review system should record, at minimum: + +- model and provider identifiers; +- prompt, rubric, and policy versions; +- orchestration mode and reasoning configuration; +- criterion definitions and ordered category anchors; +- calibration dataset and language profile; +- observed agreement, false-positive rate, calibration error, and drift; +- abstention and escalation policy; +- privacy, retention, and provider data-use controls. + +Inkspan records only bounded diagnostic/action metadata by default. Source text, replacement text, prompts, raw model output, and document envelopes are not generic telemetry. + +## Standards alignment + +- **W3C Web Annotation Data Model:** interoperable text selector semantics and change brittleness. +- **NIST AI 600-1:** lifecycle-oriented Generative AI risk identification, evaluation, monitoring, and trustworthiness controls. +- **ISO/IEC 23894:2023:** integration of AI-specific risk management into organizational activities. +- **ISO/IEC 42001:2023:** management-system requirements for responsible AI development and use, including traceability and continual improvement. +- **AERA/APA/NCME Standards:** validity, reliability, fairness, intended-use, and consequences evidence for score-based decisions. Inkspan itself does not claim conformance to a psychological testing standard; the principles inform host judge validation. + +## APA 7th references + +American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). *Standards for educational and psychological testing*. American Educational Research Association. + +Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P., & Roberts, K. (2024). *Artificial intelligence risk management framework: Generative artificial intelligence profile* (NIST AI 600-1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.600-1 + +Chen, H., & Goldfarb-Tarrant, S. (2025). Safer or luckier? LLMs as safety evaluators are not robust to artifacts. In *Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)* (pp. 19750–19766). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.acl-long.970 + +Fu, X., & Liu, W. (2025). How reliable is multilingual LLM-as-a-Judge? In *Findings of the Association for Computational Linguistics: EMNLP 2025*. Association for Computational Linguistics. https://aclanthology.org/2025.findings-emnlp.587/ + +International Organization for Standardization. (2023a). *Information technology—Artificial intelligence—Guidance on risk management* (ISO/IEC Standard No. 23894:2023). https://www.iso.org/standard/77304.html + +International Organization for Standardization. (2023b). *Information technology—Artificial intelligence—Management system* (ISO/IEC Standard No. 42001:2023). https://www.iso.org/standard/42001.html + +Liu, S., Xu, Z., Liu, Z., Yan, Y., Yu, M., Gu, Y., Chen, C., Xie, H., & Yu, G. (2026). Mitigating judgment preference bias in large language models through group-based polling. In *Findings of the Association for Computational Linguistics: ACL 2026* (pp. 1448–1464). Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.findings-acl.71 + +Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., & Zhu, C. (2023). G-Eval: NLG evaluation using GPT-4 with better human alignment. In *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing* (pp. 2511–2522). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.153 + +Shen, C., Cheng, L., Nguyen, X.-P., You, Y., & Bing, L. (2023). Large language models are not yet human-level evaluators for abstractive summarization. In *Findings of the Association for Computational Linguistics: EMNLP 2023* (pp. 4215–4233). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-emnlp.278 + +Usami, H., Hara, K., Tsuboi, A., & Matsuda, N. (2026). *LLM judges have dark current: A psychometric datasheet for LLM-as-a-Judge evaluation* [Preprint]. arXiv. https://arxiv.org/abs/2606.15610 + +Wang, P., Li, L., Chen, L., Cai, Z., Zhu, D., Lin, B., Cao, Y., Kong, L., Liu, Q., Liu, T., & Sui, Z. (2024). Large language models are not fair evaluators. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)* (pp. 9440–9450). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.511 + +World Wide Web Consortium. (2017). *Web annotation data model*. https://www.w3.org/TR/annotation-model/ + +Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). *Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena* [Preprint]. arXiv. https://arxiv.org/abs/2306.05685 + +## Claim boundary + +These sources support the need for structured, calibrated, bias-aware host evaluation and revision-bound editor integrity. They do not prove that any particular model, rubric, language, or Naruon implementation is sufficiently accurate. Those claims require the implementation plan's task-specific benchmark and current-model evidence. \ No newline at end of file diff --git a/docs/superpowers/plans/2026-08-12-writing-diagnostics-implementation.md b/docs/superpowers/plans/2026-08-12-writing-diagnostics-implementation.md new file mode 100644 index 00000000..8182bab4 --- /dev/null +++ b/docs/superpowers/plans/2026-08-12-writing-diagnostics-implementation.md @@ -0,0 +1,162 @@ +# Revision-Bound Writing Diagnostics Implementation Plan + +> **For agentic workers:** Use the repository's test-driven, exact-head, single-writer workflow. Every task begins with a realistic failing regression, implements the smallest bounded production change, removes temporary branch-only workflows before integration, and regenerates all current-head evidence after ancestry changes. + +**Goal:** Add a generic, provider-neutral Inkspan surface that validates, displays, navigates, applies, ignores, dismisses, explains, and invalidates host-supplied writing diagnostics without making semantic judgments itself. + +**Architecture:** A React-free contract validates bounded hostile input. A deterministic inverse text-projection resolver maps revision-scoped W3C selectors to one immutable ProseMirror snapshot. A semantic-neutral extension renders verified ranges. One shared React controller fences async verification by generation and invalidates the complete set after every document change. A built-in accessible panel exposes explicit actions. Standalone and collaborative editors reuse the same types and lifecycle. Version 1 accepts plain-text replacements, applies one explicitly selected diagnostic at a time, and never repairs stale selectors. + +**Technology:** TypeScript, React 18/19, TipTap/ProseMirror, Yjs, Web Crypto revision evidence, W3C `TextPositionSelector`, Vitest/jsdom, Playwright Chromium/Firefox/WebKit, Vite package subpaths, pnpm, and existing exact-head coverage/security/package/release gates. + +## Global constraints + +- Inkspan calls no model, provider, network service, database, storage service, credential broker, or host API. +- Inkspan never infers spelling, grammar, tone, clarity, pragmatics, technical quality, or actionability from text or opaque host fields. +- Keywords, regexes, phrase dictionaries, language names, domains, recipient counts, nearest-text search, quote search, and word positions are prohibited as semantic fallback or stale-selector repair. +- Host strings are untrusted plain text. Version 1 accepts no diagnostic HTML, command, JavaScript, arbitrary TipTap JSON, arbitrary transaction, or callback. +- `inkspan-prosemirror-text` version 1 is the sole v1 selector projection. +- Selector offsets are non-negative safe integers with `start <= end`; collapsed selectors remain navigable but create no inline range decoration. +- Every local or collaborative `docChanged` transaction invalidates the complete active generation. Version 1 never maps or preserves a diagnostic across changed content. +- One immutable editor snapshot supplies the current revision and every selector resolution for one generation. +- Generation, mounted-state, and editor-identity guards prevent stale promises from installing decorations, emitting actions, or moving focus. +- Diagnostics remain advisory and never block form submission, sending, persistence, export, or collaboration. +- Existing behavior remains compatible when diagnostics are absent. +- Owned production statement, branch, function, and line coverage remains exactly 100%, with beginner-readable public documentation. +- The feature stays under `Unreleased`; publication uses a separate release-only PR. + +## Task 1: React-free diagnostic contract + +**Files:** `src/writingDiagnostics.ts`, `src/writingDiagnostics.test.ts`, `src/writing-diagnostics/index.ts`, `src/index.ts`. + +- [ ] Add RED tests for valid/empty sets, exact fields, duplicate IDs, hostile arrays/objects/proxies/accessors/prototypes/symbols, resource ceilings, priorities, confidence, revision, projection, selectors, provenance, and replacement types. +- [ ] Define `advisory | important | critical`, structured `CwlEditorDocumentRevision`, structured projection identity, selector, opaque category, bounded host prose, optional plain-text replacement/confidence, and privacy-minimized provenance. +- [ ] Define stable redacted error codes for contract, limit, revision, projection, selector, conflict, and lifecycle failures. +- [ ] Validate exact own enumerable data properties and return deeply detached frozen values. +- [ ] Preserve host order; do not sort or infer semantics. +- [ ] Export identical root and React-free subpath contracts. +- [ ] Run focused tests and typecheck to GREEN. + +## Task 2: inverse canonical text-projection resolver + +**Files:** `src/writingDiagnosticProjection.ts`, its tests, `src/textPositionSelectorEvidence.ts`, `src/text-position-selector/index.ts`. + +- [ ] Add RED round-trip tests for paragraphs, headings, lists, tables, hard breaks, inline/block atoms, empty blocks, document boundaries, repeated text, astral/Korean/CJK/combining/emoji/bidirectional text, and collapsed ranges. +- [ ] Add negative tests for unsupported projection, unsafe/reversed/out-of-range offsets, grapheme splits, ambiguous structural boundaries, hostile metadata, and missing `Intl.Segmenter`. +- [ ] Build the exact `textBetween(0, size, '\n', '\uFFFC')` projection and one code-point-boundary-to-ProseMirror-position map. +- [ ] Reuse one grapheme-boundary implementation for forward and inverse paths. +- [ ] Fail closed rather than selecting nearest or similar text. +- [ ] Publish the deterministic resolver through the framework-neutral selector surface. + +## Task 3: semantic-neutral ProseMirror decorations + +**Files:** `src/extensions/WritingDiagnostics.ts`, its tests, `src/extensions/kit.ts`. + +- [ ] Add RED tests for install/focus/clear metadata, monotonic generation, exact ranges, collapsed ranges, malformed metadata, duplicate IDs, resource ceilings, editor destruction, and local/remote document changes. +- [ ] Render non-empty ranges with only: + +```text +class="cwl-writing-diagnostic cwl-writing-diagnostic--{priority}" +data-cwl-diagnostic-id="opaque-id" +``` + +- [ ] Reject all extra semantic fields and never derive spelling/grammar validity or other semantic ARIA state from host strings. +- [ ] Clear the complete generation before processing metadata on every `docChanged` transaction. +- [ ] Accept only already-validated structural ranges; perform no hashing or host callback. +- [ ] Install the extension exactly once for standalone and collaborative editor graphs. + +## Task 4: revision-bound controller + +**Files:** `src/components/useWritingDiagnosticsController.ts`, its tests, `src/types.ts`. + +- [ ] Add RED tests for absent/invalid/verifying/active/stale states, editor and callback replacement, hostile prop mutation, revision/projection mismatch, selector rejection, doc changes during hashing, concurrent promises, unmount, and callback exceptions. +- [ ] Validate the complete set before reading editor state. +- [ ] Capture one immutable envelope, derive one revision, resolve all selectors against that same snapshot, and install atomically. +- [ ] Fence every async continuation with monotonic generation, mounted state, and editor identity. +- [ ] Invalidate both verifying and active generations immediately after any local/remote document change; do not map ranges. +- [ ] Expose privacy-minimized Focus/Ignore/Dismiss/Explain controller actions and typed reason codes. + +## Task 5: accessible diagnostics panel + +**Files:** `src/components/WritingDiagnosticsPanel.tsx`, its tests, `src/components/EditorFrame.tsx`, `src/styles.css`, print-style tests. + +- [ ] Add RED accessibility tests for a named region, count, ordered items, category, priority, title, explanation, affected-range focus, previous/next navigation, Apply/Ignore/Dismiss/Explain, disabled Apply without replacement, polite status, and assertive conflict alert. +- [ ] Prove asynchronous arrival does not steal focus and explicit navigation uses roving focus. +- [ ] Render every host string as a React text node; never use raw HTML. +- [ ] Keep selected source text out of action names and attributes. +- [ ] Add forced-colors, high-contrast, reduced-motion, focus-visible, touch-target, zoom, and print behavior. Default print remains document-only; an explicit host option may include a bounded appendix. +- [ ] Add no undocumented global shortcut. + +## Task 6: standalone editor actions + +**Files:** `src/types.ts`, `src/components/CwlEditor.tsx`, `src/components/useEditorHandle.ts`, integration/accessibility/handle tests. + +- [ ] Add RED tests for optional props, hostile/valid/stale sets, explicit actions, clear, read-only/disabled modes, form submission, undo, callback replacement, and unmount. +- [ ] Preserve the raw diagnostics prop by identity until bounded controller validation. +- [ ] Add additive props for diagnostics, action/error callbacks, label, and optional print appendix. +- [ ] Add imperative Focus/Ignore/Dismiss/Explain/Apply methods through the same controller used by the panel. +- [ ] Immediately before Apply, derive and compare the exact current revision again. +- [ ] Apply one plain-text replacement through one ordinary ProseMirror transaction, derive the resulting revision, emit the complete event, and invalidate the remaining generation. +- [ ] Return typed non-mutating outcomes for stale/conflict/lifecycle cases. +- [ ] Prove source words alone generate no diagnostics. + +## Task 7: collaborative parity + +**Files:** collaborative editor/index/tests and a two-client diagnostics suite. + +- [ ] Add the same props/actions and accessible panel behavior through the shared controller and extension. +- [ ] Prove a remote Yjs insertion invalidates the complete local set before application. +- [ ] Prove remote change during pending hashing discards the stale continuation. +- [ ] Keep diagnostics, explanations, replacements, selected text, model provenance, and review state out of awareness payloads. +- [ ] Emit an action only from the client whose user invoked it. +- [ ] Preserve editor/provider/Yjs identity when diagnostics/callbacks change. + +## Task 8: framework-neutral package subpath + +**Files:** package manifest/lock, Vite subpath config, package/export/consumer/boundary tests. + +- [ ] Publish `@contextualwisdomlab/cwl-editor/writing-diagnostics` as React-free ESM/CommonJS/types. +- [ ] Export contracts, limits, validation, errors, and deterministic selector-resolution primitives only. +- [ ] Prove no React, TipTap React, provider, model SDK, network, credential, filesystem, database, or CSS side-effect dependency leaks into the pure subpath. +- [ ] Install the packed tarball into isolated ESM/CommonJS/strict-TypeScript consumers. +- [ ] Keep UI and editor handles on interactive root/collaboration entrypoints. + +## Task 9: browser, SSR, hostile-input, and no-fallback assurance + +**Files:** browser specs/fixture, security tests, server-rendering tests, `docs/TEST_STRATEGY.md`. + +- [ ] Test rendering, keyboard/touch navigation, focus, Apply, undo, strict invalidation, zoom, forced colors, and mobile targets on pinned Chromium/Firefox/WebKit. +- [ ] Prove SSR renders without browser globals, segmenter, model infrastructure, or semantic evaluation; hydration adds no duplicate IDs, focus theft, or mismatch. +- [ ] Test script/HTML strings, bidi controls, isolated surrogates, nulls, oversize, accessors, proxies, duplicate JSON members, and callback exceptions. +- [ ] Use multilingual semantic contrast fixtures; without host diagnostics the exact result is an empty diagnostic surface. +- [ ] Record browser lock, artifact identity, and the distinction between editor integrity evidence and model accuracy. + +## Task 10: canonical documentation and traceability + +**Files:** README, Architecture, PRD, TRD, API contract, threat model, operability, traceability, ADR 0028/0029/index, design, this plan, CHANGELOG, documentation-contract tests. + +- [ ] Document host semantic authority versus Inkspan deterministic integrity. +- [ ] Synchronize public type examples, strict invalidation, semantic-neutral decorations, collapsed selectors, single-action application, standalone/collaboration parity, SSR, degraded operation, privacy, and rollback. +- [ ] Remove temporary errata after folding every rule into the canonical ADR/design/plan; leave no parallel instruction set. +- [ ] Document that confidence/priority/category are host labels, not editor truth or submission policy. +- [ ] Add threats for hostile diagnostics, stale async work, replacement injection, telemetry leakage, focus attacks, collaboration races, and authority confusion. +- [ ] Map requirements to modules, tests, browser/package evidence, and release gates. +- [ ] Keep ADRs Proposed until protected implementation and acceptance exist. +- [ ] Add machine contracts that reject semantic keyword fallback, category-derived semantic ARIA, cross-edit mapping, provider ownership, stale repair, batch mutation authority, and send gating. + +## Task 11: exact-head integration acceptance + +- [ ] Reconcile the complete stack onto the latest protected main without destructive history or lost concurrent changes. +- [ ] Remove every temporary branch-specific workflow and regenerate evidence on the resulting exact head. +- [ ] Run full tests, typecheck, exact 100% owned coverage, library/demo builds, packed consumers, cross-engine browsers, Office Python 3.11/3.14 package gates, security/SAST/dependency/SBOM/provenance/secret checks, and documentation contracts. +- [ ] Inspect all current human/CodeRabbit/GHAS/Dependabot/OpenCode/Noema/Strix feedback and resolve only addressed threads. +- [ ] Require zero valid unresolved findings and qualifying independent non-author approval on the unchanged head. +- [ ] Mark Ready and merge only through protected policy; then verify protected main contains expected files and no temporary artifacts. + +## Task 12: release-only publication and host handoff + +- [ ] Open a separate release-only PR after protected feature integration and current release-train closure. +- [ ] Promote ADR status only with protected implementation and current acceptance evidence. +- [ ] Update version, final CHANGELOG, package declarations, licenses, SBOM/provenance, compatibility, and rollback. +- [ ] Publish immutable npm artifacts from the exact reviewed release head and verify all subpaths from the registry artifact. +- [ ] Record immutable version, tarball digest, source commit, manifest, browser evidence, and rollback in the Naruon companion integration. +- [ ] Never integrate a mutable branch, source archive, copied fork, local path, or unreviewed package into a host runtime. diff --git a/docs/superpowers/plans/2026-08-12-writing-diagnostics-v1-contract-errata.md b/docs/superpowers/plans/2026-08-12-writing-diagnostics-v1-contract-errata.md new file mode 100644 index 00000000..ed7e7e95 --- /dev/null +++ b/docs/superpowers/plans/2026-08-12-writing-diagnostics-v1-contract-errata.md @@ -0,0 +1,71 @@ +# Writing Diagnostics v1 Implementation Plan Errata + +Status: Required companion to `2026-08-12-writing-diagnostics-implementation.md` + +ADR 0029 resolves two contradictions discovered after the original atomic implementation plan was written. Every remaining task and acceptance review must apply this errata. Task 10 must fold these corrections into the canonical ADR, design, plan, PRD, TRD, contracts, threat model, operability, traceability, and CHANGELOG so the final protected-main documentation has no parallel instruction set. + +## Global lifecycle correction + +The following rule replaces every original-plan or design clause that permits transaction-local preservation, mapping, remapping, repair, or re-admission of an existing diagnostic: + +> Any local or collaborative transaction with `docChanged === true` invalidates the complete active diagnostic generation. Version 1 never preserves or remaps a diagnostic across changed document content. A host must submit a new set bound to the new exact revision. + +ProseMirror mapping may be used internally for ordinary editor behavior, but it is not evidence that a model judgment still targets the same meaning and is not diagnostic admission authority. + +## Task 3 correction: decoration attributes + +Replace the proposed attribute set with: + +```text +class="cwl-writing-diagnostic cwl-writing-diagnostic--{priority}" +data-cwl-diagnostic-id="opaque-id" +``` + +Do not add `aria-invalid`, spelling/grammar state, category semantics, title, explanation, replacement, confidence, provenance, model output, or HTML to decoration attributes. `categoryCode` remains opaque; it cannot be matched against words such as `spelling`, `grammar`, or `mechanics` to derive behavior or ARIA state. + +Task 3 tests must include attacker-controlled, multilingual, and lookalike category codes and prove identical semantic-neutral decoration handling. + +## Task 4 correction: controller state + +The controller's transaction subscriber invalidates both `verifying` and `active` generations before stale asynchronous work can publish. It does not map verified ranges after any document change. All old digest/selector completions are generation-fenced and discarded. + +## Task 5 correction: accessible panel + +The named panel is the semantic accessibility surface. It exposes category, priority, title, explanation, count, ordered position, affected-range navigation, and explicit Apply/Ignore/Dismiss/Explain actions as React text and native controls. + +- Do not infer semantic ARIA state from `categoryCode` or any other host text. +- Do not place selected source text in action names or attributes. +- New asynchronous diagnostics do not move focus. +- Previous/next navigation is explicit and roving; no undocumented global shortcut is added. +- Ordinary action completion uses a polite status region. +- An assertive alert is reserved for an actual application conflict. +- Information remains available without color, hover, pointer input, animation, or generated CSS content. + +## Task 6 correction: application + +Apply rechecks the exact current document revision immediately before the ordinary ProseMirror transaction. A stale generation cannot be rescued through range mapping or text search. A successful application invalidates all remaining diagnostics and produces a newly derived resulting revision. + +## Task 7 correction: collaboration + +Every remote Yjs document change invalidates the complete local diagnostic generation. A relative position or mapped ProseMirror position is not proof that the host judgment remains semantically current. Awareness payloads never carry diagnostics or review state. + +## Task 9 correction: assurance + +Cross-engine and hostile-input evidence must prove: + +- strict invalidation after every local or remote document change; +- no nearest-text, quote, keyword, category, or transaction-mapping recovery; +- no semantic ARIA derivation from opaque host fields; +- accessible information remains present through the panel and native actions; +- no diagnostics are produced when the host supplies none. + +## Task 10 reconciliation requirement + +Before the feature stack can become Ready, the original ADR 0028 and design/plan prose must be edited so they directly express ADR 0029. This errata is temporary planning evidence, not the desired final duplicate source of truth. The final documentation contract tests must fail if either of the superseded claims reappears: + +1. a diagnostic can survive `docChanged` through transaction mapping; or +2. Inkspan derives `aria-invalid` or other semantic state from an opaque category string. + +## Acceptance impact + +Existing Task 1–4 runtime direction is compatible with this correction. Downstream Task 5–12 branches must be based on exact predecessor heads that contain or explicitly consume this errata, and all exact-head CI/review evidence must be regenerated after any affected branch is refreshed. diff --git a/docs/superpowers/specs/2026-08-12-revision-bound-llm-writing-diagnostics-design.md b/docs/superpowers/specs/2026-08-12-revision-bound-llm-writing-diagnostics-design.md new file mode 100644 index 00000000..4b94190f --- /dev/null +++ b/docs/superpowers/specs/2026-08-12-revision-bound-llm-writing-diagnostics-design.md @@ -0,0 +1,305 @@ +# Revision-Bound Writing Diagnostics Design + +**Date:** 2026-08-12 +**Status:** Proposed design; not shipped behavior +**Target:** Next Inkspan feature release after the current protected-main release train + +## Objective + +Add a provider-neutral, Grammarly-like writing-diagnostic surface without turning Inkspan into a language model, email product, policy engine, persistence service, or hidden semantic classifier. + +A host application generates contextual writing proposals using its own model, rubric, authorization, privacy, retention, and review policy. Inkspan admits only structurally valid host proposals, binds them to one exact document revision and text projection, renders them accessibly, and permits explicit user actions without allowing stale asynchronous output to mutate changed content. + +Host-defined categories may describe spelling, grammar, spacing, punctuation, clarity, concision, structure, tone, pragmatics, technical precision, or actionability. Inkspan treats those fields as opaque proposal data. It does not determine whether a category, explanation, confidence, priority, or replacement is semantically correct. + +## Product behavior + +An author sees normal Inkspan editing first. When the host supplies an admitted diagnostic set: + +1. non-empty affected ranges receive non-color-only visual decorations; +2. a named diagnostics region reports the count and exposes an ordered list; +3. each item exposes category, priority, title, explanation, and optional replacement as plain text; +4. keyboard, pointer, touch, and assistive-technology users can navigate the previous or next diagnostic; +5. Focus moves to an affected structural range only after explicit user navigation; +6. Apply rechecks the exact current revision and applies one selected plain-text replacement through one ordinary ProseMirror transaction; +7. Ignore and Dismiss emit privacy-minimized host-visible actions without changing canonical content; +8. Explain reveals the supplied explanation or invokes an explicit host callback; Inkspan performs no model call; +9. asynchronous arrival never steals focus; and +10. stale diagnostics never apply through nearest-text search, keyword search, quote search, remapping, or silent position repair. + +Diagnostics remain advisory. Their presence, absence, invalidity, or staleness does not block form submission, sending, export, persistence, or collaboration in the editor package. + +## Selected architecture + +```mermaid +flowchart LR + H[Host review service] -->|revision-bound diagnostics| P[Inkspan public props] + P --> V[Deterministic contract validator] + V --> R[Exact revision and selector resolver] + R --> D[Semantic-neutral ProseMirror decorations] + D --> U[Accessible diagnostics panel] + U --> A{Explicit author action} + A -->|Apply| C[Exact current-revision check] + A -->|Ignore Dismiss Explain| E[Privacy-minimized action event] + C -->|match| T[Ordinary ProseMirror transaction] + C -->|stale conflict| X[Typed non-mutating outcome] + T --> O[Normal change revision and undo behavior] +``` + +The host may be Naruon, another CWL product, or an unrelated consumer. No host, provider, model, email, or tenant name appears in the generic runtime contract. + +## Public contract + +The v1 design mirrors the implementation types rather than maintaining a second approximate schema. + +```ts +export type CwlWritingDiagnosticPriority = + | 'advisory' + | 'important' + | 'critical'; + +export interface CwlWritingDiagnosticProvenance { + readonly workflowId: string; + readonly workflowVersion: string; + readonly judgePolicyVersion: string; + readonly orchestrationMode?: string; +} + +export interface CwlWritingDiagnostic { + readonly diagnosticId: string; + readonly documentRevision: CwlEditorDocumentRevision; + readonly textProjection: CwlEditorTextProjectionIdentity; + readonly selector: CwlEditorTextPositionSelector; + readonly categoryCode: string; + readonly priority: CwlWritingDiagnosticPriority; + readonly title: string; + readonly explanation: string; + readonly suggestedReplacement?: string; + readonly confidence?: number; + readonly provenance: Readonly; +} + +export type CwlWritingDiagnosticAction = + | 'applied' + | 'ignored' + | 'dismissed' + | 'requested_explanation' + | 'stale' + | 'conflict'; + +export type CwlWritingDiagnosticActionReasonCode = + | 'explicit' + | 'document_changed' + | 'revision_mismatch' + | 'projection_mismatch' + | 'selector_invalid' + | 'verification_failed' + | 'lifecycle_ended' + | 'diagnostic_missing'; + +export interface CwlWritingDiagnosticActionEvent { + readonly action: CwlWritingDiagnosticAction; + readonly reasonCode: CwlWritingDiagnosticActionReasonCode; + readonly diagnosticId: string; + readonly documentRevision: CwlEditorDocumentRevision; + readonly categoryCode: string; + readonly generation: number; +} +``` + +Candidate additive editor props are: + +```ts +interface CwlEditorProps { + writingDiagnostics?: readonly CwlWritingDiagnostic[]; + onWritingDiagnosticAction?: ( + event: CwlWritingDiagnosticActionEvent, + ) => void; + onWritingDiagnosticsError?: (error: WritingDiagnosticError) => void; + writingDiagnosticsLabel?: string; + printWritingDiagnostics?: boolean; +} +``` + +Candidate imperative methods use the same controller and validation path as the built-in panel. There is no trusted imperative bypass. + +## Validation boundary + +The deterministic validator fails closed and verifies: + +- exact own enumerable data properties and no unsupported fields, symbols, accessors, sparse arrays, inherited fields, or hostile reflection; +- bounded collection size, identifier length, category length, title, explanation, replacement, and provenance identifiers; +- unique diagnostic identifiers within one submitted generation; +- supported finite priority and confidence values; +- an exact lowercase SHA-256 revision object and matching strong entity tag; +- the exact `inkspan-prosemirror-text` version 1 projection identity; +- selector values are non-negative safe integers with start <= end; +- Unicode-code-point and grapheme-cluster boundaries; +- one unambiguous structural range in the exact projected snapshot; and +- plain-text replacement values only. + +Collapsed selectors are valid evidence and remain navigable, but they create no inline range decoration. Empty and non-empty selectors use the same exact revision/projection admission path. + +The validator does not decide whether an explanation is true, a replacement is better, a message is polite, or a category label is accurate. Regexes may validate bounded identifiers and revision syntax, but they cannot create, prioritize, admit, or semantically classify a diagnostic. + +## Revision and position lifecycle + +### Initial admission + +The host reviews one immutable document snapshot and returns diagnostics carrying that snapshot's exact `CwlEditorDocumentRevision`, projection identity, and W3C text-position selector. Inkspan validates the complete untrusted set before reading editor state, derives one current revision from one immutable editor snapshot, resolves all selectors against that same snapshot, and publishes only a complete verified generation. + +### Strict invalidation + +Every local or collaborative transaction with docChanged === true invalidates the complete active diagnostic generation. + +Version 1 does not preserve, map, remap, repair, or re-admit a diagnostic after changed document content. ProseMirror mapping and Yjs relative positions are useful editor mechanisms, but neither proves that a host model judgment remains semantically current. A stale async digest or selector result is discarded through the generation fence. The host must submit a new set bound to the new exact revision. + +### Explicit action + +Version 1 applies exactly one explicitly selected diagnostic at a time. + +Immediately before application, Inkspan derives and compares the current exact revision again. A matching diagnostic with a valid plain-text replacement produces one ordinary ProseMirror transaction and normal undo history. A mismatch, missing diagnostic, invalid selector, ended lifecycle, or changed document returns a typed non-mutating event. Successful application invalidates every remaining diagnostic and derives the resulting revision from the post-transaction document. + +There is no Apply All or batch mutation authority in v1. Overlapping diagnostics may be displayed independently, but each action is revalidated after every document change. + +### Re-review + +The host may use a stale/conflict action event to request a new review. Inkspan performs no network call, retry, provider fallback, or diagnostic regeneration. + +## Decoration and accessibility model + +Inline decorations contain only: + +```text +class="cwl-writing-diagnostic cwl-writing-diagnostic--{priority}" +data-cwl-diagnostic-id="opaque-id" +``` + +Inkspan does not derive aria-invalid or any other semantic accessibility state from opaque host strings. + +The editor does not infer spelling, grammar, mechanics, tone, or correctness from `categoryCode`, title, explanation, replacement, confidence, provenance, or source text. It does not place those strings in decoration attributes. The named diagnostics panel is the semantic accessibility surface. + +The panel must provide: + +- a named region and count summary; +- an ordered list with category, priority, title, and explanation; +- explicit previous/next navigation with roving focus; +- an affected-range Focus action; +- Apply, Ignore, Dismiss, and Explain native buttons; +- a disabled Apply action when no replacement exists; +- a polite status region for ordinary completed actions; +- an assertive alert only for an actual application conflict; +- no focus theft when diagnostics arrive asynchronously; and +- equivalent information without color, hover, pointer input, animation, or generated CSS content. + +Host strings render as React text nodes only. Action names may use the diagnostic title but never copy selected source text into DOM attributes. Underlines are a visual supplement, not the sole information channel. + +## Host feedback surface + +Default action events contain only opaque identifiers, exact revision evidence, opaque category code, generation, action, and bounded reason code. They contain no selected source text, replacement text, explanation, prompt, raw model output, email recipient, credential, document envelope, or tenant identifier. + +A host requiring richer audit evidence reads it from its own authorized review-session store. Generic analytics must not become a shadow copy of authored content. + +## Security and privacy + +- Treat every diagnostic object and host string as untrusted input. +- Reject accessors, prototypes, symbols, extra fields, sparse arrays, proxies, duplicate identifiers, and resource-limit violations. +- Render title, explanation, category, and replacement previews as text, never trusted HTML. +- Accept only plain-text replacement values in v1. +- Do not allow commands, JavaScript, arbitrary TipTap JSON, host callbacks, or executable markup inside a diagnostic. +- Do not place authored or model-produced text in logs, exceptions, analytics, performance marks, awareness payloads, or decoration attributes. +- Do not expose provider credentials, raw provider traces, or tenant data through provenance. +- Preserve Inkspan's no-runtime-environment-read and no-network-call contracts. + +## Semantic keyword prohibition + +Inkspan must produce zero diagnostics unless the host explicitly supplies them. It contains no semantic rule based on keywords, regexes, phrase dictionaries, sender domains, recipient counts, language names, positions, or nearest-text similarity. Opaque values that happen to contain words such as `spelling`, `grammar`, `mechanics`, `rude`, `incorrect`, or multilingual equivalents do not gain behavior or semantic ARIA authority. + +## Failure behavior + +| Condition | Inkspan behavior | +|---|---| +| No diagnostics supplied | Normal editor behavior | +| Host review pending or unavailable | Normal editor; optional host-owned status UI | +| Malformed or oversized input | Reject complete set through a redacted typed error; no editor mutation | +| Revision mismatch | Do not install or apply; return bounded stale/conflict evidence | +| Unsupported projection | Reject; no nearest-text or compatibility recovery | +| Invalid or ambiguous selector | Reject complete set; no guessed position | +| Local or remote document change | Invalidate complete active generation immediately | +| Host callback throws | Contain callback failure; preserve deterministic editor state | +| Editor destroyed | Return typed non-mutating lifecycle evidence | + +## Testing strategy + +### Contract and hostile-input tests + +- exact fields, types, limits, priorities, confidence, revision, projection, and selector validation; +- duplicate identifiers, sparse arrays, accessors, inherited fields, symbols, proxies, and hostile reflection; +- Unicode astral characters, Korean/CJK, combining marks, emoji, bidirectional text, empty/collapsed selectors, and grapheme boundaries; +- immutable detached public values and redacted errors. + +### Editor and concurrency tests + +- exact decoration attributes and no semantic ARIA derivation; +- local and remote `docChanged` invalidation; +- generation fencing for overlapping async verification; +- exact application recheck, ordinary transaction, resulting revision, and one-step undo; +- no mutation on rejected or stale input; +- privacy-minimized action events and callback-failure containment; +- zero diagnostics without host input. + +### Accessibility tests + +- named region, ordered list, count, category, priority, title, and explanation; +- keyboard navigation, roving focus, and explicit actions; +- no focus theft, polite status, conflict alert, forced colors, high contrast, reduced motion, print, zoom, and touch targets; +- no visual-only or hover-only information. + +### Collaboration, package, and browser tests + +- standalone/collaborative API parity and remote invalidation; +- no diagnostics in Yjs awareness payloads; +- SSR-safe shell and deterministic hydration; +- packed ESM/CommonJS/strict-TypeScript consumers; +- React-free `writing-diagnostics` subpath; +- Chromium, Firefox, and WebKit evidence; and +- exact 100% owned production statement, branch, function, and line coverage plus complete public JSDoc. + +## Performance constraints + +- Validation is linear in bounded diagnostic count and bounded projection size. +- At most 256 active diagnostics are admitted by default. +- One immutable snapshot and one revision derivation are shared across one verification generation. +- No document clone or digest is repeated merely to render an already admitted set. +- A document change invalidates rather than remaps the set, keeping v1 lifecycle cost deterministic. +- Telemetry records counts and timing buckets, not authored text. + +## Documentation and release requirements + +Implementation must synchronize root README, public API/JSDoc, PRD, TRD, API contract, architecture, threat model, operability, selector/revision guides, collaboration guide, test strategy, ADR index, traceability, CHANGELOG, package consumers, SBOM/provenance, rollback, and release evidence. + +The feature remains `Unreleased`. It may ship only after the complete stack is reconciled onto protected main, temporary branch-specific workflows are removed, exact-head CI/security/coverage/package/browser evidence succeeds, zero valid findings remain, qualifying independent review exists, and a separate release-only PR publishes immutable artifacts. + +## Out of scope + +- model invocation, model selection, prompt construction, rubric ownership, judge calibration, or provider failover; +- spelling dictionaries or deterministic grammar/tone classifiers; +- email/thread/recipient semantics; +- host submission, send, persistence, or compliance gates; +- persistent review sessions and cross-user aggregation; +- human-review assignment; and +- provider billing, retention, or training. + +## Primary references + +- W3C Web Annotation Data Model for Unicode-code-point `TextPositionSelector` semantics and its warning that positions are brittle across resource changes. +- TipTap v2 and ProseMirror documentation for immutable editor state, transactions, selections, and decorations. +- RFC 9110 for strong entity-tag semantics used by Inkspan revision evidence. +- Inkspan ADR 0011 for deterministic versus model-assisted authoring. +- Inkspan ADR 0018 for revision-scoped W3C selector authority. +- Inkspan ADR 0028 and ADR 0029 for host semantic authority, strict invalidation, and semantic-neutral accessibility. +- The accompanying doctoring record for LLM-judge bias and host calibration implications. + +## Approval boundary + +Approval of this design authorizes implementation work, not production claims. The feature remains unshipped until protected main contains the reconciled implementation, canonical documentation, exact acceptance evidence, independent approval, and verified release artifacts. diff --git a/src/adrQualityContract.test.ts b/src/adrQualityContract.test.ts index 04dcef71..32664c2e 100644 --- a/src/adrQualityContract.test.ts +++ b/src/adrQualityContract.test.ts @@ -23,6 +23,8 @@ const requiredAdrHeadings = [ /^## Rollback or supersession(?:\b|\s|$)/mu, ] as const; +const ADR_STATUS = /^Status: (Proposed|Accepted|Superseded)$/mu; + describe('ADR quality documentation contract', () => { it('preserves the canonical ADR quality requirements on every reconciliation branch', () => { const adrIndex = repositoryFile('docs/adr/README.md'); @@ -37,6 +39,25 @@ describe('ADR quality documentation contract', () => { expect(adrIndex).toContain('rollback or explicit supersession conditions'); }); + it('indexes every detailed ADR with its exact file and status', () => { + const adrIndex = repositoryFile('docs/adr/README.md'); + const adrFiles = detailedAdrFiles(); + + expect(adrFiles.length).toBeGreaterThan(0); + + for (const adrFile of adrFiles) { + const adr = repositoryFile(`docs/adr/${adrFile}`); + const status = adr.match(ADR_STATUS)?.[1]; + const adrNumber = adrFile.slice(0, 4); + + expect(status, `${adrFile} has no canonical status`).toBeDefined(); + expect( + adrIndex, + `${adrFile} is missing from the ADR index or has a stale status`, + ).toContain(`| [${adrNumber}](${adrFile}) | ${status} |`); + } + }); + it('applies the canonical quality sections to every detailed ADR', () => { const adrFiles = detailedAdrFiles(); diff --git a/src/writingDiagnosticsDocumentationContract.test.ts b/src/writingDiagnosticsDocumentationContract.test.ts new file mode 100644 index 00000000..75dce9fc --- /dev/null +++ b/src/writingDiagnosticsDocumentationContract.test.ts @@ -0,0 +1,105 @@ +import { readFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +import { describe, expect, it } from 'vitest'; + +/** Read one authoritative repository text artifact for deterministic assertions. */ +function repositoryFile(path: string): string { + return readFileSync(resolve(process.cwd(), path), 'utf8'); +} + +/** + * Collapse prose whitespace and code-span punctuation without weakening words. + * + * Markdown authors may correctly wrap identifiers such as `docChanged` or + * `aria-invalid` in code spans. The documentation contract checks the semantic + * prose, not that presentational punctuation choice. + */ +function normalizeProse(value: string): string { + return value.replace(/`/gu, '').replace(/\s+/gu, ' ').trim(); +} + +const designPath = + 'docs/superpowers/specs/2026-08-12-revision-bound-llm-writing-diagnostics-design.md'; +const planPath = + 'docs/superpowers/plans/2026-08-12-writing-diagnostics-implementation.md'; +const adrPath = 'docs/adr/0028-host-owned-llm-writing-diagnostics.md'; +const supersedingAdrPath = + 'docs/adr/0029-writing-diagnostics-v1-strict-invalidation.md'; + +describe('writing diagnostics documentation contract', () => { + it('keeps the public design examples synchronized with the implemented v1 types', () => { + const design = repositoryFile(designPath); + + expect(design).toContain( + "export type CwlWritingDiagnosticPriority =\n | 'advisory'\n | 'important'\n | 'critical';", + ); + expect(design).toContain( + 'readonly documentRevision: CwlEditorDocumentRevision;', + ); + expect(design).toContain( + 'readonly textProjection: CwlEditorTextProjectionIdentity;', + ); + expect(design).toContain('readonly judgePolicyVersion: string;'); + expect(design).toContain( + 'readonly reasonCode: CwlWritingDiagnosticActionReasonCode;', + ); + expect(design).toContain('readonly generation: number;'); + expect(design).not.toContain( + "export type CwlWritingDiagnosticPriority = 'suggestion' | 'important';", + ); + expect(design).not.toContain('readonly documentRevision: string;'); + expect(design).not.toContain( + "readonly projectionName: 'inkspan-prosemirror-text';", + ); + expect(design).not.toContain("readonly status: 'completed'"); + }); + + it('requires strict invalidation and semantic-neutral decoration guidance everywhere', () => { + const design = normalizeProse(repositoryFile(designPath)); + const plan = repositoryFile(planPath); + const adr = normalizeProse(repositoryFile(adrPath)); + const supersedingAdr = normalizeProse(repositoryFile(supersedingAdrPath)); + + const strictInvalidation = + 'Every local or collaborative transaction with docChanged === true invalidates the complete active diagnostic generation.'; + const semanticNeutrality = + 'Inkspan does not derive aria-invalid or any other semantic accessibility state from opaque host strings.'; + + expect(design).toContain(strictInvalidation); + expect(adr).toContain(strictInvalidation); + expect(supersedingAdr).toContain( + 'every transaction with docChanged === true, whether local or collaborative, invalidates the complete active diagnostic generation', + ); + expect(design).toContain(semanticNeutrality); + expect(adr).toContain(semanticNeutrality); + + expect(design).not.toContain('Inkspan may keep a diagnostic current'); + expect(design).not.toContain('local transaction mapping and invalidation'); + expect(design).not.toContain('Yjs remapping followed by revision rejection'); + expect(adr).not.toContain( + 'decoration mapping through local ProseMirror transactions', + ); + expect(adr).not.toContain('A safely mapped local transaction may preserve'); + expect(adr).not.toContain('valid transaction-local mapping'); + expect(plan).not.toContain('aria-invalid="spelling"'); + }); + + it('documents collapsed selectors and one-action-at-a-time application without batch authority', () => { + const design = normalizeProse(repositoryFile(designPath)); + const adr = normalizeProse(repositoryFile(adrPath)); + + expect(design).toContain( + 'selector values are non-negative safe integers with start <= end', + ); + expect(design).toContain( + 'Version 1 applies exactly one explicitly selected diagnostic at a time.', + ); + expect(adr).toContain( + 'Version 1 applies exactly one explicitly selected diagnostic at a time.', + ); + expect(design).not.toContain('a bounded batch application'); + expect(design).not.toContain('batch_overlap'); + expect(adr).not.toContain('Apply all'); + }); +});