docs: state the premise and what a refusal means

README.md

@@ -18,6 +18,18 @@ Question in, cited answer or honest refusal out. A chatbot that's right most
 of the time speaks *for* you; an answer engine that cites or declines speaks
 *from* you.
 
+**The premise.** The archive has an owner, and the owner is the author. What
+enters the archive, how much of each piece may travel, and when the system
+declines are the author's decisions, made before any question arrives. The
+engine's job is to keep those decisions in force at the one place a language
+model would dissolve them: where the model's capacity to produce a plausible
+answer gets mistaken for the authority to give it. The model's competence is
+deliberately greater than its permission. "Who decides what belongs in the
+archive?" is therefore not an open question here; the author does, by
+premise. The question the code answers is whether software can hold the line
+between what an author has authorized and what can plausibly be inferred from
+their work.
+
 This repo is the teaching-sized version of the engine behind "Ask the
 Archive" on [lukefwalton.com](https://lukefwalton.com). It runs out of the
 box on a bundled example corpus (by "Person A" — a placeholder, not a
@@ -156,6 +168,15 @@ Finally, `assertCitationsGroundedInEvidence` verifies every citation is the
 exact (id, url) pair of something actually retrieved. An invented source is
 an error, not a footnote.
 
+**What a refusal means.** `not-found` says the archive authorized no answer:
+nothing retrieved cleared the floor, or nothing that cleared it bears on the
+question. It does not say the question is unanswerable,
+that no evidence exists anywhere, or that the author has no view.
+`related-material` is the case where evidence exists and may only be pointed
+to. A refusal here is a disciplined non-assertion, not a claim to know the
+limits of one's own knowledge, and a reader is meant to trust it as the
+former.
+
 One mode gets a fourth layer. A `related-material` answer's prose is not the
 model's: after grounding, the engine replaces it with a fixed sentence
 rendered from the cited hints' label and locator
@@ -197,7 +218,15 @@ archive, one named author, a bounded corpus. The mechanisms don't depend on
 that smallness (none of them refers to corpus size), but a small demo can't
 prove that holding these boundaries stays affordable at public, plural, or
 contested scale. This repo is the bounded case on purpose, not a proof about
-the unbounded one.
+the unbounded one. Nor is it an open question here who decides what the
+archive holds and what may travel: the author does, by premise. The question
+the code settles is whether those decisions survive contact with a model that
+could infer past them.
+
+The guarantee is about what reaches the model and the wire, not about custody:
+private text already leaves for the embedding provider at ingest, and for the
+gist model when that feature is on ([`SECURITY.md`](./SECURITY.md),
+[`docs/CONTRACT.md`](./docs/CONTRACT.md)).
 
 The limit is narrower than it looks, though. What the engine guarantees is
 **soundness**: nothing enters an answer that isn't grounded in retrieved

paper.md

@@ -20,11 +20,11 @@ bibliography: paper.bib
 
 # Summary
 
-`answer-engine` is a small TypeScript package for searching a body of work its author owns, some of it published and some not, and for handing the results to a language model or an external caller without any text of the unpublished part leaving the server. Every retrievable fragment carries a disclosure policy, resolved and stored before any query: which layer its text is in (`public` or `private`) and how much of it may travel (`text`; a short `semantic` description the author released; a `locator` that says only where the moment is; or `none`). One function, `project()`, is the single crossing from retrieval to the wire type a consumer may serialize or prompt from, a union whose private variants have no field for a fragment's text; `private + text` is not a member of the policy type, so the compiler refuses it in code and the index validator refuses it in data. A private fragment can therefore still do work: it is found, it returns as a locator or an authorized description, and every hit carries provenance (creators, date, identifiers, and a locator such as `p. 184` or `12:30–14:05`) a caller can cite with no synthesis on the server. The package provides the contract types, ingest, retrieval, the two index formats, a reference consumer that cites or refuses, and an evaluation harness whose gold suite and canary sweep gate every change. Tests and demo run keyless; the package is on npm and in production.
+`answer-engine` is a small TypeScript package for searching a body of work its author owns, some of it published and some not, and for handing the results to a language model or an external caller without any text of the unpublished part leaving the server. The frame has an owner by premise: the author decides what enters the archive, how much of each piece may travel, and when the system declines, and the software keeps those decisions in force where a model would infer past them. Every retrievable fragment carries a disclosure policy, resolved and stored before any query: which layer its text is in (`public` or `private`) and how much of it may travel (`text`; a short `semantic` description the author released; a `locator` that says only where the moment is; or `none`). One function, `project()`, is the single crossing from retrieval to the wire type a consumer may serialize or prompt from, a union whose private variants have no field for a fragment's text; `private + text` is not a member of the policy type, so the compiler refuses it in code and the index validator refuses it in data. A private fragment can therefore still do work: it is found, it returns as a locator or an authorized description, and every hit carries provenance (creators, date, identifiers, locator) a caller can cite with no synthesis on the server. The package provides the contract types, ingest, retrieval, both index formats, a reference consumer that cites or refuses, and an evaluation harness that gates every change. Tests and demo run keyless; the package is on npm and in production.
 
 # Statement of need
 
-Retrieval-augmented generation [@lewis2020; @gao2023] assumes that what retrieval returns may be shown to the model, and the Model Context Protocol [@mcp2026] now lets a model the operator does not control call a search tool directly. For an author's archive it is often false: an unpublished podcast transcript, a manuscript under contract, a notebook. The operator's options have been to leave such material out of retrieval, to let it in and filter or redact what comes back, or to trust a prompt. The first two decide whether a chunk may be seen at all; neither lets a private fragment contribute to an answer by being found, located, and described while none of its text travels. The third is a promise made to the model, not a property of the data.
+Retrieval-augmented generation [@lewis2020; @gao2023] assumes that what retrieval returns may be shown to the model, and the Model Context Protocol [@mcp2026] now lets a model the operator does not control call a search tool directly. For an author's archive it is often false: an unpublished podcast transcript, a manuscript under contract, a notebook. The operator could exclude such material, filter or redact what comes back, or trust a prompt. The first two decide whether a chunk may be seen at all; neither lets a private fragment contribute to an answer by being found, located, and described while none of its text travels. The third is a promise made to the model, not a property of the data.
 
 The research question the software tests is whether that disclosure can be made a property of the data: fixed per fragment before any query, carried on the wire as the only representation a consumer can receive, and unrepresentable in the type when forbidden. The question comes from work on answerability in generative systems [@walton2026a; @walton2026b; @walton2026c; @walton2026d], Shoemaker's standing to be asked for the judgment behind an act [@shoemaker2011]: the frame a system acts under must have an owner who can be reached [@walton2026b], and an operation no one answers for should be unavailable as an ordinary operation of the system [@walton2026d]. What may travel about a retrieved private fragment is one such operation. `answer-engine` is that argument's materials, the runnable link between its design theory and a live deployment, offered to the conversation on selective disclosure in retrieval-augmented systems.
 
@@ -36,21 +36,23 @@ Two recent lines of work come closest. SD-RAG [@almasoud2026] shares the aim, se
 
 # Software design
 
-The contract (`docs/CONTRACT.md`) is the design of record. An **entity** (a song, a transcript, a book) declares a default disclosure; its **fragments** (a chapter, a transcript window) may narrow the exposure but never change the layer. Disclosure is resolved at build time by a function that only downgrades: a requested `semantic` becomes `locator` when its description is missing, failed the lint, was vetoed, went stale, or awaits a required review.
+`docs/CONTRACT.md` is the design of record. An **entity** (a song, a transcript, a book) declares a default disclosure; its **fragments** (a chapter, a transcript window) may narrow the exposure but never change the layer. Disclosure is resolved at build time by a function that only downgrades: a requested `semantic` becomes `locator` when its description is missing, failed the lint, was vetoed, went stale, or awaits a required review.
 
 **One crossing.** `retrieve()` returns scored fragments to the process that holds the index; `project()` is the only path from them to `EvidenceHit`, a discriminated union on `exposure` whose `locator` variant carries provenance and nothing else and whose `semantic` variant carries a branded `LintedGist` only the lint can construct. The private index's validator refuses `private + text` in data, where types do not reach. Scores on private hits are rounded and carry no per-plugin breakdown, because a full-precision similarity against a private vector is a measurement of it. The type fixes the form of a hit; what a sequence of hits reveals (which fragment answered, and roughly how well) is a residue the contract names and assigns to the consumer's rate limiting and the author's policy, `none` included.
 
 **Authorship and authorization are separate.** A description is drafted by a model at ingest, stored beside the index as private material, and released only by the entity's policy, a per-fragment override, or the author's edit, veto, or review. Release by policy authorizes a class of machine drafts the author need not have read; release by edit or review authorizes text the author has seen. The build-time lint is a tripwire, not a classifier: it refuses any run of five words shared with the fragment or its whole entity and catches nothing shorter. Close paraphrase, names, and spoilers are owned by policy, veto, and the evaluation's canaries.
 
 **Two artifacts, one validator.** The private index holds text and vectors and re-earns every lint at each load; the served index is the same file after a strip that blanks text and drops every projection a policy did not release, and a load-time check verifies the strip. The lint cannot run where the text is blank, so a served index is trusted to descend from a validated private one. In deployment the strip keeps text off the wire; the type states and checks the contract and binds a consumer holding the private index.
 
+**A refusal is a disciplined non-assertion.** `not-found` says the archive authorized no answer, and `related-material` says evidence exists and may only be pointed to; neither claims the matter is unknowable.
+
 **Evaluation is the admission gate.** A fixed gold set with required behaviours (answer, route to private material without quoting it, refuse) runs keyless at the retrieval tier, and a canary sweep checks every served gist against private phrases the author lists, so an exposure change is gated like a scoring change.
 
-Retrieval is brute-force cosine over an in-memory index; HTTP, transport, and site-specific ranking live in consumers.
+HTTP, transport, and site-specific ranking live in consumers.
 
 # Research impact statement
 
-The package is the substrate under Ask the Archive (lukefwalton.com/ask), a production deployment over the author's archive of roughly ten thousand embedded fragments, where published passages are public and unpublished transcript text is private: embedded for search, served as where to listen, never as the text itself. The same deployment exposes a rate-limited retrieval-only endpoint and an MCP server through which an external model receives hits at the exposure the author set, with no synthesis on the server, and it carries an unpublished novel as a private entity whose chapters are served as locators. The `semantic` exposure is exercised by the package's tests and demo and reaches the deployment only as a policy change and a rebuild. A gold suite of 311 queries, 28 of which must refuse, every row carrying hard-fail guards against leakage and misattribution, gates every change to that deployment. The software is cited as the materials of the answerability working paper [@walton2026d], documented in a technical note [@walton2026note], and archived under a concept DOI [@walton2026software]. What it shows is the feasibility of one architectural claim in those papers: that a class of normative juncture, here what may travel about a retrieved private fragment, can be moved out of model behaviour into representable operations. It does not bear on the working paper's empirical claim about how correction compounds into capability, which that paper leaves unmeasured. No use by other research groups is claimed.
+The package is the substrate under Ask the Archive (lukefwalton.com/ask), a production deployment over the author's archive of roughly ten thousand embedded fragments, where published passages are public and unpublished transcript text is private: embedded for search, served as where to listen, never as the text itself. The same deployment exposes a rate-limited retrieval-only endpoint and an MCP server through which an external model receives hits at the exposure the author set, and it carries an unpublished novel as a private entity whose chapters are served as locators. The `semantic` exposure is exercised by the package's tests and demo and reaches the deployment only as a policy change and a rebuild. A gold suite of 311 queries, 28 of which must refuse, every row carrying hard-fail guards against leakage and misattribution, gates every change to that deployment. The software is cited as the materials of the answerability working paper [@walton2026d], documented in a technical note [@walton2026note], and archived under a concept DOI [@walton2026software]. What it shows is the feasibility of one architectural claim in those papers: that a class of normative juncture, here what may travel about a retrieved private fragment, can be moved out of model behaviour into representable operations. It does not bear on the working paper's empirical claim about how correction compounds into capability, which that paper leaves unmeasured. No use by other research groups is claimed.
 
 # AI usage disclosure