Never advance the chain past a record that was not written

CHANGELOG.md

@@ -32,6 +32,35 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
   as a submission failure rather than writing it as a proof. A broken or hostile answer therefore
   leaves a recorded gap instead of a file that a later verify reports as corrupt evidence.
 
+### Removed
+- Capability tickets. `/evaluate` minted an HMAC-signed ticket, `capabilityTicket`, and nothing in
+  the product ever presented one back: the verifier function had exactly one caller, its own test.
+  The signing key was generated per process, so a ticket could not be checked by anything but the
+  process that issued it and did not survive a restart. A signed token nobody checks reads to a
+  reviewer as an authorization control and is not one, so the minting is gone rather than given an
+  endpoint no client calls. `PolicyEvaluationResponse` no longer carries the field.
+
+### Fixed
+- Audit records lost to a failing sink no longer read as tampering. The chain state advanced before
+  the sinks ran and the sink error was swallowed, so a record that never reached disk still moved
+  the chain on, and the next record carried the index jump and broken link of a DELETED record.
+  `agentwall verify` reported the same findings for a full partition as for the corpus forgery
+  `b3-record-removed`, while the process stayed up and said nothing. The chain now advances only
+  after a durable sink accepts the record, so the file stays contiguous across a loss; the refused
+  record goes to stderr under `agentwall_audit_dropped` without an integrity block; and the first
+  append that succeeds afterwards writes a gap declaration record, which both verifiers report as
+  the non-fatal `chain-gap-declared`. `/health` carries the drop counters. See
+  [The gap declaration record](docs/audit-format.md#the-gap-declaration-record).
+- The audit file sink rolls back an append that ran out of space part way through. A short write
+  left a fragment with no terminator, and the next append fused onto that line, so a full disk
+  destroyed a record that had been written on top of the one that had not.
+- A failed console write no longer terminates the service. With stdout or stderr redirected to a
+  regular file, node backs the stream with a synchronous writer whose failure Writable turns into
+  an `'error'` event rather than an exception, so the per-sink try/catch never saw it and an
+  unhandled event killed the process on the next tick. A partition full enough to stop the audit
+  file therefore took down the thing gating egress, on the record after the first one it could not
+  write.
+
 ## [0.2.0] - 2026-08-05
 
 The first tagged release. It freezes the on-disk evidence format and makes that format

README.md

@@ -257,6 +257,13 @@ and a foreign key exits 1.
 Full detail, including the conformance corpus and what verification does NOT prove, is in
 [docs/verification.md](docs/verification.md).
 
+A storage failure is deliberately not reported that way. When a record cannot be written, the
+chain does not advance past it, so the file stays contiguous and nothing in it reads as a removed
+record; the refused record goes to stderr under `agentwall_audit_dropped`, `/health` counts it, and
+the first append that succeeds afterwards writes a record declaring how many were lost. `verify`
+surfaces that as `chain-gap-declared` without failing the layer. A full partition and a deletion
+have to look different, or the alert for one gets ignored because of the other.
+
 ## Limits
 
 Stated plainly, because a security tool that oversells itself is worse than no tool.

docs/audit-format.md

@@ -311,6 +311,36 @@ Records that share a `chainIndex` inside one file are a distinct diagnosis from
 altered record. Many records but few distinct indexes is the signature of two processes each
 keeping their own chain state and appending to one file, not of an edit.
 
+### The gap declaration record
+
+A writer that produces a record and cannot store it has a choice about what to leave behind.
+Writing the next record anyway puts an index jump and a broken link into the file, which is
+byte for byte the shape of a record someone deleted; an operator reading that report is sent
+hunting a tamperer through a log nobody touched. So a conforming writer MUST NOT advance the
+chain past a record it did not store. The next record it does store takes the index and the
+`previousHash` the lost one would have taken, and the file stays contiguous.
+
+Contiguity alone makes the loss invisible, so a writer SHOULD then record what happened. A gap
+declaration is an ordinary record in every respect: it occupies its own index, it links, and
+its hash is computed exactly as any other. It is recognisable by two members:
+
+| Member | Value |
+| --- | --- |
+| `action` | `"audit:chain-gap"` |
+| `metadata.droppedRecords` | Decimal count of records that were produced and not stored, as a string |
+
+A verifier MAY report such a record, and the bundled implementations do, as a non-fatal
+finding named `chain-gap-declared`. Two rules bound what it means:
+
+- A verifier MUST NOT let a declaration excuse anything. An index gap, a link break, and a
+  hash mismatch are judged the same whether or not a declaration is present. The record is the
+  writer's account of a hole, not a licence to have one, and treating it as a licence would
+  hand an attacker a member to add to a rewritten file.
+- A verifier MUST NOT treat the absence of a declaration as proof that nothing was lost. The
+  declaration can only be written once storage accepts writes again, so a process that dies
+  during the outage never writes one. This is the completeness limit the format already has,
+  not a new one.
+
 ### Worked example: two records chained
 
 The record from the worked example above is followed on the next line by:
@@ -747,6 +777,12 @@ reports otherwise is wrong.
 - **Completeness of capture.** Every hash and every signature is computed over records that
   exist. Nothing here can show that an action which was never written down did not happen. An
   anchor proves records were not altered afterwards; it does not prove the log is complete.
+- **That a gap was declared.** A writer keeps the chain contiguous across records it could not
+  store, so a storage outage leaves no linkage failure to find. The
+  [gap declaration record](#the-gap-declaration-record) is the only in-band trace, and it can
+  be written only once storage recovers. A file that ends where the disk filled, or a process
+  that died before recovery, carries no trace at all. This is the completeness limit above,
+  stated for the case an operator is most likely to meet.
 - **Authorship, without a pinned key.** A checkpoint signature verified against the key the
   checkpoint itself carries proves only internal consistency. Anyone who can write the file
   can generate a key, sign their version, and produce a set of records that verifies
@@ -790,6 +826,8 @@ A verifier written from this document is conforming when all of the following ho
 - It reports `chained`, `linked`, and `anchored` separately, and reports counts of pending,
   confirmed, and failed anchors.
 - It distinguishes a torn final line from other parse failures.
+- It judges an index gap, a link break, and a hash mismatch identically whether or not the
+  file contains a gap declaration record.
 - It resolves a relative manifest `path` against the manifest's directory, never against its
   own working directory.
 - It checks every manifest entry against the segment it names, reports both a missing segment

scripts/security-regression.js

@@ -25,7 +25,7 @@ const securityGroups = [
     tests: [
       'tests/approval-webhook.test.ts',
       'tests/audit-chain.test.ts',
-      'tests/capability-ticket.test.ts',
+      'tests/audit-durability.test.ts',
     ],
   },
   {

src/audit/chain.ts

@@ -16,6 +16,17 @@ const HASH_STATUS = "chained-local";
 // a tampered record.
 const CANON = "cu1";
 
+/**
+ * The `action` that marks a record as the writer's declaration of a hole.
+ *
+ * A record carrying it is an ordinary chained record in every respect. It exists because a
+ * writer that could not store a record has no other way to say so inside the evidence: the
+ * chain stays contiguous across the loss, so nothing in the linkage reveals that anything
+ * happened. Named here rather than in the logger because the writer and the chain walk both
+ * need the same string, and a copy in each would drift.
+ */
+export const AUDIT_CHAIN_GAP_ACTION = "audit:chain-gap";
+
 type AuditPayloadValue = string | number | boolean | null | AuditPayloadValue[] | { [key: string]: AuditPayloadValue };
 
 function emitCanonical(value: AuditPayloadValue): string {

src/audit/file-sink.ts

@@ -5,14 +5,16 @@ import {
   writeFileSync,
   existsSync,
   fstatSync,
+  ftruncateSync,
   mkdirSync,
   openSync,
   readFileSync,
   readSync,
+  statSync,
 } from "fs";
 import { dirname } from "path";
 import { AuditEvent } from "../types";
-import { AuditChainState, findDuplicateKey } from "./chain";
+import { AUDIT_CHAIN_GAP_ACTION, AuditChainState, findDuplicateKey } from "./chain";
 
 /**
  * Durable JSONL sink for the audit chain.
@@ -173,16 +175,86 @@ export function claimWriter(
   });
 }
 
-/** One audit record per line, nothing else in the file. */
-export function createFileSink(path: string): (event: AuditEvent) => void {
+/**
+ * Undo an append that ran out of space part way through.
+ *
+ * A write that fails with ENOSPC can still have moved bytes: the kernel copies what fits and
+ * reports the shortfall, leaving a headless fragment with no terminator. The next append
+ * lands on that same line and fuses the two into one unparseable record, so a full disk
+ * destroys a record that WAS written on top of the one that was not.
+ *
+ * Only ever removes bytes this sink just wrote. `mark` is the length after the last append
+ * that completed and claimWriter guarantees no other process appends between the two, so the
+ * bytes past it are ours. Anything that does not match that picture is left alone, because a
+ * truncate aimed at the wrong offset deletes records instead of a fragment.
+ */
+function discardPartialAppend(path: string, mark: number): boolean {
+  let fd: number | undefined;
+  try {
+    fd = openSync(path, "r+");
+    if (fstatSync(fd).size <= mark) return false; // nothing of ours landed
+    if (mark > 0) {
+      const boundary = Buffer.alloc(1);
+      readSync(fd, boundary, 0, 1, mark - 1);
+      if (boundary[0] !== 0x0a) return false; // the mark is not a record boundary
+    }
+    ftruncateSync(fd, mark);
+    return true;
+  } catch {
+    return false;
+  } finally {
+    if (fd !== undefined) {
+      try {
+        closeSync(fd);
+      } catch {
+        /* ignore */
+      }
+    }
+  }
+}
+
+/**
+ * One audit record per line, nothing else in the file.
+ *
+ * Throws when the record did not reach the file. The caller decides what a refusal means for
+ * the chain; swallowing it here is what let the chain advance past records that were never
+ * stored.
+ *
+ * `append` is injectable for the same reason the lock probe is: the failure that matters here
+ * is a short write on a full filesystem, which a test process cannot stage on demand, and it
+ * is the case that decides whether a record that WAS written survives the one that was not.
+ */
+export function createFileSink(
+  path: string,
+  append: (target: string, data: string) => void = appendFileSync,
+): (event: AuditEvent) => void {
   mkdirSync(dirname(path), { recursive: true });
   claimWriter(path);
+  // Length as of the last append that completed. Tracked rather than measured per record
+  // because the repair needs it BEFORE the write, and a stat on every record would put a
+  // syscall on the proxy's per-request path to serve an error case that almost never runs.
+  let committed = existsSync(path) ? statSync(path).size : 0;
   return (event: AuditEvent) => {
-    // Flag "a" opens O_APPEND, so the kernel makes the seek-to-end and the write a
-    // single atomic operation against the file offset. That is the guarantee that keeps
-    // records from interleaving, not PIPE_BUF, which governs pipes and is only 4096 on
-    // Linux, well under a typical record carrying full detections.
-    appendFileSync(path, JSON.stringify(event) + "\n", { encoding: "utf8" });
+    // Handed over as a string, not a Buffer this function built. appendFileSync converts
+    // internally either way, and doing it here measures about 5 microseconds per record
+    // slower, which the proxy pays on every request.
+    const line = JSON.stringify(event) + "\n";
+    try {
+      // Flag "a" opens O_APPEND, so the kernel makes the seek-to-end and the write a
+      // single atomic operation against the file offset. That is the guarantee that keeps
+      // records from interleaving, not PIPE_BUF, which governs pipes and is only 4096 on
+      // Linux, well under a typical record carrying full detections.
+      append(path, line);
+    } catch (err) {
+      const discarded = discardPartialAppend(path, committed);
+      throw new Error(
+        `audit append to ${path} failed: ${(err as Error).message}` +
+          (discarded ? "; the partial write was rolled back" : ""),
+      );
+    }
+    // Bytes, not characters: a record carrying non-ASCII metadata occupies more of the file
+    // than its length, and a mark short of the real end would leave a fragment behind.
+    committed += Buffer.byteLength(line, "utf8");
   };
 }
 
@@ -385,6 +457,16 @@ export function verifyChainFile(
     if (rehash(ev) !== integ.hash) {
       problems.push(`line ${i + 1}: hash mismatch, record altered after write`);
     }
+    if (ev.action === AUDIT_CHAIN_GAP_ACTION) {
+      // The writer's own statement, inside the chain, that records it produced could not be
+      // stored. Surfaced because the chain is contiguous across such a loss and would
+      // otherwise pass in silence. It is NOT a licence: the index and link checks above ran
+      // first and still stand, so a marker cannot be used to excuse a removed record.
+      notes.push(
+        `line ${i + 1}: chain-gap-declared, the writer recorded that ` +
+          `${ev.metadata?.droppedRecords ?? "an unstated number of"} record(s) could not be written here`,
+      );
+    }
     expectedIndex = integ.chainIndex + 1;
     expectedPrev = integ.hash;
   });