diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a9ab13..b9a0d9b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,14 +15,19 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht evaluated, matching the independent Go verifier. ### Changed -- The bundled verifier's `anchored` layer judges an anchor record by the evidence instead of by what - the record says about itself. It recomputes each record's digest from the checkpoint the record - embeds and reports `digest-mismatch` when they differ, requires non-empty proof bytes behind any - submission that reached a calendar and reports `proof-missing` when they are absent, and parses the - proof against the submitted digest, reporting `proof-parse-error` when the container does not - parse. An anchor claiming `confirmed` with no proof file behind it fails the layer. A submission - recorded with an `error` is exempt: it never reached a calendar, so it has no proof to point at and - is already counted as failed. +- The bundled verifier returns the same verdict as the independent Go verifier on every case in the + conformance corpus, and the harness declares no divergences. Two parts of the report moved. The + `anchored` layer judges an anchor record by the evidence instead of by what the record says about + itself: it recomputes each record's digest from the checkpoint the record embeds and reports + `digest-mismatch` when they differ, requires non-empty proof bytes behind any submission that + reached a calendar and reports `proof-missing` when they are absent, and parses the proof against + the submitted digest, reporting `proof-parse-error` when the container does not parse. An anchor + claiming `confirmed` with no proof file behind it fails the layer. A submission recorded with an + `error` is exempt: it never reached a calendar, so it has no proof to point at and is already + counted as failed. The `chained` layer reports a partial final line as `torn-tail` and does not + fail over it, because a hard kill mid-append leaves exactly one and calling that tampering would + cry wolf on every crash; a line that does not parse anywhere else, or one that carries its + terminator, stays a fatal failure. - `agentwall anchor` parses a calendar response before keeping it, and treats one that does not parse as a submission failure rather than writing it as a proof. A broken or hostile answer therefore leaves a recorded gap instead of a file that a later verify reports as corrupt evidence. diff --git a/README.md b/README.md index ee920f7..4e780d0 100644 --- a/README.md +++ b/README.md @@ -153,9 +153,8 @@ by the same people in the same language as the writer proves the code agrees wit parses JSON with a different parser, verifies Ed25519 with a different cryptography stack, and implements [docs/audit-format.md](docs/audit-format.md) rather than importing anything from `src/`. That document is the only thing the two programs have in common, so when both accept a file, the -agreement is evidence about the FORMAT. Where they disagree one of them is wrong, and those cases -are listed in [Where the two verifiers disagree today](#where-the-two-verifiers-disagree-today) -rather than buried. +agreement is evidence about the FORMAT. Where they disagree one of them is wrong, and the +conformance harness fails on the disagreement rather than burying it. Every command below runs against evidence committed to this repository, so it reproduces from a bare checkout. Point `--audit` at your own file to check your own records; both CLI commands also @@ -188,6 +187,13 @@ left untouched: the `chained` layer then fails with `line 2: hash mismatch, record altered after write`, names the file by absolute path, and the command exits 1. +The `anchored` layer judges an anchor record by its evidence rather than by what the record says +about itself. It recomputes each record's digest from the checkpoint the record embeds +(`digest-mismatch`), requires non-empty proof bytes behind any submission that reached a calendar +(`proof-missing`), and parses that proof against the submitted digest (`proof-parse-error`). A +submission recorded with an error is exempt: it never reached a calendar, so it has no proof to +point at and is already counted as failed. + `anchored` fails until an anchor exists. `agentwall anchor` seals the live segment, signs an Ed25519 checkpoint over the head, and submits its digest to OpenTimestamps, which needs network access and no account: @@ -356,10 +362,7 @@ The run prints one line per case. Its tail: ``` ok b9-anchor-digest-altered exit=1 chained=true linked=true anchored=false ok b10-proof-truncated exit=1 chained=true linked=true anchored=false -DIVERGENCE b11-torn-tail - format expects exit=1 chained=true linked=true anchored=false - typescript returns exit=1 chained=false linked=true anchored=false - it reports a partial final line as a broken chain rather than as the torn tail a hard kill leaves behind +ok b11-torn-tail exit=1 chained=true linked=true anchored=false ok b12-duplicate-key-shadowed exit=1 chained=false linked=true anchored=false ok b13-confirmed-without-proof exit=1 chained=true linked=true anchored=false ok b14-submission-never-reached-calendar exit=1 chained=true linked=true anchored=false @@ -369,7 +372,7 @@ ok b17-sealed-segment-missing exit=1 chained=true linked=false anchored ok l1-confirmed-with-pending-proof exit=0 chained=true linked=true anchored=true ok l2-legacy-canon-unmarked exit=1 chained=false linked=true anchored=false -26 cases, typescript and go: 25 agreed, 1 declared divergence(s), 0 failure(s) +26 cases, typescript and go: 26 agreed, 0 declared divergence(s), 0 failure(s) ``` Each case is copied to a temp directory before it runs, so a verifier cannot alter what it checks. @@ -381,25 +384,11 @@ npm run gen:corpus && git status --porcelain verifier/testdata That prints nothing, because the regenerated tree is byte identical to the committed one. -### Where the two verifiers disagree today - -One corpus case gets different verdicts from the two verifiers. Both reject the file, and the -bundled verifier blames the chain instead of naming the torn tail: - -| Case | The edit | Bundled TypeScript verifier | Go verifier | -| --- | --- | --- | --- | -| `b11-torn-tail` | a partial final line, as a hard kill leaves behind | `chained` FAIL, exit 1. It condemns the whole chain over one partial write | `torn-tail` reported distinctly, `chained` PASS, exit 1 because nothing is anchored | - -That naming gap is a limit of the bundled verifier as it ships today. The harness prints every entry -in this list on each run and fails if one of them starts agreeing -([`scripts/conformance.js:40-51`](scripts/conformance.js)), so the list cannot rot into a set of -excuses, and it is why the summary line above reports one declared divergence instead of agreement on -every case. - -The bundled verifier recomputes each anchor record's digest from the checkpoint the record embeds, -requires a non-empty proof file behind any submission that reached a calendar, and parses that proof -against the submitted digest. So an altered digest, a deleted proof, and a truncated proof all fail -the `anchored` layer in both verifiers rather than in one. +Both verifiers return the same verdict on every case, which is why the summary declares no +divergences. That is agreement across the 26 cases the corpus contains, not a proof that the two +implementations are equivalent: a forgery nobody has written a case for has been put to neither of +them. The harness fails the run if they ever stop agreeing on a case it does contain +([`scripts/conformance.js:40-50`](scripts/conformance.js)). ### What verification does not prove @@ -542,7 +531,7 @@ Stated plainly, because a security tool that oversells itself is worse than no t | Anchoring is pending, not instant | An OpenTimestamps anchor stays `pending` until a Bitcoin block confirms, roughly one to six hours. `verify` reports pending as pending. Pending is not proof. | | Anchoring proves no alteration, not completeness | An anchor shows that what was written was not altered afterwards. It cannot show that everything which should have been written was. Silent omission at write time is a different, unsolved problem. | | A signature is necessary, not sufficient | It proves a key holder vouched. On a host where the audited principal can read the key, an agent with root can sign anything the operator can. Off-box anchoring is what closes that gap. | -| The bundled verifier is the less strict of the two | Four conformance cases (`b9`, `b10`, `b11`, `b13`) get different verdicts from the two verifiers. In `b9`, `b10`, and `b13` the bundled TypeScript verifier accepts evidence the format rejects; in `b11` both reject the file and the bundled one blames the chain rather than naming the torn tail. They are listed in [Where the two verifiers disagree today](#where-the-two-verifiers-disagree-today), and the conformance harness fails if any of them starts agreeing. | +| Verifier agreement is bounded by the corpus | The bundled TypeScript verifier and the independent Go verifier return the same verdict on all 26 conformance cases. That is agreement about the cases the corpus contains, not a proof that the two implementations are equivalent, and it says nothing about a forgery neither has been asked to judge. The harness fails if they ever stop agreeing on a case the corpus does contain. | | No TLS interception | CONNECT traffic is visible at hostname and port level only. Request paths, headers, and bodies stay opaque. This is deliberate: MITM would need a CA in every runtime trust store, which breaks the framework-agnostic property the proxy exists for. | | Attribution is Linux-only | It reads `/proc/net/tcp` and `/proc//fd`. There is no macOS or Windows equivalent here. The rest of the server is portable; process attribution is not. | | Channel containment is Telegram only | Slack and Discord appear in the platform schema ([`src/integrations/communication-channel/control.ts:5`](src/integrations/communication-channel/control.ts)) with no route implementation behind them. | diff --git a/scripts/conformance.js b/scripts/conformance.js index 499abfe..c565a02 100755 --- a/scripts/conformance.js +++ b/scripts/conformance.js @@ -39,16 +39,15 @@ const SKIP_GO = process.env.CONFORMANCE_SKIP_GO === "1"; /** * Cases where the bundled TypeScript verifier returns something other than the format's - * verdict. Each entry states what it returns instead and why, in one line. These are gaps in - * the bundled verifier, not opinions about the corpus. + * verdict, each stating what it returns instead and why, in one line. + * + * Empty is an assertion, not dead scaffolding: it says the two implementations currently + * disagree about nothing in this corpus. The harness enforces that in both directions. A + * change that makes them disagree fails the run because the divergence is undeclared, and + * papering over one by adding an entry here turns an empty object into a reviewable diff + * carrying a human-written reason, rather than a gap that quietly becomes normal. */ -const DIVERGENCES = { - "b11-torn-tail": { - exit: 1, - layers: { chained: false }, - why: "it reports a partial final line as a broken chain rather than as the torn tail a hard kill leaves behind", - }, -}; +const DIVERGENCES = {}; function fail(message) { process.stderr.write(`${message}\n`); diff --git a/src/audit/anchor-service.ts b/src/audit/anchor-service.ts index f70201f..bd1b55a 100644 --- a/src/audit/anchor-service.ts +++ b/src/audit/anchor-service.ts @@ -343,6 +343,10 @@ export function runVerify(paths: AnchorPaths): VerifyReport { ); segPaths.add(resolve(r.auditPath)); const chainProblems: string[] = []; + // Kept apart from the problems so a torn tail is surfaced to the operator without + // deciding the verdict. What fails the layer is evidence of an edit, and a partial + // trailing line is not that. + const chainNotes: string[] = []; let totalRecords = 0; let walked = 0; for (const p of segPaths) { @@ -375,12 +379,13 @@ export function runVerify(paths: AnchorPaths): VerifyReport { } else { for (const problem of v.problems) chainProblems.push(`${p}: ${problem}`); } + for (const note of v.notes) chainNotes.push(`${p}: ${note}`); } layers.push({ name: "chained", ok: chainProblems.length === 0, detail: `${totalRecords} records across ${walked} segment(s)`, - problems: chainProblems, + problems: [...chainProblems, ...chainNotes], }); // Layer 2: segment linkage. diff --git a/src/audit/file-sink.ts b/src/audit/file-sink.ts index cd24105..d59580f 100644 --- a/src/audit/file-sink.ts +++ b/src/audit/file-sink.ts @@ -298,6 +298,12 @@ export interface ChainVerification { ok: boolean; records: number; problems: string[]; + /** + * Findings that are reported but do not condemn the file. Only a torn tail lands here: + * it is damage a crash produces, not evidence of an edit, so it is surfaced without + * failing the layer. + */ + notes: string[]; } /** @@ -310,16 +316,25 @@ export function verifyChainFile( rehash: (event: AuditEvent) => string ): ChainVerification { const problems: string[] = []; - if (!existsSync(path)) return { ok: false, records: 0, problems: ["file does not exist"] }; + const notes: string[] = []; + if (!existsSync(path)) return { ok: false, records: 0, problems: ["file does not exist"], notes }; - const lines = readFileSync(path, "utf8") - .split("\n") - .filter((l) => l.trim() !== ""); + const raw = readFileSync(path, "utf8"); + // Whether the file ends with its terminator is the whole distinction between a hard kill + // and an edit, and a filtered split destroys it, so it is read off the raw bytes first. + // Only the trailing chunk can be unterminated: every earlier one was followed by an LF. + const chunks = raw.split("\n"); + const unterminated = raw.endsWith("\n") ? -1 : chunks.length - 1; + const lines: { text: string; torn: boolean }[] = []; + chunks.forEach((text, i) => { + if (text.trim() === "") return; + lines.push({ text, torn: i === unterminated }); + }); let expectedIndex: number | null = null; let expectedPrev: string | null = null; - lines.forEach((line, i) => { + lines.forEach(({ text: line, torn }, i) => { // Checked on the raw bytes, because JSON.parse silently collapses a duplicate member // and the evidence of it is gone the moment the line is parsed. Such a record counts // toward nothing: it does not advance the expected index or the expected link, so the @@ -337,6 +352,20 @@ export function verifyChainFile( try { ev = JSON.parse(line) as AuditEvent; } catch { + if (torn) { + // A process killed mid-append leaves exactly one partial line: the last, with no + // terminator. Calling that a broken chain sends an operator hunting a tamperer + // through a log that was never touched, and a security tool that cries wolf on + // every hard kill gets its alerts ignored. The complete records before it still + // chain, so it is named and the file stands. An unparseable line ANYWHERE else, + // or one that carries its terminator, was not produced by an interrupted append + // and stays fatal. + notes.push( + `line ${i + 1}: torn-tail, the final line has no terminator and does not parse, ` + + "which is what a hard kill mid-append leaves; the records before it are complete", + ); + return; + } problems.push(`line ${i + 1}: not valid JSON`); return; } @@ -360,5 +389,5 @@ export function verifyChainFile( expectedPrev = integ.hash; }); - return { ok: problems.length === 0, records: lines.length, problems }; + return { ok: problems.length === 0, records: lines.length, problems, notes }; } diff --git a/tests/audit-anchor-service.test.ts b/tests/audit-anchor-service.test.ts index 6f370eb..87e0295 100644 --- a/tests/audit-anchor-service.test.ts +++ b/tests/audit-anchor-service.test.ts @@ -4,6 +4,7 @@ import { tmpdir } from "os"; import { join } from "path"; import { runAnchorPass, runVerify, resolvePaths } from "../src/audit/anchor-service"; import { readManifest } from "../src/audit/rotation"; +import { chainAuditEvent } from "../src/audit/chain"; import type { HttpPoster } from "../src/audit/anchor"; import { pendingProof } from "./ots-fixtures"; @@ -580,4 +581,42 @@ describe("verify", () => { // bytes, not from the record's status field. expect(anchored?.detail).toMatch(/proofs carry 1 calendar and 0 bitcoin attestation\(s\)/); }); + + it("reports a torn tail on the chained layer without failing it", () => { + // A partial final line is what a hard kill mid-append leaves. Failing the layer over + // it tells an operator their log was tampered with when a process merely died, and a + // tool that raises tampering on every crash gets its alerts ignored. It is reported, + // because a record was genuinely lost, and it does not condemn the complete ones. + // Real hashes here, because the point is a chain that verifies apart from the tear. + const d = tmp(); + const audit = join(d, "audit.jsonl"); + const lines: string[] = []; + let previousHash: string | null = null; + for (let i = 0; i < 4; i++) { + const event = chainAuditEvent( + { + id: `e${i}`, + timestamp: "2026-08-04T00:00:00.000Z", + agentId: "agent-1", + plane: "network", + action: "test", + decision: "allow", + riskLevel: "low", + matchedRules: [], + reasons: [], + requiresApproval: false, + highRiskFlow: false, + }, + { chainIndex: i, previousHash }, + ); + lines.push(JSON.stringify(event)); + previousHash = event.integrity.hash; + } + writeFileSync(audit, `${lines.join("\n")}\n${lines[3].slice(0, 40)}`); + + const chained = runVerify({ auditPath: audit }).layers.find((l) => l.name === "chained"); + expect(chained?.ok).toBe(true); + expect(chained?.problems.join("\n")).toMatch(/line 5: torn-tail/); + expect(chained?.problems.join("\n")).not.toMatch(/not valid JSON/); + }); }); diff --git a/tests/audit-file-sink.test.ts b/tests/audit-file-sink.test.ts index 82998b0..07f674c 100644 --- a/tests/audit-file-sink.test.ts +++ b/tests/audit-file-sink.test.ts @@ -8,7 +8,9 @@ import { classifyLockOwner, createFileSink, LockOwnerProbe, + verifyChainFile, } from "../src/audit/file-sink"; +import { chainAuditEvent, rehashAuditEvent } from "../src/audit/chain"; import { AuditEvent } from "../src/types"; /** @@ -212,3 +214,89 @@ describe("audit writer lock claim", () => { expect(JSON.parse(lines[1]).id).toBe("evt-2"); }); }); + +/** + * A torn tail against interior corruption. + * + * A hard kill mid-append leaves exactly one partial line and it is always the last, with no + * terminator. Calling that a broken chain sends an operator hunting a tamperer who does not + * exist, so the two are told apart on the one signal that separates them. That boundary is the + * whole distinction, and it is pinned from both sides here: move the same damaged bytes off + * the end, or give them a terminator, and they are an edit again. + */ +describe("verifyChainFile torn tail", () => { + function chainedLines(count: number): string[] { + const lines: string[] = []; + let previousHash: string | null = null; + for (let i = 0; i < count; i++) { + const event = chainAuditEvent( + { + id: `evt-${i}`, + timestamp: "2026-08-04T00:00:00.000Z", + agentId: "agent-1", + plane: "network", + action: "http.request", + decision: "allow", + riskLevel: "low", + matchedRules: [], + reasons: [], + requiresApproval: false, + highRiskFlow: false, + }, + { chainIndex: i, previousHash } + ); + lines.push(JSON.stringify(event)); + previousHash = event.integrity.hash; + } + return lines; + } + + function auditFile(contents: string): string { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "agentwall-torn-tail-")); + tempDirs.push(dir); + const auditPath = path.join(dir, "audit.jsonl"); + fs.writeFileSync(auditPath, contents); + return auditPath; + } + + it("names a partial final line a torn tail and leaves the chain standing", () => { + const lines = chainedLines(3); + const file = auditFile(`${lines.join("\n")}\n${lines[2].slice(0, 64)}`); + + const result = verifyChainFile(file, rehashAuditEvent); + expect(result.problems).toEqual([]); + expect(result.ok).toBe(true); + expect(result.notes.join("\n")).toMatch(/line 4: torn-tail/); + }); + + it("fails a truncated line in the middle of a file as corruption", () => { + // The same damaged bytes, moved off the end. No interrupted append can put a partial + // line in front of complete ones, so this is an edit and stays fatal. + const lines = chainedLines(3); + const file = auditFile(`${lines[0]}\n${lines[1].slice(0, 64)}\n${lines[2]}\n`); + + const result = verifyChainFile(file, rehashAuditEvent); + expect(result.ok).toBe(false); + expect(result.notes).toEqual([]); + expect(result.problems.join("\n")).toMatch(/line 2: not valid JSON/); + }); + + it("fails a partial last line that still carries its terminator", () => { + // A truncation followed by a newline was not left by a kill part way through the write: + // the terminator is proof that something wrote after the record was cut short. + const lines = chainedLines(3); + const file = auditFile(`${lines.join("\n")}\n${lines[2].slice(0, 64)}\n`); + + const result = verifyChainFile(file, rehashAuditEvent); + expect(result.ok).toBe(false); + expect(result.notes).toEqual([]); + expect(result.problems.join("\n")).toMatch(/line 4: not valid JSON/); + }); + + it("keeps an intact file free of both problems and notes", () => { + const file = auditFile(`${chainedLines(3).join("\n")}\n`); + + const result = verifyChainFile(file, rehashAuditEvent); + expect(result).toMatchObject({ ok: true, records: 3, problems: [], notes: [] }); + }); +}); diff --git a/verifier/README.md b/verifier/README.md index f17674c..f7b8119 100644 --- a/verifier/README.md +++ b/verifier/README.md @@ -140,28 +140,24 @@ it checks. From the repository root: cd verifier && go build -o agentwall-verify . && cd .. node scripts/conformance.js - 26 cases, typescript and go: 25 agreed, 1 declared divergence(s), 0 failure(s) + 26 cases, typescript and go: 26 agreed, 0 declared divergence(s), 0 failure(s) `go test ./...` in this directory runs the unit tests plus a corpus walk that asserts every case's `expected.json` against this verifier alone. -## Where the two verifiers disagree today +## Where the two verifiers stand -One corpus case gets different verdicts from the two verifiers. Both reject the file, and the bundled -verifier blames the chain instead of naming the torn tail: +Both verifiers return the same verdict on every case in the corpus, and the harness declares no +divergences. That is agreement about the 26 cases the corpus contains, not a proof that the two +implementations are equivalent: a forgery nobody has written a case for has been put to neither of +them. The harness fails the run if they ever stop agreeing on a case it does contain. -| Case | The edit | Bundled TypeScript verifier | This verifier | -| --- | --- | --- | --- | -| `b11-torn-tail` | a partial final line, as a hard kill leaves behind | `chained` FAIL, exit 1. It condemns the whole chain over one partial write | `torn-tail` reported distinctly, `chained` PASS, exit 1 because nothing is anchored | - -That naming gap is a limit of the bundled verifier as it ships today. The harness declares it in -`scripts/conformance.js`, prints it on every run, and fails if it starts agreeing, so the list cannot -rot into a set of excuses. - -On the `anchored` layer the two verifiers agree case for case. The bundled one recomputes each anchor -record's digest from the embedded checkpoint, requires non-empty proof bytes behind any submission -that reached a calendar, and parses the proof against the submitted digest under the same caps as this -one, so `digest-mismatch`, `proof-missing`, and `proof-parse-error` are reported by both. +On the `anchored` layer the bundled verifier recomputes each anchor record's digest from the +embedded checkpoint, requires non-empty proof bytes behind any submission that reached a calendar, +and parses the proof against the submitted digest under the same caps as this one, so +`digest-mismatch`, `proof-missing`, and `proof-parse-error` are reported by both. On the `chained` +layer both report a partial final line as `torn-tail` rather than as a broken chain, because a hard +kill mid-append leaves exactly one and calling it tampering would cry wolf. ## Exit codes