Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,26 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht

## [Unreleased]

### Added
- An OpenTimestamps proof parser, `src/audit/ots-proof.ts`, implementing the proof grammar of
`docs/audit-format.md`. It is bounded against hostile input with caps on argument size, total
size, fork depth, operation count, and working message length, because the file being verified is
attacker-influenced by definition. RIPEMD-160 and Keccak-256 operations are declined rather than
evaluated, matching the independent Go verifier.

### Changed
- The bundled verifier's `anchored` layer judges an anchor record by the evidence instead of by what
the record says about itself. It recomputes each record's digest from the checkpoint the record
embeds and reports `digest-mismatch` when they differ, requires non-empty proof bytes behind any
submission that reached a calendar and reports `proof-missing` when they are absent, and parses the
proof against the submitted digest, reporting `proof-parse-error` when the container does not
parse. An anchor claiming `confirmed` with no proof file behind it fails the layer. A submission
recorded with an `error` is exempt: it never reached a calendar, so it has no proof to point at and
is already counted as failed.
- `agentwall anchor` parses a calendar response before keeping it, and treats one that does not parse
as a submission failure rather than writing it as a proof. A broken or hostile answer therefore
leaves a recorded gap instead of a file that a later verify reports as corrupt evidence.

## [0.2.0] - 2026-08-05

The first tagged release. It freezes the on-disk evidence format and makes that format
Expand Down
41 changes: 16 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -354,31 +354,22 @@ node scripts/conformance.js
The run prints one line per case. Its tail:

```
DIVERGENCE b9-anchor-digest-altered
format expects exit=1 chained=true linked=true anchored=false
typescript returns exit=0 chained=true linked=true anchored=true
it reports the digest the record claims was submitted and never recomputes one from the checkpoint the record embeds
DIVERGENCE b10-proof-truncated
format expects exit=1 chained=true linked=true anchored=false
typescript returns exit=0 chained=true linked=true anchored=true
it never opens a proof file, so a proof that cannot be parsed still counts as an anchor
ok b9-anchor-digest-altered exit=1 chained=true linked=true anchored=false
ok b10-proof-truncated exit=1 chained=true linked=true anchored=false
DIVERGENCE b11-torn-tail
format expects exit=1 chained=true linked=true anchored=false
typescript returns exit=1 chained=false linked=true anchored=false
it reports a partial final line as a broken chain rather than as the torn tail a hard kill leaves behind
ok b12-duplicate-key-shadowed exit=1 chained=false linked=true anchored=false
DIVERGENCE b13-confirmed-without-proof
format expects exit=1 chained=true linked=true anchored=false
typescript returns exit=0 chained=true linked=true anchored=true
it counts the status field, so an anchor claiming confirmation passes with no proof bytes behind it
ok b13-confirmed-without-proof exit=1 chained=true linked=true anchored=false
ok b14-submission-never-reached-calendar exit=1 chained=true linked=true anchored=false
ok b15-sealed-segment-rewritten exit=1 chained=true linked=false anchored=true
ok b16-live-tail-rewritten-after-checkpoint exit=1 chained=true linked=true anchored=false
ok b17-sealed-segment-missing exit=1 chained=true linked=false anchored=true
ok l1-confirmed-with-pending-proof exit=0 chained=true linked=true anchored=true
ok l2-legacy-canon-unmarked exit=1 chained=false linked=true anchored=false

26 cases, typescript and go: 22 agreed, 4 declared divergence(s), 0 failure(s)
26 cases, typescript and go: 25 agreed, 1 declared divergence(s), 0 failure(s)
```

Each case is copied to a temp directory before it runs, so a verifier cannot alter what it checks.
Expand All @@ -392,23 +383,23 @@ That prints nothing, because the regenerated tree is byte identical to the commi

### Where the two verifiers disagree today

Four corpus cases get different verdicts from the two verifiers. In three of them the bundled
TypeScript verifier accepts evidence the format rejects, which makes the Go verifier the stricter of
the two today. In the fourth both reject the file, and the bundled verifier blames the chain instead
of naming the torn tail:
One corpus case gets different verdicts from the two verifiers. Both reject the file, and the
bundled verifier blames the chain instead of naming the torn tail:

| Case | The edit | Bundled TypeScript verifier | Go verifier |
| --- | --- | --- | --- |
| `b9-anchor-digest-altered` | the anchor record's `digest` field altered | `anchored` PASS, exit 0. It reports the digest the record claims was submitted and never recomputes one from the checkpoint the record embeds | `digest-mismatch`, `anchored` FAIL, exit 1 |
| `b10-proof-truncated` | the OTS proof truncated inside a length prefix | `anchored` PASS, exit 0. It never opens a proof file, so a proof that cannot be parsed still counts as an anchor | `proof-parse-error`, `anchored` FAIL, exit 1 |
| `b11-torn-tail` | a partial final line, as a hard kill leaves behind | `chained` FAIL, exit 1. It condemns the whole chain over one partial write | `torn-tail` reported distinctly, `chained` PASS, exit 1 because nothing is anchored |
| `b13-confirmed-without-proof` | an anchor claiming `confirmed` with its proof file deleted | `anchored` PASS, exit 0. It counts the status field, so a claim of confirmation passes with no proof bytes behind it | `proof-missing`, `anchored` FAIL, exit 1 |

The three acceptance gaps are limits of the bundled verifier as it ships today. The harness prints
every entry in this list on each run and fails if one of them starts agreeing
([`scripts/conformance.js:40-66`](scripts/conformance.js)), so the list cannot rot into a set of
excuses, and it is why the summary line above reports four declared divergences instead of agreement
on every case.
That naming gap is a limit of the bundled verifier as it ships today. The harness prints every entry
in this list on each run and fails if one of them starts agreeing
([`scripts/conformance.js:40-51`](scripts/conformance.js)), so the list cannot rot into a set of
excuses, and it is why the summary line above reports one declared divergence instead of agreement on
every case.

The bundled verifier recomputes each anchor record's digest from the checkpoint the record embeds,
requires a non-empty proof file behind any submission that reached a calendar, and parses that proof
against the submitted digest. So an altered digest, a deleted proof, and a truncated proof all fail
the `anchored` layer in both verifiers rather than in one.

### What verification does not prove

Expand Down
3 changes: 2 additions & 1 deletion docs/audit-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -810,7 +810,8 @@ The bundled implementation of this format is in
[`src/audit/file-sink.ts`](../src/audit/file-sink.ts) for the writer and the per-file chain
walk, [`src/audit/rotation.ts`](../src/audit/rotation.ts) for the manifest,
[`src/audit/signing.ts`](../src/audit/signing.ts) for checkpoints,
[`src/audit/anchor.ts`](../src/audit/anchor.ts) for anchor records and proof persistence, and
[`src/audit/anchor.ts`](../src/audit/anchor.ts) for anchor records and proof persistence,
[`src/audit/ots-proof.ts`](../src/audit/ots-proof.ts) for the proof grammar above, and
[`src/audit/anchor-service.ts`](../src/audit/anchor-service.ts) for the three-layer verify.

Where that code and this document disagree, this document is correct and the code has a bug.
15 changes: 0 additions & 15 deletions scripts/conformance.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,26 +43,11 @@ const SKIP_GO = process.env.CONFORMANCE_SKIP_GO === "1";
* the bundled verifier, not opinions about the corpus.
*/
const DIVERGENCES = {
"b9-anchor-digest-altered": {
exit: 0,
layers: { anchored: true },
why: "it reports the digest the record claims was submitted and never recomputes one from the checkpoint the record embeds",
},
"b10-proof-truncated": {
exit: 0,
layers: { anchored: true },
why: "it never opens a proof file, so a proof that cannot be parsed still counts as an anchor",
},
"b11-torn-tail": {
exit: 1,
layers: { chained: false },
why: "it reports a partial final line as a broken chain rather than as the torn tail a hard kill leaves behind",
},
"b13-confirmed-without-proof": {
exit: 0,
layers: { anchored: true },
why: "it counts the status field, so an anchor claiming confirmation passes with no proof bytes behind it",
},
};

function fail(message) {
Expand Down
93 changes: 89 additions & 4 deletions src/audit/anchor-service.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import { createHash } from "crypto";
import { appendFileSync, existsSync, mkdirSync, readFileSync } from "fs";
import { dirname, join, resolve } from "path";
import { anchorToOpenTimestamps, fetchPoster, type AnchorRecord } from "./anchor";
import { appendFileSync, existsSync, mkdirSync, readFileSync, statSync } from "fs";
import { basename, dirname, isAbsolute, join, resolve } from "path";
import { anchorDigest, anchorToOpenTimestamps, fetchPoster, type AnchorRecord } from "./anchor";
import { verifyChainFile } from "./file-sink";
import { chainAuditEvent, findDuplicateKey } from "./chain";
import { OtsParseError, parseOtsProofFile } from "./ots-proof";
import { loadOrCreateKeys, signCheckpoint, verifyCheckpoint, type Checkpoint } from "./signing";
import {
adoptExistingSegments,
Expand Down Expand Up @@ -269,6 +270,26 @@ function countIndexReuse(path: string): { distinct: number; worst: number } | nu
}
}

/**
* Find the proof file an anchor record names.
*
* `proofPath` holds whatever the producer wrote, relative to a working directory this
* verifier neither knows nor needs to share, because an evidence directory gets copied
* between hosts. So a short fixed candidate list is tried and the first file that exists
* wins. Deriving the name from the digest instead is refused: naming a proof after its
* digest is a writer convention rather than a rule of the format, and the recorded path
* is the only thing that finds a proof named any other way. Nothing here writes.
*/
function resolveProofPath(proofPath: string, r: ResolvedPaths): string | null {
const candidates = isAbsolute(proofPath) ? [proofPath] : [];
candidates.push(
join(r.proofDir, proofPath),
join(r.proofDir, basename(proofPath)),
join(dirname(r.anchorLogPath), proofPath),
);
return candidates.find((c) => existsSync(c)) ?? null;
}

export interface LayerVerdict {
name: string;
ok: boolean;
Expand Down Expand Up @@ -402,6 +423,8 @@ export function runVerify(paths: AnchorPaths): VerifyReport {
let pending = 0;
let confirmed = 0;
let failed = 0;
let calendarAttestations = 0;
let bitcoinAttestations = 0;
if (existsSync(r.anchorLogPath) && existsSync(r.keyPath)) {
const manifestEntries = readManifest(r.manifestPath);
// One composite set per distinct sealed-segment count. An anchor log holds many
Expand Down Expand Up @@ -434,6 +457,19 @@ export function runVerify(paths: AnchorPaths): VerifyReport {
if (rec.checkpoint) {
const v = verifyCheckpoint(rec.checkpoint, keys.publicKey.export({ type: "spki", format: "der" }).toString("base64"));
if (!v.ok) anchorProblems.push(`checkpoint ${rec.checkpoint.chainIndex}: ${v.problem}`);
// Recompute the digest from the checkpoint the record carries, rather than
// reporting the one the record states. Taken on trust, `digest` lets a forger
// point a record at a checkpoint its proof never covered: the proof still
// parses, because nothing tied the two together. Recomputing is what makes an
// off-box timestamp attest to THIS checkpoint.
const submitted = anchorDigest(rec.checkpoint);
if (rec.digest !== submitted) {
anchorProblems.push(
`checkpoint ${rec.checkpoint.chainIndex}: digest-mismatch, the record says it submitted ` +
`${String(rec.digest).slice(0, 16)} and the checkpoint it embeds hashes to ` +
`${submitted.slice(0, 16)}, so the proof does not attest to this checkpoint`,
);
}
const composites = compositesFor(rec.checkpoint.chainIndex);
if (!composites) {
anchorProblems.push(
Expand All @@ -448,6 +484,11 @@ export function runVerify(paths: AnchorPaths): VerifyReport {
"no longer describes the evidence on disk",
);
}
} else {
anchorProblems.push(
"digest-mismatch, an anchor record embeds no checkpoint, so there is nothing to " +
"recompute its digest from and nothing for its proof to be about",
);
}
// A total submission failure is recorded as status "pending" WITH an error,
// because the record is written either way. Counting that as pending would
Expand All @@ -456,6 +497,48 @@ export function runVerify(paths: AnchorPaths): VerifyReport {
if (rec.error) failed++;
else if (rec.status === "confirmed") confirmed++;
else if (rec.status === "pending") pending++;

// `status` is what the record says about itself, and a record is exactly as
// trustworthy as the host that wrote it. The calendar's response IS the proof, so
// an anchor that reached a calendar has proof bytes behind it and one that did not
// carries `error` and is already counted failed. Without this, "confirmed" with an
// empty proof directory verifies, which is the overclaim this layer exists to
// refuse: it would report Bitcoin-grade evidence for a line of JSON.
if (!rec.error) {
const named = typeof rec.proofPath === "string" ? rec.proofPath : "";
const found = named ? resolveProofPath(named, r) : null;
if (!found || statSync(found).size === 0) {
anchorProblems.push(
`anchor ${rec.chainIndex}: proof-missing, it records status "${rec.status}" and ` +
(named
? `the proof it names (${basename(named)}) is absent or empty`
: "names no proof file") +
", so no off-box bytes stand behind the claim",
);
continue;
}
// Parse it. Unopened, a proof is a file name: truncate the bytes and the anchor
// still counts, which reduces the whole layer to trusting that an HTTP request
// once happened.
let parseProblem: string | null = null;
try {
const attestations = parseOtsProofFile(found, Buffer.from(String(rec.digest), "hex"));
if (attestations.length === 0) parseProblem = "it parses but reaches no attestation";
for (const a of attestations) {
if (a.kind === "pending") calendarAttestations++;
else bitcoinAttestations++;
}
} catch (err) {
parseProblem =
err instanceof OtsParseError ? err.message : `unreadable: ${(err as Error).message}`;
}
if (parseProblem) {
anchorProblems.push(
`anchor ${rec.chainIndex}: proof-parse-error, ${basename(found)} ${parseProblem}, ` +
"so the bytes on disk are not the timestamp the record claims",
);
}
}
}
}
const attempted = confirmed + pending + failed;
Expand All @@ -466,7 +549,9 @@ export function runVerify(paths: AnchorPaths): VerifyReport {
attempted === 0
? "nothing anchored off-box yet"
: `${confirmed} confirmed, ${pending} pending a Bitcoin block` +
(failed ? `, ${failed} FAILED to reach a calendar` : ""),
(failed ? `, ${failed} FAILED to reach a calendar` : "") +
`; proofs carry ${calendarAttestations} calendar and ${bitcoinAttestations} ` +
"bitcoin attestation(s), neither kind confirmation on its own",
problems: anchorProblems,
});

Expand Down
16 changes: 16 additions & 0 deletions src/audit/anchor.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { createHash } from "crypto";
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
import { join } from "path";
import { parseOtsProof } from "./ots-proof";
import type { Checkpoint } from "./signing";

/**
Expand Down Expand Up @@ -155,6 +156,21 @@ export async function anchorToOpenTimestamps(
failures.push(`${cal}: empty proof body`);
continue;
}
// Parse the body before it is kept. A response that does not lead from the
// digest we submitted to an attestation is not a timestamp, whether the
// calendar is broken or an on-path attacker answered for it. Writing it anyway
// would put bytes on disk that a later verify reports as a proof parse error,
// so the operator would be told their evidence is corrupt when in truth it was
// never a proof. Try the next calendar instead.
try {
if (parseOtsProof(res.body, Buffer.from(digest, "hex")).length === 0) {
failures.push(`${cal}: proof reaches no attestation`);
continue;
}
} catch (err) {
failures.push(`${cal}: unparseable proof, ${(err as Error).message}`);
continue;
}
let proofPath: string | undefined;
if (proofDir) {
mkdirSync(proofDir, { recursive: true, mode: 0o700 });
Expand Down
Loading