ledger-signals: witness verify-report false-positives on a legitimate historical report (issue #188)

[ruvnet]

Closes #188.

Hypothesis

Given the committed report corpus under docs/dream-cycle/, when a new governance test iterates every file and asserts any file with more than one literal ## Witness heading line is on an explicit, documented, hash-verified grandfather list (and that the list contains no stale/unreproducible entry), then the test should fail today pointing at 2026-09-10-compiler-parity-report.md's ambiguous heading, and pass once that one entry is added — without changing splitAtWitnessSection's existing fail-closed ambiguity behavior (a separate, already-passing test deliberately locks that in for a genuinely adversarial duplicate-heading case) and without editing any already-merged historical report's content. Confirmed, not amended after evaluation began.

Candidate

New file only: scripts/witness-corpus.test.mjs (+105 lines, 1 file, 0 deletions), wired automatically into the existing npm run test:governance glob. Three node:test cases:

1. No committed report has more than one ## Witness heading unless on the grandfather map.
2. The grandfather map names no file that no longer needs it (stale-entry detection).
3. Every grandfathered report's witness is still independently reproducible by hand — generic/data-driven over every map entry (re-derives each entry's hash from its own byte marker + the file's current bytes), not hardcoded to one filename, so a future entry without a matching hash fails here mechanically — mirroring @dream-machine/ledger's legacyPrefixDigest fail-closed design (added after an independent critic's review, see Reward-Hack Check).

The grandfather map has exactly one entry today: 2026-09-10-compiler-parity-report.md, predating splitAtWitnessSection/witness verify-report (2026-09-15, PR #111). Its first ## Witness occurrence (line 93) is a forward-reference placeholder ("See STEP 16 below...") written before the tool existed, not a real second section (the real one is line 118). The originally-published report_sha256 is independently, byte-level reproducible from the file's current content.

Evaluation Receipt

Real evaluator: npm test (vitest run && npm run test:governance), run live before (file absent) and after (file present).

	Baseline (b2abcc1)	Candidate
vitest	809	809 (unchanged)
governance	195	198 (+3 new, 0 regressions)
npm run lint	clean	clean
npm run typecheck	clean	clean

Non-tautology (4 live negative controls, each restored and diff-verified byte-identical afterward):

1. Emptied the grandfather map → test 1 fails, names the real offending file exactly.
2. Added a second, real, single-heading filename to the map → test 2 fails (stale-entry detection).
3. Added a fabricated entry with a wrong hash → tests 2 and 3 fail closed (the exact scenario an independent critic named — see below — now caught mechanically).
4. Corrupted the real entry's expected hash → test 3 fails with the mismatch.

Credentials reality check: OPENROUTER_API_KEY present tonight — LLM_EVAL not blocked. This candidate makes no model calls; verified entirely by the real test suite and direct cryptographic re-derivation.

Baseline

Parent commit b2abcc14a632d7e0948962065a7c6c8920cf306c (main), measured live before the candidate file existed (nothing to stash). Session's checkout started shallow; unshallowed via git fetch --unshallow first (same already-tracked class as issue #149/PR #150) before any measurement.

Darwin Lineage

DARWIN=not-applicable — a single new regression test over a fixed document corpus has no evolvable population or numeric objective, consistent with unbroken precedent on this surface (#116/#127/#132/#137/#145/#154/#167/#176). Entrypoint probed live (npx --yes @metaharness/darwin --version responds; reachable, not blocked).

Evidence

OBSERVATION (manual corpus audit finds exactly one ambiguous-heading file) → OBSERVATION (live repro) → OBSERVATION (byte-level hand reproduction proves the published hash correct — tool false-positive, not corruption) → HYPOTHESIS (frozen above) → MEASUREMENT (1004→1007 tests live before/after, 0 regressions) → MEASUREMENT (4 live negative controls) → INFERENCE (independent critic, separate subagent context) → DECISION (critic: CLEAR + 1 disclosed non-blocking gap) → DECISION (gap closed same session) → DECISION (ship; ACCEPT).

Reward-Hack Check

Independent adversarial critic (separate subagent context, no authorship stake, instructed to try to break the candidate): CLEAR, one disclosed non-blocking gap found and closed same session.

• Confirmed the diff is purely additive (git status --porcelain/git diff --stat showed only the one new file throughout); independently re-read packages/witness/src/report-witness.test.ts in full and confirmed its duplicate-heading-rejection tests are untouched/unweakened.
• Independently reproduced the grandfathered file's hash from scratch with its own throwaway script (not trusting this candidate's own assertion).
• Independently proved non-tautology by emptying the grandfather set and re-running, then restoring and diffing byte-identical.
• Gap found: first draft's grandfather list was a bare filename Set — tests 1/2 would pass for a future fabricated entry with a plausible comment but no automatic proof of authenticity. Closed same session: converted to a Map carrying each entry's own byte marker + expected hash; the reproducibility test is now generic/data-driven over every entry. Re-verified the critic's own named attack (a fabricated entry with a wrong hash) now fails tests 2 and 3 mechanically, then removed and the file restored to its real, single-entry state.
• No gold data, threshold, or evaluator config touched anywhere. No undocumented cache. No hidden cost.

Security Review

No prompt-injection, MCP-authority, or credential surface touched. No new filesystem-write surface (read-only, fixed repo-relative path via fileURLToPath(new URL(...)), no traversal surface). No network/exec surface. No regex used (no ReDoS exposure on file content — exact-string ===/indexOf/lastIndexOf only). No production/runtime code path changed — new test infrastructure only.

Regression Analysis

0 pre-existing tests modified, removed, or weakened anywhere in the repo. vitest count unchanged (809); governance count increased only by the 3 new tests (195→198). Lint/typecheck clean. No wasm/NAPI degradation (build was clean, not run again tonight since nothing in packages/ changed).

ADR

None — a new governance/regression-test addition, not an architectural decision (consistent with every prior correctness/coverage fix on this surface having no ADR).

Gist

GIST=LOCAL — no gh CLI gist capability this session (GH_TOKEN invalid) and no gist-creation MCP tool available. GitHub MCP access itself works fine (get_me, issue/PR tools all succeeded) — FALLBACK=false. Full report committed at docs/dream-cycle/2026-10-16-ledger-signals-report.md.

Issue

#188

Witness

report_sha256 : 34e07d7b29706b44ece0b62e5843a32c7bf1b4ee8848b1cb7027ccbf20a13238
session_commit: b2abcc14a632d7e0948962065a7c6c8920cf306c
witness       : 3fc2fa96d63069b3d20d36e1ce3a519c0cee9c1b24bb99e5694eadbb40291bb3

Reproduce: dream-machine witness verify-report docs/dream-cycle/2026-10-16-ledger-signals-report.md --commit b2abcc14a632d7e0948962065a7c6c8920cf306c

Merge Policy

Human review required. This session never self-merges and never autonomously promotes candidate state. dream.config.json's autoMerge: false; no automerge-safe label applied.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01WLunD924RoaiUkFET9A8hG

---

Generated by Claude Code