feat(prior-art): admit what closed systems are known to do, graded as consensus

[jtomaszewski]

The problem

prior-art requires every claim in the verdict to be opened by you. That rule is what makes the skill worth more than asking the question directly — and it quietly deletes an entire class of system from every survey.

SAP, Palantir, Oracle, Monitor, most vertical ERPs: no public source, docs that are marketing or paywalled or render only under JS. All four source-class sweeps come back empty there by construction, not because nothing is known. Under the old rules that lands as "couldn't verify" and the signal goes to zero.

The result reads as neutral while being drawn entirely from whatever happens to be on GitHub — and it leans the same way every time: toward small open projects, away from the incumbents the reader is actually choosing between. That's a bias in the survey, not caution.

It showed up live on a real run. Researching how ERPs model project cost vs. schedule, the verdict reported "SAP couldn't be verified — two attempts, empty HTML". True, and useless: that SAP PS separates the WBS (cost rollup) from the activity network (schedule) is exactly the axis the whole recommendation turned on, and every SAP consultant alive would have said so.

The fix

The old folklore grade was doing two incompatible jobs: blog-to-blog echo ("most teams do X") and professional knowledge about a closed system that thousands of practitioners would correct on sight. Splitting them:

• New strength consensus — no source reachable, but the claim is error-corrected by use.
• New provenance recall — you knew it; no page was involved at any point.

The admissibility test is not "can you cite it" (you can't — that's the premise) but would someone who runs this system daily object to this exact sentence? "SAP PS separates the WBS from the activity network" survives. "SAP added that in 4.6C" doesn't — nobody's work depends on that being right in your head.

So the line runs through grain, not confidence:

Admissible	architectural shape, the vocabulary the system uses, the trade-off it's famous for
Never	field names, counts, defaults, versions, dates, prices — and never a quote (a remembered quote is a fabricated quote)
Never	the reason. Recall gives you what a closed system does; the why is rarely published, and a reconstructed rationale is the most fluent thing an LLM produces

Why this isn't a loophole

Two containments, both structural rather than vibes:

1. Declare it before the sweep. Written first, a recall claim is a prediction — confirmed it upgrades to documented, contradicted it's a finding worth more than either, unreachable it stays consensus and the reader knows the search was tried. Written after, it's unfalsifiable and arrives in exactly the shape the verdict wanted, which is what confabulation looks like from the inside.
2. Never load-bearing alone. consensus may corroborate, contrast, or occupy the quadrant nothing else reaches. If deleting every consensus line would change the recommendation, the survey isn't finished — say that instead of leaning on them.

And they're graded in the verdict sentence, not a footnote. The failure this skill exists to prevent is laundering, not uncertainty; a reader who can see the grade can discount it.

Hard rule 1 is unchanged in substance — nothing still enters the verdict on a subagent's word, and every documented claim is still opened and quoted.

Also in this change

• An empty sweep now has three meanings, not two: nothing exists / the tool failed / the vendor doesn't publish. Conflating the third with the first is how the bias got in.
• A failed verification means different things by class. In an open codebase, not finding the feature is strong evidence it isn't there. For a vendor that publishes nothing, it tells you about their documentation, not their design.
• The grade-ranking note extends to consensus — about a system's shape it routinely beats documented about one of its fields, for the same reason the existing note gives for inferred beating documented: the shape is the part thousands of people would have noticed was wrong.

Verification

Prose-only change to one SKILL.md; no code paths, no scripts, no tests in this repo to run. Checked by hand: the diff applies cleanly on top of #48, the two new grades are referenced consistently across steps 4, 5, 6 and the hard rules, and no existing rule was weakened except rule 1, which was rewritten to say what it always meant.

Not verified: how the new grade behaves on a live run. Worth watching for the predictable failure — consensus creeping in after a sweep comes back thin, which is exactly the case the pre-declaration rule is there to catch and the one a reviewer can't see from the diff.

Draft: the one open question is whether consensus should also be called out as an exception where step 6 says folklore "may never justify a decision", or whether rewriting hard rule 1 covers it.

🤖 Generated with Claude Code