CHECK_META is not the only taxonomy: four hardcoded copies leave 12 checks on no category page

[serge-ivo]

Rewritten 2026-08-11 against main @ d1c5314. The goal was right; two of the four original evidence bullets were not. Both are corrected below, and the strongest evidence — which the original body missed entirely — has been added: there are five hardcoded copies of the category taxonomy, and the two that drive the report UI leave 12 checks on no category page at all. The issue is re-anchored from "split the Quality bucket" to "make CHECK_META the only taxonomy and derive the copies from it", because the split is unsafe until the copies are gone.

Problem

schema/src/check-meta.ts is documented as the single source of truth for categories and weights. It is not. Four other places hardcode their own version of the taxonomy, and they disagree with the schema and with each other. The user-visible result: 12 of the 37 checks appear on no category page in the HTML report, five of which carry real weight and move the composite score. A user sees a score drop, opens every category page, and cannot find the check that caused it.

Concretely, styling is worth 1 point, frontend-health 2, env-validation 1, git-hygiene 1, memory-safety 1 — 6 of Quality's 30 points — and none of them is reachable from the report's category navigation.

Evidence

Copy 1 — schema/src/check-meta.ts (the intended source of truth)

37 checks, 7 categories, weights summing to 100. Quality is 30 points across 16 members:

error-handling, complexity, duplication, docs, react, flutter, accessibility, best-practices, env-validation, git-hygiene, memory-safety, html-quality, frontend-health, styling, container-health, cloudflare-workers.

That is the original issue's core complaint and it is accurate: a low Quality score can mean UI accessibility, repo hygiene, resource lifecycle, docs, container readiness, or a Workers config problem. No single owner can act on it.

Copy 2 — cli/src/report/html.ts:32-50 (drives the HTML report pages and sidebar)

export const GROUPS: { id: string; label: string; file: string; checks: string[] }[] = [
	{ id: "foundations", label: "Foundations", file: "foundations.html", checks: ["structure", "lint", "types", "type-safety", "standards"] },
	{ id: "quality", label: "Quality", file: "quality.html",
		checks: ["complexity", "duplication", "error-handling", "react", "accessibility", "docs", "best-practices"] },
	...

25 checks in 7 groups. catScores at html.ts:80-85 maps only over GROUPS, and the sidebar at html.ts:94-110 renders only catScores. Every check not named in GROUPS is unreachable from category navigation. The 12 missing from CHECK_META:

Check	Weight	Schema category	Scored?
env-validation	1	Quality	yes
git-hygiene	1	Quality	yes
memory-safety	1	Quality	yes
frontend-health	2	Quality	yes
styling	1	Quality	yes
flutter	0	Quality	advisory (gated)
html-quality	0	Quality	advisory
container-health	0	Quality	advisory (gated)
cloudflare-workers	0	Quality	advisory (gated)
sqlite-d1	0	Security	advisory (gated)
design-consistency	0	AI Analysis	Pro
file-cohesion	0	AI Analysis	Pro

Plus dead-code, the synthetic 38th row emitted by cli/src/core.ts:164-172, which is in no taxonomy at all.

html.ts also labels the group "AI Readiness" where the schema category is "LLM Readiness" — the same bucket under two names in the same product.

Copy 3 — app/src/components/ReportViewer.tsx:54-61 (drives the dashboard radar)

const categories = [
  { label: "Foundations", checks: ["structure", "lint", "types", "type-safety", "standards"] },
  { label: "Quality", checks: ["complexity", "duplication", "error-handling", "react", "accessibility", "docs", "best-practices"] },
  ...

Six categories, 20 of 37 checks, no AI Analysis group at all. This is an independent transcription of copy 2 that has already drifted from it.

Copy 4 — cli/CLAUDE.md:163-170

A weight table, correctly labelled as a convenience copy ("Do not hand-edit weights here without checking @vibecodeqa/schema's CHECK_META"). It currently matches the schema. It is a copy nonetheless and will drift the moment membership changes.

Copy 5 — cli/src/check-meta.test.ts:51

const validCats = ["Foundations", "Quality", "Testing", "Architecture", "Security", "LLM Readiness", "AI Analysis"];

An allowlist assertion. This is the one that makes any category change a five-file edit, and it is why #66 and #67 were both scoped away from introducing new categories.

Corrections to the original body

• "cli/docs/checks.md still says Quality is 26 points" — cli/docs/checks.md does not exist. cli/docs/ contains analyzer-release-validation.md, exclusion-policy.md, internal-analyzer-contract.md, language-profiles.md, out-of-process-analyzers.md, repo-discovery.md. The public check doc is vibecodeqa/docs/docs/checks.md in the vibecodeqa repo, and it already says ## Quality (30% of score) (line 121), matching the schema. It also already says "38 checks… the weights of the 37 checks that carry category metadata sum to 100" (line 6) and carries a "verified against @vibecodeqa/cli 0.54.4 on 2026-08-08" callout. There is no docs/schema weight disagreement. That acceptance criterion is already met; do not spend time on it.
• "html-quality, frontend-health, styling … do not read like the same category" — the framing implied they lack metadata. All three are in CHECK_META (schema/src/check-meta.ts:402, :414, :426), all in Quality, at weights 0, 2 and 1. The only runner in cli/src/core.ts with no CHECK_META entry is dead-code (core.ts:164), which falls through getCheckMeta's default at schema/src/check-meta.ts:505-518 to category: "Other", weight: 5. It escapes the score only because cli/src/score.ts:19 skips details.synthetic, and cli/src/core.ts:378 maps it to available-unscored. The weight-5 fallback is a live landmine: any future runner registered without metadata silently joins a phantom "Other" category worth 5 points.

What to do

Ordered cheapest-first. Steps 1–3 are worth doing on their own even if the split in step 4 never happens.

1. Derive GROUPS from CHECK_META. Replace the literal in cli/src/report/html.ts:32-50 with a function that groups Object.values(CHECK_META) by category, deriving id/file from a slug of the category name. Keep an explicit display-order array of category names so page order stays stable. This alone puts all 37 checks on a category page and removes the LLM Readiness/AI Readiness naming split. Decide what to do with dead-code — either give it a CHECK_META entry at weight 0, or render it explicitly on the page of its sourceCheck (performance).

2. Derive the app's radar the same way. app/src/components/ReportViewer.tsx:54-61 should read categories from @vibecodeqa/schema's CHECK_META, which the app can already import. Filing note: this step is in the app repo — open it as a linked app issue rather than doing it here, per the repo-ownership rule.

3. Delete copy 5, or generate it. cli/src/check-meta.test.ts:51's validCats should be derived from a single exported category list in the schema, so adding a category is one edit and not five.

4. Only then, split Quality. With the copies derived, a membership change is a single edit to CHECK_META. The original body's proposed split is a reasonable starting point but should not be adopted verbatim — see Sequencing.

5. Regenerate cli/CLAUDE.md:163-170 from the schema at the end, or replace the table with a pointer to the schema.

Alternatives considered and rejected

• Fix the copies by hand and keep them literal. Rejected: they have already drifted three ways (schema vs html.ts vs ReportViewer.tsx) with nobody noticing that a quarter of the checks fell off the report. A hand-fix restores the same failure mode.
• Split Quality first, fix rendering later. Rejected: splitting membership while GROUPS is a literal increases the number of orphaned checks. The derivation must land first.
• Make html.ts fall back to an "Other" page for unlisted checks. Rejected as a permanent answer — it hides the drift instead of removing it — but it is an acceptable one-line stopgap if step 1 slips, and it would make the current 12 orphans visible immediately.
• Move the taxonomy into a JSON data file consumed by all repos. Rejected: @vibecodeqa/schema is already the shared package and already exports CHECK_META and getCategoryWeights(). A second artifact is a sixth copy.

Sequencing — do this after #62 and #68

Both change which checks exist and which category owns a rule, so splitting categories before they land means doing the split twice:

• Decompose standards runner into scoped owners instead of one catch-all check #62 decomposes standards into scoped owners — it explicitly reassigns dangerous-API rules to security/react, file-size rules to complexity/file-cohesion, and TODO rules to comment-staleness. Foundations' membership and standards' 3 points both move.
• Define ownership boundaries across frontend, accessibility, HTML, styling, and design checks #68 defines ownership across react/accessibility/html-quality/frontend-health/styling/design-consistency — five of those six are current Quality members and four are among the 12 orphans above. Define ownership boundaries across frontend, accessibility, HTML, styling, and design checks #68 effectively decides the composition of any "Frontend UX" bucket this issue would create.

Steps 1–3 (derivation) have no dependency on either and can ship now. Step 4 (the split) should wait.

Acceptance criteria

• Every check in CHECK_META appears on exactly one category page in the generated HTML report; a fixture asserts Object.keys(CHECK_META) is fully covered by the derived groups, so a future runner cannot fall off silently.
• dead-code has a defined home — either a CHECK_META entry or an explicit render rule — and getCheckMeta's "Other"/weight 5 fallback is never reached in a real scan. Add a test that asserts it.
• Category labels are identical in schema, HTML report and app radar (no LLM Readiness / AI Readiness split).
• cli/src/check-meta.test.ts no longer hardcodes category names.
• cli/src/check-meta.test.ts:57 (weights sum to 100) still passes after any split.
• Skipped, not-applicable, advisory and Pro checks remain visually distinct from scored ones — #65 and #52 delivered that contract; a derived GROUPS must not regress it.
• cli/CLAUDE.md's table matches the schema after the change.
• A migration note records the category rename mapping so historical reports in .vibe-check/history/ can still be read by cli/src/history.ts and cli/src/trend.ts.

Downstream — the public check count

All three of #61, #66 and #67 touch the number of checks VCQA claims to run, and that number is currently wrong nearly everywhere. This issue is the home for fixing it; #66 and #67 link here rather than duplicating.

Verified counts on main:

• 38 — runner registry (cli/src/core.ts, allRunners), i.e. what a scan actually emits.
• 37 — CHECK_META entries (schema/src/check-meta.ts). The 38th is the synthetic dead-code; cli/src/cli.test.ts:9 encodes exactly this: const SYNTHETIC_CHECK_COUNT = 1;
• 34 — the number still published to users.

Already correct, needs no change:

• vibecodeqa/docs/docs/checks.md — says 38/37 with a full inventory table, verified 2026-08-08.
• cli/README.md:15 — deliberately quotes no number ("canonical checks across 7 categories").

Still says 34 (31 occurrences across 11 files in the vibecodeqa repo, plus 3 elsewhere):

• vibecodeqa/index.html (meta description, og:description, twitter:description, body copy)
• vibecodeqa/scan.html, vibecodeqa/skills.html, vibecodeqa/tools.html, vibecodeqa/stacks/flutter.html
• vibecodeqa/compare/index.html, compare/vs-eslint.html, compare/vs-snyk.html, compare/vs-sonarqube.html, compare/vs-codeclimate.html
• vibecodeqa/docs/zensical.toml:20 (site_description) and :51 (nav entry "The 34 checks" = "checks.md")
• app/src/App.tsx:122 — "34 checks across 7 categories."
• mcp/README.md:58 — "Full scan — all 34 checks"
• ops/ARCHITECTURE.md:140 — "34 checks across 7 categories"

Recommended fix, and the reason this belongs to the taxonomy issue rather than a copy-edit pass: stop quoting a number in prose that a human maintains. Follow cli/README.md's lead — say "canonical checks across N categories" in marketing copy, and let vibecodeqa/docs/docs/checks.md be the one page that carries the exact figure, ideally generated from CHECK_META at build time. A one-off find-and-replace of 34 → 38 will be wrong again the week #66 or #67 lands.

Filing note: the marketing HTML, zensical.toml, app/src/App.tsx, mcp/README.md and ops/ARCHITECTURE.md are in four other repos. Track them as linked issues in vibecodeqa, app, mcp and ops; this issue owns the CLI/schema side and the decision about whether a number is quoted at all.

Constraints the implementer should know

• Schema is upstream. cli/src/check-meta.ts is a re-export shim over @vibecodeqa/schema. A schema change needs a schema release before the CLI can consume it, and app imports the same package.
• Trunk-based (cli/CLAUDE.md:21-45): commit to main, no PR.
• Repo ownership: presentation and views are app; the taxonomy and its derivation are cli/schema. Step 2 must be filed as an app issue.
• #73 (closed) still governs: do not close until published to npm and validated through the published CLI.

Open question for the maintainer

What is the final category set? The original body proposed eight (Foundations, Maintainability, Frontend UX, Repo & Delivery, Testing, Security, Architecture, LLM Readiness). That is a product decision with a real cost — every category is a page in the report, a spoke on the radar, and a historical-comparison break. Steps 1–3 do not require answering it. Step 4 does, and #68's outcome should inform the Frontend UX boundary.

Related

• Decompose standards runner into scoped owners instead of one catch-all check #62 — decomposes standards; changes membership. Sequence before step 4.
• Define ownership boundaries across frontend, accessibility, HTML, styling, and design checks #68 — frontend ownership boundaries; changes membership. Sequence before step 4.
• Duplicate migration numbers never affect a score: the rules are gated to SQLite/D1 at weight 0 #66, fetch() without a timeout is undetected — including the analyzer's own 10 un-timed call sites #67 — each may add a check; both were scoped to avoid adding a category because of the five copies documented here.
• Unavailable premium and non-applicable checks should be skipped, not scored F/0 #52 (closed) — unavailable/Pro checks rendering as F/0.
• Define scoring policy for stack-gated checks such as Workers, D1, Flutter, and containers #65 (closed) — the scoreMode contract a derived GROUPS must preserve.
• Extract default exclusion policy into a maintained shared registry #49, Add default generated/temp/agent artifact exclusions across all analyzers #44 (closed) — generated/temp artifact exclusions.
• Discovery evidence UI and deterministic scan explanation #60 (closed) — discovery evidence UI.