Skip to content

Test breadth: count a file as tested only when a same-named test imports it - #43

Merged
joshsiaw merged 4 commits into
mainfrom
cato/c6-breadth-graph-pairing
Oct 13, 2026
Merged

joshsiaw merged 4 commits into
mainfrom
cato/c6-breadth-graph-pairing

Conversation

@joshsiaw

Copy link
Copy Markdown
Contributor

What changes

Test breadth (C6) used to count a source file as tested when any test with the same basename existed anywhere in the repo. One page.test.tsx marked every page.tsx as tested.

For JavaScript and TypeScript, a file now counts as tested only when a same-named test imports it, read from the code graph (graphify, already pinned for C1). The C6 scan builds the graph itself when it needs it.

When it falls back

The graph does not resolve every repo's test imports. Vue single-file components, aliases defined only in bundler config and dynamic imports are the known gaps. So each stack checks itself. Of the files that have a same-named test, if the graph sees the import for fewer than half, the stack falls back to name pairing. The report gives the share and the reason.

Every other stack (Elixir, Ruby, Python) keeps name pairing.

What does not change

  • The scoring formula and anchors.
  • Discipline, the CI-gate half of C6.
  • C1. Its link rules moved into a shared classifyLink, and C1 reports are byte-identical before and after on the repos tested.
  • Any system on name pairing scores exactly as before. Its audit gains one line when a stack fell back.

Expected effect

On JS/TS repos where the graph resolves well, breadth drops by a few points, because files credited by name alone stop counting.

Checks

  • Unit tests for every new piece. Each load-bearing one was confirmed to fail when its rule was deliberately broken.
  • Old and new engine run side by side on real JS/TS and Elixir repos.
  • The criterion page is updated in the same change.

Cost: one extra graphify run per scan for repos with JS/TS tests (about a minute on a large monorepo).

🤖 Generated with Claude Code

joshsiaw and others added 4 commits October 13, 2026 14:21
Test breadth paired a source file with any same-named test anywhere in the
repo, so a test for one page.tsx marked every page.tsx as tested. For
JavaScript and TypeScript the scan now builds the code graph and requires the
same-named test to import the file.

The graph does not resolve every repo's test imports (Vue single-file
components, aliases defined only in bundler config, dynamic imports), so each
stack checks itself: if the graph sees the import for fewer than half of its
same-named test pairs, it falls back to name pairing and the report says so.
Other stacks keep name pairing. A system on name pairing scores exactly as
before.

C1's link rules move into classifyLink so the module graph and the file
graph share one definition. The criterion page is updated to match.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…llbacks

Review fixes: test pairing reads .vue components and does not apply the stdlib
name filter to a test's imports; the graph is only built for stacks that count;
the threshold compares the unrounded rate; a fallback is named in the audit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@joshsiaw
joshsiaw added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit 0cf5aa5 Oct 13, 2026
5 checks passed
@joshsiaw
joshsiaw deleted the cato/c6-breadth-graph-pairing branch October 13, 2026 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants