Repository navigation
test: mutation-testing pilot for server/wire-parsers (Stryker + tap-runner) - #710
Conversation
…unner) Adds an opt-in `npm run test:mutation` (not in npm test or CI) scoped to server/wire-parsers/**. The suite is node:test, so the tap runner is used instead of vitest; each file runs under test/helpers/home-guard.js. Triage of surviving mutants added two edge-case test files; score 64.43% -> 97.19% (anthropic 69.66 -> 98.60, openai 61.98 -> 96.58). The 34 remaining survivors are equivalent and listed in docs/mutation-testing.md. No production code changes. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 82739a694c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "@stryker-mutator/core": "^10.0.0", | ||
| "@stryker-mutator/tap-runner": "^10.0.0", |
There was a problem hiding this comment.
The new npm run test:mutation command installs Stryker 10, but the lockfile declares Node >=22.0.0 for both @stryker-mutator/core and the TAP runner, while this project advertises Node >=18 and its CI matrix includes Node 20. Stryker’s v10 release notes explicitly state that Node 20 support was dropped, so contributors using a supported Node 18/20 environment cannot reliably use the advertised pilot, and engine-strict installs will reject it. Pin a Node-20-compatible Stryker release or clearly separate and enforce a Node 22 requirement for this command.
Useful? React with 👍 / 👎.
…ode prerequisite for Stryker pilot Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…ends on #710) (#714) * test: mutation-testing pilot for server/wire-parsers (Stryker + tap-runner) Adds an opt-in `npm run test:mutation` (not in npm test or CI) scoped to server/wire-parsers/**. The suite is node:test, so the tap runner is used instead of vitest; each file runs under test/helpers/home-guard.js. Triage of surviving mutants added two edge-case test files; score 64.43% -> 97.19% (anthropic 69.66 -> 98.60, openai 61.98 -> 96.58). The 34 remaining survivors are equivalent and listed in docs/mutation-testing.md. No production code changes. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: extend mutation testing to pricing.js and default-rates.js Adds server/pricing.js and server/default-rates.js to the Stryker scope with their unit test files. Every test process gets a nonexistent CCXRAY_PRICING_CACHE and a no-network preload, so a mutant that breaks the cache/TTL checks cannot download LiteLLM or touch pricing-cache.json. New edge tests cover ratesFromLiteLLMEntry, fetchPricing's cache and (stubbed) network paths, the synchronous context-table load, prefix matching, cache-file validation and the 1h cache-creation rate. pricing.js 41.38% -> 83.62%, default-rates.js 65.57% -> 88.93%; remaining survivors are listed as equivalent/unreachable in docs/mutation-testing.md. Co-Authored-By: Claude Opus 5.5 <[email protected]> * test: assert socket forwarding in anthropic detectSession; document Node prerequisite for Stryker pilot Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * test: guard net.Socket#connect in no-network preload; fix mutation doc file count Co-Authored-By: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
A mutation-testing pilot (Stryker) scoped to
server/wire-parsers/**. The changes are tests, config and docs only, with no production code. The newnpm run test:mutationscript is not part ofnpm testand not in CI.Setup
@stryker-mutator/coreand@stryker-mutator/tap-runner(^10). The lockfile only gains new entries; no existing package version changes.node:test, which the vitest runner can't execute. Each test file runs asnode --require ./test/helpers/home-guard.js --test-reporter=tap <file>, so the HOME guard is kept.coverageAnalysis: perTestlimits each mutant to the test files that cover it.stryker.config.mjs:mutate: server/wire-parsers/**, 12 related test files listed explicitly,concurrency: 2, incremental cache atreports/stryker-incremental.json.reports/and.stryker-tmp/are git-ignored.docs/mutation-testing.md(with a pointer fromdocs/testing.md).Scores (1,209 mutants)
anthropic.jsopenai.jsindex.jsTime and memory (8 cores / 15 GiB,
nice, concurrency 2)Triage
test/wire-parsers-{anthropic,openai}-edges.test.js, all with synthetic inputs. They cover behaviour no test was asserting:registerPromptVersionrulesbuildEntryFields// Stryker disablecomments: 0. They would have required production-file edits.Verification
stryker run --force: same scores, 71 s.test/launch-decision.test.jsfails ifCCXRAY_FAIL_CLOSEDis inherited from the environment. That happens on main too.Recommendation on expanding (each as its own follow-up)
server/pricing.js+default-rates.js: next. Highest value, since they produce cost figures. Expect about 1,000 mutants, 2–3 min and about 0.6 GB. Tests should inject rates (setRateTable) or use a tempCCXRAY_PRICING_CACHEso results don't depend on the on-disk cache.server/config*: high value (context-window maths). Expect about 900 mutants and 3–5 min. Use unit tests only.CCXRAY_*variables.