ci: validate every package on PRs, stop cancelling main runs, and add a global complexity ceiling

[aviggiano]

Problem

• Pull requests skipped three of the eight release lanes. package-gates (artifacts, config, topology, prompts, evals, modal, dashboard, security, references, packed install), cli, and benchmark-history-typecheck (which holds the workspace typecheck) ran only on pushes to main, so their regressions first failed after merge.
• Pushes to main cancelled each other. 24 of the last 40 main push runs were cancelled (as of 2026-09-29), so most main commits never got a complete validation.
• main is red in package-gates, and no pull request caused it. smol-toml 1.9.0, published 2026-09-22T17:15Z, breaks the config loader, and the packed-install consumer resolves dependencies without the workspace lockfile. package-gates last passed on main in run 35789754192 (2026-09-22) and has failed since run 36490897026 (2026-09-28), for example in 36495460966. Running package-gates on pull requests would not have prevented this; it would have turned every open pull request red at once, which is why this PR fixes the loader first.
• The complexity lint was both leaky and blocking (Add cyclomatic complexity lint #960). The complexity and size rules ran only in the changed-lines strict lint.
• Vitest packages used the 5 s default timeout. main went red on it in runs 34967618488 (modal ci-config and modal-documents) and 34634052856 (modal worker-result).
• CLI tests leaked their run directories. On main, the single test status surfaces a terminal product and live workflow lifecycle divergence left 446 MB and 34,091 entries in TMPDIR.
• The lanes are dominated by fsync. The analysis profiled one runtime test: 222 s of about 260 s wall time was spent in 55,998 fsyncs.
• Super-linter failed every pull request that edited CHANGELOG.md. It lints each changed Markdown file in full with MD013 at 400 characters, and CHANGELOG.md has many entries longer than that on a single line. No merged PR has edited CHANGELOG.md since super-linter arrived in chore: adopt external static-analysis gates #1005, and most of the open fix PRs fail External static analysis for this reason alone. This PR's first run hit the same 46 errors, on CHANGELOG.md and on existing 621-character table rows in agent-adapter-boundaries.md.

Root cause

• scripts/ci/release-validation-lanes.mjs filtered lanes by event. Only the runtime lanes had pull_request: true.

• Every push shared one concurrency group (github.ref) with cancel-in-progress: true. Changing only cancel-in-progress would not fix it:

  • GitHub also cancels a pending run in a group when a newer run queues.
  • main pushes arrive in bursts, for example three within 9 seconds at 22:11 on 2026-09-28.

• ESLint reports complexity, max-lines-per-function and max-statements on a function's first line (and max-lines on the first line past the limit), and the diff plugin keeps a message only when its line changed. Two consequences:

  • Adding a branch inside a function that is already over budget passes.
  • Editing only that function's signature fails, which forces an unrelated refactor onto a fix.

• Packed-install failure. smol-toml 1.9.0 builds every parsed table with Object.create(null). The config loader has two assumptions that break on such tables:

  • isPlainObject accepts only Object.prototype.
  • Object.entries(table).sort() stringifies values, which a null-prototype object cannot do.

  The packed-install consumer has no lockfile, so it resolves ^1.7.1 to 1.9.0, and ultrafuzz init fails with project must be a table; run must be a table; .... The workspace lockfile still pins 1.7.1, which is why only the packed install caught it.

• Leaked run directories. The CLI tests' tempProject() called mkdtempSync and nothing ever removed the directory. Sealed snapshot directories are mode dr-x, so a plain rm -rf cannot delete them anyway.

Change

One commit per concern:

1. fix(config): normalize TOML tables once, at the parse call. parseProjectConfigToml wraps the parse in structuredClone(parse(text)), which returns the same data as ordinary objects. The rest of the loader is unchanged. __proto__ keys stay own data properties (checked with 1.9.0 directly).
2. ci: run every lane on every event.
   • The lane table has no event filter, no PULL_REQUEST_REQUIRED_GATES and no --event argument. It keeps its test that it runs each gate scripts/validate-release.mjs defines exactly once.
   • release-validation needs only release-validation-lanes, so the lanes start without waiting for the build and super-linter jobs. release-gates still requires every job.
   • The PR-only runtime and CLI smoke runners and their package scripts are deleted, because the lanes run those suites in full.
   • The build job is now always named Build gates, and its budget drops from 30 to 15 minutes. It took 2.8-3.6 minutes on every run of this PR.
   • Runs other than pull requests get their own concurrency group (github.run_id), so only a pull request's superseded runs are cancelled.
   • The ci.yml line-by-line assertions in packages/modal/test/ci-config.test.ts (185 lines) and the timeout_minutes === 120 change detector are deleted. The remaining guard parses ci.yml and checks that neither the release-validation job nor any of its steps has an event or pull_request condition, and that release-gates requires the lanes' result without one.
3. ci: run validate:release under eatmydata. The job installs it only when the image lacks it (command -v eatmydata || { sudo apt-get update && sudo apt-get install -y eatmydata; }); the current image ships eatmydata 131-1ubuntu1, so the step does not touch apt today.
   • LD_PRELOAD reaches the test processes, which is where startRun and runCli copy and fsync the sealed snapshot (both run in-process in the runtime and CLI tests).
   • It does not reach the Smithers engine processes: smithersCommandEnv (packages/runtime/src/smithers.ts) builds their environment from an allowlist without LD_PRELOAD, so those still sync. I checked this by reading the code, not by tracing.
4. ci: report unused exports without blocking. pnpm -w knip --include exports,types,duplicates --no-exit-code reports 52 unused exports, 38 unused exported types and 4 duplicate exports on this tree and exits 0. The first version exited 1 under continue-on-error, which put a Process completed with exit code 1. failure annotation on every Build gates run; the run on this head has none. continue-on-error stays so a knip crash in this step still cannot block. The step can become blocking after the dead-code PRs land.
5. test: add --testTimeout=30000 to the vitest run scripts of config, topology, prompts, evals, evmbench and modal. A follow-up commit deletes the modal assertion scripts.test === "vitest run", which failed package-gates in this PR's first run. The property it guarded, that the unit-test script never runs the real Modal smoke, is still asserted by not.toContain("smoke").
6. test(cli): clean up per test. temporaryRoot(prefix, t) (in packages/cli/test/temporary-root.ts) registers a t.after hook. The hook restores owner write permission and removes the root and the <root>-fake-bin directory that the fake-engine fixtures create beside it. The tests in cli.test.ts, lifecycle-commands.test.ts and audit-profile-commands.test.ts pass their context in; those files hold every CLI test that launches a run. temporary-root.test.ts builds the shape a launched run leaves (read-only files in dr-x directories) plus the sibling, and asserts both are gone when the test ends.
7. build(lint): a global complexity ceiling (Add cyclomatic complexity lint #960), no baseline. pnpm -w lint now fails any function under packages/ or scripts/ above cyclomatic complexity 90. That is today's maximum once the open simplification batch lands (synchronizeLinkedWorkflowRun; the 122-point verifyCurrentCampaignTimeoutEvidence is deleted by refactor(runtime): delete host re-implementations of registry artifact gates #1178). Nothing is grandfathered or suppressed, following chore: adopt external static-analysis gates #1005's no-baseline stance and the approach Add cyclomatic complexity lint #960 links to (chore(lint): enforce cyclomatic complexity ceiling modem-dev/hunk#861). Lower the ceiling in eslint.config.js as the worst functions are simplified. Changed lines stay under the stricter diff-limited budgets (lint:strict:ci: complexity 20, size rules, type-aware strict rules). An earlier revision of this PR baselined 1,003 existing violations in eslint-suppressions.json; that was dropped because chore: adopt external static-analysis gates #1005 deliberately added no baseline or suppression registry.
8. ci: turn off MD013 only. .github/linters/.markdown-lint.yml is super-linter's pinned v8.7.0 default markdownlint template with only MD013 turned off, the same trade-off .github/linters/.yamllint.yml already makes for YAML line length. Only 4 of the 109 tracked Markdown files have lines over 400 characters (CHANGELOG.md, agent-adapter-boundaries.md, provider-harness-research.md, CODE_OF_CONDUCT.md).
9. docs: CHANGELOG entries. docs/reference/development.md and docs/reference/agent-adapter-boundaries.md are updated for the lane change, and development.md says which processes eatmydata covers.

Deliberately not built (and why)

• A tighter always-on budget. A ceiling of 90 enforces little by itself; the previous revision of this PR measured 1,003 violations of the 20/80/500 budgets. Tightening needs either a baseline (which chore: adopt external static-analysis gates #1005 ruled out) or refactoring the hotspots first, so the ceiling starts at the current maximum.

• No per-event lane policy. With every lane on every event, a PR-required gate list has nothing to configure, so it is deleted rather than set to "all".

• The lane matrix stays in release-validation-lanes.mjs. With the event filter gone the script only prints a constant, and inlining it into ci.yml would delete the selection job and the script's entry point. test: model-free end-to-end campaign with controller kill and resume on the real pinned engine #1187 edits that script now, so that is a follow-up.

• Lane timeouts stay at 120 minutes. Runner variance is 2-4x between runs, so lowering them needs more evidence than this PR's runs.

• The test: give Modal validation cases individual timeout budgets #1126 per-case vitest timeouts (15-30 s) are left alone. Some of them are in files other open PRs edit. They are now as tight as or tighter than the new default, and locally 5 modal cases timed out at exactly those budgets under load, so removing them is a reasonable follow-up.

• packages/runtime/test/temporary-root.ts keeps removing its roots at process exit. Per-test cleanup there needs a check that no runtime test shares a root across tests. Because runtime shards still accumulate run directories, the step that deletes the Android and .NET SDKs to reclaim disk stays.

• The workspace lockfile stays on smol-toml 1.7.1. A dependency bump belongs in its own PR, and the fix works with both versions.

• The snapshot changed while reading false positive is not fixed here. workflow-integrity.ts compares nlink and ctime of the source files it copies, and those change whenever another process hard-links the same pnpm-store file. It can fail a real launch on a developer machine; it is a runtime fix in files other open items own.

• No further CI restructuring. package-gates still repeats the dependency-advisories and ci-scripts gates that the build job already runs, and the network advisory check is still inside the build job. Both deserve their own change.

Verification

Each fix was checked against unmodified origin/main (b6dd1da9) and against this branch.

Change	On origin/main	On this branch
Config fix: packages/config/test/toml-null-prototype.test.ts, with smol-toml's parse mocked to return 1.9-shaped tables	Fails at import time with the same failed default config validation error as CI	Passes; the full config suite passes 95/95
Config fix: node scripts/validate-packed-install.mjs (fresh registry resolution)	Fails with packed init failed ... project must be a table	packed install validation passed for 12 packages at 0.1.0
Lanes: scripts/ci/release-validation-lanes.test.ts run against each lane script	2 pass, 1 fail	Passes
Lanes: matrix emitted for a pull request	5 lanes (runtime-supporting, runtime-1..4)	8 lanes, the same set for every event
Vitest timeout: throwaway 6 s test (not committed)	Bare vitest run: Test timed out in 5000ms	Passes under the new package script
CLI cleanup: TMPDIR left behind	One test: 446 MB, 34,091 entries	All tests of the three changed files: 0 entries
CLI cleanup: TMPDIR left by the 8-run test report bundle --require-verified fails closed on every present invalid event journal, run alone	3.5 GB	4 KB (the empty TMPDIR itself)

packages/cli/test/temporary-root.test.ts is new, so it cannot run on main. It passes on this branch and fails under both of two mutations of the helper: skipping the permission restore makes the hook throw EACCES, and skipping the sibling fails its assertion.

PR-gating guard, mutation-tested. I ran the previous and the current version of the guard in release-validation-lanes.test.ts against mutated copies of ci.yml:

Mutation of ci.yml	Previous guard	Current guard
None	Passes	Passes
if: github.event_name != 'pull_request' on the "Validate release lane" step	Passes	Fails
The same if on the release-validation job	Fails	Fails
github.event_name != 'pull_request' && added to the release-gates requirement step	Fails	Fails
The requirement step's condition wrapped in ${{ }} (equivalent)	Fails	Passes

Other checks run on the final head:

• pnpm -w lint, npx prettier --check ., CI=1 ESLINT_PLUGIN_DIFF_COMMIT=origin/main pnpm -w lint:strict:ci, pnpm -w knip and node scripts/docs-check.mjs pass.
• pnpm -w knip --include exports,types,duplicates --no-exit-code exits 0 with the report; without --no-exit-code it exits 1.
• actionlint 1.7.12 on ci.yml is clean (without shellcheck installed locally; super-linter runs it in CI). markdownlint-cli 0.49.0 with .github/linters/.markdown-lint.yml is clean on the changed Markdown files.
• bun test scripts/ci/release-validation-lanes.test.ts passes 3/3. Earlier in this PR: the modal ci-config.test.ts passes 12/12, pnpm --filter @ultrafuzz/{config,modal} typecheck and the CLI tsc -p tsconfig.test.json pass.
• pnpm -w test:ci-scripts passes 95 of 96 locally. The failure is scripts/ci/safe-archive-bun.test.ts > repeatedly extracts a synthetic archive without stalling Bun callbacks, which hits its own 15 s budget at load average 14 on this shared machine. It fails the same way on origin/main here, covers code this PR does not touch, and passes 96/96 in the Build gates job of every CI run of this PR.
• Super-linter. The first CI run failed only on MD013: 46 errors, all on existing lines of CHANGELOG.md and agent-adapter-boundaries.md. markdownlint-cli 0.49.0 reproduces those 46 errors locally with the pinned default template, and reports none with .github/linters/.markdown-lint.yml.
• Full modal suite, run locally. 5 of 677 tests timed out at their own 15 s and 30 s per-case budgets, in node-provider.test.ts and public-worker.test.ts. The suite took 716 s here at load average 33-40, against 130 s on the CI runner, where those files passed. The one file I changed, smoke.test.ts, passes 14/14.
• Locally I ran the full config and modal suites and the three changed CLI files, but not the full runtime or CLI suites, nor the other packages' suites. This PR's CI runs are the full runs.

CI effect, measured on this PR's six runs.

• Run 1 is 36503175731, which failed only the modal script-string assertion and super-linter, both fixed since.
• Runs 2-5 are 36506450057, 36510165058, 36512609711 and 36516391013. Each passed every job.
• Run 6 is 36520602838, on the final head. Its commits change only text, lint:fix, the knip and eatmydata steps, and the guard test. It passed every job; its Build gates job ran 96/96 CI script tests and carries no failure annotation, and in the shard 1 log the Install eatmydata step printed /usr/bin/eatmydata and skipped apt.

Durations are for the Validate release lane step, in minutes. The baseline is the same lane in seven recent runs: main 36495460966 and 35789754192, and PR runs 36279711018, 36491801277, 36491760245, 36498046011 and 36497862407. The CLI and package lanes have only two baseline runs, because they never ran on PRs.

Lane	Run 1	Run 2	Run 3	Run 4	Run 5	Run 6	Baseline range (median)
runtime shard 1	15.9	19.0	18.5	21.9	22.1	18.6	24.4-34.5 (28.1)
runtime shard 2	14.9	20.9	20.9	22.0	18.7	17.9	26.3-29.2 (27.1)
runtime shard 3	13.0	12.6	19.9	19.9	14.9	14.1	22.2-27.4 (26.6)
runtime shard 4	22.3	15.1	21.6	21.3	12.9	21.7	20.5-35.4 (27.7)
runtime supporting	19.4	19.2	18.9	14.8	19.0	18.6	21.5-44.7 (23.9)
CLI	36.6	37.2	25.3	36.1	36.2	36.4	34.7, 40.4
package gates	9.1	7.0	6.7	11.8	11.7	8.0	11.3, 11.6
benchmark history and typecheck	1.9	2.2	2.9	3.0	2.9	2.9	2.0, 2.6

• The runtime lanes got faster consistently.
  • Shards 1-3 came in below every baseline run in all six runs, 19-53% under their medians.
  • Shard 4 was inside its range four times and below it twice.
  • The supporting lane was 19-38% under its median, below every baseline run.
• The CLI and package lanes are mixed.
  • The CLI lane was inside its two-run baseline five times and well below it once. Its per-test sum was 9-10% below main's in runs 1 and 2 (39.6 and 40.3 min, against 44.1 on main 36495460966), and 38% below in run 3.
  • With two baseline runs, I cannot separate an eatmydata effect on those lanes from runner variance.
• PR wall time was 39.2, 39.1, 27.8, 37.7 and 37.8 min for runs 1-5, with all eight lanes. Run 6 took 58.0 min end to end, but its lane-selection job waited 19 min for a runner (created 04:12:38, started 04:31:40); from the lanes' start to the end it took 38.4 min. The four most recent successful PR runs before this PR took 36.6-38.8 min, and ran only the five runtime lanes after the 8-minute build-and-smoke prerequisite. The CLI lane is the critical path.

Local run of the three changed CLI test files (TMPDIR isolated, eatmydata, 98 min at load average 30-46):

• 98 of 101 tests and subtests passed, and TMPDIR held 0 entries afterwards.
• The 3 failures were all WORKFLOW_SUBMISSION_FAILED: workflow execution file ... changed while reading, raised while ultrafuzz run copied the sealed snapshot.
  • The check at packages/runtime/src/workflow-integrity.ts:2334 compares nlink and ctimeNs before and after the read.
  • The source files are pnpm-store hard links shared by every worktree on this machine, and concurrent pnpm installs (other agents', and one of mine) add links to them.
  • The same test passed on main and on this branch in CI, and failed a different subtest each time it was re-run here, so I don't attribute it to this change. It is a real launch-time false positive; see "Deliberately not built".

Risk / compatibility

• More runner time. Pull requests use three more lane runners, and the lanes also run when lint fails. Wall time did not grow materially: 27.8-39.2 min across this PR's first five runs (run 6 queued for runners for 19 min), against 36.6-38.8 min for recent PR runs that ran five lanes. The CLI lane is now the critical path, so the next PR-latency win is the CLI suite itself (serial --test-concurrency=1, and a full run launch per test).

• More concurrent runners. Pushes to main no longer queue behind each other, so bursts use more runners at once.

• Pull requests now depend on fresh registry resolution. The packed-install gate in package-gates installs without the lockfile. A newly published semver-compatible release that breaks the install, as smol-toml 1.9.0 did, now fails every open pull request at once instead of only main.

• Merge order with the open work-item PRs. From git merge-tree and a lint of each merged tree, heads as of 2026-09-29 04:27 UTC; the heads keep moving:

  • Besides CHANGELOG.md, these conflict textually:
    • fix(runtime): agent adapters never hang or reroute on telemetry and environment quirks #1173, refactor(evals): delete the dead reporter/telemetry pipeline and publication leftovers #1174 and fix(runtime): resume keeps agent auth config and a no-op attach no longer rewrites run state #1177 in packages/cli/test/cli.test.ts, on test headers this PR changed to async (t) / tempProject(t).
    • fix(runtime): agent adapters never hang or reroute on telemetry and environment quirks #1173 also has a modify/delete conflict on packages/runtime/scripts/run-pr-smoke-tests.mjs, which this PR deletes.
    • test: model-free end-to-end campaign with controller kill and resume on the real pinned engine #1187 conflicts in scripts/ci/release-validation-lanes.{mjs,test.ts}, packages/cli/package.json and docs/reference/development.md.
  • fix(runtime): resume keeps agent auth config and a no-op attach no longer rewrites run state #1177 adds a tempProject() call without t, which will not compile once this merges; it needs t when it rebases.
  • This PR merges after the simplification batch: on current main the maximum complexity is still 122, so its own lint only passes once refactor(runtime): delete host re-implementations of registry artifact gates #1178 has landed.

• eatmydata install. Today the step does not run apt. GitHub's notice on these jobs says ubuntu-latest moves to Ubuntu 26 from 2026-10-19; if that image lacks eatmydata, the step runs apt-get update and installs it, which depends on the Ubuntu mirrors.

• The build job's name on PRs changed from PR build and Node.js 24 runtime smoke to Build gates. main has no branch protection, and the org ruleset has no required status checks (checked through the API), so nothing keys on the old name.

• No product runtime behaviour changes, apart from the config loader accepting smol-toml 1.9 tables.

Closes #960
Refs #923, Refs #997, Refs #1000

Refs #923 rather than Closes. The 30-minute budget in #923 no longer exists: #994 split cli into its own lane at 75 minutes, and #1137 raised it to 120. The measurements above show eatmydata does not materially shorten the CLI suite, and it remains the longest lane.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no outstanding finding or new actionable issue was identified.

Findings

1.  Existing function exceeds new ceiling ▶

Fix with agent prompt

### Issue 1
eslint.config.js:undefined-16
If `verifyCurrentCampaignTimeoutEvidence` still has the 122-point complexity stated in the PR description, the new 90-point limit makes the required `pnpm -w lint` gate fail. That function is unchanged from the supplied base, so this PR cannot pass that gate against the base until the separate simplification lands or the ceiling is adjusted.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR makes every release-validation lane run on pull requests, prevents main pushes from cancelling one another, and adds a global complexity ceiling. It also normalizes parsed TOML tables, adjusts test timeouts and temporary-directory cleanup, and updates CI linting and documentation.

• Release lanes now start independently of build and static-analysis jobs, while the final gate still requires their results.
• CLI test cleanup covers sealed run directories and sibling fake-bin directories.
• The prior complexity-ceiling concern no longer applies: the function identified in that thread is absent from the current tree.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Select[Select release lanes] --> Lanes[Run every validation lane]
  Build[Build gates] --> Final[Release gates]
  Static[External static analysis] --> Final
  Select --> Final
  Lanes --> Final

Loading

_{Reviews (8) · Last reviewed commit: "Merge origin/main into claude/w21-ci-qua..."}

[greptile-apps]

Existing function exceeds new ceiling If verifyCurrentCampaignTimeoutEvidence still has the 122-point complexity stated in the PR description, the new 90-point limit makes the required pnpm -w lint gate fail. That function is unchanged from the supplied base, so this PR cannot pass that gate against the base until the separate simplification lands or the ceiling is adjusted.

Prompt To Fix With AI

This is a comment left during a code review.
Path: eslint.config.js
Line: 16

Comment:
**Existing function exceeds new ceiling** If `verifyCurrentCampaignTimeoutEvidence` still has the 122-point complexity stated in the PR description, the new 90-point limit makes the required `pnpm -w lint` gate fail. That function is unchanged from the supplied base, so this PR cannot pass that gate against the base until the separate simplification lands or the ceiling is adjusted.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!