ci: [measurement] combined end-state of the nightly-deferral set (#2877+#2878+#2880)

[b7r6]

What and why

This is a measurement draft. Please don't merge it.

It combines the CI-deferral PRs on one branch, so that a single CI run shows what a PR's wall clock would be with all of them in place:

• ci: defer cargo, integration, and coverage lanes to nightly #2877: Cargo, Cargo Dev, integration and Coverage move to nightly.
• ci: run Bazel Native nightly and on merge, not on every PR #2880: Bazel Native moves to nightly and merge.
• ci: skip the sanitizer lane on docs-only changes #2878: the sanitizer lane skips docs-only changes. This one has since merged.

The individual PRs are the ones to review.

How was this verified?

The source branches touch separate workflow files, so this branch is exactly their union.

Its run on 2026-10-04 finished in 14 minutes end to end, and Bazel Dev was the slowest check. For comparison, today's run on #2910, with every lane on the PR, was paced by Coverage at 41 minutes.

Risk

None. This branch exists to be measured and closed. Merging it by accident would be the same as merging the PRs it combines. I'll close it once #2877 and #2880 are settled.

AI assistance

Claude assembled this branch and drafted this description under my direction.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
nativelink	Ready	Preview	Oct 4, 2026 3:20am UTC
nativelink-aidm	Ready	Preview	Oct 4, 2026 3:20am UTC

[b7r6]

Measured end-state — corrected methodology (supersedes the earlier version of this comment, which compared a single sample against the wrong baseline population: main push runs, a different trigger and cache mode).

Method. Unit of measurement: one head-SHA's complete check-run set. Critical path = max workflow wall-clock in the set; cost = sum of job-level durations. Baseline arm: the six PR commits #2878, #2881–#2885 (same evening, same runner pool, same class of change; each ran the full current lane set). End-state arm: three runs of this branch (initial + two no-op amends). Arms were sampled interleaved on the same Saturday evening.

Critical path (min):

arm	n	samples	median	pole workflow
baseline	6	24.3, 24.7, 25.7, 32.3, 32.9, 40.2	29.0	Nix — all 6
end-state	3	13.7, 14.2, 15.8	14.2	Nix — all 3

Zero overlap: every end-state sample is below every baseline sample (exact one-sided Mann-Whitney, U=0, p = 1/84 ≈ 0.012). Median −52%; worst observed 40.2 → 15.8.

Why this is structural, not sampling luck: in every sample of both arms the critical path is the Nix workflow itself. Baseline Nix carries Cargo Dev and Integration×3 inside it; this change removes those jobs, so end-state Nix executes a strict subset of baseline Nix's jobs. Per commit, under equal cache state, end-state wall-clock is therefore pointwise ≤ baseline — the samples only estimate the size of the gap.

Runner cost (job-level sums per PR push): baseline 287, 272 (n=2, ≈280 median); end-state 83, 74, 71 (≈76 median). ≈ −200 runner-minutes per PR push (−73%), including the xlarge Coverage runner. (The earlier "80–120" figure was an underestimate.)

Caveats, stated: end-state n=3, baseline cost n=2; cache temperature uncontrolled (mitigated by interleaved same-night sampling); CodeQL is a shared high-variance lane in both arms (observed poling at 29 min on #2880 when Nix ran warm) and is unchanged by this set.

Per-PR signal retained: Bazel Dev, installation, the RBE toolchain matrix, asan, LRE, CodeQL, and all lints. (Bazel Native and the cargo lanes are the deferred set — nightly + merge-to-main + on-demand.) Review units remain #2877, #2878, #2880; this draft exists only for the numbers above and should be closed, not merged.