Epic: tests give faster feedback with less redundant work #1849

Outcome

Offline CI and local tests provide the same meaningful product, security and compatibility evidence with less elapsed time and repeated work. Each guarantee is tested at the lowest sufficient level, with representative integration proof connecting detailed lower-level coverage to the real application. Environment lifetimes match test intent and safely amortize setup through sharing, private cloning or complete reset. Scenario matrices are justified by independent failure modes; theme parity is covered centrally in Storybook rather than multiplying unrelated runtime scenarios. Fast feedback, expensive final confirmation and endurance have separate execution controls; endurance can run alongside ordinary proof.

Why now

The maintainer requested a full duration catalog, then selected separating connection recovery from light/dark parity and asked for an optimization epic with scoped issues, without implementing changes yet. The offline local profile at 5cf1d548f7bd3a314eb36394dc5db0268db97da6 measured a recording-cap test at 602.65 seconds and four equivalent ten-disconnect scenarios at 92.86–94.57 seconds each. Successful exhaustive CI run 38549344893 took 27m40s across selected jobs and about 157.6 runner-minutes. Individual test-body time, setup/build time, job wall time and aggregate critical path are distinct measurements.

Current state — 2026-10-26

The implementation children from the earlier optimization pass are closed. The delivered work and its attributed receipts are recorded on each child and in the dated comments here. In summary: the recovery suite runs two responsive paths without a theme multiplier (#1851/#1905); Storybook owns light/dark parity; recording limit correctness is separated from the ten-minute endurance run (#1854/#1860/#1862); runtime packages build once per test invocation (#2458, superseding #2319); the 87 deliberate wait windows are inventoried and justified (#1855 through #2627-#2639 and #2647); browser and unit weights were refreshed (#2417, #2736). Each delivered receipt is scoped to its own source and runner; none is a controlled whole-CI benchmark.

The earlier pass still has two documentation/acceptance obligations:

1. docs: correct browser weight refresh prerequisites #2550 corrects the contributor weight-refresh recipe, which still describes two browser jobs while the browser domain has three shards (PROOF_SHARDS, scripts/proof-shards.ts:3) and the refresher requires a successful attributed run and every fresh shard (refreshBrowserCosts, scripts/refresh-browser-costs.ts:15-29).
2. ci: audit the test-cost epic's coverage and performance acceptance #2763 audits this epic's definition of done: named before/after source snapshots, controlled and uncontrolled comparisons kept apart, the coverage map for every bullet, and an explicit per-bullet completion verdict.

The 2026-10-25 multi-agent test-value audit added 15 separately scoped implementation children: #2961, #2962, #2964, #2965, #2966, #2967, #2973, #2974, #2975, #2976, #2977, #2978, #2979, #2980 and #2981. They cover frozen migration assertions, repeated lint scans and CLI queries, duplicated real-consumer/pointer cases, and moving detailed markup or query matrices to cheaper owners. Their issue bodies contain exact retained-proof maps, source anchors, focused receipts where measured and required mutation controls. These are proposed reductions; no post-change or whole-CI speedup is claimed.

The epic closes when the original obligations, the new children and the definition of done are satisfied and the maintainer accepts the audit's verdicts. The 2026-10-25 source audit does not itself fulfill #2763's before/after performance acceptance.

Shared context

• The delivered recovery suite retains two responsive paths (const cells, packages/ui-react/tests/connection-continuity-runtime.test.ts:321-324); its invariants include DOM identity, geometry, scroll, focus, selection, attachments and follow-ups (function same, same file:706-722). Width selects different rendered trackers (function drawn, same file:342-344). The four-cell timings in Why now describe the original profile, before ui-react: remove theme duplication from connection-continuity recovery tests #1851/test(ui-react): run continuity recovery once per viewport #1905's delivered theme reduction.
• Storybook already supplies two theme projects (const storyProject, packages/ui-kit/vitest.config.ts:57-79; story project calls at 211-212) and an accessibility error gate (a11y, packages/ui-kit/.storybook/preview.ts:68). Rendering twice does not alone prove all geometry and interaction parity.
• The delivered recording split retains sustained native encoding, persistence, full-length decode and cleanup (test("ten-minute native endurance, packages/ui-react/tests/browser/recordings.endurance.tsx:11-65). Fast limit correctness is separate; ci: separate fast feedback, final confirmation and parallel endurance suites #1854 now delivers standalone execution policy through chore(ci): gate candidate confirmation and isolate endurance proof #1862, with the separately verified actual-main proof linked below.
• Binding context: ROADMAP.md (What binds future work), docs/process/testing.md, docs/decisions/ci-utility.md, design-kit D9/D21, D32/D39 and docs/decisions/example-corpus.md.
• Related findings outside this epic: ci: include layout scenarios in measured browser shard weights #1836 (layout shard weights) and ci: attribute and reduce repeated workspace builds in UI runtime tests #1837 (repeated workspace-build attribution) keep their existing ownership; cross-reference their results rather than duplicate them. This epic owns test cost; Epic: Keyless verification completes deterministically and isolates host state #736 and its nested browser epic Epic: browser verification is deterministic and attributable #2753 own deterministic isolation and failure attribution.

Constraints and non-goals

Preserve unique behavioral assertions, justified responsive/pointer/auth/backend cases, offline guards, native browser proof, lifecycle cleanup, production limits, contract envelopes, pinned images, default discovery and required proof gates. Remove redundant parameter combinations only with an assertion-to-coverage mapping. Any change to a currently required endurance obligation or decision record must explicitly reconcile that obligation, not silently skip it. Do not add a blanket passing-test cache, shorten product deadlines, increase concurrency or shard counts without evidence, or make provider calls. Paid opt-in image-provider tests remain cataloged as unmeasured.

Breakdown

The native sub-issue list is the plan and the progress record. The new test-value implementation children can land independently, with the retained owners named in their bodies preserved across other cleanups. #2550 owns the documentation correction. #2763 owns the final coverage/performance acceptance audit after the relevant changes are delivered. Any further unmet definition-of-done bullet becomes a new child here.

Definition of done

• Every native child is complete and its assertion mapping explains what was retained, moved or deduplicated. Remaining genuine theme-specific checks state a concrete reason.
• Shared/cloned/reset environments have measured benefit and proven case independence, dirty/failure cleanup and resource ownership. Tests whose intent is startup, isolation or teardown retain the required fresh lifecycle.
• Detailed variants use unit/component/small integration proof where sufficient; a documented guarantee/boundary map explains how representative E2E coverage completes the proof. No unique guarantee disappears through sampling or weaker doubles.
• Storybook owns reusable light/dark layout and interaction parity, with meaningful failing-first/mutation evidence and accessibility checks in both themes.
• Runtime suites preserve independent failure-mode coverage without an automatic Cartesian theme multiplier.
• Every deliberate wall-clock wait has a named temporal assertion, minimum justified duration and reason controlled time/completion signals cannot provide equivalent proof; expensive real-time confirmation follows the explicit tier policy.
• Recording limit correctness has fast proof; any remaining real-time endurance coverage has an explicit purpose and reviewed execution policy.
• Intermediate pushes use the fast tier; expensive final confirmation proves the exact candidate. Endurance is independently discoverable/scheduled, runs alongside other proof under measured resource bounds, and does not distort ordinary sharding.
• Comparable before/after results record source, runtime, resources, individual durations, setup, total runner consumption and CI critical path. Report savings and remaining bottlenecks without treating historical different-SHA runs as a controlled benchmark.
• All required selected checks pass on the delivered source. Closed children alone do not establish performance improvement or retained coverage.

Unscheduled tracking work; no release milestone is selected.

2026-10-24: rewrote the body to the current outcome and remaining obligations; added #2763 to own the coverage/performance acceptance audit that no child owned; the dated reconciliations that were in the body are preserved in a comment.

2026-10-24 triage at main af6300478a07379178bd30221a31c0b917df2283: Refreshed moved current-source pointers by inspecting their named anchors. Historical receipts retain their source identities; scope, rulings, priorities and outstanding prerequisites are preserved. No runtime result or readiness is inferred from this navigation repair.

2026-10-26: reconciled the 15 native test-value follow-ups from the 2026-10-25 review. Earlier implementation receipts remain historical evidence; current open work and final acceptance are distinguished. No source/test behavior or measured CI result changed in this tracker update.