Skip to content

[P2-33] Establish reproducible rendering performance baselines and gates #38

Description

@phranck

Problem

Current performance tests use absolute Date thresholds on shared CI runners, often print statistics without enforceable regression criteria, and do not exercise a full AppRunner/RenderLoop path. ForEach benchmarks currently measure empty output. Claimed write reductions and syscall behavior are not tied to reproducible fixtures.

Timing the incorrect String/ANSI renderer before the correctness migration would lock in misleading baselines.

Proposed solution

  • Run release-build benchmark fixtures only after correctness/output assertions pass.
  • Record deterministic work counters for body, measure, render, commit, surface cells, allocations, terminal writes, and wakeups.
  • Add representative small/large keyed collections, Unicode/style compositing, resize, focus/input, cached trees, images, and idle-app scenarios.
  • Use repeated samples with median/P95 and versioned baselines on controlled or scheduled runners.
  • Prefer algorithmic work/allocation counters in pull requests; reserve noisy wall-clock gates for stable benchmark infrastructure.

Acceptance criteria

  • No benchmark can pass while rendering empty or incorrect output.
  • Baselines cover full runtime commits, not only isolated helpers.
  • Idle CPU/wakeup count, memory growth, large keyed collection scaling, layout/compositing, and terminal-write volume have explicit regression limits.
  • Results record platform, architecture, compiler, build mode, fixture revision, and variance.
  • Performance claims in documentation link to a reproducible benchmark.
  • Release benchmarks and all Swift 6.0 quality gates complete without warnings/errors.

Dependencies

Runs after correctness/runtime #6/#8-#15, surface #11, keyed/lazy rendering #12/#25, cache #14, and scheduler #26.

Parallelization

Fixture design and counters can be prepared earlier. Enforced baselines must wait for the final correctness architecture to avoid benchmarking known-broken paths.

Commit structure

  • Split this issue into small, thematic, independently revertible commits wherever the work can remain coherent.
  • Every commit must build and keep its applicable tests/gates green. Do not commit an intentionally failing regression test; use local/known-issue characterization or land the test with the smallest fix.
  • Keep characterization/fixtures, mechanical renames or moves, semantic changes, and documentation/migration updates separate when each step remains green.
  • The issue boundary is not a commit boundary; multiple commits are expected for independently reversible changes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority: mediumImportant follow-up after critical and high dependencies

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions