Skip to content

Test: Judge the startup fill by pairwise ratios to the no-delay run beside it - #108

Merged
b-re-w merged 1 commit into
developfrom
fix/startup-fill-test-noise
Nov 13, 2026
Merged

b-re-w merged 1 commit into
developfrom
fix/startup-fill-test-noise

Conversation

@b-re-w

@b-re-w b-re-w commented Nov 13, 2026

Copy link
Copy Markdown
Member

Pins SPEC.md letify-link, Session item 7 (bulk in flight per subflow, startup phase). The check failed on 2026-11-13 on macOS (29.25 MiB/s) and ubuntu publish-crates verify (31.09 vs 32 needed) because the ceiling block was measured apart from the delayed block and one 0.5 s window is noisy.

Change: five interleaved pairs (no-delay run, delayed run, alternating order), each judged by its ratio over 1.0-2.0 s; median ratio >= 0.8 and first-2 s mean ratio >= 0.5. Thresholds unchanged; ceiling is the measured one capped at 40 MiB/s.

Evidence:

  • Startup phase reverted (cap_mode starts in Grow, as before 5c9891e): median ratio 0.30 (11.6-12.0 MiB/s vs 39.9), test fails.
  • Current code: ratio 0.97-1.09 (24 idle runs 6+8 of median), first-2 s mean 0.65-0.73; under taskset 2 cores plus 2 busy loops (10 runs) 1.00-1.14 and 0.69-0.79; 1 core plus 3 busy loops (6 runs) 1.04-1.12, 0.71-0.75.
  • fmt and clippy as in letify-link.yml pass.

…eside it

Pins SPEC.md letify-link, Session item 7, bulk in flight per subflow (the startup phase
of the bulk cap). The check failed twice on 2026-11-13 on shared runners: 29.25 MiB/s on
macOS and 31.09 MiB/s on the ubuntu publish-crates verify, against 32 needed. The ceiling
was measured in a block of three runs apart from the three delayed runs, so a change in
runner load between the blocks was not cancelled, and the 0.5 s window was short enough for
one scheduler stall to move it.

The test now runs five pairs, each a no-delay run and a delayed run back to back, in
alternating order. Each pair is judged by its own ratio over the 1 s from 1.0 to 2.0 s, and
the median of the five ratios must be at least 0.8. The mean of the first 2 s must stay at
or above 0.5 of the same ceiling. The thresholds are unchanged. The ceiling is the
measured one, capped at the 40 MiB/s the paths carry.

With the startup phase reverted (cap_mode starting in Grow, as before 5c9891e) the median
ratio is 0.30 (11.6 to 12.0 MiB/s against 39.9) and the test fails. With the startup phase
the ratio is 0.97 to 1.09 on an idle machine and 1.00 to 1.12 under busy loops.
@b-re-w
b-re-w merged commit dd57380 into develop Nov 13, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant