Skip to content

cvcGL: fix the macOS grid golden settle flake - #761

Merged
transfix merged 2 commits into
masterfrom
fix/ari-golden-grid-settle
Oct 22, 2026
Merged

transfix merged 2 commits into
masterfrom
fix/ari-golden-grid-settle

Conversation

@transfix

@transfix transfix commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

The flake

On macOS CI, cvcgl_ari_golden sometimes failed grid's "one more frame draws the same pixels (the UI has settled)" check on the first ctest attempt, with 8 px off by 1. --repeat until-pass:3 hid it, because the retry passed. It hit 2 of 4 macOS lanes on master (run 39610796467, 796517b), 2 of 4 on #760 (run 39612556385), and 1 of 4 here (run 39654691408).

Cause: the Apple Software Renderer, not the UI

  • The first commit adds a fingerprint of what ImGui hands GL: draw lists (vertices, indices, commands, clip rects, texture ids) plus each texture's size and pixels. The settle line now reports it.
  • On the failing macOS attempt (package-macos / libcvc-debug, attempt 1): 8 px differed by 1 (first at (39,61): 38,38,38 then 38,39,39), and ImGui drew the same UI both times (572 vertices, texture 512x128). The renderer drew identical input two ways.
  • The differing pixels move around. The earlier runs had one column at x=117, y 230..237 (the "d" of the "Inner borders" title). This run had scattered pixels in x 39..567, y 61..107.
  • It has only ever hit grid, the first document each process renders. In the same failing process, menus, scene, widgets and windows all settled at +-0. So it's a transient of the renderer's first frames that the test can't pin.
  • Linux llvmpipe never shows it: 96 runs of grid, 32 at a time, with byte-identical frames and draw data.

The fix

The settle check keeps its +-0 tolerance. When the next frame differs and the UI fingerprint is identical, the later frame becomes the capture and the test renders another, up to 10 times. The log says how many frames that took and what the first difference was. A UI that changed still fails at once, and so does a rasterizer that never settles. docs/ARIADNE.md §12 describes this.

This does loosen one thing: a document may now take up to 10 extra frames to settle, but only when ImGui's draw data proves the UI didn't change. The pixel tolerance stays at +-0.

Verified locally

cvcpkg Mesa 25.3.6 llvmpipe on Xvfb, CVC_REQUIRE_RENDER=1:

  • cvcgl_ari_golden 34/34; every document reports "ImGui drew the same UI both times".
  • ctest ^cvcgl_ 72/72.
  • A fault-injected scratch copy of the test (not committed) exercised each path:
    • +1 noise on one pixel in the first 2 or 4 renders: passes, "settled after 2 / 4 more frame(s)".
    • Noise on every render: fails, "had not settled after 10 more frame(s)".
    • A changed draw-list fingerprint: fails at once, "ImGui's draw lists changed".

Verified on CI

On 18a0734 (CI/CD run 39710264633), all 25 checks passed. cvcgl_ari_golden passed on the first attempt on all four macOS lanes.

  • The "Ariadne golden-image numbers" step shows grid took 7 frames on libcvc-debug and libcvc-release-grpc-static, against the usual 6, and 6 on the other two.
  • So the renderer noise fired on 2 of 4 lanes, as before, and one extra frame with the same UI absorbed it each time.
  • That step prints only the frame count and the golden lines, so the "settled after 1 more frame(s)" note itself isn't in the job log.

cvcgl_ari_golden requires the frame after the capture to be pixel-identical
("the UI has settled"). On macOS CI, grid fails that on the first ctest
attempt on about half the lanes: 8 px in one column (x=117, y 230..237,
the anti-aliased edge of the "d" in the "Inner borders" title) off by 1.
The retry passes, and Linux has never shown it (96 runs, 32 in parallel,
identical frames).

The settle line now also says whether the two frames handed GL the same
UI: a fingerprint of ImGui's draw lists (vertices, indices, commands,
clip rects, texture ids) and of every texture's size and pixels. A settle
failure with the same UI comes from the rasterizer, not the document. It
also names the first differing pixel and its two colours.
#761's diagnostic caught the macOS flake on CI run 39654691408
(package-macos / libcvc-debug, attempt 1): grid's two frames differed by
1 in 8 px (first at (39,61): 38,38,38 then 38,39,39) while ImGui had
handed GL byte-identical draw lists and textures both times. So the
Apple Software Renderer drew the same UI two ways. It has only ever hit
the first document in a process, and the later documents in that same
process settle at +-0, so it is a transient of the renderer's first
frames that this test cannot pin.

The settle check keeps +-0. When the next frame differs but the UI
fingerprint is identical, the later frame becomes the capture and the
test renders another, up to 10 times, and the log says how many it took.
A UI that changed still fails at once, and so does a rasterizer that
never settles. docs/ARIADNE.md section 12 describes both.
@transfix
transfix marked this pull request as ready for review October 22, 2026 05:11
@transfix
transfix merged commit 5cbc71a into master Oct 22, 2026
26 checks passed
@transfix
transfix deleted the fix/ari-golden-grid-settle branch October 22, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant