ci: the --output render path has no CI coverage beyond the engine id

[TMHSDigital]

The gap

Smoke runs each example's default check path. The --output render path is exercised only at authoring time, on one version, by the author. A version-dependent render bug is therefore invisible to CI by construction — which is how an inverted EEVEE engine id shipped in two showcase pieces and survived until someone tried to render on 5.x.

tests/check_engine_id.py now closes the engine-id part of that gap at zero cost: it evaluates all 84 mappings in the tree against both eras. It does not cover the rest of the render path — the framing helper, material and node wiring, normal-map hookup, image write.

Why a render canary was not added

Measured on this machine, showcase/shipping-crate --output, wall-clock for the whole process including the check path:

Engine	4.5.11	5.1.2	5.2.1	Render-only delta
EEVEE (default)	2 914 ms	4 004 ms	3 515 ms	1 152 / 1 889 / 1 890 ms
Cycles, 32 samples CPU	14 449 ms	—	12 559 ms	~11 000 / ~10 900 ms

Check-only baseline for the same piece: 1 762 / 2 115 / 1 625 ms.

Two things block the obvious version of this:

1. The cheap canary does not cover the motivating class. EEVEE at ~1.2–1.9 s per version is nearly free, but blender-smoke.yml deliberately does not render EEVEE. Its own comment says why:

Cycles (CPU) so this is reliable on GPU-less runners; the EEVEE-id regression itself is gated in run_smoke.py via engine assignment.

and in tests/smoke/run_smoke.py:

Render the non-black frame with Cycles (CPU): reliable on GPU-less headless runners, where an EEVEE GPU render aborts the process (no EGL).

2. The covering canary is the expensive one. Cycles is the engine CI can rely on, and Cycles never touches the EEVEE id — so a Cycles canary would not have caught either inversion. It costs ~11 s per version locally for a single piece, against a current showcase-suite total of 20.9 s on 5.2.1. GitHub runners are slower than this machine, so the real figure is plausibly 2–4× that, which would put one canary at or past the ~30 s per-version budget on its own.

So the cheap option does not cover, and the covering option is not cheap — and neither covers the class that motivated the work, because check_engine_id.py already does that better and for free.

What a canary would still buy

Everything in the render path that is not the engine id:

• gallery_framing.check_framing — the exit-10 gate, including its matte render
• normal-map node wiring (wire_normal) and material assignment at render time
• image write and the webp/png format branch
• anything that only touches bpy.ops.render.render

None of that is covered today on any version.

Options, if this is picked up

1. Cycles canary on one piece, one version only (not per version). ~11 s once rather than three times. Catches non-engine render-path breakage; catches nothing version-specific.
2. Add a resolution override to showcase pieces so a canary can render at, say, 320×180. The scripts hardcode 1280×720; examples/swatch-grid already takes --width, so there is precedent. Would make a per-version Cycles canary affordable, at the cost of touching 28 scripts.
3. Establish whether EEVEE actually aborts on current runners. The Cycles-only decision is documented but the comment does not carry a date or a measurement. If EEVEE works on today's ubuntu-latest with the libraries the workflow already installs, the ~1.9 s per-version canary becomes available and covers the engine id too.

Option 3 is the one worth measuring first, because it is the only one that makes the cheap canary viable.

Not blocking

The engine-id class — the one that actually shipped a bug twice — is covered as of tests/check_engine_id.py. This issue is about the remainder.

[TMHSDigital]

Option 3 measured: EEVEE does not abort on today's runners, but it is slower than Cycles there

I ran a throwaway workflow on a probe branch (now deleted), run 35803188502, on ubuntu-latest. It rendered showcase/shipping-crate --output at the default 1280×720 on both LTS series. Step 1 installed exactly the smoke workflow's library list. Variant B then added libgl1-mesa-dri libegl-mesa0 libglx-mesa0.

Series	Engine	Display	Extra Mesa	Result	Wall (whole process)
4.5	EEVEE	headless	no	exit 0	60.3 s
4.5	EEVEE	xvfb-run	no	exit 0	39.6 s
4.5	EEVEE	headless	yes	exit 0	38.0 s
4.5	Cycles	headless	no	exit 0	23.5 s
5.2	EEVEE	headless	no	exit 0	88.7 s
5.2	EEVEE	xvfb-run	no	exit 0	67.4 s
5.2	EEVEE	headless	yes	exit 0	64.3 s
5.2	Cycles	headless	no	exit 0	41.1 s

• The "EEVEE GPU render aborts the process (no EGL)" premise no longer holds. Every EEVEE variant exited 0 with a real image. The log carries 3× EGL Error (0x3009): EGL_BAD_MATCH warnings, but the render completes on software GL. That comment in tests/smoke/run_smoke.py:138 and blender-smoke.yml dates from 9ad35cd (2026-06-19) and carries no measurement.
• The CI EEVEE output is correct. Against a local EEVEE render of the same piece on Blender 5.2.1 LTS (Windows), the CI images differ by mean abs 0.00092–0.00098 with 0.02% of pixels past 2%, which is at the noise floor. 4.5 and 5.2 agree with each other and with the local render.
• But the cheap canary is not cheap on runners. EEVEE runs on llvmpipe there, so it costs 38–89 s per version at 1280×720, slower than Cycles CPU (23–41 s). The ~1.2–1.9 s figure in this issue was measured on a GPU machine.

What that means for the options

• Option 3 as written (EEVEE canary at full res): viable, not cheap, about 1–1.5 min per version per piece.
• Option 2 (resolution override): now the useful lever for both engines. EEVEE on llvmpipe and Cycles CPU both scale roughly with pixel count. At 320×180 (1/16 the pixels), a per-version EEVEE canary that also covers the engine id should come down to a few seconds. That is an inference, not measured. examples/swatch-grid --width is the precedent.
• The EEVEE canary would exercise check_framing (exit 10), material wiring, image write and the engine id, which Cycles never touches.

Artifacts (PNGs and logs) are attached to the run for 90 days.