[Visual Test] Label preflight pagination aborts workflow before screenshot capture on run #30887567592

[MichaelFisher1997]

Workflow run

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30887567592

Head SHA: 8d205ff0b27a951ac7136cb72796ecd8e2e91ed4 (the only commit on dev: the nix flake → devenv migration, PR #951)

Exact failure

The failure is not in Vulkan or screenshot readback. The game command never ran, so build-output.log was never created. The diagnose agent found the workspace contained only weston.log (weston startup, then caught signal 15 from the Stop headless Wayland compositor step), no screenshot.png, and no captured log.

The failing step is Ensure visual-test label exists at .github/workflows/visual-test.yml:55-66. It runs:

if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75"
fi
if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75"
fi

The gh label list invocation returns the GitHub default 30 labels per page. The repository currently has 41 labels, and run-visual-test is on page 2. The precheck therefore reports the label as missing even though it exists. The subsequent gh label create then fails with:

label with name "run-visual-test" already exists; use `--force` to update its color and description

Because the step uses set -euo pipefail (it runs via the gh CLI which exits non-zero on a duplicate create), the step exits 1. Subsequent steps (Setup Lavapipe Vulkan, Run menu screenshot capture, the screenshot existence check, the golden-image diff, the artifact uploads) are skipped. Only the Stop headless Wayland compositor step runs (it has if: always()), and it kills the weston process with SIGTERM — which is the signal recorded in weston.log:62.

Why there is no Vulkan, swapchain, or screenshot error

VulkanSwapchain.createSwapchain at modules/engine-graphics/src/vulkan_swapchain.zig:127, the headless 1920x1080 color-attachment image at the same file lines 128-174, screenshot-mode init at src/game/app.zig:268-276, HomeScreen.init at modules/game-ui/src/screens/home.zig:33-40, and screenshot.requestCapture at modules/engine-graphics/src/vulkan/screenshot.zig:24-44 were never executed. The Zig binary was never launched; only the YAML pipeline ran.

Additional observations

• gh label list --repo OpenStaticFish/ZigCraft --limit 100 returns 41 labels; the default gh label list returns 30. run-visual-test is at index 31 of 41, so the page-1-only check cannot see it. Verified locally on dev.
• The previous issue [Visual Test] Label pagination aborts workflow before screenshot capture #968 (run #30795834741) described the same root cause. That issue is still OPEN and the Ensure visual-test label exists step in visual-test.yml has not been modified since the devenv migration commit 8d205ff (the only commit on dev), so the regression is recurring.
• The diagnose prompt is also stale but did not cause this run failure: .github/prompts/visual-test-diagnose.md:7 says screenshot.ppm, while the actual command at .github/workflows/visual-test.yml:81 uses the supported screenshot.png. The current encoder would still reject .ppm at modules/engine-graphics/src/vulkan/screenshot.zig:39 if a future run reverted to .ppm.

Suggested fix

Make the bootstrap idempotent without relying on paginated list output:

- name: Ensure visual-test labels exist
  run: |
    gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75" \
      --force
    gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75" \
      --force
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

Alternatively, query each exact label through gh api repos/$GITHUB_REPOSITORY/labels/<name> before creating it, or add --limit 100 to the existing gh label list call to cover the current page count.

Also update .github/prompts/visual-test-diagnose.md to describe the actual PNG output path and the current HomeScreen module at modules/game-ui/src/screens/home.zig (the prompt still references the old src/game/screens/home.zig).

[github-actions]

Classification: Bug — CI infrastructure regression in .github/workflows/visual-test.yml:55-66 (label bootstrap pagination + non-idempotent create), not a Vulkan/rendering failure.

Validation:

• Root-cause analysis is correct. gh label list defaults to --limit 30; with 41 labels the existing run-visual-test (index 31) is invisible to the page-1 grep, so the subsequent gh label create fails with already exists. set -euo pipefail plus the non-zero exit aborts the job before any Zig binary runs, which matches the recorded evidence (only weston.log present, no build-output.log/screenshot, SIGTERM in weston log from the unconditional Stop headless Wayland compositor step at :132).
• Confirmed locally against dev: the workflow file has not been touched since the devenv migration commit 8d205ff referenced in this report.
• The diagnose prompt staleness (screenshot.ppm at .github/prompts/visual-test-diagnose.md:7 and the old src/game/screens/home.zig path at line 29) is a separate, latent bug. It did not cause this run but should be fixed in the same change — the encoder at modules/engine-graphics/src/vulkan/screenshot.zig:39 would reject any reversion to .ppm.

Duplicate check: This is the same regression as #968 (run #30795834741) and should be closed as a duplicate of it rather than treated independently. Any fix must close both.

Scope: Narrow — a single-line workflow patch plus a prompt refresh. No engineering decomposition needed.

Recommended next implementation (smallest correct patch), to be filed against #968/this issue:

1. In .github/workflows/visual-test.yml:55-66, replace both gh label list … | grep precondition + gh label create pairs with unconditional gh label create … --force (idempotent on every run; no pagination dependency). Keep the GH_TOKEN: ${{ secrets.OPENCODE_PAT }} env line.

2. In .github/prompts/visual-test-diagnose.md:

   • Line 7: change -Dscreenshot-path=screenshot.ppm to -Dscreenshot-path=screenshot.png and drop the "PPM is then converted to PNG" sentence at line 9.
   • Lines 8 and 26-29: replace src/game/screens/home.zig with modules/game-ui/src/screens/home.zig and any other path references to match the game-ui module structure.
   • Line 21: drop "PPM write" from the failure-pattern list or replace with "PNG write".

3. Re-run the visual-test workflow on the resulting PR (label it run-visual-test) to confirm Ensure visual-test label exists exits 0 and the Lavapipe/Rayleigh-screenshot steps actually execute.

The --force form is preferable to --limit 100 because it stays correct as the label count grows and removes the read-then-write race entirely.

opencode session  |  github run