Repository navigation
ci: fuzz step reddens on the coordinator's own deadline with no finding #466
Description
What happens
The gate's fuzz step (scripts/fuzz.sh, go test -fuzz … -fuzztime=10s) intermittently fails with no finding:
--- FAIL: FuzzNewBigVal_AcceptedFormsAreJSONValid (10.05s)
context deadline exceeded
FAIL
FAIL github.com/dexpace/morphic/ir 10.053s
make: *** [Makefile:113: fuzz] Error 1
No failing input is minimized or written to testdata/fuzz/, the run reports PASS on the next attempt with the same code, and the target passes a 20s local search. The elapsed time is the budget itself (10.05s, 11.08s): the fuzz coordinator hit -fuzztime while a worker was still executing an input and reported the deadline as a failure instead of stopping cleanly — Go's fuzzing engine has no grace period for that, so under runner load it surfaces as a red gate.
Two occurrences on 2026-09-12/13:
- CI, feat(ir)!: add Payload.Required and Parameter.Provenance #437 (
FuzzNewBigVal_AcceptedFormsAreJSONValid): https://github.com/dexpace/morphic/actions/runs/34693832565 — the rerun of the identical head was green. - Local
make gateon the head of fix(compilers/openapi): report a bare deprecation extension key #463 (FuzzCycleDetector, 11.08s):make fuzzrerun green, all four targets.
In the twelve failed gate runs before these, the message appears in none, so this is new-ish or newly frequent; either way one hiccup now reddens an unrelated PR's whole gate, and a reviewer reading "fuzz failed" has to open the log to learn nothing was found.
Where it lives
scripts/fuzz.sh runs each target once with the whole budget and no slack:
go test "$pkg" -run '^$' -fuzz "^${target}\$" \
-fuzztime="$fuzztime" -fuzzminimizetime="$minimize_time"Options
- Distinguish the deadline from a finding. A real finding writes a reproducer under
testdata/fuzz/<Target>/and prints its path; the deadline case writes nothing. On a non-zero exit, treatcontext deadline exceededwith no new corpus file as one bounded retry (once), and anything else as the failure it is. This keeps the 10s budget and cannot hide a finding, since a finding is reproduced from the written seed on the retry too. - Give the coordinator slack — e.g.
-fuzztime=8sinside a 10s wall budget — which shrinks the search for a property nobody measured.
Option 1 is the one that keeps the search as it is and makes the failure mode legible. Either way the script's comment should name the failure mode, so the next reader of a red fuzz step knows what "context deadline exceeded" with no reproducer means.
Activity
fuad-daoud commentedon Nov 2, 2026
Evidence for deciding this, from a review of #1301 (which moves the fuzz search to a nightly job and gives a deadline with no finding no handling of its own).
The pinned toolchain suppresses the error. In go1.27.2,
src/internal/fuzz/fuzz.go:129-137,stopdrops the error whenerr == ctx.Err() || err == fuzzCtx.Err() || isInterruptError(err). The comment there reads: "Also check ctx.Err() because when ctx's deadline expires, there is a window where ctx.Err() is set but fuzzCtx (a child) has not yet been canceled. See go.dev/issue/75804."The deadline paths in
worker.goeither returnctx.Err()(lines 117, 163, 307 and 315) or setw.interruptedbefore signalling (stop, line 452), so they are suppressed too.grep -c 75804 src/internal/fuzz/fuzz.go, per toolchain:Both failures here predate the fix reaching this repository. From
go.mod'sgoline overgit log origin/main -- go.mod:20778b20(build: move to Go 1.27 and refresh dependencies #471, 2026-09-21), which moved the repository to 1.27.1;050709f0(build: move to Go 1.27.2, golangci-lint v2.14.0, openapi v1.25.5 #804).The two occurrences recorded here (2026-09-12 and 2026-09-13) both ran on 1.26.3.
CI history. Of the failed gate runs created from 2026-09-20 to 2026-11-01, excluding #1301's two trial-branch runs, there are 37. Of those, 13 reached the fuzz step, and none failed in it. That is weaker than it sounds: a run re-run to green would not appear.
What did not discriminate. 80 runs each of a 150ms search, built by go1.26.6 and by go1.27.2, gave 0 failures on either. The evidence rests on the source and the timeline, not on a reproduction.
If #1301's approach lands, a red night is a finding or a failure worth reading. This issue could then close as fixed upstream by go.dev/issue/75804 from 1.27.1 on.