Repository navigation
ci: ready PR can retain green draft-mode proof with product jobs skipped #2363
Description
Out of scope
No weakening of exact candidate identity, no skipped product proof being promoted to coverage, no paid CI and no change to package behavior. Drafts stay cheap: a draft PR's run keeps running only the cheap rejection/metadata gates.
Context
During #2356 / PR #2359, head 59eb04cbf725749d93a4d8bd3262c3ca9c18e06e was pushed while draft and then marked ready. Live PR state was ready, mergeable and CLEAN when the watcher reported all 12 checks passed. However, the latest completed CI 39069053895, attempt 1, used PR_DRAFT: true, cadence: draft and no retained proof. Verify, pack, unit, runtime, runtime_host, browser, layout, captures and endurance were all skipped. Its proof job 117502911568 nevertheless returned success. No merge was performed on that evidence; the worker returned the PR to draft to request a fresh confirmation.
Both metadata and proof logs checked out 1b9203d0c6630f5e43b7ff6b4de779ae111dccf3, parents base 0d780605fe1ea9a281cce947b1f0074f3f3e7acc and the head above, tree a183600290c080d9d8e1e31eb0544187b2f63dcc.
Event order (triage reconstruction, 2026-10-18)
From the PR timeline and the Actions run API for that head:
| Time (UTC) | Observation |
|---|---|
| 20:58:48 | convert_to_draft |
| 20:59:53 | head_ref_force_pushed: the synchronize event fires while the PR is a draft, so its payload has pull_request.draft: true |
| 20:59:55 | ready_for_review: this payload has draft: false, so proofCadence selects confirmation |
| 20:59:57 | CI run 39069053680 created (the lower id). It was cancelled at 20:59:58 with no jobs |
| 20:59:57 | CI run 39069053895 created (the higher id). It ran with PR_DRAFT: true, cadence draft, and proof succeeded at 21:01:38 |
Observed: both runs share the per-PR concurrency group (concurrency, .github/workflows/ci.yml:18-20, cancel-in-progress for pull requests). The run created second cancelled the first. The surviving run carried the draft payload.
Inferred: run 39069053680 was the ready_for_review run. It is the only other pull_request CI event for this head in that window, and the draft payload can only come from the synchronize event. The run API does not expose the event action, and a run cancelled before any job has no logs, so this attribution is not directly observable. The order is the point: the run for the earlier event (synchronize, 20:59:53) was queued after the run for the later event (ready_for_review, 20:59:55). It therefore cancelled the confirmation request. GitHub does not guarantee that workflow runs for near-simultaneous events are queued in event order.
The later transitions on the same head (21:03:05 draft → run 39069292228 cancelled; 21:03:43 ready → run 39069337278 cancelled at 21:15:45 by the 21:15:24 draft run 39070168750) were queued in event order and do not show the inversion.
Why the draft success reads as merge-grade
proofCadence(export function proofCadence,scripts/ci-plan.ts:313-321) choosesdraftfrom the event payload alone.- The aggregate (
if (process.argv[2] === "aggregate"),scripts/ci-proof.ts:78-93) callsrequireCandidate→validateCandidate(export async function validateCandidate,scripts/candidate-proof.ts:6-11). That returns without any live readiness check for any cadence other thanfeedbackorconfirmation. With every product job unselected,proofProblemsfinds nothing to fail, soproofsucceeds. - The aggregate check is named
prooffor every pull-request cadence. A draft-mode success therefore looks the same in the check list as a confirmation success. mainhas no branch protection or ruleset at triage time; merge admission is the agent procedure in.agents/skills/work-issue/references/ci-and-merge.md. That procedure says readysynchronizeleavesproofnonpassing (lines 32-34), but nothing makes a stale draft-modeproofnonpassing once the PR is ready.
Governing ruling
The maintainer's 2026-10-14 ruling in #1849/#1854 (docs/decisions/ci-utility.md:97-108) says drafts run cheap gates only. Only opened, reopened or ready_for_review on a ready PR requests confirmation. A ready PR's non-confirmation run deliberately fails proof with the action to take, and fast feedback "is useful, but never sufficient merge evidence". The same section gives scheduled/manual runs a distinct aggregate check name "so they cannot impersonate a PR's candidate confirmation". So the required behavior is already decided: on a PR that is ready, proof must not pass unless that run is a confirmation. This issue applies that ruling to the draft cadence; it does not need a new ruling.
Scope
Make a pull_request run whose cadence is draft unable to report a passing proof once the PR is no longer a draft:
- In the aggregate, a draft-cadence run reads the live PR read-only, the same way confirmation already does. It refuses when the live PR is not open, is no longer a draft, or has a head other than the event's head. The refusal message names the action, like the existing
feedbackmessage: convert to draft, finish pushes, then mark ready for review. - A draft-cadence run on a PR that is still a draft keeps its cheap success. Feedback and confirmation behavior are unchanged.
- If the live read fails or returns an invalid response, the run fails closed instead of passing.
- Update
docs/decisions/ci-utility.md(Scheduling and selection) and.agents/skills/work-issue/references/ci-and-merge.mdto state that a draft-modeproofon a ready PR fails, and own any citations the change shifts.
Why this covers both orders: if the PR is ready before the draft run's aggregate reads it, the live check refuses. If the read happens first, the ready_for_review run is queued after the draft run was queued. It becomes the newest member of the concurrency group and supersedes the draft run.
Another mechanism is acceptable only if it meets the same acceptance criteria. One example is a distinct aggregate check name for draft cadence, following the scheduled/manual precedent, so a ready PR shows no proof row. Record the choice and its residual races in the PR.
Acceptance criteria
- Explain the ready/draft event ordering with actual event/head/base and run/job identities, keeping causality that is not observable marked as such (Event order section above, 2026-10-18).
- Offline aggregate controls with a draft-cadence event payload: live PR draft → passes; live PR ready → fails with the named confirmation-required message; live head differs → fails; live read error or invalid response → fails. These cover the delayed or out-of-order case where the payload says draft and the live PR says ready.
- A mutation that removes the live readiness check makes the "live PR ready" control fail on that assertion; the PR records the receipt and the restore.
- Cheap draft gates, ready
synchronizefeedback, exact confirmation identity, every selected job and validated domain-retention semantics are unchanged. Existingci-plan,candidate-proofandci-prooftests pass unmodified apart from new cases. - Decision record and merge reference updated as in Scope.
Verification
Offline workflow/admission controls failing first on the selected mechanism, then restored; bun run check:pr --base origin/main before ready. Hosted evidence on the PR itself: a draft push whose proof passes cheaply, and a ready confirmation whose proof passes with every selected job. Do not treat a check count or CLEAN state as complete proof.
Contract impact
Internal CI/merge-admission only. No CLI --json, MCP, schema_version or frontmatter change.
2026-10-18 triage at main 775eb17aab740fa759c2182f306c38fc17acbcbe: reconstructed the event order from the PR timeline and run API (first criterion now checked). Folded the 2026-10-14 cadence ruling into a concrete behavior spec, replacing "needs scoping", and added offline controls and a mutation criterion. Attached to #1849, whose definition of done requires that expensive final confirmation proves the exact candidate.
Activity
schlessera commentedon Oct 18, 2026
Claim
draftproof-2363-0a14, intended branchfix/2363-draft-proof-live-readiness, from fetched main775eb17aab740fa759c2182f306c38fc17acbcbe. Implementing the Scope above: draft-cadenceproofreads the live PR and refuses once it is ready, with offline controls and a mutation receipt. No change to feedback/confirmation behavior, candidate identity or job selection.test(ci): drop the spawned draft aggregate control
schlessera commentedon Oct 18, 2026
2026-10-18 status for claim
draftproof-2363-0a14: PR #2398 is ready at5e75daec79298942498b1acf1f83640667a0838fon base68e12f27eb249c9fe7262a498a0397c04bbda8f4. Confirmation run 39085021845 passed 13 of its 15 product jobs. Both failures are attributed outside this change:authored narrow grouped notes lightexceeded the 15 s deadline. That is ci: authored narrow grouped diagram notes exceed WebKit case deadline #2386 (fix in test(ui-react): separate narrow callout geometry and capture #2393); the receipt is posted there.provider-history.test.tsfixture copy hitENOENTon git'sobjects/maintenance.lock. Filed as tests: fixture repositories start detached git maintenance that races snapshot copies #2405 (detached auto-maintenance in fixture repositories).Live evidence of the fix itself: draft run 39084898164 reported its aggregate as
draft gateswith every product job skipped, and its live readiness read passed against the real draft PR. While the confirmation ran, the head had noproofrow at all. Before this change, that same moment would have shown the draft run's passingproof.No rerun requested. Next: after #2393 merges, refresh onto main, rerun preflight and request confirmation again. #2400 tracks the aggregate-wiring test this PR could not add cheaply.
fix(ci): fail draft-mode proof once the live PR is ready
test(ci): cover every draft action and the hosted draft-proof entry p…
fix(ci): name the draft aggregate apart from candidate proof
test(ci): drive the draft readiness check through its hosted callers
test(ci): drop the spawned draft aggregate control
2 remaining items
test(ci): cover every draft action and the hosted draft-proof entry p…
fix(ci): name the draft aggregate apart from candidate proof
test(ci): drive the draft readiness check through its hosted callers
test(ci): drop the spawned draft aggregate control
fix(ci): keep the audited ci.yml producer digest
fix(ci): fail draft-mode proof once the live PR is ready
test(ci): cover every draft action and the hosted draft-proof entry p…
fix(ci): name the draft aggregate apart from candidate proof
test(ci): drive the draft readiness check through its hosted callers
test(ci): drop the spawned draft aggregate control
fix(ci): keep the audited ci.yml producer digest
fix(ci): keep draft-mode success from standing as a ready PR's proof (#…
schlessera commentedon Oct 18, 2026
Delivered by #2398, squash
685d83e22cc452e97a2c92ef7ae683f34ceb2371, verified on fetched main. Its parent isc4d31c6bd1aeed60fd3cda02f9c8586fe8a509aeand its tree4ff5bc48ee9f7179b61f4869d4dae959c7bf9aeeexactly matches the confirmed synthetic checkoutac658a664.Ready run 39115057376, attempt 1:
BRAIN_CANDIDATE_RECEIPTnames headdbec504b2and basec4d31c6bd, which equalled live main at merge. 15 of 16 selected jobs passed (verify, pack, all four unit shards, runtime, runtime_host, browser (1) and (3), both layout shards, captures, endurance). Browser (2) failed only onmodule settings: source-preserving edits and guarded navigation in dark at 320px, which ran 15,741 ms against its 15,000 ms deadline. That is the long-standing #1229 flake (receipt posted there), in code this change does not touch. The aggregate failed only on that job.Merged under the maintainer's 2026-10-18 ruling to merge on the 15/16 receipts with that failure attributed to #1229. This is not a passing aggregate and is not recorded as one.
Live evidence for this fix, collected during delivery:
draft gates, neverproof(runs 39084898164, 39114974312).proofrow until the confirmation aggregate reported. A watcher had treated the draft-modeproofon test(ui-react): separate narrow callout geometry and capture #2393's old workflow as passing.