Skip to content

fix(ci): read the contract gate's base from the PR, not the event - #2660

Merged
schlessera merged 1 commit into
mainfrom
fix/2650-native-stack-base-044
Oct 22, 2026
Merged

schlessera merged 1 commit into
mainfrom
fix/2650-native-stack-base-044

Conversation

@schlessera

@schlessera schlessera commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Closes #2650
Refs #2141

What changed

The contract gate now compares a PR with the base the PR itself declares, read from GET /repos/{repo}/pulls/{number} with the workflow's read-only token on every trigger, instead of github.base_ref. Before this, an upper layer of a native stack passed on opened but failed on every edited/labeled run. Those events reported main, so the inherited contract-doc edit from the parent layer was charged to the upper layer (#2644, runs 39777521171 and 39779073432). Now a missing PR number fails the gate with exit 2 and a named diagnostic, and so do a refused or failed lookup (5xx and network errors are retried twice), a response for another PR, a base in another repository, an invalid branch name, or a base the checkout has not fetched. It never falls back to main. The event's value is logged next to the resolved base, so a disagreement shows in the run log.

This PR also delivers #2141's criterion that "contract comparison for an upper native layer uses its own parent", with inherited and own-contract controls. Per the maintainer ruling of 2026-10-22, the rest of #2141 is parked rather than ported here: the frozen-prefix, synthetic-chain and ORT-tree candidate confirmation, and native integrate.ts support. The old PRs #2187, #2285 and #2286 are closed with a pointer here.

Why this shape

  • Read the PR object, not another event field. GitHub's docs say stacked PRs are "evaluated as if each pull request in the stack targets the base of the stack". They don't say what github.base_ref holds for upper layers, and on test: measure first-party egress paths and route Git over SSH through the turn proxy #2644 it changed between opened and edited. The PR object is the authority the stack rules name ("the PR's declared base"). ci.yml was not changed. Its draft runs on test: measure first-party egress paths and route Git over SSH through the turn proxy #2644 got the parent SHA (converted_to_draft run 39784882558: BASE_SHA 757c0d9, the parent head), and it does not trigger on edited.
  • No new permission. contents: read stays exact, and the workflow lint still enforces it. ci.yml already reads pulls/N with the same token for its draft-gate readiness check.
  • No dependency install in the contract job. The lookup is a small fetch inside check-contract-pr.ts. It does not import proof-reuse.ts, whose import graph would need bun install.
  • The local form check-contract-pr.ts <base-ref> <head-sha> is unchanged.

Proof

Receipts below; local logs are kept by the coordinator.

Failing-first. I ran the new workflow-level harness against the unchanged contract.yml (BASE_REF: origin/${{ github.base_ref }}), with the event reporting main for a stacked upper layer whose parent changed docs/integration-contract/http.md:

(fail) the contract gate compares each PR with the base it declares (#2650) > every trigger of an upper layer diffs against its parent, even when the event names main
-   "exit": 0,   +   "exit": 1,
+ "docs/integration-contract/http.md changed, but the title does not start with `CONTRACT:`.

That is the exact #2650 failure. The own-contract, ordinary and fail-closed tests also failed. Restored: tests/ci-workflows.test.ts + tests/contract-gate.test.ts, 94 pass / 0 fail, 557 assertions.

Workflow-level harness (tests/ci-workflows.test.ts). It runs the installed step: env expressions are substituted from an event context, then the step runs through bash -e in a fixture repo that has origin/main and origin/feat/1665-lower. The PR lookup is a loopback server, and the repository network guard is untouched. Cases:

  • the stacked upper layer passes for every trigger (opened, edited, synchronize, reopened, labeled, unlabeled), whether the event names main or the parent;
  • the upper layer's own contract edit still fails without the title and label, and its log does not mention the inherited file;
  • an ordinary PR against main behaves as before, in both directions;
  • a missing PR number, a 404, an unfetched base, a foreign base repo, or a response for another PR each exit 2 with that diagnostic and never print a resolved base;
  • the fork hostile-title control still holds.

Mutations (each restored; named test fails on the expected assertion):

  • M1, base taken from EVENT_BASE_REF again: "every trigger of an upper layer…" fails with exit 1 and the inherited http.md diagnostic.
  • M2, a failed lookup falls back to origin/main: "an unreadable or unfetched declared base fails the gate instead of choosing main" fails, exit 2 → 1.
  • M3, base-repository check removed: "an answer that does not identify the PR's base in this repository fails closed" fails, because the promise resolved.
  • M4, contract-step lint block removed: "the contract gate cannot take its base from the event (ci: the contract gate compares a stacked upper layer against main after an edited event #2650)" fails because the problem is missing.

Workflow lint. bun scripts/check-ci-workflows.ts passes. The lint now refuses a contract step that gets its base from github.base_ref or pull_request.base, or that lacks PR_NUMBER or GH_TOKEN; the event value is allowed only as EVENT_BASE_REF.

bun run check:pr --base origin/main

Result at head 91f2ce2dd on base 2ba99c607: passed, 989 fast + 108 affected tests, 0 fail ("Local preflight passed").

Hosted. This PR's own contract run 39798005870 passed at head 91f2ce2dd and logged Contract base: origin/main, declared by PR #2660; the event agrees. That confirms the pulls/N lookup works with the contents: read token. Draft gates run 39798002286 passed. I did not record a run on a stacked fixture PR. The workflow-level harness covers that, as the issue's acceptance criteria allow.

Checklist

Anything left open

🤖 Generated with Claude Code

For an upper layer of a native stack, `edited` events report the stack's
base (`main`) as `github.base_ref`, so the contract gate diffed the layer
against `main` and blamed it for its parent's contract-doc edit (#2644).

The gate now reads the PR's declared base from `pulls/N` with the
read-only token on every trigger and diffs against that remote-tracking
ref. A missing PR number, a refused or failed lookup, a foreign or invalid
base, or an unfetched base fails the gate (exit 2) instead of falling back
to `main`. The event's value is logged only to show a disagreement.

The workflow lint refuses a contract step that takes its base from event
context, and a workflow-level harness executes the installed step against a
loopback PR lookup for every trigger, stacked, ordinary and failure cases.

Closes #2650
Refs #2141

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@schlessera schlessera added this to the 0.44.0 milestone Oct 22, 2026
@schlessera schlessera added type: fix Something is broken and must behave the way it is documented. area: ci CI workflows, the gates they run, release scripts, packaging. labels Oct 22, 2026
@schlessera
schlessera marked this pull request as ready for review October 22, 2026 14:54
@schlessera
schlessera merged commit 30c1d50 into main Oct 22, 2026
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, the gates they run, release scripts, packaging. type: fix Something is broken and must behave the way it is documented.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: the contract gate compares a stacked upper layer against main after an edited event

1 participant