Skip to content

CI: a merge gate that needs review, evidence and the owner's OK at each head - #106

Merged
krishhgg merged 19 commits into
mainfrom
ci/merge-gate
Oct 13, 2026
Merged

krishhgg merged 19 commits into
mainfrom
ci/merge-gate

Conversation

@krishhgg

@krishhgg krishhgg commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

What changes

Main requires no status check today, so a pull request with red CI could merge, and every "ready to merge" so far was a hand check. This adds what the main ruleset will require once the ruleset change lands (a separate step after this merges):

Status Passes when
merge-gate/greptile Greptile reviewed the head and scored it 5/5. Read from the newest description edit by greptile-apps in GitHub's edit history, which must hold exactly one score and one reviewed commit, so a block the author typed or changed never counts.
merge-gate/accepted The owner added the accepted label, and nobody removed it since.
merge-gate/evidence The maintainer checked a real-Mac report for the head, or the pull request only touches docs/, Tests/, .github/, .greptile/, README, CONTRIBUTING, SECURITY, LICENSE or .gitignore.
merge-gate/config Every changed file is a kind no check reads as settings, or the author is the owner, or the owner OK'd it at the head. The safe kinds are an allowlist: Swift sources and tests, text docs (md, html, svg), skills, the product's scripts (not the lid gate script CI runs), plists under Resources/, and the root README and LICENSE (SECURITY.md and CONTRIBUTING.md set policy, and Greptile reads them as guidance for every review). Among them, these still count: a dot-named file; an AGENTS.md, CLAUDE.md or GEMINI.md (and their override and local forms); a diff that adds a ShellCheck directive or a Swift @diagnose or @warn in any spelling; a file GitHub sends no diff for (too big, or binary, which one NUL byte makes a script), unless it was deleted; a changed symlink or submodule, read from the head's tree; and any non-ASCII name. The safe kinds match only in their exact case, so scripts/backstop.SH, which CI's scripts/*.sh checks skip but a Mac opens as scripts/backstop.sh, counts; the names that count match in any case. Anything else, such as .github/, Package.swift or an image, waits for the owner, since no reviewer can read an image and a script could source one. A fork's pull_request run uses the fork's workflow files, so a changed ci.yml could report a job as passed without running it.
merge-gate/codex The maintainer's Codex review of the head has nothing open.
  • .github/scripts/merge-gate.sh
    • evaluate posts what GitHub data settles and pending with a reason for the rest. The owner's newest success or failure on evidence, config or codex always wins. The changed files come from comparing main with the head commit, not from the pull request.
    • post lets the maintainer set codex, evidence or config, only on the current head, then starts the workflow for that pull request.
    • check prints every gate. It works each gate out again from GitHub's data, so a removed label, a lower Greptile score or the owner's newer failure fails before the workflow catches up. It also checks what GitHub doesn't: that each CI job's newest run tested a merge with the current main, and that a ruleset the person running it can't bypass requires every check. For each required check it counts what GitHub counts, including a status that shares a check run's name. Names match in any case, as GitHub matches status names, and the newest check run under each spelling must pass, so a passing Mac-Mini can't hide a failing mac-mini. Last, if every row passed, it reads the pull request, the gates and main again and fails if any changed during the check.
    • merge refuses unless check passes, then merges pinned to the checked head. GitHub still enforces every required check at the merge. What it can't see is a change the workflow hasn't posted yet, so a label removed, a lower Greptile score or a main that moved in the few seconds between the check's last reads and the merge goes unseen.
    • rerun toggles draft so CI tests against the current main without a push.
  • .github/workflows/merge-gate.yml runs evaluate on pull_request_target events, hourly as a safety net, and on dispatch. Every run for one pull request shares a concurrency group, so two runs never interleave. It checks out main and reads pull requests through the API only. It never checks out or runs a pull request's code.
  • ci.yml
    • Each job writes a "Tested merge" note naming the main and head commits it tested; check reads it.
    • CI runs on ready_for_review too (for rerun).
    • ShellCheck and the bash 3.2 checks cover .github/scripts.
    • New steps run the merge gate tests under macOS /bin/bash 3.2 and Linux bash 5.
    • The release build and the lid check now run before the tests, so no pull request code has run in the job yet when they do. Test code could otherwise skip them, for example by writing BASH_ENV to $GITHUB_ENV.
    • zizmor's install runs python3 -I, so a venv.py on the branch can't stand in for Python's.
    • The ShellCheck job fails unless every script starts with #!/bin/bash or #!/usr/bin/env bash, and lints with --shell=bash, so a changed shebang can't switch ShellCheck to another shell's rules.
    • The ShellCheck job installs ShellCheck 0.11.0, checked by its sha256, and fails unless the findings that directives hide in scripts/ match .github/shellcheck-hidden.txt, which counts as config. A directive covers the next command, so a { after an existing one and a } at the end of the file would put the rest of the script under it without touching the directive.
  • .github/scripts/merge-gate-test.sh and merge-gate-fake-gh.sh: 74 tests that run the gate against a stand-in for gh that answers from files. They cover pagination as gh does it, and answers that change between reads.
  • .github/scripts/shellcheck-hidden.sh makes that list. It lints a copy of scripts/ in which each # that whitespace and shellcheck follow becomes #~, so no directive survives, and prints each finding with its file, check and line. The copy must parse as the script does, or a changed line could end a here-document early and hide the code after it, so bash 5.2's --pretty-print, which parses without running anything and drops comments, must print the copy and the script the same. A # shellcheck in a string, a here-document or backticks fails the check. shellcheck-hidden-test.sh runs it on 29 cases, among them a directive in each place ShellCheck honors one (after ;, &, (, a case pattern's ), inside $(), a here-document trick in each direction, a .shellcheckrc, SHELLCHECK_OPTS, $BASH_ENV, a bash older than 5.2 and a ShellCheck that fails. Each directive case first checks that its directive is what hides the finding: ShellCheck must report it once the directive names another check.
  • CONTRIBUTING.md: "How a pull request merges" and "Real-Mac evidence". Also a new pull request template.

How it was tested

Real-Mac evidence

Not needed: only .github/ and CONTRIBUTING.md change.

🤖 Generated with Claude Code

RetriggerConfidence Score: 5/5

The reviewed changes appear safe to merge; no new blocking issue was found.

Summary

This PR adds five merge gates for review, owner acceptance, Mac evidence, config approval and Codex review.

  • The merge gate posts five checks for each pull-request head.
  • Maintainers can check every gate and merge at the checked head.
  • CI spots ShellCheck warnings hidden by script directives.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Pull request head] --> B[CI tests merged code]
  A --> C[Read review, label, files and owner decisions]
  C --> D[Post five gate statuses]
  B --> E[check]
  D --> E
  E --> F{All checks pass on current main?}
  F -->|No| G[Refuse merge]
  F -->|Yes| H[Read head, gates and main again]
  H --> I[Merge pinned to checked head]
Loading

Reviews (18) · Last reviewed commit: "Merge gate: check that the copy without ..." · Reviewed by Greptile

…ch head

A pull request into main should only merge when every check passes at
its head commit. Today main requires no status at all, so this adds the
pieces the ruleset change will require:

- .github/scripts/merge-gate.sh posts and reads five merge-gate/*
  statuses: greptile (5/5 at the head, read from Greptile's own
  description edits, so an author-typed block never counts), accepted
  (the owner's label), evidence (a real-Mac report the maintainer
  checked, or a docs/tests/tooling-only change), config (changes to
  .github/, .greptile/ or Package.swift need the owner's OK, since a
  fork's CI runs its own workflow files) and codex (the maintainer's
  review). Its check command prints every gate, including that CI ran
  after main's newest commit and that the ruleset requires each check;
  merge refuses unless check passes.
- .github/workflows/merge-gate.yml runs evaluate on pull_request_target
  events, hourly and on dispatch. It checks out main and reads pull
  requests through the API only.
- CI also runs on ready_for_review, so `merge-gate.sh rerun` can test a
  pull request against the current main without a push, and ShellCheck
  covers .github/scripts.
- CONTRIBUTING.md says how a pull request merges and what a real-Mac
  report holds; a pull request template asks for one.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A throwaway copy of main's rulesets on #107 showed that plain merges
fail because of the "only krishhgg may merge" update rule, which the
owner bypasses only through --admin, and that --admin does not get past
a required status or an open conversation in the ruleset with no bypass
list.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Comment thread .github/scripts/merge-gate.sh Outdated
Comment thread .github/scripts/merge-gate.sh Outdated
Comment thread .github/scripts/merge-gate.sh Outdated
Comment thread .github/scripts/merge-gate.sh Outdated
Comment thread .github/workflows/ci.yml Outdated
krishhgg and others added 4 commits October 12, 2026 15:18
- Greptile: count a score only from greptile-apps' newest edit, and only
  if that description holds one score and one reviewed commit, so text
  the author adds cannot supply it.
- Owner decisions: the owner's newest success or failure on evidence,
  config or codex wins on every path, and comes back if a run posted
  over it.
- accepted: the newest add or removal of the label must be the owner
  adding it.
- Files come from comparing main with the head commit, renames count
  both names, and 300 or more files assume the worst.
- The workflow lists pull requests, then evaluates each in its own
  concurrency group, so runs for one pull request never interleave.
- CI writes a "Tested merge" note naming the main and head it tested;
  check needs exactly one on each job's newest run, naming the current
  main, and points at `merge-gate.sh rerun`, which starts CI again
  without a push.
- check reads every page, counts a gate success only from the workflow
  or the owner, evaluates extra required checks, and counts a rule only
  if it comes from a ruleset whoever runs check cannot bypass.
- merge-gate-test.sh runs 36 cases against a stand-in for gh; CI runs
  them with /bin/bash 3.2 on macOS and bash 5 on Linux, and the macOS
  syntax and bash 4 checks now cover .github/scripts.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
check now works out every gate again from GitHub's data and fails a
posted success that no longer holds: a removed label, a lower Greptile
score or the owner's failure posted while a workflow run was reading.
post starts the workflow after posting, so a run that read the statuses
before the post cannot leave its result on top.

check and merge read main again at the end and stop if it moved.

Every required context and app pair from the rules is evaluated: the
newest check run from each app and, for an unpinned context, the newest
status must all pass, as GitHub counts them. Greptile edits that share
the newest time and disagree wait for the next edit. A status from a
deleted account no longer shifts the tab-separated fields.

The fake gh now models GraphQL pagination and answers that change
between reads. 48 tests pass under /bin/bash 3.2, and each fix has a
test that fails without it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
check's reads take several seconds, so a label removed, a lower
Greptile score or an owner failure posted after it worked out the gates
went unseen. When every row passes, check now reads the head, the gates
and main once more, last, and fails if any changed. merge relies on
that, which leaves only the second until GitHub merges.

Every required check's newest status must pass too, also for the CI
names and app-pinned contexts: GitHub needs both a check run and a
status with one name to pass, and a status does not say which app
posted it.

Status fields are read with tabs and line breaks as spaces and nothing
escaped, so a restored owner description no longer posts again on every
run.

The fake gh works out a commit's combined status from its statuses. 54
tests pass under /bin/bash 3.2, and each fix has a test that fails
without it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
GitHub treats status contexts in any case as one, so a failure posted
as MERGE-GATE/evidence or "shellcheck scripts" slipped past exact
comparisons. Gate statuses and the combined status now read in lower
case, and a required check's runs match in any case, though only a run
with the exact name counts as there.

check's last read now also fails if the pull request turned draft,
changed base or lost its clean merge during the check.

The comments now say what is left: GitHub enforces every required
check at the merge, and only a change the workflow has not posted yet,
made in the few seconds of check's last reads and the merge, goes
unseen.

59 tests pass under /bin/bash 3.2, and each fix has a test that fails
without it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Comment thread .github/scripts/merge-gate.sh Outdated
krishhgg and others added 10 commits October 12, 2026 17:01
The fifth Codex review found two ways around the config gate: Greptile
reads a .greptile/ folder at any depth, and SwiftPM builds from a
Package@swift-<version>.swift that matches the toolchain. Both now
need the owner's OK, as do greptile.json anywhere, the lid gate script
CI runs, a .shellcheckrc anywhere, and zizmor.yml.

A test's jq used a negative array index, which CI's bash 4 scan
flagged, stopping both macOS jobs; it now picks the check run by id.

60 tests pass under /bin/bash 3.2.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The sixth Codex review found two more ways around the config gate.
zizmor skips what .gitignore lists, so a fork could hide merge-gate.yml
from the lint job. And the Mac runners check out to a case-insensitive
disk, where [email protected] satisfies SwiftPM's lookup and a case
variant of any config file lands on the real one.

is_config now matches paths in any case and also counts .gitignore,
.ignore and .gitattributes anywhere (the last can hide files from a
reviewer's diff as generated). CONTRIBUTING lists them.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The seventh Codex review found two more ways around the config gate.
The Mac runners' disk folds Unicode as well as case, so scripts/
spelled with U+017F (a long s) lands on the real lid gate script; any
name with a byte outside printable ASCII now counts as config. And the
lint job ran python3 -m venv from the checkout, where a venv.py on the
branch would load instead of venv and fake the zizmor install; it now
runs python3 -I, which keeps the checkout off Python's path.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Codex rounds 5 to 7 each found another file that steers a check, so the
config gate is now an allowlist: Swift sources and tests, docs, skills,
the product's scripts and resources, and the root README, CONTRIBUTING,
SECURITY and LICENSE pass, and anything else waits for the owner. Within
those, a dot-named path, an AGENTS.md, CLAUDE.md or GEMINI.md, and the
lid gate script still count.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- A diff that adds a ShellCheck directive, or a Swift @warn or @diagnose
  attribute, counts as config, as does a changed file whose diff GitHub
  leaves out. A control character in a name now counts too.
- docs/spec.md, SECURITY.md, CONTRIBUTING.md and docs/release-validation.md,
  which .greptile/files.json gives Greptile as guidance for every review,
  count as config; a test checks every unscoped files.json path.
- CI runs the release build and the lid check before the tests, so test
  code cannot skip them through $GITHUB_ENV or by editing the checkout.
- check reads the newest check run under each spelling of a required
  name, so a passing Mac-Mini cannot hide a failing mac-mini.
- CONTRIBUTING lists AGENTS.override.md and CLAUDE.local.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- Any file GitHub sends no diff for counts as config unless it is an
  image or was removed: one NUL byte makes a script binary, and GitHub
  then reports it with no diff and no changed lines.
- The Swift warning control check matches @diagnose and @warn in any
  spelling Swift accepts, such as @`diagnose`, and without arguments.
- docs/spec.md and docs/release-validation.md go back on the allowlist:
  contributors are meant to update them with their changes, and counting
  them sent 8 of 13 passing open pull requests to the owner. SECURITY.md
  and CONTRIBUTING.md still count.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
ShellCheck picks its rules from a script's first line, so a pull request
that changed backstop.sh to #!/bin/ksh was linted as ksh while the
LaunchAgent runs it with /bin/bash (the tenth Codex review of #106). The
lint job now fails unless every script starts with #!/bin/bash or
#!/usr/bin/env bash, and runs ShellCheck with --shell=bash.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Codex's eleventh review: renaming scripts/backstop.sh to
scripts/backstop.SH passed the config gate, since is_config lowercased
names before the allowlist. CI's scripts/*.sh globs skip the new name,
so it escaped the shebang check, ShellCheck and the Mac syntax checks,
while the Mac's case-insensitive disk still opens it as backstop.sh.

The allowlist now matches names exactly, case included. The names that
count as config still match in any case. The patchless image check
drops its ascii_downcase to match.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Codex's twelfth review: scripts/unchecked.sh as a symlink to a binary
docs/payload.png passed the config gate. Both names were on the
allowlist, the image needed no diff, and CI followed the link and linted
the image's hidden script with its "# shellcheck disable=all".

The gate now reads the head's tree and counts a changed symlink (mode
120000) or submodule (160000) as config, and assumes the worst if
GitHub cuts the tree short. Images leave the allowlist and any file
without a diff counts unless removed: no reviewer can read one, and a
script could source it.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Codex's thirteenth review found that a pull request could widen an
existing ShellCheck directive without touching it: a "{" right after
install.sh's SC2016 directive and a "}" at the end put the rest of the
script under it, and the gate only looks for added directives.

The ShellCheck job now lists every finding a directive hides in
scripts/, by linting a copy in which every comment starts "#~", and
fails unless the list matches .github/shellcheck-hidden.txt, which
counts as config. ShellCheck is pinned to 0.11.0, checked by sha256,
since each version finds different things.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Comment thread .github/scripts/shellcheck-hidden-test.sh Outdated
krishhgg and others added 3 commits October 12, 2026 23:50
Codex's fourteenth review found that each case's check that ShellCheck
really hides the finding piped ShellCheck into grep under pipefail.
ShellCheck exits 1 when it reports anything, so the check never failed,
and a case whose directive hid nothing still passed.

Each case now lints its script twice, as written and with the directive
naming another check, and fails unless only the second reports the
finding. That showed that a directive right after "|" or "&&" is a parse
error, not a hidden finding, so those cases go, and that one after a
background "&" is honored, so that case comes in.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The list read each line from the copy whose comments start "#~", so a
line with a comment showed "#~" where the script has "#". Contributors
copy this list into .github/shellcheck-hidden.txt, so it now reads the
line from the script itself, which has the same line numbers. The list
on main has no comments, so it stays the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Codex's fifteenth review found that rewriting every "#" that starts a
comment could change how the copy parses: a line "#E" inside a
here-document that ends at "#~E" became "#~E" in the copy, ended it
early, and let a second here-document swallow code that a widened
directive hid in the real lint. The list then stayed the same.

The copy now changes only a "#" that whitespace and "shellcheck" follow,
and bash 5.2's --pretty-print, which parses without running anything
and drops comments, must print the copy and the script the same. A
"# shellcheck" in a string, a here-document or backticks fails the
check. bash reads $BASH_ENV even with --pretty-print, so the script
unsets it, and it refuses a bash older than 5.2.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@krishhgg
krishhgg merged commit b6a7499 into main Oct 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant