Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8cec0a2
fix(review): fail closed when required check is not a verdict
seonghobae Aug 14, 2026
24d144c
fix(noema): confine CodeGraph context path to the workspace
seonghobae Aug 14, 2026
e9fc895
test(review): prove draft PRs skip central OpenCode dispatch
seonghobae Aug 14, 2026
45fc61d
fix(ci): use valid checkout pin for draft review repair
seonghobae Aug 14, 2026
ae8251b
fix(review): dispatch OpenCode for draft pull requests
github-actions[bot] Aug 14, 2026
99c4381
docs(review): record verified draft dispatch contract
seonghobae Aug 14, 2026
8a2ee80
test(noema): prove existing review cannot bypass primary approval
seonghobae Aug 14, 2026
7a43aca
test(noema): cover approved existing-review branch
seonghobae Aug 14, 2026
a67aa6d
fix(noema): require primary approval before existing verdict
github-actions[bot] Aug 14, 2026
a578df5
docs(review): record Noema approval-order verification
seonghobae Aug 14, 2026
f0dcf6d
test(review): reproduce repository dispatch payload overflow
seonghobae Aug 14, 2026
fa558eb
chore(review): remove superseded payload repair workflow
seonghobae Aug 14, 2026
82d9552
test(review): prove LineageWeave remains centrally routable
seonghobae Aug 14, 2026
2de7a3c
fix(ci): publish verified LineageWeave route with workflow token
seonghobae Aug 14, 2026
9b056c5
test(review): export verified LineageWeave route patch
seonghobae Aug 14, 2026
3ddfc82
chore(review): remove superseded one-shot route repair
seonghobae Aug 14, 2026
0fbede0
chore(review): remove superseded route publisher
seonghobae Aug 14, 2026
afba545
chore(review): remove superseded route patch exporter
seonghobae Aug 14, 2026
d9d6a3c
chore(review): remove unrelated materializer test changes
seonghobae Aug 14, 2026
27903c9
fix(noema): fail closed on drafts without primary approval
seonghobae Aug 16, 2026
d5fb65e
merge(main): resolve changelog conflict with hourly repair
seonghobae Aug 16, 2026
801820d
fix(review): spend dispatch budget on empty Reviews tabs first
seonghobae Aug 16, 2026
fe7a5e0
fix(review): treat GitHub run-name as an in-progress OpenCode dispatch
seonghobae Aug 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 40 additions & 3 deletions .github/workflows/opencode-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ concurrency:

permissions:
contents: read
pull-requests: read

jobs:
required-workflow-bootstrap:
Expand Down Expand Up @@ -53,7 +54,43 @@ jobs:
name: opencode-review
needs: [coverage-evidence]
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
steps:
- run: >-
echo "Review approval remains a separate current-head PR review
requirement produced by the authenticated dispatch workflow."
- name: Fail closed without a current-head OpenCode verdict
env:
GH_TOKEN: ${{ github.token }}
TARGET_REPOSITORY: ${{ github.event.pull_request.base.repo.full_name || github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
if [ "${{ github.event.action }}" = "closed" ]; then
echo "PR closed; a current-head OpenCode verdict is not required."
exit 0
fi
if [ -z "${PR_NUMBER:-}" ] || [ -z "${HEAD_SHA:-}" ]; then
echo "::error::Missing PR number or head SHA; cannot verify a current-head OpenCode verdict."
exit 1
fi
reviews="$(gh api --paginate "repos/${TARGET_REPOSITORY}/pulls/${PR_NUMBER}/reviews")"
verdict="$(printf '%s\n' "$reviews" | jq -s --arg sha "$HEAD_SHA" '
(add // [])
| [
.[]
| select(
(.user.login // "" | ascii_downcase) as $user
| $user == "opencode-agent" or $user == "opencode-agent[bot]"
)
| select((.commit_id // "" | ascii_downcase) == ($sha | ascii_downcase))
| .state
]
| map(select(. == "APPROVED" or . == "CHANGES_REQUESTED"))
| first // empty
')"
if [ -z "$verdict" ]; then
echo "::error::No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. This required check is not a review and must not succeed until the authenticated dispatch posts a current-head verdict."
exit 1
fi
echo "Current-head OpenCode verdict: ${verdict}."
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ Semantic Versioning where the repository publishes a release.

### Fixed

- Noema now validates the current-head primary OpenCode approval before accepting an existing Noema verdict, preventing a secondary review from making the required gate look successful on its own.
- Draft pull requests now receive same-head Strix and OpenCode review dispatches while remaining excluded from branch updates, auto-merge changes, direct merge, and review-state cleanup.
- The required `opencode-review` check now fails closed unless `opencode-agent` already posted `APPROVED` or `CHANGES_REQUESTED` on the current head, so a stub success can no longer look like a review (ContextualWisdomLab/contextual-orchestrator#176).
- The merge scheduler now spends its review-dispatch budget on pull requests with no OpenCode verdict on any commit before leftover increments that already have a previous-head APPROVED or CHANGES_REQUESTED, so one-dispatch-per-run no longer starves an empty Reviews tab.
- The scheduler treats GitHub's full `run-name` (`OpenCode Review Dispatch owner/repo#N@sha`) as an in-progress same-head dispatch, so a later sweep cannot `cancel-in-progress` a review that already passed coverage.
- Noema no longer exits 0 when the current head has no primary OpenCode approval, including on draft pull requests; that skip was the green `noema-review` check with an empty Reviews tab.
- `load_codegraph_context` now confines `NOEMA_CODEGRAPH_CONTEXT_PATH` to `GITHUB_WORKSPACE` (or cwd) with `..` rejection and realpath checks, so a Strix path-traversal report on that helper cannot read files outside the review workspace.
- Materialized base Python locks only when every package line is an exact SHA-256 pin or a bounded relative `-r`/`--requirement` include. A lone `--require-hashes` directive, a dotted include such as `./lock.txt`, or `-r other-hashes.txt` no longer enters the trusted build context.
- Refused a conflict-scope repository root whose immediate parent is a symbolic link, so a swapped parent cannot redirect the canonical worktree after the last-component check (CWE-367).
- Bounded the Strix quality self-test's deterministic timeout fixtures to 3-second process and 5-second fake-sleep budgets so exact-head policy evidence completes inside the existing job limit without changing production Strix scanner timeouts, providers, credentials, or review semantics.
Expand Down
93 changes: 93 additions & 0 deletions docs/doctoring/required-review-check-is-not-a-verdict.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Required OpenCode/Noema checks are not reviews

검토 기준일: **2026-08-14**

## Incident

On ContextualWisdomLab/contextual-orchestrator#176 the required
`opencode-review` and `noema-review` checks were green, but the Reviews
API had no APPROVE or REQUEST_CHANGES. Authors treated the check name as
a review verdict (GitHub, n.d.-a). That is weaker than the modern-review
expectation that a review is an explicit, current-head judgment
(Bacchelli & Bird, 2013).

## Decision

The required `opencode-review` job on
`.github/workflows/opencode-review.yml` never runs the model. Privileged
review stays in `opencode-review-dispatch.yml`. The required job now
reads current-head reviews with `pull-requests: read` and **fails
closed** unless `opencode-agent` / `opencode-agent[bot]` already posted
`APPROVED` or `CHANGES_REQUESTED` on that SHA. A COMMENTED review, a
review on an old SHA, or no review at all cannot make the check green.

`scripts/ci/noema_review_gate.py` no longer returns 0 when the current
head has no primary OpenCode approval. That skip was exit 0, so the
required `noema-review` check looked like a successful review. Draft
status is checked only after that primary-approval gate, so a draft
without an OpenCode verdict cannot turn `noema-review` green. The gate
also validates the primary approval before accepting an existing Noema
review; a secondary verdict cannot independently turn the required gate
green.

Human `repository_dispatch` as `seonghobae` remains rejected; only
`github-actions[bot]` may start the privileged dispatch. After a real
verdict is posted, re-run the required `opencode-review` job so the
fail-closed check can observe it.

The one-dispatch-per-run budget used to walk pull requests in created-at
order, so leftover increments that already had a previous-head verdict
consumed the slot while a later PR with an empty Reviews tab waited. The
scheduler now stable-sorts that budget: no OpenCode APPROVED or
CHANGES_REQUESTED on any commit first, then previous-head re-reviews,
then current-head verdicts. COMMENTED-only evidence is not a verdict and
keeps the empty-Reviews priority.

A second same-head `repository_dispatch` used to cancel the first through
workflow `cancel-in-progress` because `active_review_run_refs` compared
the GitHub `name` field to the short alias `OpenCode Review Dispatch`.
Live runs set `name` to the interpolated run-name. The matcher now
accepts that prefix so a queued or in-progress same-head review is
`already_running`.

## Draft pull-request review contract

Draft status is a merge-readiness signal, not a request to suppress early
feedback. The central scheduler therefore dispatches same-head Strix first
and then authenticated OpenCode review for draft pull requests. The draft
path is deliberately review-only: it cannot update the head branch, enable
or disable auto-merge, merge, dismiss reviews, or resolve review threads.
Marking a pull request ready remains the explicit boundary for merge
automation.

## Verification contract

- `tests/test_opencode_required_verdict_gate.py` pins
`current_head_opencode_verdict` and `decide_required_verdict_check`.
- `tests/test_noema_review_gate.py` requires exit 1 when there is no
primary OpenCode approval, even when the current head already has a Noema
review, while retaining the idempotent success path after a valid primary
approval exists.
- `tests/test_opencode_agent_contract.py` pins the required workflow
fail-closed error string.
- `tests/test_pr_review_merge_scheduler.py` proves that a draft pull request
receives same-head Strix and OpenCode dispatch while branch updates,
auto-merge mutation, direct merge, review dismissal, and thread cleanup
remain unreachable, and that a never-reviewed pull request consumes the
one-dispatch budget before a leftover increment that already has a
previous-head OpenCode verdict.
- Both repairs were exercised test-first: the draft-dispatch contract failed
against the old unconditional skip, and the Noema ordering contract failed
against the old secondary-review-first branch. The exact repaired source
then passed 988 tests, 7,056 production statements, 2,834 production
branches, and the public-docstring gate at 100%.

## References (APA 7th)

Bacchelli, A., & Bird, C. (2013). Expectations, outcomes, and challenges of
modern code review. In *Proceedings of the 35th International Conference on
Software Engineering* (pp. 712–721). IEEE.
https://doi.org/10.1109/ICSE.2013.6606617

GitHub. (n.d.-a). *About status checks*. GitHub Docs. Retrieved
August 14, 2026, from https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks
45 changes: 39 additions & 6 deletions scripts/ci/noema_review_gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import urllib.parse
import urllib.request
from collections.abc import Sequence
from pathlib import Path
from typing import Any


Expand Down Expand Up @@ -380,13 +381,36 @@ def review_thread_context(pr: dict[str, Any]) -> str:
return "\n".join(lines)


def codegraph_context_root() -> Path:
"""Return the workspace root that may contain CodeGraph context files."""
raw = os.environ.get("GITHUB_WORKSPACE", "").strip() or os.getcwd()
return Path(raw).resolve()


def confined_codegraph_context_path(path: str, root: Path) -> Path | None:
"""Return the resolved path when it cannot escape the workspace root."""
candidate = Path(path)
if ".." in candidate.parts:
return None
try:
resolved = candidate.resolve()
except OSError:
return None
if not resolved.is_relative_to(root):
return None
return resolved


def load_codegraph_context() -> str:
"""Load optional precomputed CodeGraph context for structural review evidence."""
path = os.environ.get("NOEMA_CODEGRAPH_CONTEXT_PATH", "").strip()
if not path:
return ""
confined = confined_codegraph_context_path(path, codegraph_context_root())
if confined is None:
return "CodeGraph context unavailable: path escapes the workspace."
try:
with open(path, encoding="utf-8") as handle:
with confined.open(encoding="utf-8") as handle:
return truncate_text(handle.read(), MAX_REVIEW_CONTEXT_CHARS)
except OSError as exc:
return f"CodeGraph context unavailable: {exc}"
Expand Down Expand Up @@ -582,7 +606,12 @@ def submit_review(repo: str, number: int, pr: dict[str, Any], actor: str, verdic


def inspect_and_review(repo: str, number: int) -> int:
"""Inspect PR state and submit Noema's LLM review when gates are clean."""
"""Inspect PR state and submit Noema's LLM review when gates are clean.

Missing current-head primary OpenCode approval fails closed, including
on draft pull requests, so the required check cannot look reviewed
without a Reviews-tab verdict.
"""
pr = fetch_pr(repo, number)
actor = current_actor()
if actor in PRIMARY_REVIEW_AUTHORS:
Expand All @@ -591,15 +620,19 @@ def inspect_and_review(repo: str, number: int) -> int:
"Noema review skipped so GitHub receives an independent reviewer."
)
return 0
if not current_primary_approval(pr):
print(
"Current head does not have a primary OpenCode approval; "
"Noema cannot skip as success because that made the required "
"check look like a review."
)
return 1
if pr.get("isDraft"):
print("PR is draft; Noema review skipped.")
print("PR is draft; Noema review skipped after primary OpenCode approval.")
return 0
if existing_noema_review(pr, actor):
print("Current head already has a Noema review; nothing to do.")
return 0
if not current_primary_approval(pr):
print("Current head does not have a primary OpenCode approval; Noema review skipped.")
return 0
if has_current_changes_requested(pr):
print("Current head has requested changes; Noema review skipped.")
return 0
Expand Down
75 changes: 71 additions & 4 deletions scripts/ci/opencode_dispatch_status.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,58 @@
)


OPENCODE_VERDICT_STATES = frozenset({"APPROVED", "CHANGES_REQUESTED"})
MISSING_VERDICT_MESSAGE = (
"No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. "
"This required check is not a review and must not succeed until the "
"authenticated dispatch posts a current-head verdict."
)


def current_head_opencode_verdict(
reviews: Sequence[dict[str, Any]], head_sha: str
) -> str | None:
"""Return the latest current-head OpenCode APPROVED or CHANGES_REQUESTED state."""
expected = (head_sha or "").lower()
if not expected:
return None
for review in reversed(reviews):
author = str((review.get("user") or {}).get("login") or "").casefold()
if author not in OPENCODE_APP_APPROVAL_AUTHORS:
continue
if str(review.get("commit_id") or "").lower() != expected:
continue
state = str(review.get("state") or "").upper()
if state in OPENCODE_VERDICT_STATES:
return state
return None


def decide_required_verdict_check(
*,
expected_head: str,
pull_request: dict[str, Any],
reviews: Sequence[dict[str, Any]],
) -> dict[str, str]:
"""Fail closed unless OpenCode already published a current-head verdict."""
live_head = str((pull_request.get("head") or {}).get("sha") or "")
if not expected_head or live_head.lower() != expected_head.lower():
return {
"state": "failure",
"description": (
"OpenCode required-check target is stale or the live PR head "
"is unavailable."
),
}
verdict = current_head_opencode_verdict(reviews, expected_head)
if verdict is None:
return {"state": "failure", "description": MISSING_VERDICT_MESSAGE}
return {
"state": "success",
"description": f"Current-head OpenCode verdict: {verdict}.",
}


def _has_current_approval(reviews: Sequence[dict[str, Any]], head_sha: str) -> bool:
"""Return whether the latest OpenCode decision is a verified approval."""
for review in reversed(reviews):
Expand Down Expand Up @@ -67,23 +119,38 @@ def decide_status(


def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
"""Parse commit-status evidence inputs."""
"""Parse commit-status or required-verdict evidence inputs."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--model-outcome", required=True)
parser.add_argument("--coverage-result", required=True)
parser.add_argument(
"--mode",
choices=("dispatch-status", "required-verdict"),
default="dispatch-status",
)
parser.add_argument("--model-outcome")
parser.add_argument("--coverage-result")
parser.add_argument("--expected-head", required=True)
parser.add_argument("--pull-request-file", required=True, type=Path)
parser.add_argument("--reviews-file", required=True, type=Path)
return parser.parse_args(argv)


def main(argv: Sequence[str] | None = None) -> int:
"""Print one JSON commit-status decision."""
"""Print one JSON decision and exit 1 when the required verdict is missing."""
args = parse_args(argv)
pull_request = json.loads(args.pull_request_file.read_text(encoding="utf-8"))
reviews = json.loads(args.reviews_file.read_text(encoding="utf-8"))
if not isinstance(pull_request, dict) or not isinstance(reviews, list):
raise SystemExit("pull request evidence must be an object and reviews evidence an array")
if args.mode == "required-verdict":
decision = decide_required_verdict_check(
expected_head=args.expected_head,
pull_request=pull_request,
reviews=reviews,
)
print(json.dumps(decision, separators=(",", ":")))
return 0 if decision["state"] == "success" else 1
if not args.model_outcome or not args.coverage_result:
raise SystemExit("--model-outcome and --coverage-result are required")
print(
json.dumps(
decide_status(
Expand Down
Loading
Loading