Skip to content

fix(automation): react on review-body mentions via GraphQL - #990

Closed
seonghobae wants to merge 12 commits into
mainfrom
fix/mention-router-review-body-reaction
Closed

fix(automation): react on review-body mentions via GraphQL#990
seonghobae wants to merge 12 commits into
mainfrom
fix/mention-router-review-body-reaction

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Trusted maintainers invoke @cwl-noema-review from a submitted review body. Those mentions already dispatch and react through GraphQL addReaction. A second mention on the same review received Reaction already exists, which looked like a failed eyes reaction.

This increment treats an already-reacted GraphQL error as success and requires data.addReaction before claiming the optional reaction landed. An empty GraphQL body is a warning after dispatch has already been queued. Issue comments and review comments keep their REST reaction paths. The existing receipt issue comment still posts.

Verification

  • Already-reacted GraphQL errors on addReaction return true. Empty {} or missing data.addReaction return false and warn. Mixed already-reacted plus a real error stays a warning.
  • coverage run -m pytest tests && coverage report --show-missing twice at 100% (989 passed, scripts/ci 100% statement/branch), then interrogate 100%.

@cwl-noema-review

Trusted @cwl-noema-review and @opencode-agent mentions on pull-request
review comments and submitted reviews were ignored because parse_event
required issue.pull_request. Accept those surfaces, sweep them, drop the
case-sensitive workflow body filter, and require a per-changed-file
review walk with staged compute allocation.
Live run 31670687388 queued @cwl-noema-review for #954 then failed the
job on POST .../reactions with 403 Resource not accessible by
integration, so no receipt was posted. Treat the reaction as optional,
give the local job pull-requests: write for conversation receipts, and
keep durable dispatch evidence authoritative.
Record that an exceptional eyes-reaction response cannot be treated as a
missed dispatch. Force the trusted-uv installer tests onto the linux
x86_64 runner path and add the control-plane architecture diagram.
Live router run 31672030631 queued Noema for #9560c253f0 and then
failed OpenCode with HTTP 422: repository_dispatch allows 10
client_payload properties and the review-only flags made 14. Nest
those flags under review_contract, bind the wrapper to that object,
and leave the flat invocation-key claim unchanged.
Doctoring already required skipping pending reviews, but parse_event
and the sweep treated a dismissed body with submitted_at as a live
mention. Dispatch only submitted, non-dismissed review bodies.
route-local-agent-mention on main dies after a successful dispatch
because GITHUB_TOKEN lacks the reactions permission (HTTP 403).
Keep the reaction non-fatal and declare the write so the eyes
receipt can actually appear.
Inline @cwl-noema-review mentions now POST /pulls/comments/{id}/reactions.
A 403 there is a warning after dispatch. Submitted review bodies still
have no REST reaction endpoint.
Submitted @cwl-noema-review review bodies have no REST reaction
endpoint. Resolve the review node_id and addReaction EYES. A 403 or
GraphQL error is a warning after dispatch.
@seonghobae
seonghobae enabled auto-merge (squash) August 13, 2026 11:05
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 00a20c61-a439-4b7b-bf1d-e9b30ab85f6f

📥 Commits

Reviewing files that changed from the base of the PR and between 6eb06cd and 65196cc.

📒 Files selected for processing (22)
  • .github/workflows/agent-mention-opencode-dispatch.yml
  • .github/workflows/agent-mention-router.yml
  • .gitignore
  • AGENTS.md
  • ARCHITECTURE.md
  • CHANGELOG.md
  • CLAUDE.md
  • ci-review-prompt.md
  • code-reviewer-prompt.md
  • docs/automation/review-agent-comment-invocation.md
  • docs/doctoring/review-agent-mention-surfaces.md
  • scripts/ci/agent_mention_router.py
  • scripts/ci/agent_mention_sweep.py
  • scripts/ci/materialize_base_python_requirements.py
  • scripts/ci/opencode_review_prompt_template.md
  • tests/test_agent_mention_complete_payload_binding.py
  • tests/test_agent_mention_idempotency.py
  • tests/test_agent_mention_router.py
  • tests/test_agent_mention_sweep.py
  • tests/test_agent_mention_workflow_contract.py
  • tests/test_materialize_base_python_requirements.py
  • tests/test_opencode_agent_contract.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Submitted review payloads already carry node_id. GraphQL addReaction
must use that id and skip an extra REST GET before the eyes reaction.
GraphQL addReaction on a submitted review body is success only when
data.addReaction lands or every error is already-reacted. An empty
GraphQL body is a warning after dispatch, not a missed mention.
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review

1 similar comment
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review

@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review exact current head d6cfb1d3 — review-body mention GraphQL eyes; drop invalid reactions:write mention scope. Head moved after e87085f2. Independent Noema APPROVE on this SHA is required. Do not wait on two-approval. Reviewers remain edit: deny. NVIDIA_NIM_API_KEY only.

Materialize a base Python lock only when every package line is an exact
SHA-256 pin or a two-token relative -r/--requirement include of a
candidate lock path. A lone --require-hashes directive, ./dotted paths,
and -r other-hashes.txt no longer enter the trusted build context.
@seonghobae
seonghobae force-pushed the fix/mention-router-review-body-reaction branch from d6cfb1d to aadefbb Compare August 13, 2026 18:38
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review exact current head aadefbb7 — review-body mention GraphQL eyes; drop invalid reactions:write; coverage accepts only bounded relative requirement includes. Head moved after d6cfb1d3. Independent Noema APPROVE on this SHA is required. Do not wait on two-approval. Reviewers remain edit: deny. NVIDIA_NIM_API_KEY only.

Review-body mention heads still declared a non-existent GITHUB_TOKEN
scope. Issue-comment eyes use issues: write; review-comment eyes use
pull-requests: write. NVIDIA_NIM_API_KEY only.
@seonghobae

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review exact current head 65196ccd — review-body mention GraphQL eyes; drop invalid reactions:write. Head moved after aadefbb7. Independent Noema APPROVE on this SHA is required. Do not wait on two-approval. Reviewers remain edit: deny. NVIDIA_NIM_API_KEY only.

Copy link
Copy Markdown
Contributor Author

Closing as superseded by canonical cumulative #991. #991 retains submitted-review GraphQL eyes, treats only wholly already-reacted errors as idempotent success, rejects empty/mixed error payloads, and additionally reuses a validated webhook/sweep node_id before bounded REST lookup. Keep one clean cumulative router PR.

@seonghobae seonghobae closed this Aug 14, 2026
auto-merge was automatically disabled August 14, 2026 09:18

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant