Skip to content

fix: preserve incomplete reviews and orchestrate bounded coverage - #47

Merged
AppSolves merged 4 commits into
mainfrom
fix/bounded-review-orchestration
Nov 15, 2026
Merged

AppSolves merged 4 commits into
mainfrom
fix/bounded-review-orchestration

Conversation

@AppSolves

@AppSolves AppSolves commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Valid incomplete model reviews previously aborted the isolated engine before the trusted host could retain their summary/findings. Preserve those validated partial results while keeping the review job failed and publication strictly COMMENT. Invalid, quarantined, wrongly scoped or unverifiable outputs cannot approve or merge. Preserve primary execution/proxy failures and earlier safe findings when later work fails.

The trusted host creates immutable evidence groups (16 files / 2 MB, at most 32), retains all shared read-only source/dependencies and binds results to original units and input hashes. One pinned Codex thread executes the groups with at most two targeted continuations globally. Multiple groups require a separate cross-group synthesis; a sole group integrates the explicit final whole-PR check into its own response, eliminating a redundant model turn. The outer host independently validates coverage/attempts/synthesis and rechecks drained context and output completeness. Missing required evidence cannot be waived. Findings retain original source locations, are deduplicated and cannot be silently removed by synthesis; output loss prevents approval.

Final hardening:

  • Replace the identical-image fixture with four genuinely different valid PNG pixel patterns and Git blob identities. The real pinned app-server retained all four distinct encoded inputs across six groups, continuation and final synthesis (16 scripted provider requests); first-turn-only attachment did not reproduce image loss. Keep that behavior rather than blindly duplicating attachments/history.
  • Fix a demonstrated proxy weakness: counting attachments alone allowed repetition/substitution to replace original visual evidence. Bind required fingerprints from the first trusted native request and require their continued presence, plus the existing attachment minimum. Identical native encodings are allowed when present in the trusted first input, avoiding an invented one-to-one Git/JPEG identity. No hashes, image bytes or credentials enter public diagnostics.
  • Attach host-generated, strictly validated group counts, synthesis state and fixed unfinished-reason enums to incomplete COMMENT reports. Model output cannot supply the status or use it to authorize operations. Preserve summary/inline findings and re-normalize the full result after metadata is added; overflow remains explicitly incomplete and fails the job.

Budget/security contract: one proxy and one 20-minute timer for all work; the existing 100 inference-request, 24 context-lookup and 768 GitHub-call limits are not reset or increased. A one-group native scripted inspection/report uses one turn/two requests. The six-group fixture uses seven baseline turns/14 scripted requests, or eight/16 with its one continuation. Actual model tool use, retries, latency and tokens can be substantially higher. The maximum 32-group plan can require 35 turns with continuation, leaving limited request/time headroom; no universal large-PR completion guarantee is claimed. Runtime arithmetic and remaining compaction/re-encoding uncertainty are documented in docs/adaptive-context.md.

Validation (Node 24):

  • npm run check: typecheck, lint, all 242 tests and build passed.
  • npm run test:clean: independent npm ci --ignore-scripts / full check with regenerated Env passed.
  • npm run test:workers, npm run test:web, npm run catalog:check: passed.
  • Docker build / npm run test:sandbox: actual pinned Codex 0.169.0, real inspection/isolation and 14 scripted-provider scenarios passed, including distinct images, continuation, failed synthesis, evidence tampering, shared-budget exhaustion and cross-group findings. No live AI inference.
  • npm audit --audit-level=moderate: zero vulnerabilities; Actionlint, Wrangler dry-run and diff checks passed.

Authentication, isolation, Git/OIDC integrity, credentials, quotas, merge policy and production data are unchanged. The opt-in provider harness understands the private execution envelope and still requires explicit live/paid authorization and an explicit bounded allowance. No Worker trust/configuration pin or downstream caller was changed. No merge, deployment, live inference or release is part of this task. Real-model semantic quality, latency/token consumption and large-history compaction behavior remain controlled live acceptance items; a later authorized rollout must coordinate the actual source merge SHA normally and begin with observed advisory/no-merge reviews.

GitHub CI for exact head 79e2d26: push run https://github.com/AppSolves/codexbot/actions/runs/43771068065 and pull-request run https://github.com/AppSolves/codexbot/actions/runs/43771071268 both completed successfully, including sandbox and Trivy security checks. PR remains open; no auto-merge, deployment or caller changes.

@AppSolves
AppSolves merged commit 560083b into main Nov 15, 2026
4 checks passed
@AppSolves
AppSolves deleted the fix/bounded-review-orchestration branch November 15, 2026 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant