Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Security and behavior changes

## Unreleased — bounded review orchestration

- Harden required visual-input retention against duplicate/substituted images; native multi-turn acceptance now uses four distinct PNGs across groups, continuation and synthesis.
- Integrate final whole-PR checks for one-group reviews without raising budgets. Add bounded host-generated coverage/reason status to incomplete COMMENT reports and recheck output size after adding metadata.

- Preserve valid incomplete model summaries/findings through the private result channel. The trusted runner still fails incomplete analysis; publication remains COMMENT without approval or merge.
- Build immutable host-owned evidence groups, run bounded same-thread continuation and mandatory cross-group synthesis, and independently validate coverage before accepting completeness. All groups share the existing proxy, request/time limits and context lookup budget.
- Preserve primary analysis failures when the workflow reports failure. Add deterministic and real pinned Codex/Docker coverage for large PRs, images, continuation, synthesis, evidence integrity and global budget exhaustion.

## Unreleased — final context completion

- Add metadata-only optional existence probes without weakening mandatory file evidence, immutable verification or existing request/API budgets. Verified absent candidates no longer invalidate otherwise complete context; infrastructure/verification errors stay distinct.
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ Configure a trigger plus up to five aliases matching `@[A-Za-z0-9]{1,39}`. Match

Use `--model MODEL_ID` (or `--model=MODEL_ID`) to override the repository's model for one review; replace MODEL_ID with an available Codex catalog identifier. An unsupported model or effort fails without substitution or billing fallback. Missing inspection, incomplete context or an inconclusive result fails the review job while the separate publisher may still publish a safe COMMENT.

Reviews use bounded groups of immutable changed-file evidence and a final cross-group synthesis in the same isolated Codex session. Unfinished analysis retains validated findings and its explanation as a non-approving COMMENT; it still fails the review check. Grouping does not reset usage budgets or authorize merging.

Set `bot.trigger: "@CBReview"` to use the example above. The legacy default `@codexbot` remains supported; new setup chooses `@CBReview<repository_id>`. Use a name without “codex” when another Codex integration is installed. CodexBot cannot control that independent connector; see [trigger isolation and rollout](docs/review-rollout.md).

## Status
Expand Down
8 changes: 4 additions & 4 deletions docs/acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,16 @@ The optional Linux-only harness accepts an operator-controlled owner-only JSON c
```sh
# OAuth, only after deployment authorization; no API-key fallback:
CHATGPT_DEPLOYMENT_AUTHORIZED=true npm run acceptance:provider -- --mode oauth \
--credential-file /private/operator/session.json --model YOUR_MODEL --max-requests 3 --allow-live-oauth
--credential-file /private/operator/session.json --model YOUR_MODEL --max-requests 8 --allow-live-oauth

# Separate intentional paid API test:
npm run acceptance:provider -- --mode api-key --credential-file /private/operator/api-key.json \
--model YOUR_MODEL --max-requests 3 --allow-paid-api
--model YOUR_MODEL --max-requests 8 --allow-paid-api
```

Build the pinned image first and use a nonroot Linux host. The fixture sends no real private source or GitHub writes. Requests are limited to an explicit 1–3, each input to 50 KB; the API-key harness adds a 2048-output-token cap. These are usage-unit ceilings, not a guaranteed currency ceiling: select the model using current pricing and an operator-controlled project/account budget. OAuth must omit unsupported API fields. Failure or truncation never becomes a passing check.
Build the pinned image first and use a nonroot Linux host. The fixture sends no real private source or GitHub writes. Requests are limited to an explicit operator-selected 1–8, each input to 50 KB; the API-key harness adds a 2048-output-token cap. These are usage-unit ceilings, not a guaranteed currency ceiling: select the model using current pricing and an operator-controlled project/account budget. OAuth must omit unsupported API fields. Failure or truncation never becomes a passing check.

The provider harness now reviews a synthetic authorization regression, not an empty no-tool prompt. It supplies a diff, both source versions and trusted project instructions; the change makes a role check always truthy. Success requires actual inspection, complete schema-valid findings and a finding on `src/access.js` RIGHT line 3. The model is passed through the same per-review override path. Reading and reporting normally need at least two provider requests; a one-request budget is an intentional failure test. A model that exceeds the chosen three-request ceiling fails rather than borrowing more quota. This real-provider gate is opt-in and has not been executed for the approval-policy fix; scripted Responses in CI prove native command execution/isolation, not AI defect recognition. Never run this harness with an API key without explicit paid-usage authorization.
The provider harness now reviews a synthetic authorization regression, not an empty no-tool prompt. It supplies a diff, both source versions and trusted project instructions; the change makes a role check always truthy. Success requires actual inspection, complete schema-valid findings and a finding on `src/access.js` RIGHT line 3. The model is passed through the same per-review override path. A one-group review integrates the final whole-PR check in the same turn. The native scripted inspection/report pattern uses two provider requests; real models may need more. Multiple groups require a separate synthesis under the same global allowance; a smaller budget remains an intentional failure test. The example explicitly allows eight requests; choose the budget before authorizing usage. Groups never reset that allowance. A model exceeding the operator-selected ceiling fails rather than borrowing more quota. This real-provider gate is opt-in and has not been executed for the approval-policy fix; scripted Responses in CI prove native command execution/isolation, not AI defect recognition. Never run this harness with an API key without explicit paid-usage authorization.

For OAuth success, require real streamed inference to a valid completed structured result using **only** the eligible SIWC access credential and verify the corresponding ChatGPT usage allowance/account/workspace. Confirm no API-key secret exists and no API-billed inference occurred in that isolated test window. A synthetic stream, successful token exchange, model list or HTTP 200 alone is insufficient. Check explicit denied plan scope, expired/revoked grants, actual refresh/rotation across two runs, disconnect/revoke and exhausted allowance; each must fail closed without paid fallback. Repeat API-key success and revoked/invalid-key behavior independently with bounded intentional billing.

Expand Down
27 changes: 27 additions & 0 deletions docs/adaptive-context.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Bounded context exploration and review presentation

## Bounded review orchestration

After collection passes, the trusted Actions host writes a private `review-plan.json` and immutable `review-units/` inputs. Changed paths are sorted to keep nearby code together, then grouped within 16 files / 2 MB per input, at most 32 groups. Inputs contain the actual textual patches and complete collected base/head changed-file text, or SHA-bound verified PNG metadata, plus applicable trusted instructions, the whole-PR path overview and an inventory of shared collected source. Every group keeps access to all original source, callers, dependencies, the file index and the same bounded context socket; grouping never removes cross-file context. Trusted instruction input is bounded to 1 MB; unrepresentable essential evidence fails before inference. Original Git/snapshot and binary verification remain required.

The orchestrator is immutable application code, not a model-provided plan. It checks hashes of host-prepared inputs and referenced text/image files before delivery. The actual pinned app-server processes groups sequentially in one ephemeral thread, with one required successful inspection command per turn and matching thread/turn identity. It allows at most two targeted continuations **across the whole review**, including synthesis. Explicit missing mandatory or visual evidence cannot be cleared by a later model claim. For multiple groups, a separate final synthesis reconciles all unit results and cross-group behavior; it cannot waive unfinished units. A sole group instead receives an explicit final whole-PR/cross-file check in its own response, avoiding a redundant turn. Its verified evidence, inspection and completeness gates still apply. No model field such as “all groups inspected” controls scheduling or coverage.

One credential proxy and one engine timer cover all turns: the existing 100 inference-request / 20-minute ceiling is not multiplied per group. The same 24 context requests, 768 GitHub calls, text/image byte limits and model/effort selection remain in force. Protocol receipts and scope checks establish supplied evidence and completed scheduled responses, not proof of a model's semantic reasoning. More turns can change actual consumption within those ceilings; no real-provider cost or quality improvement is claimed from synthetic tests.

Valid incomplete results return to the trusted runner as data. It independently checks every expected unit ID/hash, bounded attempts and synthesis status against its original plan, drains context lookups, and rechecks host and model completeness. Incomplete work fails the Actions review job and may publish only COMMENT, preserving validated summary and findings. Invalid or quarantined output remains rejected. Later execution failures preserve earlier safe findings and the primary execution diagnosis; generic workflow failure does not overwrite it. Findings are deduplicated by original path/side/line and normalized title; more than 40 findings or other output loss is explicitly incomplete. The model cannot delete earlier valid findings during synthesis.

Diagnostics and incomplete COMMENT reports expose only bounded group counts, synthesis status and fixed unfinished-reason enums. The host overwrites that advisory status from its validated plan/results; model output cannot supply it or use it to authorize approval. They never expose prompts, source, arbitrary reasons or provider bodies. Scripted Docker acceptance exercises all scheduled groups for 93 changes / four distinct valid PNGs, read-only shared callers, continuation, incomplete synthesis, altered evidence and one global proxy allowance. This is not real-model quality acceptance. Any future rollout requires normal source review, CI and coordinated immutable Worker/caller pins; this change does not deploy or change existing pins.

The existing immutable collection and isolated Codex execution remain in place. Changed base/head files and trusted default-branch instructions are still required evidence. Direct JS/TS dependencies, simple Python relative imports and Rust sibling modules are resolved without executing project code. Python/Go/Java test naming heuristics supplement the existing nearby tests/manifests.

The collector writes `/workspace/file-index.json`: at most 12 nonrecursive directory walks and 5,000 regular-file paths. A partial supplementary index does not invalidate a small PR. It does not imply exhaustive semantic coverage. No full recursive tree or unrelated blob download is required.
Expand Down Expand Up @@ -41,3 +53,18 @@ Final completion is checked again by the trusted host after the isolated engine


Optional discovery never uses a model-supplied required/optional flag. The existing `/file` endpoint stays mandatory and rejects extra fields such as `required: false`; the separate `/probe` operation supplies existence only, not readable or verified contents. A probe can report an object exists even when a later content read rejects its Git type, size or blob hash. A successful probe cannot waive initial changed-file/instruction/dependency requirements or clear a previous required failure. Both operations share the unchanged 24-request and GitHub API budgets. Exhausted optional probes fail without invalidating otherwise complete evidence; a denied required fetch remains fatal. There is no extra inference pass or model/budget change. Host checks do not claim to prove exhaustive model reasoning about every possible dependency.

## Runtime envelope and image retention

For G planned groups, successful baseline scheduling uses T = G + (G > 1 ? 1 : 0) turns. At most two targeted continuations are added globally. In native scripted inspection-then-report acceptance, each turn uses two provider requests. That observed minimum is not a prediction of real-model tool iterations, latency or tokens: all actual calls and retries still share the unchanged 100-request and 20-minute limits.

| Shape (byte limits can create more groups) | Groups | Baseline turns | Scripted requests | Turns / scripted requests with two continuations |
| --- | ---: | ---: | ---: | ---: |
| One bounded group | 1 | 1 | 2 | 3 / 6 |
| Medium example | 3 | 4 | 8 | 6 / 12 |
| 93-file/four-image fixture | 6 | 7 | 14 | 9 / 18 |
| Maximum plan | 32 | 33 | 66 | 35 / 70 |

The maximum plan leaves only 30 requests beyond that two-step/35-turn pattern and an arithmetic average of roughly 34 seconds per turn within 20 minutes, before startup overhead. There is no guaranteed large-PR completion or wall-clock/token cost estimate. Real models may need more inspection, compaction or reasoning; exhaustion produces a failed, non-approving partial review. Do not increase limits to hide this. A controlled rollout should first observe representative advisory/no-merge reviews and split work that does not fit the envelope.

The native acceptance now creates four different PNG pixel patterns and verifies four distinct Git blob identities and four distinct encoded image inputs on every request across groups, continuation and final synthesis (16 requests in the large/one-continuation fixture). The first-turn-only attachment behavior is supported by the tested pinned app-server history; image loss was not reproduced. The proxy additionally binds fingerprints of the first trusted required visual inputs and rejects subsequent loss, repetition-as-replacement or substitution. Distinct Git images can have identical native encodings (for example after lossy conversion); the guard retains the minimum attachment count and the first trusted fingerprint set rather than inventing a one-to-one Git-to-encoding identity. Fingerprints and image bytes never enter public diagnostics. No images are blindly reattached per turn, which would inflate history. Native context compaction/re-encoding behavior at real large input sizes remains unmeasured; loss of required evidence fails closed rather than authorizing approval.
2 changes: 2 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ See [operations](operations.md), [security](security.md) and [acceptance](accept

## Adaptive context and browser management

Review execution uses a host-generated immutable coverage plan, bounded sequential app-server turns in one isolated thread and final cross-group synthesis. The host independently validates evidence receipts, coverage and final context/output completeness. Valid partial reviews survive as failed-job COMMENT results; neither partial findings nor a model assertion can authorize approval or merge. See [orchestration limits and acceptance](adaptive-context.md#bounded-review-orchestration).

The collector also writes a supplementary regular-file index (12 directories / 5,000 paths, no recursive tree dump). Explicit Python relative imports and Rust sibling modules join JS/TS dependencies; nearby Python/Go/Java test names are considered. A separate host Unix socket accepts bounded base/head `/file` requests for required contents and `/probe` requests for optional existence for this authorized immutable repository. It never forwards arbitrary URLs, executes project code or exposes credentials. At most 24 requests share the existing byte/blob/API limits and per-SHA cache. Required files that cannot be obtained invalidate completeness. A metadata-only probe may establish absence without fetching source; it cannot waive required evidence. Probe API/tree failures remain fail-closed rather than being mislabeled as missing paths. The reviewer mount stays read-only and network=none. The read token now lasts through bounded context exploration; shutdown cancels/drains lookups and revokes it before temporary source cleanup. Failed token cleanup fails the review job.

The system prompt asks for introduced, evidence-backed bugs traced through unchanged callers and dependencies, with trusted default-branch guidance subordinate to execution/output safety. Additional comment context is advisory. The model/effort CLI precedence is unchanged; a generated pinned-native catalog powers dashboard validation and distinguishes catalog membership from account entitlement.
Expand Down
6 changes: 6 additions & 0 deletions docs/validation.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Validation record and release gates

## Bounded orchestration (synthetic acceptance)

Final hardening replaces the identical-image fixture with four distinct valid PNG pixel patterns, verifies four Git identities and retention of all four encoded image inputs on every native request across six groups, one continuation and synthesis (16 scripted provider requests). The pinned first-turn-only attachment flow passed; actual image loss was not reproduced. A separate deterministic regression did reproduce the proxy accepting four copies of one image in place of four originals; required input fingerprints now reject loss, duplicate-as-replacement and substitution. One-group native completion takes one turn/two scripted inspection/report requests, while multi-group synthesis and global budget denial remain enforced. Incomplete GitHub reports expose only bounded counts, synthesis state and fixed reason enums, preserve summary/inline findings and cannot use metadata to authorize approval. Metadata is included in the existing size/DLP normalization before submission; a 59,950-byte review that overflows when annotated remains a failed, non-approving result with explicit output-loss flag. Real-model latency/token use and compaction/re-encoding at large production sizes remain unmeasured. No real inference or rollout was performed.

Deterministic tests cover host-owned immutable grouping, evidence tampering, exact unit/turn scope, global continuation limits, unresolved mandatory/visual evidence, valid partial output, lost execution after partial findings, duplicate findings and independent host coverage validation. Publication tests require COMMENT with preserved summary and inline findings for incomplete analysis, and failed-job tests retain nonzero review-job outcome. The actual pinned native Codex/Docker harness uses scripted Responses for small clean/findings/incomplete reviews, a 93-file / four-PNG case, changed callers across groups, targeted continuation, incomplete synthesis, changed evidence and a shared proxy budget exhausted between turns. Every scenario retains the existing write/network/credential denial checks. This is protocol, coverage and isolation evidence, not a claim of real-model semantic quality or measured production cost improvement. No live inference or production rollout belongs to this validation pass.

## Browser onboarding and context revision: local validation, 2026-11-15

The final feature checkout passed `npm run check` (170 tests, zero skipped; typecheck, lint and build), `npm run test:clean` (independent `npm ci --ignore-scripts` and regenerated Env), `npm run test:workers`, `npm run test:web`, `npm run catalog:check`, Actionlint, `npm audit --audit-level=moderate` (zero findings), Docker build, `npm run test:sandbox`, Wrangler deployment dry-run and `git diff --check`. The final image also passed the strict Trivy High/Critical gate with no exemptions. Local reports, screenshots and images are ignored, not release artifacts.
Expand Down
Loading