Skip to content

test: verify replan call limits by capture route - #9175

Merged
loopx-agent merged 1 commit into
mainfrom
codex/replan-qualification-budget-20261121
Nov 21, 2026
Merged

loopx-agent merged 1 commit into
mainfrom
codex/replan-qualification-budget-20261121

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Problem

Two scripted replan cases report failures after the CLI workflow qualifies. Their assertions tie the capture allowance and writeback format to the same flag and require the workflow to use its entire tool budget.

Change

Verify the ordinary and native-read limits against the captured fixture and reconcile the reported total with the observed sequence. Validate the writeback format against the delivered replan view. Keep the exact semantic work sequence and the existing limits of seven ordinary calls and three native reads in a captured run.

Validation

  • Current main reproduced both capture-on failures. Both capture-off controls passed.
  • Six focused CLI tests passed, covering capture on and off, progressive readback, and deep-path recovery.
  • Both captured end-to-end cases used seven ordinary calls and one native read. The uncaptured runs used seven ordinary calls.
  • The tests use a scripted transport and isolated LoopX CLI state.
  • Ruff and the quick premerge canary passed.

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.2-sol · OpenAI (runtime_reported, xhigh)

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机 / Motivation

维护 replan 验收的开发者需要知道已成功的 CLI 流程有没有超出实际工具预算。基线有两条捕获用例在工作和结算完成后失败,因为测试把可用预算当成必须消费的次数。候选分别核对普通调用与原生读取,并按实际送达的 replan 视图检查写回。范围限于合成脚本和真实本地 CLI 的测试断言,实际模型与正式安装 Host 留在各自的验收流程。原工作序列与结算验收继续由现有端到端用例保护。

改动思路 / Approach

保留两条端到端用例及其精确工作序列,共用一个短断言核对调用总数和两类上限。捕获设置决定可读次数,送达的视图决定写回形态。保留对捕获确实发生和 off 模式没有原生读取的要求,避免宽松的总数断言掩盖一种调用超限。

具体改动 / Changes

审查 21869aaf0d043fec38ebb814a811c59ecc49c76c 对 64353773dc8b1ca82ff531cc7653dec6c3d41c0b 的单文件差异,新增 26 行、移除 12 行。 验收原则

  • route-budget 由观测序列复核合计,普通调用最多七次,捕获方式可用三次原生读取。
  • writeback-shape 原生渐进视图检查可解析的写回命令,内联视图检查现有 evidence-linked 写回契约。 写回断言
  • settled-work 两条现有真实 CLI 用例继续核对精确工作的动作或后继,以及原 Turn 结算。覆盖扫描确认现有深路径恢复也复用这条用例,本提案无需新增 smoke 或重复 fixture。

对主干的风险 / Risks

精确基线两条 capture-on 用例按预期失败,两个 off 对照通过。精确候选的整个模块 53 项通过,Ruff 与 diff 检查通过。额外的断言反例拒绝错误计数、第四次原生读取、第八次普通调用、捕获后未读取,以及关闭捕获时意外读取。保留七次实际工作序列,正式 Host、真实模型和托管检查未在此次评审执行。

我的整体评价 / Verdict

APPROVE。 现有测试守住两类预算和写回形态,允许成功流程提前收口,并继续验证原 Turn 的实际结算。无需额外产品改动。

English verdict: APPROVE at 21869aa. The tests enforce the two call budgets and the delivered writeback shape without requiring an otherwise successful replan to exhaust its allowance.

@loopx-agent
loopx-agent merged commit 30e9a55 into main Nov 21, 2026
30 of 36 checks passed
@loopx-agent
loopx-agent deleted the codex/replan-qualification-budget-20261121 branch November 21, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant