Skip to content

Clarify r3 deduction-trace retry feedback / 明确 r3 扣分依据重试反馈 - #67

Merged
yyuan308 merged 1 commit into
mainfrom
codex/r3-retry-contract-feedback
Aug 25, 2026
Merged

yyuan308 merged 1 commit into
mainfrom
codex/r3-retry-contract-feedback

Conversation

@yyuan308

Copy link
Copy Markdown
Collaborator

Summary / 摘要

  • On a structural retry, r3 grading now repeats the exact deduction-trace invariant: full credit uses deduction_trace=null; every non-full leaf needs a non-empty trace whose deducted points sum exactly to max_score - score.
  • r3 评分在结构重试时会重申精确扣分依据约束:满分使用 deduction_trace=null;每个非满分叶子项必须有非空 trace,且扣分总额严格等于 max_score - score。
  • The feedback contains contract rules only. It never includes a prior response, answer evidence, student identifier, or private validator details.
  • 该反馈只包含契约规则,不包含先前模型输出、作答证据、学生标识或私有校验细节。

Validation / 验证

  • python -m unittest discover -s tests/benchmark/core — 479 passed.
  • Adds regression coverage for the retry invariant and uses the same feedback in text and headless runners.
  • 增加重试约束回归测试,并让文本与 headless 运行器使用同一反馈。

Privacy and limits / 隐私与局限

  • Public code and synthetic tests only; no Data/, submissions, identities, transcripts, gold scores, raw outputs, or metrics.
  • 仅含公共代码和合成测试;不包含 Data/、作答、身份、转录、gold、原始输出或指标。
  • This is output-contract reliability work, not a rubric revision or an accuracy claim.
  • 这是输出契约可靠性改进,不是 rubric 修订,也不构成准确率主张。

Next step / 下一步

  • After merge, run DeepSeek V4Flash from a fresh private output directory against the frozen r3 G1 packet. M1-Codex continues separately; heldout remains untouched.
  • 合并后,从全新私有输出目录使用冻结的 r3 G1 packet 运行 DeepSeek V4Flash。M1-Codex 独立继续,heldout 保持不访问。

@yyuan308
yyuan308 merged commit b411e3b into main Aug 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant