fix(stride): avoid authority marker substring matches - #3835
Conversation
huangruiteng
left a comment
There was a problem hiding this comment.
Findings
- Blocker — caller-supplied free text still becomes authority-change truth.
_has_authority_change_markertokenizes an untypedclassificationinto an unordered set, then treats anyreplan/visiontoken, orgateplus a broad decision token, as a real authority transition. This fixes substring collisions but loses negation and word order:not_replan,vision_check,gate_not_approved,gate_status_recorded_without_transition, andno_gate_changedall resetbounded_slices_since_changeeven though none proves an authority delta. I reproduced all five on the exact head; each returned1where the no-change baseline returns3. That conflicts with #3203's qualification rule that keyword/prose heuristics must not be semantic truth and with the typed-state review contract. Please carry a typed authority-change field/enum in the run receipt and consume that here; if M1 cannot change the schema, use a closed allowlist of complete classification values emitted by owned writers, with unknown values staying no-change. Do not infer authority from arbitrary token co-occurrence.
动机
本 PR 要修复 authority stride 观测里的误判:旧实现用 substring 查找 replan、vision、gate,所以 revision_review、supervision_check、waiting_at_approval_gate 会被错误当作重规划或权限变化。这个问题会让 bounded_slices_since_change 偏小,从而虚构“最近发生过 steering”的观测事实。动机正确,而且属于 #3203 observation-only 阶段必须解决的 correctness gap。
改动思路
实现把 substring matching 改成 token matching:先用 ASCII 字母数字正则拆词;replan/vision 作为 whole-token marker;gate 只有单独出现,或与一组 decision/transition token 共现时才算 authority change。整体仍保持 shadow-only,没有改变 Todo、scheduler、quota、gate 或执行行为。
这个方向能解决原始三个已知碰撞,但没有把数据源从自由文本升级为 typed evidence,因此只是缩窄 heuristic,而不是消除 heuristic。尤其 token 被放进 set 后,否定、顺序和上下文都不可恢复。
具体改动
_AUTHORITY_CLASSIFICATION_TOKEN_RE:把 classification 拆成[a-z0-9]+token,解决revision内含vision的 substring 问题。_AUTHORITY_TOKEN_MARKERS:把replan和vision设为任意 whole-token 即命中的 marker;这也导致not_replan、vision_check的 false positive。_AUTHORITY_GATE_DECISION_TOKENS:给gate配置 accepted/approved/changed/recorded 等宽泛共现词;它无法识别not approved或recorded without transition。_has_authority_change_marker:集中封装匹配逻辑,但输入仍是 caller-supplied free text,输出却被当成 authority boolean。build_stride_observation:用 helper 定位最后一次“变化”,随后重置bounded_slices_since_change;因此 helper 的误判会直接污染最终观测。- 测试新增 4 个负例与 6 个正例,覆盖了原始 substring bug 和预期 marker,但缺少否定及无关共现的反例。
正向 walkthrough:revision_review 被拆成 revision/review,不再命中 vision,三条无 authority delta 的记录得到 bounded_slices_since_change = 3,原始 bug 得到修复。
负向 walkthrough:gate_not_approved 被拆成 gate/not/approved;实现看到 gate + approved 就返回 true,忽略 not,最终把计数重置为 1。not_replan 更直接:只要出现 replan token 就返回 true。这不是 fail-closed 的 authority observation。
关键代码讲解
_AUTHORITY_CLASSIFICATION_TOKEN_RE改善了词边界精度,但正则只能回答“有哪些词”,不能回答“是否真的发生了 authority transition”。_AUTHORITY_GATE_DECISION_TOKENS把领域语义编码成开放文本共现表;recorded、changed等词在其他上下文中也常见,因此不是可靠 transition contract。_has_authority_change_marker用set丢弃顺序和重复信息,然后输出 boolean;这里是本 PR 的关键 correctness boundary,也是 blocker 所在。build_stride_observation信任该 boolean,并把最后一个命中位置作为 authority segment 边界,所以误判会系统性低估 stride。- 两组 parametrized tests 能证明 substring collision 已修复,但测试 oracle 仍沿用了“marker token 即真值”的假设,没有覆盖 typed-state contract 所要求的反例。
对主干的风险
这是 shadow-only 观测,不会直接改变调度或权限,因此没有立即的执行权限风险;但它产出的 authority metric 会作为后续评估和自适应设计的基础。若合入,任意 caller 都能通过命名里的否定词或描述性词组制造一次假的 authority reset,污染实验数据,并可能让后续控制策略建立在错误标签上。相比旧实现,误判面缩小了,但核心的 untyped semantic-truth 风险仍然存在。
Validation
| Surface | Result |
|---|---|
| GitHub checks | 9 checks green/skipped as expected |
| Focused stride + goal-vision pytest | 58 passed |
| Seeded negation/co-occurrence counterexamples | 5 failed, all five false positives reproduced |
| Ruff + Python compile | passed |
| Focused mypy | passed |
| Diff hygiene | passed |
| Premerge direct/catalog/risk-profile checks | 12 passed |
| Change-quality exact scope | receipt cqr_7e3952e91ce0f9d6ae38, fail/invalid; merge gate blocked |
我的整体评价
PR 的范围清晰、原始 substring bug 确实被修复、现有回归和仓库 canary 也都干净;但 #3203 的验收目标不是“更精细的关键词匹配”,而是不能把关键词或 prose 当 semantic truth。当前实现仍会对合法自由文本产生可复现的 authority false positive,所以整体 verdict 是 REQUEST_CHANGES / hold。建议把 authority delta 建模成 typed receipt field/enum;若阶段约束暂不允许 schema 变更,至少只接受受控 producer 发出的完整 exact classification 值,并为否定、等待、记录但未转换等情况补充反例。
English verdict: REQUEST_CHANGES on exact head c69a52d460557924871bec9cfed8006293a3443e. The substring collisions are fixed, but untyped classification tokens still act as authority truth; five negated/descriptive counterexamples falsely reset the stride. Repository tests and 12 premerge checks pass, while the exact-scope change-quality receipt is invalid because the semantic blocker remains.
Signed-off-by: yuefengw <[email protected]>
Signed-off-by: yuefengw <[email protected]>
c69a52d to
e588d7f
Compare
|
Implemented the requested fail-closed correction on the rebased head Changes:
Validation:
The exact-scope semantic blocker is addressed. Please re-review the new head. 中文说明:已改为受控 writer 的完整 classification 闭合白名单;未知值一律按 no-change,维护者列出的 5 个误报均已补充回归测试。该 M1 仍为 observation-only,没有 schema、scheduler、quota、gate 或权限行为变化。 |
Signed-off-by: yuefengw <[email protected]>
Signed-off-by: yuefengw <[email protected]>
e588d7f to
d10ae4b
Compare
|
Follow-up: the final head is All required checks are green, including Sign-off, pytest, Windows PowerShell, build, dependency review, and Sonar. |
huangruiteng
left a comment
There was a problem hiding this comment.
结论:APPROVE。我在 exact head d10ae4b569ea48a8f4c3dfd5fd0bf24b5fa74a99 重新核对了先前 blocker;新的完整值白名单消除了否定词和描述性 classification 被当成 authority truth 的问题,未发现新的 blocking finding。
动机
本 PR 修复 authority stride 的观测误判。主干原先对 caller-supplied classification 做 substring 匹配,revision_review、supervision_check、waiting_at_approval_gate 等并未发生 authority delta 的记录也会重置 bounded_slices_since_change。上一轮 whole-token 方案仍会把 not_replan、vision_check、gate_not_approved、gate_status_recorded_without_transition、no_gate_changed 误判为 authority change。exact head 改为只承认当前受控 writer 使用的四个完整 classification 值,未知或描述性值一律按 no-change 处理;这正是 M1 不扩展 run schema 时所需的 fail-closed 最小修复。非目标仍保持清晰:不改变调度、Todo、quota、notification、gate 或执行权限。
改动思路
观测链路仍是 read_run_index 读取公开 run index,build_stride_observation 按 agent_id 过滤记录,并从最后一个可信 authority-change 记录计算 stride。变化只发生在真假判定边界:_has_authority_change_marker 不再拆词或推断语义,而是先做现有 compact/casefold 规范化,再与 AUTHORITY_CHANGE_CLASSIFICATIONS 的闭集做 exact membership。
我核对了生产来源:operator-gate writer 通过 classification_for_decision 固定生成 operator_gate_approved、operator_gate_rejected、operator_gate_deferred;autonomous-replan obligation 固定投影 bounded_replan_progress,而无有效 delta 的 replan writeback 会被现有 qualification 降级为 replan_noop。因此正向路径能够重置一次 authority segment,未知、否定、记录型或 no-op 分类则不会。更完整的 typed authority-change receipt field 仍是后续理想方向,但本 PR 的 closed compatibility boundary 已消除当前可复现误判,并未增加第二套运行时 authority。
具体改动
- 生产代码:
loopx/control_plane/runtime/stride_observation.py为 +25/-4。删除开放 substring 判定,引入四值frozenset和一个 fail-closed helper;build_stride_observation的其余读取、过滤、计数、freshness 与 shadow-only 输出均未变化。 - 测试:
tests/control_plane/test_stride_observation.py为 +106/-6。新增 15 个未知/否定/描述性反例,覆盖上一轮报告的五个 false positive 以及replan_noop、记录型 replan、vision checkpoint 等近邻值;另用四个受控正例证明合法 authority decision 会重置 stride。 - Scope fit:生产 helper 被
build_stride_observation的 active projection path 直接调用,不是 coverage-only 代码;两文件、+131/-10 与问题规模相称,没有新增 schema、CLI surface、持久状态或迁移负担。
关键代码讲解
AUTHORITY_CHANGE_CLASSIFICATIONS:把兼容期可作为 authority evidence 的完整值集中成闭集;值来自现有 operator-gate 与 autonomous-replan writer,而不是从任意 prose 中归纳。_has_authority_change_marker:保留输入规范化,但唯一决策是 exact membership。not_replan、gate_not_approved等即使包含 marker token 也会 fail closed。build_stride_observation:只把 helper 返回 true 的最后一条 run 当成 segment 边界;输出仍为 read-onlyhierarchical_stride_observation_v0,失败/重试责任仍在上游 writer 与后续 typed-schema 演进。- 两组 parametrized boundary tests:一组证明未知值保持
bounded_slices_since_change == 3,另一组证明四个受控值重置后结果为1,正负 oracle 没有从实现输出反推。
对主干的风险
最强回归场景是遗漏一个真实 authority writer 的 classification,导致 false negative、让 stride 看起来比实际更长。当前代码路径扫描未发现闭集之外的生产 authority writer:route_continuation_replan_recorded、successor_replan_recorded、vision checkpoint 等是 delivery/记录语义,不应仅凭名字升级为 heavy authority;未知值继续 no-change 是有意的安全降级。回滚只需恢复旧 classifier,但会重新引入已证明的 false positive,因此更合适的长期恢复路径是增加 typed authority-change 字段并迁移本闭集。
验证结果:GitHub exact-head 9 项检查成功;focused stride + goal-outcome-continuity pytest 为 30 passed;Ruff、Python compile、diff hygiene 均通过;LoopX premerge 的 4 项 direct checks 与 12 项 catalog/risk-profile smokes 全部通过,0 failure、0 skip、0 manual hold;change-quality exact-scope receipt cqr_36ac4d2f0d6535d97303 有效且 gate passed。default-off isolation 不适用(无 feature gate/opt-in 变更);authority naming 与实际 gate/replan event 对齐;domain-neutrality、guidance-vs-obligation 未引入新风险。PR 描述中“standalone replan/vision/gate 仍被识别”的旧表述已落后于 exact head,建议合并前更新,但不影响代码 verdict。
我的整体评价
先前 blocker 已完整解决:分类真值不再来自 token 共现,所有已报告反例和相邻风险值都被锁定为 no-change,同时受控正例保留。实现直接、可回滚,复用了现有 normalization 与 writer vocabulary,helper 虽小但承载了明确的 fail-closed domain invariant,没有值得在本 PR 继续扩张的 companion refactor。剩余风险只在兼容期闭集需要随新 authority writer 演进;typed receipt field 可作为后续 schema 工作,不应继续阻塞这个 M1 correctness fix。
English verdict: APPROVE on exact head d10ae4b569ea48a8f4c3dfd5fd0bf24b5fa74a99. The previous blocker is resolved: authority resets now require one of four complete controlled classification values, while negated, descriptive, no-op, and unknown classifications fail closed. Focused tests, GitHub checks, the exact-scope quality receipt, and the full selected premerge gate all pass. The PR description still names obsolete standalone examples and should be refreshed before merge, but no blocking code finding remains.
|
The fail-closed boundary is now exactly where it should be. Hardening points, all deferrable:
|
Summary
This is the remaining M1 observation-correctness hardening for #3203 and RFC #3204.
replanandvisionas complete classification tokens.gateas an authority change only when it is standalone or paired with an explicit gate decision/transition token.Why
classificationis a bounded, caller-supplied field rather than a typed enum. The previousmarker in classificationcheck treated values such asrevision_review,supervision_check, andwaiting_at_approval_gateas authority changes. That resetauthority.bounded_slices_since_changeeven though no authority delta was recorded.The new classifier is still deliberately read-only and conservative at the ambiguous
gateboundary. It does not infer authority changes from summaries, notes, command names, or file paths.Scope and Compatibility
shadow_onlybehavior and existing observation/evaluation schemas are unchanged.replan,vision,gate,bounded_replan_progress,vision_refresh_accepted, andoperator_gate_approvedremain recognized.Validation
pytest -q tests/control_plane/test_stride_observation.py tests/control_plane/test_goal_outcome_continuity_characterization.py— 21 passedruff check loopx/control_plane/runtime/stride_observation.py tests/control_plane/test_stride_observation.py— passedpython -m compileall -q loopx/control_plane/runtime/stride_observation.py tests/control_plane/test_stride_observation.py— passedloopx check— no errors; public boundary scan cleanloopx canary premerge --from-git-diff— passed, 12 checks selected, 0 blocking failures, 0 manual holdsThe full
tests/control_planerun finished with 2,092 passed and 3 baseline/environment failures unrelated to these files:The canary classified one maintainability smoke result as an inherited advisory; it did not mention either changed file, and the smoke passes when run directly.
Tracks #3203.