Skip to content

fix(stride): avoid authority marker substring matches - #3835

Merged
huangruiteng merged 4 commits into
huangruiteng:mainfrom
yuefengw:codex/3203-stride-marker-boundary
Sep 4, 2026
Merged

fix(stride): avoid authority marker substring matches#3835
huangruiteng merged 4 commits into
huangruiteng:mainfrom
yuefengw:codex/3203-stride-marker-boundary

Conversation

@yuefengw

@yuefengw yuefengw commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

This is the remaining M1 observation-correctness hardening for #3203 and RFC #3204.

  • Replace substring containment when deriving authority changes from caller-supplied run classifications.
  • Match replan and vision as complete classification tokens.
  • Treat gate as an authority change only when it is standalone or paired with an explicit gate decision/transition token.
  • Add regression coverage for the reported false positives and for existing explicit positives.

Why

classification is a bounded, caller-supplied field rather than a typed enum. The previous marker in classification check treated values such as revision_review, supervision_check, and waiting_at_approval_gate as authority changes. That reset authority.bounded_slices_since_change even though no authority delta was recorded.

The new classifier is still deliberately read-only and conservative at the ambiguous gate boundary. It does not infer authority changes from summaries, notes, command names, or file paths.

Scope and Compatibility

  • No run-record schema changes.
  • No changes to scheduler cadence, Todo selection, quota spend, notification, execution, or user/protected-operation authority.
  • shadow_only behavior and existing observation/evaluation schemas are unchanged.
  • Explicit values such as replan, vision, gate, bounded_replan_progress, vision_refresh_accepted, and operator_gate_approved remain recognized.

Validation

  • pytest -q tests/control_plane/test_stride_observation.py tests/control_plane/test_goal_outcome_continuity_characterization.py — 21 passed
  • ruff check loopx/control_plane/runtime/stride_observation.py tests/control_plane/test_stride_observation.py — passed
  • python -m compileall -q loopx/control_plane/runtime/stride_observation.py tests/control_plane/test_stride_observation.py — passed
  • loopx check — no errors; public boundary scan clean
  • loopx canary premerge --from-git-diff — passed, 12 checks selected, 0 blocking failures, 0 manual holds

The full tests/control_plane run finished with 2,092 passed and 3 baseline/environment failures unrelated to these files:

  • 2 fine-grained CLI tests selected the system Python 3.9 instead of the configured Python 3.13 runtime.
  • 1 effect-runtime doctor test observed the current Node/readiness environment before reaching its assertion.

The canary classified one maintainability smoke result as an inherited advisory; it did not mention either changed file, and the smoke passes when run directly.

Tracks #3203.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • Blocker — caller-supplied free text still becomes authority-change truth. _has_authority_change_marker tokenizes an untyped classification into an unordered set, then treats any replan/vision token, or gate plus a broad decision token, as a real authority transition. This fixes substring collisions but loses negation and word order: not_replan, vision_check, gate_not_approved, gate_status_recorded_without_transition, and no_gate_changed all reset bounded_slices_since_change even though none proves an authority delta. I reproduced all five on the exact head; each returned 1 where the no-change baseline returns 3. That conflicts with #3203's qualification rule that keyword/prose heuristics must not be semantic truth and with the typed-state review contract. Please carry a typed authority-change field/enum in the run receipt and consume that here; if M1 cannot change the schema, use a closed allowlist of complete classification values emitted by owned writers, with unknown values staying no-change. Do not infer authority from arbitrary token co-occurrence.

动机

本 PR 要修复 authority stride 观测里的误判:旧实现用 substring 查找 replanvisiongate,所以 revision_reviewsupervision_checkwaiting_at_approval_gate 会被错误当作重规划或权限变化。这个问题会让 bounded_slices_since_change 偏小,从而虚构“最近发生过 steering”的观测事实。动机正确,而且属于 #3203 observation-only 阶段必须解决的 correctness gap。

改动思路

实现把 substring matching 改成 token matching:先用 ASCII 字母数字正则拆词;replan/vision 作为 whole-token marker;gate 只有单独出现,或与一组 decision/transition token 共现时才算 authority change。整体仍保持 shadow-only,没有改变 Todo、scheduler、quota、gate 或执行行为。

这个方向能解决原始三个已知碰撞,但没有把数据源从自由文本升级为 typed evidence,因此只是缩窄 heuristic,而不是消除 heuristic。尤其 token 被放进 set 后,否定、顺序和上下文都不可恢复。

具体改动

  • _AUTHORITY_CLASSIFICATION_TOKEN_RE:把 classification 拆成 [a-z0-9]+ token,解决 revision 内含 vision 的 substring 问题。
  • _AUTHORITY_TOKEN_MARKERS:把 replanvision 设为任意 whole-token 即命中的 marker;这也导致 not_replanvision_check 的 false positive。
  • _AUTHORITY_GATE_DECISION_TOKENS:给 gate 配置 accepted/approved/changed/recorded 等宽泛共现词;它无法识别 not approvedrecorded without transition
  • _has_authority_change_marker:集中封装匹配逻辑,但输入仍是 caller-supplied free text,输出却被当成 authority boolean。
  • build_stride_observation:用 helper 定位最后一次“变化”,随后重置 bounded_slices_since_change;因此 helper 的误判会直接污染最终观测。
  • 测试新增 4 个负例与 6 个正例,覆盖了原始 substring bug 和预期 marker,但缺少否定及无关共现的反例。

正向 walkthrough:revision_review 被拆成 revision/review,不再命中 vision,三条无 authority delta 的记录得到 bounded_slices_since_change = 3,原始 bug 得到修复。

负向 walkthrough:gate_not_approved 被拆成 gate/not/approved;实现看到 gate + approved 就返回 true,忽略 not,最终把计数重置为 1not_replan 更直接:只要出现 replan token 就返回 true。这不是 fail-closed 的 authority observation。

关键代码讲解

  1. _AUTHORITY_CLASSIFICATION_TOKEN_RE 改善了词边界精度,但正则只能回答“有哪些词”,不能回答“是否真的发生了 authority transition”。
  2. _AUTHORITY_GATE_DECISION_TOKENS 把领域语义编码成开放文本共现表;recordedchanged 等词在其他上下文中也常见,因此不是可靠 transition contract。
  3. _has_authority_change_markerset 丢弃顺序和重复信息,然后输出 boolean;这里是本 PR 的关键 correctness boundary,也是 blocker 所在。
  4. build_stride_observation 信任该 boolean,并把最后一个命中位置作为 authority segment 边界,所以误判会系统性低估 stride。
  5. 两组 parametrized tests 能证明 substring collision 已修复,但测试 oracle 仍沿用了“marker token 即真值”的假设,没有覆盖 typed-state contract 所要求的反例。

对主干的风险

这是 shadow-only 观测,不会直接改变调度或权限,因此没有立即的执行权限风险;但它产出的 authority metric 会作为后续评估和自适应设计的基础。若合入,任意 caller 都能通过命名里的否定词或描述性词组制造一次假的 authority reset,污染实验数据,并可能让后续控制策略建立在错误标签上。相比旧实现,误判面缩小了,但核心的 untyped semantic-truth 风险仍然存在。

Validation

Surface Result
GitHub checks 9 checks green/skipped as expected
Focused stride + goal-vision pytest 58 passed
Seeded negation/co-occurrence counterexamples 5 failed, all five false positives reproduced
Ruff + Python compile passed
Focused mypy passed
Diff hygiene passed
Premerge direct/catalog/risk-profile checks 12 passed
Change-quality exact scope receipt cqr_7e3952e91ce0f9d6ae38, fail/invalid; merge gate blocked

我的整体评价

PR 的范围清晰、原始 substring bug 确实被修复、现有回归和仓库 canary 也都干净;但 #3203 的验收目标不是“更精细的关键词匹配”,而是不能把关键词或 prose 当 semantic truth。当前实现仍会对合法自由文本产生可复现的 authority false positive,所以整体 verdict 是 REQUEST_CHANGES / hold。建议把 authority delta 建模成 typed receipt field/enum;若阶段约束暂不允许 schema 变更,至少只接受受控 producer 发出的完整 exact classification 值,并为否定、等待、记录但未转换等情况补充反例。

English verdict: REQUEST_CHANGES on exact head c69a52d460557924871bec9cfed8006293a3443e. The substring collisions are fixed, but untyped classification tokens still act as authority truth; five negated/descriptive counterexamples falsely reset the stride. Repository tests and 12 premerge checks pass, while the exact-scope change-quality receipt is invalid because the semantic blocker remains.

@yuefengw
yuefengw force-pushed the codex/3203-stride-marker-boundary branch from c69a52d to e588d7f Compare September 3, 2026 06:28
@yuefengw

yuefengw commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Implemented the requested fail-closed correction on the rebased head e588d7f56.

Changes:

  • Replaced token/substr/co-occurrence matching with an exact closed allowlist: bounded_replan_progress, operator_gate_approved, operator_gate_rejected, and operator_gate_deferred.
  • Unknown/descriptive classifications are no-change, including not_replan, vision_check, gate_not_approved, gate_status_recorded_without_transition, no_gate_changed, replan_noop, typed-ACK labels, checkpoint labels, and command-template labels.
  • Kept M1 observation-only; no run schema, scheduler, quota, gate, permission, or replan behavior changes.
  • Added regression coverage for all five reported false positives and the controlled positive values.

Validation:

  • focused stride tests: 27 passed
  • replan/vision regressions: 65 passed
  • monitor/blocked-successor regressions: 50 passed
  • Ruff + Python compile: passed
  • loopx canary premerge --from-git-diff: 12/12 selected checks passed; 0 failures, 0 manual holds

The exact-scope semantic blocker is addressed. Please re-review the new head.

中文说明:已改为受控 writer 的完整 classification 闭合白名单;未知值一律按 no-change,维护者列出的 5 个误报均已补充回归测试。该 M1 仍为 observation-only,没有 schema、scheduler、quota、gate 或权限行为变化。

@yuefengw
yuefengw force-pushed the codex/3203-stride-marker-boundary branch from e588d7f to d10ae4b Compare September 3, 2026 06:32
@yuefengw

yuefengw commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up: the final head is d10ae4b56 (the two commits were rewritten only to add the required DCO trailers). The implementation and test content is otherwise unchanged from the review update above.

All required checks are green, including Sign-off, pytest, Windows PowerShell, build, dependency review, and Sonar.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

结论:APPROVE。我在 exact head d10ae4b569ea48a8f4c3dfd5fd0bf24b5fa74a99 重新核对了先前 blocker;新的完整值白名单消除了否定词和描述性 classification 被当成 authority truth 的问题,未发现新的 blocking finding。

动机

本 PR 修复 authority stride 的观测误判。主干原先对 caller-supplied classification 做 substring 匹配,revision_reviewsupervision_checkwaiting_at_approval_gate 等并未发生 authority delta 的记录也会重置 bounded_slices_since_change。上一轮 whole-token 方案仍会把 not_replanvision_checkgate_not_approvedgate_status_recorded_without_transitionno_gate_changed 误判为 authority change。exact head 改为只承认当前受控 writer 使用的四个完整 classification 值,未知或描述性值一律按 no-change 处理;这正是 M1 不扩展 run schema 时所需的 fail-closed 最小修复。非目标仍保持清晰:不改变调度、Todo、quota、notification、gate 或执行权限。

改动思路

观测链路仍是 read_run_index 读取公开 run index,build_stride_observationagent_id 过滤记录,并从最后一个可信 authority-change 记录计算 stride。变化只发生在真假判定边界:_has_authority_change_marker 不再拆词或推断语义,而是先做现有 compact/casefold 规范化,再与 AUTHORITY_CHANGE_CLASSIFICATIONS 的闭集做 exact membership。

我核对了生产来源:operator-gate writer 通过 classification_for_decision 固定生成 operator_gate_approvedoperator_gate_rejectedoperator_gate_deferred;autonomous-replan obligation 固定投影 bounded_replan_progress,而无有效 delta 的 replan writeback 会被现有 qualification 降级为 replan_noop。因此正向路径能够重置一次 authority segment,未知、否定、记录型或 no-op 分类则不会。更完整的 typed authority-change receipt field 仍是后续理想方向,但本 PR 的 closed compatibility boundary 已消除当前可复现误判,并未增加第二套运行时 authority。

具体改动

  • 生产代码:loopx/control_plane/runtime/stride_observation.py 为 +25/-4。删除开放 substring 判定,引入四值 frozenset 和一个 fail-closed helper;build_stride_observation 的其余读取、过滤、计数、freshness 与 shadow-only 输出均未变化。
  • 测试:tests/control_plane/test_stride_observation.py 为 +106/-6。新增 15 个未知/否定/描述性反例,覆盖上一轮报告的五个 false positive 以及 replan_noop、记录型 replan、vision checkpoint 等近邻值;另用四个受控正例证明合法 authority decision 会重置 stride。
  • Scope fit:生产 helper 被 build_stride_observation 的 active projection path 直接调用,不是 coverage-only 代码;两文件、+131/-10 与问题规模相称,没有新增 schema、CLI surface、持久状态或迁移负担。

关键代码讲解

  1. AUTHORITY_CHANGE_CLASSIFICATIONS:把兼容期可作为 authority evidence 的完整值集中成闭集;值来自现有 operator-gate 与 autonomous-replan writer,而不是从任意 prose 中归纳。
  2. _has_authority_change_marker:保留输入规范化,但唯一决策是 exact membership。not_replangate_not_approved 等即使包含 marker token 也会 fail closed。
  3. build_stride_observation:只把 helper 返回 true 的最后一条 run 当成 segment 边界;输出仍为 read-only hierarchical_stride_observation_v0,失败/重试责任仍在上游 writer 与后续 typed-schema 演进。
  4. 两组 parametrized boundary tests:一组证明未知值保持 bounded_slices_since_change == 3,另一组证明四个受控值重置后结果为 1,正负 oracle 没有从实现输出反推。

对主干的风险

最强回归场景是遗漏一个真实 authority writer 的 classification,导致 false negative、让 stride 看起来比实际更长。当前代码路径扫描未发现闭集之外的生产 authority writer:route_continuation_replan_recordedsuccessor_replan_recorded、vision checkpoint 等是 delivery/记录语义,不应仅凭名字升级为 heavy authority;未知值继续 no-change 是有意的安全降级。回滚只需恢复旧 classifier,但会重新引入已证明的 false positive,因此更合适的长期恢复路径是增加 typed authority-change 字段并迁移本闭集。

验证结果:GitHub exact-head 9 项检查成功;focused stride + goal-outcome-continuity pytest 为 30 passed;Ruff、Python compile、diff hygiene 均通过;LoopX premerge 的 4 项 direct checks 与 12 项 catalog/risk-profile smokes 全部通过,0 failure、0 skip、0 manual hold;change-quality exact-scope receipt cqr_36ac4d2f0d6535d97303 有效且 gate passed。default-off isolation 不适用(无 feature gate/opt-in 变更);authority naming 与实际 gate/replan event 对齐;domain-neutrality、guidance-vs-obligation 未引入新风险。PR 描述中“standalone replan/vision/gate 仍被识别”的旧表述已落后于 exact head,建议合并前更新,但不影响代码 verdict。

我的整体评价

先前 blocker 已完整解决:分类真值不再来自 token 共现,所有已报告反例和相邻风险值都被锁定为 no-change,同时受控正例保留。实现直接、可回滚,复用了现有 normalization 与 writer vocabulary,helper 虽小但承载了明确的 fail-closed domain invariant,没有值得在本 PR 继续扩张的 companion refactor。剩余风险只在兼容期闭集需要随新 authority writer 演进;typed receipt field 可作为后续 schema 工作,不应继续阻塞这个 M1 correctness fix。

English verdict: APPROVE on exact head d10ae4b569ea48a8f4c3dfd5fd0bf24b5fa74a99. The previous blocker is resolved: authority resets now require one of four complete controlled classification values, while negated, descriptive, no-op, and unknown classifications fail closed. Focused tests, GitHub checks, the exact-scope quality receipt, and the full selected premerge gate all pass. The PR description still names obsolete standalone examples and should be refreshed before merge, but no blocking code finding remains.

@now-ing

now-ing commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

The fail-closed boundary is now exactly where it should be. AUTHORITY_CHANGE_CLASSIFICATIONS (stride_observation.py:21-30) plus exact membership in _has_authority_change_marker replaces every parsing heuristic with one closed set, and the new tests use a real oracle rather than echoing the implementation: test_boundary_unknown_classifications_stay_no_change locks 15 unknown/negated/descriptive values to bounded_slices_since_change == 3, while test_boundary_controlled_authority_classifications_reset_stride covers all four accepted values. Nice detail — the reworked test_boundary_authority_changes_require_explicit_markers fixture now pairs bounded_progress_report (index 0) against bounded_replan_progress (index 2), so the near-miss discrimination is pinned by data, not only by the parametrization. Reproduced on d10ae4b5: both PR-named test files, 30 passed; probes confirm operator_gate_approved v2, operator gate approved, and OPERATOR_GATE_APPROVED, all stay no-change.

Hardening points, all deferrable:

  1. The closed set duplicates writer literals with no mechanical tie. Three of the four values are produced solely by classification_for_decision (operator_gate.py:55-62) — I verified {classification_for_decision(d) for d in OPERATOR_GATE_DECISIONS} yields exactly the three operator_gate_* values — and the fourth exists only as an f-string literal at autonomous_replan_obligation.py:171. Nothing fails today if a new decision joins OPERATOR_GATE_DECISIONS or a classification gets renamed: the stride silently stops resetting (false negatives invisible to every current test). Since this compat window explicitly expects the set to track its writers, a one-line characterization test asserting the emitted decision classifications stay inside AUTHORITY_CHANGE_CLASSIFICATIONS would make that drift mechanical (deriving the subset via the function itself also works if the layering allows the import).

  2. The docstring says "exact value from the controlled writer set" but the matcher casefolds (stride_observation.py:47): Operator_Gate_Approved and OPERATOR_GATE_APPROVED also reset the stride (reproduced), while OPERATOR_GATE_APPROVED, does not. Controlled writers emit lowercase only, so this is harmless today — but either drop .casefold() to make the boundary literal, or document the match as case-insensitive-exact so contract and code agree.

  3. Optional placement: runtime classification vocabularies are otherwise centralized in runtime/status_classifications.py (re-exported through loopx.status). Hosting this set there keeps the vocabulary in one registry once it grows past four values.

@huangruiteng
huangruiteng merged commit aff3d21 into huangruiteng:main Sep 4, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants