Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -638,6 +638,140 @@ decision through the same `previous_review` requirement. Feature-off context
and legacy decision references remain unchanged. This repairs feedback
delivery; execution quality and goal benefit still require separate evidence.

<a id="work-meta-goal"></a>

#### Meta-goals and single-Todo acceptance (proposal)

A **meta-goal** is a planning objective for pursuing one or more existing goals.
It specifies what should be true of a plan, a choice among candidate plans, or
the relationship between plans, and how to tell when that condition is met.
A decision question identifies an uncertainty that could change the choice.
The same meta-goal can span Turns as questions and actions change.
[Wilensky, §2.2](https://www2.eecs.berkeley.edu/Pubs/TechRpts/1980/Archive/ERL-m-80-33.pdf)

| Meta-goal | Example and success condition | Possible Stride action |
|---|---|---|
| Resolve a conflict between goals | Two Goals may need the same limited test window. Find an order or alternative route that meets both sets of acceptance criteria and deadlines. If the commitments cannot both be met, the authorized Goal owners decide the trade-off. | `verify` whether the windows overlap, or `switch` to test a mechanism that can run independently. |
| Combine overlapping plans | Two plans require equivalent validation against the same input revision. Run it once with coverage for both sets of criteria, and let each acceptance decision reference the shared result. | `verify` coverage if it is uncertain. Confirmed coverage can support `grow` on the shared route within its validated scope. |
| Choose the highest-value feasible continuation | Compare expected goal value within the same acceptance, deadline, authority and resource constraints. When expected outcomes, risks and later options are equivalent, prefer the lower full cost. | Compare `grow` on a supported route with a `switch` experiment. Use `verify` when the evidence could improve that choice enough to justify the probe. |

Stride chooses the next action in pursuit of the meta-goal. `verify` gathers
evidence that can change the choice at a justified cost. `grow` expands work on
a supported route. `switch` tests a materially different mechanism. The same
meta-goal can call for different actions as evidence arrives. Plan changes
such as reordering work or sharing a dependency use the existing Todo paths.
Goal commitments, acceptance, budget and authority remain with their owners.

**Value of a continuation.** Compare the incremental utility delivered by the
current slice and the expected additional utility from later choices. One
Bellman-style expression for this comparison is

$$
Q(s,b,a)=\mathbb E\!\left[\Delta U_{\mathrm{now}}+
D(\Delta t)V_{\mathrm{future}}(S',b-K)\mid s,b,a\right].
$$

Here `s` and `b` are the current state and remaining resources. An action
uses resources `K` and leads to state `S'` with updated results and evidence.
`D(Δt)` discounts utility over the time until the next decision. Use one when
time is governed solely by the deadline. Define utility from the Goal's
acceptance criteria. The future term counts the additional utility available
from feasible choices after the current slice.

Information has value through better later choices. Estimate that value by
comparing the later decision with and without the new observation at the same
time and resource point. Count it once in the evidence-conditioned future
term. An action can deliver utility and informative evidence together.
When comparing a probe with acting directly, include the probe's resource
use and delay. Terminal-outcome objectives and stopping follow the existing
[budgeted planning model](target-yardstick-stride-decisions-v0.md#budgeted-planning).

**First acceptance trial.** Use one Todo to deliver a faster scheduler under
its existing correctness and budget constraints. The meta-goal is to select
the optimization route that warrants further investment. The decision
question is whether the candidate's gains carry over to peak load. Compare
tuning immediately with spending part of the same budget on a peak-load
probe that could change which route to scale. State the objective and choice
in the existing Stride plan and reasons. For the probe path, choose `verify`
and use `next_probe` to compare the candidate and incumbent under matched peak
conditions, with a prediction, minimum evidence, adoption criterion and cost bound.

**Worked scheduler example.** Give the Todo 10 budget units and a hard
deadline, with $D=1$. In this example, a 10% accepted peak-throughput gain
over the incumbent is worth 10 utility points. The prior probability that
the current mechanism's gain transfers to peak load is $P(T)=0.60$.
At remaining budget $r$, `grow` earns $r$ points if $T$ holds and retains the
incumbent at 0 points otherwise. `switch` earns $r/2$ points in either state.
Both payoffs assume $r$ covers implementation, any switch, and final
validation under the same acceptance criteria.

The probe costs 1 unit, leaves 9 for either route, and delivers 0 immediate
throughput points. Assume the matched peak comparison passes its correctness
checks and returns one of two valid results. A $+$ means the candidate reaches
at least the declared 5% gain, and a $-$ means it falls below it. Assume
symmetric sensitivity and specificity $P(+\mid T)=P(-\mid \neg T)=q=0.90$.
Let $p$ be the transfer probability given the evidence so far. At any
positive remaining budget $r$, `grow` beats `switch` when $p>0.50$, since
their expected values are $rp$ and $r/2$.

| Valid probe result | Probability | Transfer probability after the result | Next choice |
|---|---:|---:|---|
| $+$ | 0.58 | $0.54/0.58 \approx 93.1\%$ | `grow` |
| $-$ | 0.42 | $0.06/0.42 \approx 14.3\%$ | `switch` |

| Continuation | Total budget | Expected accepted gain (points) |
|---|---:|---:|
| Direct `grow` | 10 | $0.60\times10=6.00$ |
| Direct `switch` | 10 | $10/2=5.00$ |
| `verify`, then choose | $1+9=10$ | $0.54\times9+0.42\times4.5=6.75$ |

In the last row, 0.54 is the joint probability of transfer and a positive
signal. The alternative earns 4.5 points in either state after a negative
signal. Using prior evidence with the same 9 units gives
$\max(0.60\times9,4.5)=5.40$. The information is worth $6.75-5.40=1.35$
points, the budget it consumes costs $6.00-5.40=0.60$ in foregone utility,
and the net gain over direct `grow` is 0.75 points.

For symmetric accuracy $0.50\le q\le0.60$, `grow` remains optimal after
either result, giving the information zero decision value. When $q>0.60$,
the results fall on opposite sides of the 50% decision threshold.
For probe cost $k$, the probabilities $P(T,+)=0.60q$ and $P(-)=0.60-0.20q$
give expected value $(10-k)[0.60q+(0.60-0.20q)/2]=(10-k)(0.30+0.50q)$.
With feasible follow-up work, probe when this exceeds 6.00. At $q=0.90$,
this requires $k<2$ units. At $k=1$, it requires $q>11/15$ (about 73.3%).
Equality gives the same expected value as direct `grow`. If a probe can
return an inconclusive result, include that outcome's probability and its
continuation value at the remaining budget in the comparison.

In a later authorized Turn, use `decision_followthrough` and `previous_review`
to tie the next choice to the exact candidate, workload and observed result.
Read back all three outcomes across the trial cases.

- The evidence favors expanding the current route and shows useful headroom.
Choose `grow` and execute a larger coherent slice within the remaining budget.
- The evidence favors testing a different mechanism. Choose `switch` and run a
bounded test of that mechanism.
- The cost bound is reached with insufficient evidence. Record the choice as
unresolved and handle settlement or waiting through the existing owner.

Use the existing Todo, Turn admission and settlement paths. Independently
verify the action taken. For executed follow-up work, check whether it
followed the decision and how its result compares with the original acceptance
criteria. Record complete costs, including inconclusive probes, in all three cases.

**Follow-up work.** For cross-Todo planning, start with two Todos within one
Goal. Check that shared validation covers each Todo's acceptance criteria and
read back both consumers' use. Cross-Goal changes follow the authority of the
affected Goals.

For cross-task strategy evolution, use verified decision episodes to propose a
bounded, versioned Stride strategy change through its owner. An independent
evaluator compares it with the current version on held-out tasks under the same
model, starting conditions and budget. Read back actual adoption and outcomes
in later authorized work. Use those observations and the evaluation verdict to
decide whether to revise, retain or withdraw the candidate and what to try next.
[Capability evolution research](frontier-science-research-program-v0.md#t07-diversity-preserving-capability-evolution)

#### An iterative strategy and engineering boundary

The first strategy is **model-guided, evidence-conditioned work planning**.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -546,6 +546,105 @@ Turn、义务、settlement identity 与已接受的回执必须一致;派生
context 和历史决策引用保持兼容。这里修复的是反馈送达,执行质量和目标收益
仍需独立证据。

<a id="work-meta-goal"></a>

#### 元目标与单 Todo 首版验收(提案)

**元目标**是关于如何实现一个或多个既定目标的规划目标。它规定一个计划、候选计划的
选择或计划之间的关系应达到什么状态,以及如何判断该状态已经达到。决策问题指出
哪项不确定性可能改变选择。同一个元目标可以跨 Turn 延续,具体问题和动作随证据变化。
[Wilensky §2.2](https://www2.eecs.berkeley.edu/Pubs/TechRpts/1980/Archive/ERL-m-80-33.pdf)

| 元目标 | 例子与达成条件 | 可选的 Stride 动作 |
|---|---|---|
| 解决目标之间的冲突 | 两个 Goal 可能需要使用同一个有限的测试窗口。找到能同时满足双方验收和期限的顺序或替代路线。若双方承诺确实无法同时兑现,由获授权的 Goal 归属方决定取舍。 | 用 `verify` 核对窗口是否重叠,或用 `switch` 测试能独立运行的另一种机制。 |
| 合并重叠计划 | 两个计划需要对相同输入版本做等价验证。执行一次覆盖双方验收条件的验证,各自的验收决定均可引用该结果。 | 覆盖范围不确定时用 `verify`。已确认的覆盖可以支持在验证范围内用 `grow` 扩大共享路线。 |
| 选择预期价值最高的可行接续 | 在相同验收、时限、权限和资源约束下比较预期目标价值。预期成果、风险和后续选择等价时,选完整成本更低的方案。 | 比较用 `grow` 扩大已有依据的路线与用 `switch` 测试另一种机制。若证据能改善这一选择且值得探针成本,使用 `verify`。 |

Stride 选择推进元目标的下一步。`verify` 用合理成本获取可能改变选择的证据。`grow`
扩大已有证据支持的路线。`switch` 测试实质不同的机制。同一个元目标可随新证据改用
不同动作。重排工作、共享依赖等计划变更沿用既有 Todo 路径。Goal 承诺、验收、预算
和权限仍由原归属方管理。

**接续方案的价值。** 比较当前切片带来的增量效用,以及后续选择能带来的预期增量效用。
可以用 Bellman 形式表达这一比较。

$$
Q(s,b,a)=\mathbb E\!\left[\Delta U_{\mathrm{now}}+
D(\Delta t)V_{\mathrm{future}}(S',b-K)\mid s,b,a\right].
$$

`s` 和 `b` 分别是当前状态和剩余资源。动作消耗资源 `K`,得到包含新结果和新证据的
状态 `S'`。`D(Δt)` 按到下一次决策的时间折现效用。若时间只受截止期限约束,取 1。
效用由 Goal 的验收标准定义。未来项计入当前切片结束后仍可通过可行选择实现的增量效用。

信息通过改善后续决策产生价值。在相同的时间和资源条件下,比较看到新观测与沿用
已有证据时的后续选择,估计由此增加的预期效用。这部分价值在基于新证据的未来项中
计一次。一个动作可以同时产出有用结果和决策证据。比较探针与直接推进时,计入探针
的资源消耗和延迟。终局成果目标及停止条件沿用
[预算规划模型](target-yardstick-stride-decisions-v0.zh-CN.md#budgeted-planning)。

**首版验收。** 用一个 Todo 在既有正确性与预算约束下交付更快的调度器。元目标是
选出值得进一步投入的优化路线。决策问题是候选方案的收益能否迁移到峰值负载。
比较立即调优与在同一预算内先做峰值探针,后者可能改变要扩大的路线。在既有 Stride
计划与理由中写清目标和选择。采用探针时选择 `verify`,用 `next_probe` 在相同峰值
条件下比较候选与基线,并声明预测、最低证据、采用条件和成本边界。

**调度器算例。** 假设该 Todo 有 10 个预算单位,时间只受硬截止期限约束,取 $D=1$。
效用以通过原验收的峰值吞吐增幅计分。相对基线增长 10% 计 10 分。当前机制的收益能
迁移到峰值负载的先验概率为 $P(T)=0.60$。剩余预算为 $r$ 时,若 $T$ 成立,`grow`
获 $r$ 分,否则保留基线,计 0 分。`switch` 在两种状态下均获 $r/2$ 分。两条路线
都用 $r$ 覆盖实施、必要的切换和最终验证,沿用相同验收标准。

探针花 1 个单位,留 9 个单位给后续路线,当期吞吐增益计 0 分。假设同一峰值负载下的
对照通过正确性检查,并返回两种有效结果之一。候选达到预先声明的 5% 增幅记为 $+$,
低于该增幅记为 $-$。假设灵敏度和特异度相同,$P(+\mid T)=P(-\mid \neg T)=q=0.90$。
令 $p$ 为依据已有证据估计的迁移概率。只要剩余预算 $r$ 大于 0,$p>0.50$ 时
`grow` 的期望值 $rp$ 就超过 `switch` 的 $r/2$。

| 有效探针结果 | 概率 | 观测后的迁移概率 | 下一步 |
|---|---:|---:|---|
| $+$ | 0.58 | $0.54/0.58 \approx 93.1\%$ | `grow` |
| $-$ | 0.42 | $0.06/0.42 \approx 14.3\%$ | `switch` |

| 接续方案 | 总预算 | 期望验收收益(分) |
|---|---:|---:|
| 直接 `grow` | 10 | $0.60\times10=6.00$ |
| 直接 `switch` | 10 | $10/2=5.00$ |
| 先 `verify` 再选 | $1+9=10$ | $0.54\times9+0.42\times4.5=6.75$ |

最后一行的 0.54 是收益可迁移且探针为正的联合概率。探针为负时,切换机制在两种
状态下都获 4.5 分。同样只剩 9 单位时,沿用先验信息的最优期望为
$\max(0.60\times9,4.5)=5.40$。信息价值为 $6.75-5.40=1.35$ 分,消耗 1 单位
预算放弃的效用为 $6.00-5.40=0.60$ 分,相对直接 `grow` 的净收益为 0.75 分。

当对称准确率 $0.50\le q\le0.60$ 时,两种结果下 `grow` 均仍是最优选择,信息的
决策价值为零。$q>0.60$ 时,两个结果分落在 50% 选择阈值两侧。联合概率
$P(T,+)=0.60q$,负结果概率 $P(-)=0.60-0.20q$。成本为 $k$ 的探针期望值为
$(10-k)[0.60q+(0.60-0.20q)/2]=(10-k)(0.30+0.50q)$。后续工作可行且期望值
超过 6.00 时,探针值得做。$q=0.90$ 时要求 $k<2$ 个单位。$k=1$ 时要求
$q>11/15$,约为 73.3%。等号表示与直接 `grow` 的期望值持平。若探针可能无结论,
比较时还要计入这种结果的概率和其剩余预算下的后续价值。

在后续获准的 Turn 中,通过 `decision_followthrough` 和 `previous_review` 将下一次
选择与确切的候选、负载和观测结果关联。在不同验收案例中读回三种结果。

- 证据更支持扩大当前路线,且仍有实际收益空间。选择 `grow`,在剩余预算内执行更大的连贯工作切片。
- 证据支持测试另一种机制。选择 `switch`,执行该机制的有界测试。
- 到达成本边界时证据仍不足。记录选择未决,通过原归属方处理结算或等待。

沿用既有 Todo、Turn 准入和结算路径。独立核验实际采取的动作。对于已执行的后续工作,
检查它是否遵循该决定,以及结果与原验收标准是否相符。三种情况均记录完整成本,包括
未能得出结论的探针。

**后续工作。** 跨 Todo 规划先从同一 Goal 内的两个 Todo 开始。检查共享验证能否
覆盖各自的验收条件,并读回双方的实际使用。跨 Goal 变更遵循相关 Goal 的权限边界。

跨任务演进使用已核验的决策案例,由 Stride 策略归属方提出有界的版本改动。独立评测方
固定模型、起始条件和预算,在留出任务上比较候选与当前版本。通过后续获准工作读回实际
采用和结果。结合评测结论与这些观测,决定修订、保留或撤回候选,以及下一轮试什么。
[能力进化研究](frontier-science-research-program-v0.zh-CN.md#t07-保留多样性的能力进化)

#### 可迭代策略与工程边界

第一版策略是**模型依据证据规划完整工作**:显式选择 grow(扩大有支持的路线)、
Expand Down
Loading