Skip to content

[PR] fix: stop preflight marathon rounds that would blow the client's patience wall clock (#3573) - #3590

Open
ranxianglei wants to merge 11 commits into
masterfrom
2026-10-17_preflight-client-patience
Open

ranxianglei wants to merge 11 commits into
masterfrom
2026-10-17_preflight-client-patience

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: zhipuai-lb/glm-5.3)

修什么

一句话:老会话刚升到 v0.1.198 后,第一轮“预压缩马拉松”会一轮接一轮地折上下文,每轮要等一次 25–44 秒的模型摘要调用;客户端(dsh 一类)约 60 秒就断开,于是摘要调到一半被掐死、这一轮白干,用户看到的就是“每轮卡一分钟、反复断流”。这个 PR 让马拉松在“再跑一轮就会超过客户端耐心”的地方主动停:已完成的部分已经存盘,客户端重试时从更小的视图接着折,几轮内收敛,会话恢复。

同属 #3573 的另一半修复(另一半=鬼表,#3586:折叠后 usage 读数虚高把会话重新点燃)。两个 PR 互相独立、可分别合并。

机制

验证

  • 新增 tests/issue3573-preflight-client-patience.test.ts 2 用例(真代理端到端,外部摘要上游,窗口 12K/载荷 ~48K):
    1. 慢摘要(250ms/次)+ 400ms 预算缝:每次尝试恰好跑完第一轮即停(停点日志 + 502 + 无冷却毒化),折叠数逐次增长,重试循环最终 200 转发;A/B 反证:预算放到 60s(=修复前形态)同一用例必挂(单次尝试直接跑完马拉松)。
    2. 快摘要(5ms/次)+ 大预算:整个马拉松单次尝试完成、零停点日志——正常路径完全不受影响。
  • 亲缘回归 85 个用例全绿(issue2678/2682/2952/1987/1820/3573×2/1372/1841/1933/2639/3461/external-summary×N),typecheck + build 干净。

Fixes #3573

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-17_preflight-client-patience (48d5926)

Latest build — install this exact version

npm install -g [email protected]

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
48d5926 [email protected] build
0996c53 [email protected] build
8787747 [email protected] build
b1e1cf9 [email protected] build
142cd06 [email protected] build
8a59472 [email protected] build
3b34378 [email protected] build
7d915c5 [email protected] build
7689e38 [email protected] build
393f4d1 [email protected] build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3590.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 全仓检索(中英文多组关键词,open+closed)只命中本 PR 与姊妹 PR #3586(折叠后 usage 鬼表)——症状与机制都不同(本 PR = 马拉松墙钟失控;#3586 = 折叠后压力表虚高),issue 正文也声明两者独立可分别合并。无历史 issue/PR 做过轮级墙钟准入。

审核进行中(独立于 issue 侧结论重做):已通读 src/preflight.ts 全量 diff + 轮循环/判定串联、src/server.ts fail-fast 消费端(状态码/retryable/dead-end 冷却)、持久化链(markDirty→scheduleSave 经 runPrepare)、新测试文件全文;方向关(①)初判成立——修的是「轮循环只看轮数不看墙钟」这个机制本身,不是掐症状。目前发现一个边界路径的诚实性缺陷(wall-budget 停轮后仍超窗时,判定文案误报「预算耗尽 N 轮」,relaxed 子情形还会被映射成 413 终态而非 502 可重试——恰好破坏本 PR 自己「重试续折收敛」的故事),按小问题直接修纪律将在 PR 分支补一个标注 commit。

接下来:本地定向测试(issue3573 + 亲缘回归)+ typecheck/build → CI 结果复核 → 修 F1 → 八问结论回复。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论(独立重审,不复述 issue 侧分析):可合并。 新 head c38381881 全量 CI 绿(17 success / 13 skipped / 0 fail)。审核中发现 1 处边界路径诚实性缺陷,已按小问题直接修纪律在 PR 分支补标注 commit c38381881(见第 5 问)。

结构化八问:

  1. 是什么问题:老会话升级 v0.1.198 后,首轮预压缩马拉松一轮接一轮折上下文,每轮摘要调用要等 25–44s 上游延迟([Bug][preflight] folds run oldest-first, so a 180s pass burns its budget on an almost-exhausted head range while the largest foldable ranges are never dispatched #2682 实测,与范围大小无关);客户端(dsh)约 60s 就断,每次尝试跑不完就被掐死,用户看到反复卡死断流。
  2. 什么原因导致的:缺陷的生产者是轮循环的准入条件——只有 aborted/poolQueueDrop/轮数上限三个维度,没有墙钟维度。单轮成本由上游延迟决定,"轮数收敛"≠"墙钟收敛",这是机制缺一个维度,不是某个阈值没调对。
  3. 影响范围与严重程度:触发条件 = 慢上游摘要(>~25s/次)× 载荷超窗需多轮折叠(典型即升级后老会话)。严重度:非永久不可用——客户端重试可恢复,但旧行为下每次尝试都跑进分钟级再被掐断,体验是反复卡死;修复后几次快速重试内收敛。快上游路径完全不受影响(测试 2 钉住)。
  4. 怎么修复的:每轮开始前做预测准入「已耗时 + 上一轮实测×2 > 50s 墙预算」则停轮(第一条轮永远放行——没有测量值;紧急 digest 纯 CPU 通道豁免);停止不是失败——已完成折叠已持久化,fit 检查要么放行要么 502 快速失败(dead-end 冷却门要求 compressedRanges===0,不会毒化),客户端重试从更小视图续折。tier-recovery 漏斗([bug][preflight] Raw-range futility gate may strand tier-enabled sessions with distillable T1 blocks (502 deadlock) #2952)在同一调用点会再做上游调用,经 eligibleForTierRecovery() || wallBudgetHit 显式一并压制,否则绕过预算。
  5. 是否有回归问题:逐项审过既有行为,全部保持:① 轮循环 aborted/poolQueueDrop 退出与轮数上限语义不变;② tier-recovery 正常触发路径不变(仅新增 wallBudgetHit 抑制);③ 判定级联既有分支(budgetHit/relaxed/no-range)顺序与文案不变;④ server 端 fail-fast 状态码映射、retryable、dead-end 冷却门不变——wall 停轮必已落 ≥1 折,冷却不会被误武装(代码验证 + 测试断言无 COOLDOWN_MARK);⑤ 持久化链(runPrepare→markDirty→scheduleSave)未触碰,502 路径折叠落盘靠既有机制(测试断言每轮 block 数递增);⑥ plugin/proxy 双模式共用服务端 preflight,改动模式中立。审核发现 F1(已直接修,commit c383818):wall 停轮后若仍超窗,判定落入通用分支——常见子情形误报「the compress budget was exhausted after N rounds」(引用从未到达的轮数上限),relaxed 子情形更会置 deterministic → 被映射成 413 终态(无头宿主按惯例直接放弃重试),恰好杀死本 PR 自己的重试收敛故事。修法:级联首位加专属 wallBudgetHit 分支(kind exhausted + retryable:true + 诚实文案,不设 deterministic → 保持 502;冷却门不受影响),并在测试中钉住新文案(每个停轮尝试的 body 必须含 "client-patience wall budget")。
  6. 是否破坏缓存:不涉及——不动前缀缓存/增量游标/去重键;502 fail-fast 不写任何缓存状态,dead-end 标记逻辑未变。
  7. 是否改了设计方向:不涉及——架构/边界/数据流未转向,只给既有轮循环补上墙钟准入维度,与 [Bug][preflight] folds run oldest-first, so a 180s pass burns its budget on an almost-exhausted head range while the largest foldable ranges are never dispatched #2682 的成本测量一致。已知属性(F2,仅记录不修):预测界(上一轮实测×2)在病态延迟方差下(前一轮便宜+当前轮昂贵)可能仍超客户端耐心,优雅退化为修复前行为,属预测式准入的固有性质。
  8. 是否引入了新的配置:无——不加 env/config/CLI;PREFLIGHT_WALL_BUDGET_MS = 50_000 为模块常量,_setPreflightWallBudgetForTest 是测试专用 seam(仓库既有约定,finally 复位)。

测试有效性:两个用例都有牙——用例 1 逐次断言 502 + NOT_FORWARDED + 无冷却毒化 + 折叠逐次增长 + 停点日志,且必须 ≥2 次尝试才收敛到 200(单次尝试不可能收敛,这正是机制本身);A/B 反证(预算放 60s 必挂)目前是 PR 描述声明而非可执行测试,seam 在、可随时补跑,接受。用例 2 钉住快上游零影响。本地:F1 后 issue3573 2/2 过、typecheck 干净;亲缘回归 19 文件 186 例全绿(F1 前 head);最终 head c38381881 远程 CI 全量绿(authoritative)。

行为变更披露(规则 4):旧→新:修复前马拉松不管墙钟跑到轮数/预算耗尽(单次请求可能数分钟,被客户端超时掐在半路);修复后慢上游马拉松每次尝试约 1 轮即停,外部表现从「一次长请求反复被掐」变为「数次快速 502 + 收敛重试」。不做自动重试的客户端看到的结局有变化,但它们此前本来就会在马拉松中途死于自身超时(更差);快上游路径完全不变(测试 2 钉住)。

收工判定:可合并——方向关通过(修的是「轮循环不看墙钟」这个机制本身,不是掐症状)、回归审计干净、CI 全绿、F1 已修并钉住。姊妹 PR #3586(鬼表)独立,可分别合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:与 #4520 / #4530(master docs-sync 全红)同根因——缺种子行已由 PR #4524 修复并于今日 11:27Z 合入 master(merge commit 8ae56a0)。本 PR 分支不含任何文档改动,无可修内容,未重复修复;此处只做「验证 + 触发重跑」。

结构化八问

  1. 是什么问题:本 PR 的 docs-sync 检查红了。表面看是本 PR 的问题,实际是 stale-base 受害——master 自 ad98e7e(Plugin lanes: pin identity to client msgids, render outbound from the server-owned history | 插件车道服务端正史:消息身份钉在客户端 id 上,出站前缀由正史渲染 #4491/[PR] feat: plugin-lane server-owned history — msgid pinning + transcript render endpoint | 插件车道服务端正史:msgid 钉身份 + 正史渲染端点 #4505 插件车道服务端正史)合入后,任何包含该 master 状态的合并树 docs-sync 必红。
  2. 什么原因导致的:[PR] feat: plugin-lane server-owned history — msgid pinning + transcript render endpoint | 插件车道服务端正史:msgid 钉身份 + 正史渲染端点 #4505 合入时手改 CONFIGURATION.md / .zh-CN.md 的生成块,塞入一行 rollout.features.pluginLaneServerHistory,但从未在 website/config-reference/rollout.yaml 写对应种子条目,且该行含反引号(种子校验器明令禁止)。生成器纯靠种子重建表格 → 再生成必然删掉该行 → diff 红。已实证:在未改动的 master(c81c17d)上直接跑生成器,产生同样的 diff——master 自身就是红的。
  3. 影响范围与严重程度:所有在途 PR 的 docs-sync/docs-coverage 全红,阻塞合并流程;非运行时行为问题,不影响已发布功能(该功能 percent:0 默认关闭且本身完好)。
  4. 怎么修复的:根修复由 [PR] docs: restore dropped pluginLaneServerHistory seed row — unblocks docs-sync on master #4524 完成(补回种子条目 + 生成器重写两份 MD、去反引号),已合入 master。我确认本 PR 分支零文档改动后,在新合并树(本 PR head 393f4d1 + 新 master 8ae56a0)本地验证:merge 无冲突、生成器对 4 个受控文件逐字节复现通过、coverage 检查通过——docs-sync 将绿。随后向本 PR 分支推了一个空提交(7689e38,message 注明用途),让 GitHub 基于修复后的 master 重算 merge ref 并重跑 CI。
  5. 是否有回归问题:无。本分支仅增一个零 diff 的空提交,不碰代码与行为;master 侧 [PR] docs: restore dropped pluginLaneServerHistory seed row — unblocks docs-sync on master #4524 为纯文档/种子修复,无行为面。
  6. 是否破坏缓存:不涉及(未触碰任何运行时路径)。
  7. 是否改了设计方向:无。
  8. 是否引入了新的配置:无。

状态:新 CI 运行 40090677602 已在跑(docs-sync 因 runner 排队尚未开始执行);依据本地逐字节复现,该项将绿。若运行结束后另有失败项,请再转发。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):目标 issue #3573 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

@ranxianglei ranxianglei reopened this Oct 24, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

Reopening under the condition written in your 10-24 closing comment ("如判断有误请直接 reopen 并留言"). Evidence the closure was premature:

  1. This PR passed review on 10-17 — the v0.1.198 升级后老会话丢失压缩状态:197 折叠过的会话以 117 万 token 原文重放,预压缩超过客户端耐心,老会话看起来「炸了」 #3573 thread records "审核通过,可合" for exactly this branch, before the cleanup batch closed it.
  2. Current master (146b34f) contains no wall-clock admission control in the preflight marathon path. The only deadline machinery present is the external-summary pool's shared per-pass deadline (src/preflight.ts:256, :2356) — that caps pool congestion within a pass, not total marathon duration against client patience.
  3. P1 [Bug] Excessive preflight compression time on ultra-large contexts; client mid-disconnect causes repeated request forwar | [Bug] 超大上下文 preflight 压缩耗时过长,客户端中途断开导致请求反复无法转发 #3978 remains OPEN, and its own floor-5 plan pairs this PR (wall half) with [PR] fix: persist each committed preflight fold so a dead marathon leaves durable progress (#3978) #4046 (persistence half) as the two halves of one loop closure. [PR] fix: persist each committed preflight fold so a dead marathon leaves durable progress (#3978) #4046 is still open awaiting human merge; with this PR closed, the A-half is ownerless.
  4. Post-review field data (edisonzerolam, 10-22, v0.1.204): 8 rounds / 288 s, deficit still 1.18M tokens at client abort — precisely the shape this PR bounds.

Next step (execution, no new decision): rebase 2026-10-17_preflight-client-patience onto current master, run targeted typecheck/build/tests, push, then human merge as usual. If the intent was to drop this half entirely (superseded by something specific), say so in-thread and I'll stop here rather than rebase into a superseded design.

ranxianglei and others added 3 commits October 25, 2026 05:59
…ence wall clock (#3573)

A summarization round costs 25-44s against a real upstream regardless of
range size (#2682), while the common client timeout class (dsh ~60s) only
fits ONE such round. An over-window preflight kept admitting rounds until
the client disconnected mid-round: the attempt died, the in-flight summary
call was wasted, and the user saw an opaque 60s hang-retry loop (#3573).

Rounds that make upstream calls are now admitted only while the wall clock
leaves room for the next one (estimate: 2x the last round's measured time,
first round always admitted, digest regime exempt — CPU-only by design).
A stopped attempt keeps its completed folds (they are persisted), so the
client's retry resumes from the smaller view and the marathon converges
across attempts; the fail-fast stays a plain 502 with no dead-end cooldown,
and the tier-recovery funnel is suppressed for the stopped attempt so it
cannot sneak extra upstream calls past the budget.

Fixes #3573 (companion to #3586 — same issue, independent half).
…rdict (#3573 review)

When the client-patience wall budget stops the marathon while the payload still overflows the window, the post-loop verdict fell through to the generic branches: 'the compress budget was exhausted after N rounds' (citing a round cap that was never reached), or — in the relaxed-regime subcase — a deterministic exhaustion that server.ts maps to 413, a terminal verdict headless hosts honor by giving up. That kills exactly the retry-convergence story this fix exists for: the folds ARE persisted and the next request resumes from a smaller view.

Give the wall-budget stop its own first-in-cascade verdict: kind exhausted with retryable true and an honest detail (rounds run, folds persisted, retry resumes). No deterministic flag -> stays 502; the dead-end cooldown gate (compressedRanges === 0) is untouched because a wall-budget stop always has at least one landed fold. The issue3573 test now pins the honest wording in every stopped attempt's body.
@ranxianglei
ranxianglei force-pushed the 2026-10-17_preflight-client-patience branch from 7689e38 to 7d915c5 Compare October 24, 2026 22:15
…aster-red #4599, same one-liner as #4603)

Master-born red since #4545 landed: 0fe7589 removed the non-dsh lanes'
consumers of ClientBuildCheck (owner decision: keep the note to the dsh lane),
leaving the type referenced only inside client-build-note.ts itself, and the
4d34abf de-export sweep missed it. Every PR rebased onto tip inherits the
unused-exports failure (tracked in #4599; fix PR #4603 pending merge). This
branch carries the identical one-line fix as its own labeled commit per the
few-line-fix discipline (duplicate fixes across PRs are acceptable).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:本失败与 #4602/#4603(master born-red unused-exports)同机制,已在 CI 总控 #4599 登记;#4559/#4607 等分支已各自带同一行修复(重复修复可接受)。未立新 issue,仅在本 PR 分支顺手修 + #4599 留结论。

结构化八问

  1. 是什么问题:rebase 后的新 head(7d915c5)在 unused-exports 门禁红了。表面是本 PR 的 CI,实际是 master 继承性失败——master 自 [PR] dsh lane: note in-session when running an unverified @deepseek-ai/dsh client build (soft, exact-match) #4545 合入后该项自身就红。
  2. 什么原因导致的:[PR] dsh lane: note in-session when running an unverified @deepseek-ai/dsh client build (soft, exact-match) #4545 引入 type-only 类型 ClientBuildCheck,随后 0fe7589(owner 决策:client-build note 只保留 dsh 车道)删掉了其余车道的消费点,该类型的引用只剩它自己文件内部;4d34abfaf 的去导出清扫漏了它 → 门禁判 INTERNAL_ONLY。凡 rebase 到 master tip 的 PR 都继承此红。本 PR 自己的改动(preflight 墙钟准入)与该符号无关——本 PR 新增的 _setPreflightWallBudgetForTest 有正常引用、不触发门禁。
  3. 影响范围与严重程度:所有在途 PR 的 unused-exports 门禁全红,阻塞合并流程;无运行时影响(type-only、零外部引用,去 export 关键字不改任何行为)。
  4. 怎么修复的:src/agent/client-build-note.ts:32 去掉前导 export(一行),独立标注 commit 3b34378c5,与 [PR] fix(ci): drop the unused export of ClientBuildCheck #4603/[PR] fix(pi): inline mode now intercepts codex Responses WebSocket traffic (#4526) #4559/[PR] fix(storage): block interval bounds survive post-fold resaves (#4592) #4607 同一修法。master 侧根治由 [PR] fix(ci): drop the unused export of ClientBuildCheck #4603 承担,CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 已留言。
  5. 是否有回归问题:无。改动仅为一个零外部引用类型的导出修饰符(文本计数前后均证实只在本文件内使用);本地门禁 ✓ clean (6 waived) + tsc --noEmit 全绿。
  6. 是否破坏缓存:不涉及(未触碰任何运行时路径)。
  7. 是否改了设计方向:无。
  8. 是否引入了新的配置:无。

状态:已 push(新 head 3b34378c5,fast-forward),等 CI 重跑。提示:按 #4599 盘点,master 另有常红项(test 矩阵 LIVE_FILES tripwire 等)——若重跑后本 PR 在这些项上再红,同属继承性失败,走 #4599 流程处理,不属于本 PR 分支可修范围。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复(10-17 首轮已答;姊妹修复 #3586 鬼表 / #4046 持久化半独立,机制与症状均不同)。

执行 floor 10(reopen 评论)声明的计划:rebase 到当前 master → 定向验证 → push → 等人工合并。已完成前三步,分支新 head 3b34378。

做了什么

验证

八问

  1. 是什么问题:不是新缺陷——陈旧清理误关后 reopen 的 PR 落后 master 10 个提交,需 rebase+验证+push 重新达到可合并态(floor 10 已声明此计划)。
  2. 什么原因导致的:不涉及(无新根因;CI 红均为 master 出生即红的继承项,见 CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 盘点)。
  3. 影响范围与严重程度:仅本 PR 的检查矩阵(信号层面),无用户面影响;合并前用户侧行为不变。
  4. 怎么修复的:顺序 cherry-pick 线性化(保作者、保内容、丢 merge/空 commit),[PR] fix: measure completed preflight round before wall admission #3732 两缺陷修复并入,门槛让位按风暴惯例独立成 commit 可事后丢弃。
  5. 是否有回归问题:零冲突;四处机制位置新行号复读通过;本地 224/224 含 [PR] fix: measure completed preflight round before wall admission #3732 加固断言。已知观察项:[PR] fix(ci): clear the two master-red gates in one branch — unused-exports (#4602) + the #3890 LIVE_FILES tripwire (#4604) #4606 修好 tripwire 后分片真正开跑时,若 sentinel 清单里的 born-red 用例 tests/preflight-client-abort.test.ts 在本 head 失败,需对照 [PR] fix(ci): clear the two master-red gates in one branch — unused-exports (#4602) + the #3890 LIVE_FILES tripwire (#4604) #4606 后的 master 归因,再判是否本 PR 引入。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:无(墙钟准入机制自 10-17 起未变,[PR] fix: measure completed preflight round before wall admission #3732 只修其测量时机与计数差一)。
  8. 是否引入了新的配置:无(PREFLIGHT_WALL_BUDGET_MS 常量 + 测试 seam,10-17 已披露)。

行为变更:相对 10-17 披露无新增(慢上游马拉松改为「每尝试约一轮 + 快速 502 + 重试续折收敛」,快路径不受影响,测试 2 钉住)。

结论:代码面就绪,等人工合并。 前置条件是 master 变绿(#4606 合入);届时 rebase 丢弃让位 commit、test 分片全量远端跑一遍即可。持久化另一半 #4046 仍独立等待人工合并。

ci-check added 5 commits October 26, 2026 04:13
master window commits 293ae65/de5e03659 bumped kernel/package.json to
0.0.165 without regenerating the lockfiles; lock-sync re-resolution then
rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and
fails. Pure recorded-version sync via npm install --package-lock-only;
no dependency or semantic change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant