Skip to content

[Klaud Cold] Update dsr1-fp4-b200-sglang SGLang image to v0.5.21-cu130 / [Klaud Cold] 将 dsr1-fp4-b200-sglang 的 SGLang 镜像更新至 v0.5.21-cu130 - #3673

Closed
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-d92e15065c55779d-ed0569e9ff1aef5d
Closed

Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-d92e15065c55779d-ed0569e9ff1aef5d

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:v0.5.19-cu130 to lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469.
Baseline: 2026-09-16 · lmsysorg/sglang:v0.5.19-cu130
Mean latency · Sources: API 1, API 2

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 3bfc5c 407.4 45.51 208.32 5.27
8k/1k c1 TP4 EP1 620784 N/A N/A N/A N/A
8k/1k c2 TP4 EP1 2a3d39 N/A N/A N/A N/A
8k/1k c2 TP4 EP1 bab69f 704.42 79.05 257.26 6.04
8k/1k c4 TP4 EP1 684a05 N/A N/A N/A N/A
8k/1k c4 TP4 EP1 a8bdd9 1,179.46 131.21 288.36 7.15
8k/1k c8 TP4 EP1 06004a N/A N/A N/A N/A
8k/1k c8 TP4 EP1 425d06 1,828.31 205.53 379.49 9.17
8k/1k c16 TP4 EP1 83585a N/A N/A N/A N/A
8k/1k c16 TP4 EP1 af369a 2,584.54 286.38 650.3 12.92
8k/1k c32 TP4 EP1 bea97b 3,487.5 390 1,034.79 18.97
8k/1k c32 TP4 EP1 e5e256 N/A N/A N/A N/A
8k/1k c64 TP4 EP4 502b5d N/A N/A N/A N/A
8k/1k c64 TP4 EP4 a9e5b1 4,918.77 545.69 1,319.72 27.24
8k/1k c128 TP4 EP4 c77207 6,637.64 735.22 1,834.97 40.75
8k/1k c128 TP4 EP4 e8c737 N/A N/A N/A N/A
8k/1k c256 TP4 EP4 368d10 8,365.57 929.78 6,928.52 60.21
8k/1k c256 TP4 EP4 e506d2 N/A N/A N/A N/A

Note: 8k/1k c1 TP4 EP1 620784, 8k/1k c2 TP4 EP1 2a3d39, 8k/1k c4 TP4 EP1 684a05, 8k/1k c8 TP4 EP1 06004a, 8k/1k c16 TP4 EP1 83585a, 8k/1k c32 TP4 EP1 e5e256, 8k/1k c64 TP4 EP4 502b5d, 8k/1k c128 TP4 EP4 e8c737, 8k/1k c256 TP4 EP4 e506d2: unavailable.

Eval: N/A

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:v0.5.19-cu130 更新为 lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469。
**基线:**2026-09-16 · lmsysorg/sglang:v0.5.19-cu130
平均延迟 · 来源: API 1, API 2;数值及异常说明见上表。

Pin lmsysorg/sglang:v0.5.21-cu130 by digest and replace the
--cuda-graph-max-bs and --disable-piecewise-cuda-graph aliases, which
v0.5.21 removed, with --cuda-graph-max-bs-decode and
--cuda-graph-backend-prefill=disabled, their v0.5.19 targets.

将 dsr1-fp4-b200-sglang 的 SGLang 镜像更新为按 digest 固定的
lmsysorg/sglang:v0.5.21-cu130,并将 v0.5.21 已移除的
--cuda-graph-max-bs 和 --disable-piecewise-cuda-graph 别名替换为其在
v0.5.19 中对应的 --cuda-graph-max-bs-decode 和
--cuda-graph-backend-prefill=disabled。

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 37009882020 / attempt 1 · 2026-10-02 13:30 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · 15b01bb526bd · Mean latency
Change: Pin SGLang v0.5.21-cu130 by digest and replace the removed --cuda-graph-max-bs and --disable-piecewise-cuda-graph aliases with their v0.5.19 targets --cuda-graph-max-bs-decode and --cuda-graph-backend-prefill=disabled (v0.5.19 aliases, v0.5.21 fields); the bundled image source is commit e00930c5 and the old flags fail argparse there; measured ref d090548.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 620784 44.44 (N/A) 224.01 (N/A) 5.39 (N/A)
8k/1k c64 TP4 EP4 502b5d 547.16 (N/A) 1,315.66 (N/A) 27.16 (N/A)

Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c32 95.45% (N/A) N/A/1,319 (old/new)
gsm8k/em_strict · c128 95.38% (N/A) N/A/1,319 (old/new)

Note: All rows: Δ N/A: no matched eval baseline.

Next: Stop before the final sweep: check-final rejects the matrix because the 9 published baseline points (c1-c256, run 34557177019 at 25f8f98) predate the #3352 srt-recipe migration and cannot be regenerated.

中文

初次尝试 · 已通过 · Run 37009882020 / attempt 1 · 2026-10-02 13:30 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · 15b01bb526bd · 平均延迟
**变更:**按 digest 固定 SGLang v0.5.21-cu130,并将已移除的 --cuda-graph-max-bs 与 --disable-piecewise-cuda-graph 别名替换为其在 v0.5.19 中对应的 --cuda-graph-max-bs-decode 与 --cuda-graph-backend-prefill=disabled;镜像内源码为 e00930c5,旧参数在其中无法通过 argparse;测量提交为 d090548。实测数值及异常说明见上表。
**下一步:**在最终 sweep 前停止:check-final 拒绝该矩阵,因为 9 个已发布基线点(c1-c256,run 34557177019,提交 25f8f98)早于 #3352 的 srt-recipe 迁移,无法重新生成。

为 dsr1-fp4-b200-sglang 的 SGLang v0.5.21 更新添加 perf-changelog 记录。

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

failed · Cleanup pending. Stop and confirm owned runs before closing.

中文

failed · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 2, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-d92e15065c55779d-ed0569e9ff1aef5d branch October 2, 2026 13:31
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

failed · Repairs: 0 · Runs: 37009882020, 37013298309
All owned runs ended. PR closed; branch deleted for retry.

中文

failed · 修复次数:0 · 运行:37009882020, 37013298309
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant