[AgentX][Dsv4] SGLang update - #3426
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
Oasis-Git
added
the
full-sweep-enabled
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36463674051 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36463674051 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and found no bugs. Because it touches configs/nvidia-master.yaml (a CODEOWNERS-protected file) and adds a new model-branch to a benchmark script with several performance-tuning constants, a human look would still be worthwhile.
What was reviewed:
- The new
$MODEL-conditional branches in dsv4_fp4_b300_sglang_mtp.sh (SPEC_ARGS/ROUTER_ARGS refactor, HiCache ratio, prefill-decode-interval, chunked-prefill flags, MEM_FRACTION_STATIC). - Router failure isolation:
--disable-health-checkalready applies to both model variants (line ~310), so the new EAGLE-only--disable-circuit-breakerdoesn't remove an existing guard. - The unconditional
MEM_FRACTION_STATIC=0.84for DeepSeek-V4-Pro sits inside the TP>=8 branch's sibling scope, but the new nvidia-master.yaml recipe only exercises tp:8, so the documented TP<8 (DEP4) 0.902-minimum concern isn't reached by this config. - perf-changelog.yaml addition is appended at the tail, consistent with the append-only convention.
Extended reasoning...
The diff extends a B300/FP4/SGLang agentic benchmark script with a second model branch (EAGLE spec-decoding vs DSPARK), adds one new recipe to the CODEOWNERS-protected configs/nvidia-master.yaml, and appends one perf-changelog entry at the tail (no rewritten history). No security-sensitive surface (auth, crypto, permissions) is touched; the risk is purely in benchmark tuning correctness (spec-decoding params, mem-fraction overrides, router flags). Deciding factor for defer over approve: configs/nvidia-master.yaml is owned by specific CODEOWNERS, and the change carries several magic tuning constants whose correctness can't be fully validated from the diff alone.
This review covers commit e83cbd3, which is no longer the latest commit on this pull request; later commits are not covered by it.
更新 B300 DSV4 性能。
补充 B300 DSV4 TP8 低延迟并发点 1、2、4、8、16、32。
DSV4 低延迟配置使用明确的 decode CUDA graph 参数，修复新镜像的 CLI 歧义错误。
添加 B300 DSV4 TP4 低延迟测试点，并保留现有 TP8 和 DP8 配置。
033cc79 to
2e46f34
Compare
|
InferenceX has switched away from unmaintainable bash scripts to YAML files that don't repeat the same stuff over and over again. Please merge the latest |
|
Sorry, over the weekend, there was 2 major refactors to clean up the technical debt accumalated over the past 11 months of moving at the speed of light. We don't see any major refactors in the forthseeable future besides cleaning up AMD multinode AgentX pile of bash. As much, due to the refactors, u would need to ask your agent to rebase from remote main@latest. Thank you in advance for ur understanding |
…ng-agentx-update # Conflicts: # benchmarks/single_node/agentic/dsv4_fp4_b300_sglang_mtp.sh # inferencex-e2e/perf-changelog.yaml
cquil11
added
full-sweep-enabled
|
/use 36463674051 |
|
/stage-results 36463674051 |
|
@cquil11 staged run 36463674051: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-09-28~r36463674051 This run remains available across future |
|
/reuse-sweep-run 36463674051 |
# Conflicts: # inferencex-e2e/perf-changelog.yaml
4 participants
B300 curve update
Port to the native srt-slurm layout: merged
maininto this branch and replaced the deletedbenchmarks/single_node/agentic/dsv4_fp4_b300_sglang_mtp.shchanges with the recipeinferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv4/sglang/b300-fp4-eagle/agentic.yaml. It has oneoverride_*variant per matrix point (TP4 and TP8 c1-32, DEP8 HiCache c64-512) and keeps the script's DeepSeek-V4-Pro EAGLE settings at every point. The golden AL (2.49) is injected by the SRT adapter on throughput runs. Every search-space entry ofdsv4-fp4-b300-sglang-agentic-hicache-eaglenow setssrt-recipe, and the perf-changelog entry is updated.