Skip to content

[GLM-5.2 GB300] Preserve draft precision and extend scheduler watchdog / [GLM-5.2 GB300] 保留 draft 精度并延长调度 watchdog #3402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
edwingao28 merged 3 commits into from
Oct 2, 2026
+16 −2
Merged

[GLM-5.2 GB300] Preserve draft precision and extend scheduler watchdog / [GLM-5.2 GB300] 保留 draft 精度并延长调度 watchdog #3402

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fc03e75
fix: disable GLM-5.2 GB300 draft MoE FP8 conversion
edwingao28 Sep 27, 2026
2dd6885
chore: synchronize GLM-5.2 GB300 with main
edwingao28 Oct 2, 2026
f89ed78
chore: reappend GB300 changelog after main sync
edwingao28 Oct 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...encex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/gb300-fp4/agentx/agg.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ roles:
SGLANG_HICACHE_DEBUG_LOG: '1'
SGLANG_HICACHE_DEBUG_SAMPLE_RATE: '16384'
SGLANG_MOE_NVFP4_DISPATCH: '1'
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '1'
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '0'
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
PIP_BREAK_SYSTEM_PACKAGES: '1'
SGLANG_DG_CACHE_DIR: /deepgemm_cache
Expand Down
4 changes: 3 additions & 1 deletion ...arks/multi_node/srt-slurm-recipes/glm5.2/sglang/gb300-fp4/agentx/disagg-mtp-variants.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ base:
speculative-num-steps: 1
speculative-eagle-topk: 1
speculative-num-draft-tokens: 2
watchdog-timeout: 1800
enable-metrics: true
enable-cache-report: true
decode:
Expand All @@ -106,7 +107,7 @@ base:
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK: '512'
SGLANG_MOE_NVFP4_DISPATCH: '1'
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '1'
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '0'
SGLANG_ENABLE_THINKING: '1'
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
SGLANG_REASONING_EFFORT: max
Expand Down Expand Up @@ -147,6 +148,7 @@ base:
model-loader-extra-config: '{"enable_multithread_load": true}'
mem-fraction-static: 0.9
disaggregation-decode-extra-slots: 0
watchdog-timeout: 1800
enable-metrics: true
enable-cache-report: true
health_check:
Expand Down
12 changes: 12 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9245,3 +9245,15 @@
- "Disable FP8 conversion of the GLM-5.2 NextN/MTP draft MoE on B200 AgentX so the draft runs at its shipped precision; images, topology, workload and acceptance are unchanged."
- "关闭 B200 GLM-5.2 AgentX 对 NextN/MTP draft MoE 的 FP8 转换，使 draft 保持原始发布精度；镜像、拓扑、工作负载和验收不变。"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3400

- config-keys:
- glm5.2-fp4-gb300-dynamo-sglang-agentic-agg
- glm5.2-fp4-gb300-dynamo-sglang-agentic-disagg
scenario-type:
- agentic-coding
description:
- "Disable FP8 conversion of the GLM-5.2 NextN/MTP draft MoE on GB300 AgentX so the draft runs at its shipped precision; images, topology, workload and acceptance are unchanged."
- "Disaggregated prefill and decode use the 1800 s scheduler watchdog of the GLM-5.2 B200/GB200 recipes instead of the 300 s default."
- "关闭 GB300 GLM-5.2 AgentX 对 NextN/MTP draft MoE 的 FP8 转换，使 draft 保持原始发布精度；镜像、拓扑、工作负载和验收不变。"
- "分离式 prefill 和 decode 采用 GLM-5.2 B200/GB200 配方的 1800 秒调度 watchdog，替代默认 300 秒。"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3402
Comment on lines +9257 to +9259

@claude claude Bot Oct 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) New perf-changelog entry mixes two Simplified-Chinese description lines with the English ones, violating the repo's English-only rule for this file.

Why this was flagged

AGENTS.md:126 states new inferencex-e2e/perf-changelog.yaml entries must be English-only and must not add Chinese translations or bilingual descriptions. The entry appended at inferencex-e2e/perf-changelog.yaml:9238-9250 (config-keys glm5.2-fp4-gb300-dynamo-sglang-agentic-agg/-disagg) includes two Chinese description lines at 9249-9250 alongside the English ones at 9247-9248. Any changelog consumer or maintainer relying on English-only entries (e.g. tooling that parses/greps descriptions, or CODEOWNER review per the PR_REVIEW_CHECKLIST rule) gets a bilingual entry the base branch's convention forbids. Fix: remove the two Chinese description lines so the entry is English-only, consistent with every other entry in the file.

Verification: The new perf-changelog entry appended at inferencex-e2e/perf-changelog.yaml:9238-9250 contains two Simplified-Chinese description lines (9249-9250) alongside the English lines at 9247-9248. This contradicts AGENTS.md:126, which requires new entries to be English-only with no Chinese translations or bilingual descriptions. The violation is introduced by this PR; the base branch has no such entry.

Loading