Skip to content

Close a Qwen3-Coder XML thinking budget with a line break (#1914) - #1924

Closed
leehack wants to merge 1 commit into
mainfrom
fix/1914-thinking-budget-qwen3-coder-newline
Closed

leehack wants to merge 1 commit into
mainfrom
fix/1914-thinking-budget-qwen3-coder-newline

Conversation

@leehack

@leehack leehack commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

When a thinking budget runs out, llamadart forced a bare </think>. llama.cpp (cd959bae9) forces \n</think> for the Qwen3-Coder XML family, and without the line break Qwen3.5 keeps thinking or leaks the raw call as content. A budget on ChatFormat.qwen3CoderXml (Qwen3.5, Nemotron 3 Nano, StepFun 3.5 Flash) now appends \n to the forced message. Fixes #1914; lane 1 of #1918.

  • New trait ChatFormatTraits.budgetThoughtClosePrefix (default '', '\n' for qwen3CoderXml), read through a top-level budgetThoughtClosePrefix(formatIndex); public thinkingTagsFor is unchanged.
  • The planner appends it to forcedMessage only when the caller set no endTag and the plan renders the prompt (not native-chat, not schema backend): LiteRT-LM throws on any forced message.
  • Budget token counting is unchanged (maintainer decision).
  • Docs: changelog (both files), generation-and-streaming, runtime-parameters, template-engine-internals, ThinkingBudget.forcedMessage Dartdoc. Also fixed the stale claim that WebGPU and LiteRT-LM reject a budget.

Production-readiness scope

  • User-facing scope: a thinking budget on Qwen3-Coder XML templates closes the thought as llama.cpp does.
  • Supported platforms/paths: rendered-prompt generation (native llama.cpp, WebGPU).
  • Unsupported or intentionally unavailable paths: native LiteRT-LM chat is untouched and still rejects forcedMessage.
  • Out of scope / follow-ups: upstream's other end sequences (<tool_call>, DSML) as a single native end sequence; budget tags for gpt-oss, Apriel and the Gemma 4 start tag.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
  • Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant.
  • Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
  • Public capability changes update the validation coverage registry: N/A, no capability change.
  • Security/privacy review completed: no secrets or paths involved.
  • Follow-up work is tracked above.

PR type guidance

  • Bugfix PR: root cause above; regression tests below; affected paths: rendered-prompt backends.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze and check_platform_boundaries.dart
  • dart test -p vm on the changed tests and test/unit/tooling/ (all pass except 6 windows_registry_qualification_test cases that need pwsh, absent locally)
  • dart test -p chrome test/unit/core/engine/chat_completion_request_planner_test.dart
  • coverage-lib: 93.79% lines (threshold 70%); the only failures in the full VM run were 6 windows_registry_qualification_test cases needing pwsh
  • ./tool/testing/run_template_parity_suites.sh
  • run_local_e2e.dart --scenario gguf-chat-features-smoke with Qwen3.5-2B-Q4_K_M
  • ./tool/docs/build_site.sh: not run locally (port 8080 in use by another lane); left to CI

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze format, analyzer, import boundaries macOS, Flutter 3.47.1 PASS clean
root-vm planner and parity tests, tooling suite VM PASS 6 pwsh-less Windows registry tests excluded; full root VM run left to CI
root-chrome planner test Chrome PASS
coverage-lib lib coverage VM 93.79%
template-parity vendored template parity VM PASS run_template_parity_suites.sh
structured-output-adversarial upstream emission parity of the forced close VM; Qwen3.5/Nemotron 3 Nano/StepFun vs QwQ/MiniMax-M2/DeepSeek-V3.1 controls PASS thinking_budget_close_parity_test.dart against test/fixtures/thinking_budget_close_upstream.json (llama.cpp cd959bae9); fails first on the three Qwen3-Coder XML templates
gguf-chat-features-smoke real GGUF thinking budget with tool call Metal, Qwen3.5-2B-Q4_K_M PASS toolCallWithThinkingBudget finish tool_calls, 1 call, no content leak
docs-site site build N/A port conflict locally; CI

Review Notes

  • Independent review status: pending auditor
  • CI status / head SHA: pending; head 388678c575bfce28139b1b3c3deda78e0e6c5c2a

High-risk regression review

  • Classification: high-risk (backendRuntime, structuredOutput)
  • Implementation task: lane 1914 worker
  • Independent blocking QA task: pending
  • Exact head / current base: 388678c575bfce28139b1b3c3deda78e0e6c5c2a / edaf8b5629
  • Production-branch deletion, bypass, or miswire proof: mutants all fail a test: M1 drop trait (parity + planner), M2 render-always (planner native-chat case), M3 ignore endTag (planner), M4 replace instead of append (planner), M5 prefix first (planner)
  • Affected-family real-model/artifact evidence: Qwen3.5-2B-Q4_K_M smoke above; upstream llama-server emissions recorded in the fixture; Nemotron 3 Nano and StepFun weights unavailable, covered by upstream thinking-tag fixtures for their vendored templates
  • Explicit unavailable-family or other N/A evidence: Nemotron 3 Nano, StepFun 3.5 Flash weights
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

llama.cpp forces \n</think> for the Qwen3-Coder XML family; a bare </think>
let Qwen3.5 keep thinking or leak the raw call as content.
@leehack

leehack commented Nov 11, 2026

Copy link
Copy Markdown
Owner Author

Landed via batch integration PR #1926 (rebase merge). Integration audit PASS at 6439022; this PR's audited head is recorded in #1926's constituent table.

@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1924.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen3.5 tool call with a tiny thinking budget streams the raw call text as content

1 participant