[leehack]

Summary

After a prompt that opens no thought, llama.cpp cd959bae9 (native v0.9.0) takes a thought only where the reply starts with the start tag and keeps every other thinking tag as content. llamadart's handlers took thoughts anywhere (extractThinking), so a bare </think>, a <think> after text, or a second </think> after a leading thought became reasoning, and the stream could end differently from the final parse and repeat text in both channels.

• New format trait takesThoughtOnlyAtReplyStart, set only on formats verified against the oracle: generic, content-only, Llama 3, GLM 4.5, Laguna, LFM2, Kimi K2, DeepSeek R1 / V3.2 / V4, Granite, MiniCPM5, Mistral Nemo, Apertus, Hunyuan V3, Hermes and Qwen3-Coder XML.
• For those formats, parseRendered (and the engine) runs the handler inside withThoughtOnlyAtReplyStart, a zone flag that switches extractThinking to the leading-thought rule. The flag reaches every extractThinking call site (handlers, parseXmlToolCalls, the incomplete-call check) without changing the public ChatTemplateHandler signatures.
• The stream ends that leading thought at its first end tag and streams the rest as content, as it already does for Seed-OSS and PEG replies, so every chunking ends as the parse does.
• Unchanged: ChatTemplateEngine.parse (no render, so it can't tell what the prompt opened), forced-open and thinking-off-closed prompts, DeepSeek V3, and formats without the trait.

Fixes #1865 (items 1 and 2). Refs #1874.

Production-readiness scope

• User-facing scope: Replies to prompts that open no thought (for example Qwen3 with thinking on, or Llama 3 / generic templates) now parse and stream as llama-server does: only a leading thought is reasoning, and text never streams as both content and reasoning.
• Supported platforms/paths: All backends that use the Dart chat parser (native llama.cpp, WebGPU bridge, LiteRT-LM chat parsing); VM and Chrome suites pass.
• Unsupported or intentionally unavailable paths: None added. Known differences that stay: where llama.cpp parses no reasoning at all for a template (no reasoning in the template, or thinking off), it keeps even a leading <think> thought as content, and llamadart still takes it as reasoning (as before; recorded in closed_thought_upstream.md).
• Out of scope / follow-ups: Forced-open prompts in handler formats (generic, GLM 4.5, Laguna, DeepSeek R1/V3.2/V4, MiniCPM5, MiniMax M2, Nemotron V2, Hunyuan V3) still take thoughts after the first end tag, and their stream can differ from the parse: same class, separate shape, #1881. Qwen3-Coder JSON-in-XML calls after a bare end tag stay with Reasoning parity follow-ups from #1524/#1525 audit: mutants X2/X7, Ling <think> in args, partial-tag drop #1531 (lane 4). MiniMax M3, Kimi K3, Gemma 4 and Solar Open keep their own, different upstream rules and are not changed.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable. (N/A: no new option; parse behavior only.)
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (parseRendered Dartdoc, both guides, both changelogs, llamadart-chat-streaming skill.)
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope). N/A: no capability change.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Bugfix PR: Root cause: handlers call extractThinking, which takes a thought at any <think> and treats an end tag before any start tag as ending a prompt-opened thought; only Hermes, Qwen3-Coder XML and DeepSeek V3 got the Streaming a bare </think> after a prompt with no open thought duplicates text and depends on chunking #1858 bare-end-tag rule, and none got the mid-reply rule. The stream split tags as they arrived, while the final parse recursed over the whole reply, so the final reconciliation re-sent text. Regression tests: oracle fixture cases (below), failing on base for every new entry (370 failures) and passing on head.

Test Plan

• dart run tool/prepare_workspace.dart
• dart format --output=none --set-exit-if-changed .
• dart analyze
• dart test -p vm -j 1 --exclude-tags local-only (local -j 1 run: only windows_registry_qualification_test (6 cases) fails, identically on base f78cafa97 on this macOS host)
• dart test -p chrome --exclude-tags local-only (4660 passed)
• coverage-lib matrix row: see Matrix Evidence
• Other targeted/local-only validation: ./tool/testing/run_template_parity_suites.sh (all five stages pass); dart run tool/testing/check_platform_boundaries.dart; dart test -p vm test/unit/tooling/package_skills_test.dart; oracle sweeps and mutation run below.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
high-risk-exact-head-independent-qa	independent review of exact head	-	Pending	Requested from an independent auditor before mark-ready
structured-output-adversarial	template parity, upstream parser parity, streaming partial/final	macOS VM + Chrome, oracle at llama.cpp cd959bae9	Pass	run_template_parity_suites.sh; sweeps below; fixture cases recorded from the oracle
coverage-lib	lib line coverage	macOS VM	Pass	93.66% line coverage (33427/35688), threshold 70

Upstream comparison (llama.cpp cd959bae9 oracle, common_chat_peg_parse + common_chat_session)

Corpora over all 76 upstream models/templates (thinking on/off x kwargs x 5 tool modes) and the vendored test/fixtures/templates: the #1858 bare end-tag corpus extended with #1865 shapes (finished + partial), the #1525 thinking corpus, and a format-native tool-call corpus. Each finished row compares the oracle session with parseRendered and with the stream at whole, per-char, 2-char, 3-char and every 2-split chunking.

Corpus	Finished parse = llama.cpp (base -> head)	Partial parse = llama.cpp (base -> head)	Every chunking = parse (base -> head)	Regressions
#1865 bare/mid-reply, upstream	36450 -> 59634	37613 -> 60605	73470 -> 96792 of 104160	12 (see note)
#1525 thinking, upstream	51758 -> 66984	-	83688 -> 99882 of 103212	0
#1865 bare/mid-reply, vendored	7764 -> 18012	8192 -> 18392	15058 -> 23592 of 24000	0
#1525 thinking, vendored	13042 -> 20152	-	17096 -> 23696 of 24120	0
native tool calls around tags, upstream	1778 -> 3286	1778 -> 2932	5584 -> 6902 of 7040	0

Note: the 12 rows are partial Command R+ replies (</think>), where llama.cpp returns empty content for every partial reply of that template and rejects every finished one; the base matched by chance.

Every finished row of a trait format after a prompt that opens no thought now streams as it parses. Remaining parse differences in those formats are all outside #1865: a leading thought where llama.cpp parses no reasoning for the template, Apriel 1.6 and IFM K2 Horizon (dedicated reasoning-prefix parsers upstream), Hermes' untagged JSON call, Apriel 1.5 and Qwen3-Coder tool-call shapes (#1531). Remaining stream differences are forced-open or thinking-off-closed prompts and non-trait formats, unchanged from base.

Per format (prompts that open no thought, #1865 + #1525 upstream corpora, finished rows):

Format	Parse = llama.cpp (base -> head)	Every chunking = parse (base -> head)
generic	1356 -> 6294 / 10992	5142 -> 10992
contentOnly	1688 -> 8104 / 12688	5616 -> 12688
llama3	396 -> 1872 / 2520	1296 -> 2520
glm45	684 -> 2784 / 3054	1158 -> 3054
laguna	792 -> 3066 / 3066	1170 -> 3066
lfm2	1320 -> 5960 / 7320	3240 -> 7320
kimiK2	1800 -> 7200 / 7320	3240 -> 7320
deepseekR1	168 -> 818 / 1118	504 -> 1118
deepseekV32	180 -> 890 / 1034	354 -> 1034
deepseekV4	360 -> 1780 / 2068	708 -> 2068
granite	600 -> 2440 / 2440	1080 -> 2440
minicpm5	168 -> 818 / 1118	504 -> 1118
mistralNemo	288 -> 1424 / 1952	864 -> 1952
apertus	2460 -> 3880 / 4540	3180 -> 4540
hunyuanV3	1230 -> 1940 / 2270	1590 -> 2270
hermes	11694 -> 14116 / 17956	16118 -> 17956
qwen3CoderXml	1392 -> 1600 / 2368	2116 -> 2368

Durable tests

• test/fixtures/closed_thought_upstream.json no_thought_templates: one vendored template per trait format (Gemma 2, SmolLM3, Llama 3.2, GLM-4.6, Laguna XS.2, LFM2-8B-A1B, Kimi K2, R1 Distill Qwen 32B, V3.2, V4, Granite 3.3, MiniCPM5, Mistral Nemo, Apertus, Hy3) plus new cases for the Streaming a bare </think> after a prompt with no open thought duplicates text and depends on chunking #1858 entries, recorded from the oracle (finished, without tools, and partial). closed_thought_upstream_test checks the parse and every chunking against them, and now compares upstream_without_tools with a render without tools, which covers content-only.
• Five templates vendored at the pinned ref b10549 (identical at cd959bae9): DeepSeek V3.2, V4, Granite 3.3, Laguna XS.2, Mistral Nemo. Their caps are pinned from Dart detection, and the forced-open grammar test now lists DeepSeek V3.2/V4 and Laguna with the required-call grammars that take any text before the calls.
• thinking_utils_test: the leading-thought rule (ASCII-only leading whitespace, unended thought, escapes, custom tags, forced-open untouched). closed_thought_upstream_test: only trait formats switch rule, and only for rendered parses.
• Changed expectations: the LFM2 boundary scenario with thoughts after the reply start (now content; inner calls stay inert), and a DeepSeek R1 latency check that meant a prompt-opened thought now renders one.

Mutation check

19 of 20 mutants killed; per-format trait removal killed for all 17 formats. Survivor P3 (apply the rule after a forced-open prompt) is equivalent: every trait handler passes thinkingForcedOpen to extractThinking, which ignores the rule for a forced-open thought (U5 kills that).

ID	Mutant	Result
P1	rendered parse never applies the rule	killed
P2	rule also without promptOpensNoThought (public parse, thinking-off-closed)	killed
P3	rule also after a forced-open prompt	survived (equivalent)
P4	every format takes the rule	killed
U1-U6	no whitespace skip / Unicode whitespace skip / last end tag / no unescape / rule for forced-open / unended thought as content	killed
S1-S4	stream keeps old split / thought not ended at first end tag / rule after forced-open or closed-off / rest still split	killed
R*	trait removed from each of the 17 formats	killed

Evidence: oracle harness, sweep rows (base and head) and summaries in the maintainer evidence folder llamadart-evidence/issue-1865/.

Review Notes

• Independent review status: pending (high-risk; draft until an independent auditor reviews this exact head).
• CI status / head SHA: a6ea031, CI green (32 pass, 9 skipped).
• Follow-up: forced-open handler formats: A forced-open thought in most handler formats still takes later thinking tags as thoughts #1881

High-risk regression review

• Classification: high-risk (backendRuntime, structuredOutput)
• Implementation task: 0.19.4 release plan: LiteRT-LM Web tool calling and parity fixes #1874 lane 3
• Independent blocking QA task: pending
• Exact head / current base: a6ea031 / f78cafa
• Production-branch deletion, bypass, or miswire proof: mutants P1, P4, S1, S2 and per-format trait removal fail the oracle fixture tests.
• Affected-family real-model/artifact evidence: primary upstream emissions: llama.cpp cd959bae9 oracle over all 76 upstream templates and the vendored templates; no model weights needed for a parser-only change.
• Explicit unavailable-family or other N/A evidence: N/A
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

Independent audit

Exact head a6ea0316b6913b247fe4a744d0ebbfa152088a2d against base f78cafa97: PASS, 0 blocking, 0 PR-caused P1, 0 threads. All 1026 fixture values re-derived from a llama.cpp cd959bae9 oracle; 131-template corpus shows no match→mismatch, public parse identical, forced-open/thinking-off renders unchanged, 678 stream rows fixed; zone approach judged sound; exclusions and changed expectations justified; Qwen3-0.6B GGUF and .litertlm outputs identical base vs head. 31/35 mutants killed (survivors equivalent or a P3 gap). Coverage 93.66%. Readiness evidence v2: local run exits 2. Non-blocking items tracked on #1861.

[github-actions]

Chat app preview for a6ea031: https://leehack-llamadart-chat-pr-1882.static.hf.space

Space: https://huggingface.co/spaces/leehack/llamadart-chat-pr-1882

[leehack]

Landed on main via batch integration PR #1895 (rebase merge, main ba4e0e1b; tree identical to the audited integration head 5ffd1224).