[leehack]

Summary

• Native .litertlm getMetadata()['tokenizer.chat_template'] now reports the template the LiteRT-LM runtime renders for text-only chats. That is the bundle's embedded template for everything except Qwen3 text bundles. When llamadart parses with a different template (the file-name family template or ModelParams.chatTemplate), it is reported as llamadart.chat_template.parsing.
• ChatTemplateRenderer parses with customTemplate > ModelParams.chatTemplate > llamadart.chat_template.parsing > tokenizer.chat_template. create() renders and parses exactly as before: format, parser, grammar, stops, thinking marks, JSON Schema tool gating, tool-result gating and the rendered-fallback prompt.
• engine.chatTemplate(), and through it ChatSession budgeting, takes prompt and tokenCount from a plain Jinja render of the runtime template: no format routing, generic tool instruction, gates or grammar. Every other field still comes from the parsing template. If the runtime template doesn't render in Dart, whether it throws an Exception or an Error (for example {{ 10 // 0 }}), the parsing prompt is returned and a warning is logged.
• One helper (_templateRouting) now decides Qwen3 text routing for metadata, generateChat and prepareJsonSchemaChat. As a result, native JSON Schema chat now refuses bundles whose text chats the runtime renders with the bundle template, matching generation (changelog). Bundles with no family match and no override still report no template.
• _modelToolResultMedia and LiteRtLmService.applyChatTemplate read the parsing key first, so their output is unchanged. The example chat app detects tool support from the parsing key.
• Docs: Dartdoc for ModelParams.chatTemplate, getMetadata and chatTemplate; model-families.md (replaces the Explain which chat template LiteRT-LM bundles render (#1836) #1842 caveat); skills/llamadart-tool-calling/SKILL.md; changelog bullet in both files.

Fixes #1836. Refs #1827 (its smoke part shipped in #1833; this makes the pre-closed <think> visible in chatTemplate()) and #1845 (lane 3). Design and maintainer decisions: design, decisions.

Production-readiness scope

• User-facing scope: native LiteRT-LM metadata and chatTemplate() show what the runtime renders. Parsing is unchanged, and the parsing template is available under a documented key.
• Supported platforms/paths: native LiteRT-LM (.litertlm) on all native platforms. llama.cpp/GGUF and WebGPU are unaffected (no parsing key, same code path).
• Unsupported or intentionally unavailable paths:
  • LiteRT-LM Web keeps its accounting template, unchanged.
  • Bundles with no family match and no override still report no template, because there is no parsing template to keep create() unchanged.
  • The static metadata describes text-only chats. A media request to a Qwen3 text bundle renders the embedded template.
  • chatTemplate() is a Dart render of the runtime template, not the runtime's exact bytes. This is documented.
• Out of scope / follow-ups: Native LiteRT-LM generate() wraps the raw prompt in the bundle template, so create() fallbacks send a double-templated prompt #1853, a separate bug found while validating: native generate() wraps the raw prompt in the bundle template, so create() fallbacks send a prompt rendered by two templates.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (README does not mention this.)
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope). N/A: no LlamaEngineCapabilities change; validation_coverage_test.dart passes.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots. The new key carries template source only.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Feature PR: N/A
• Bugfix PR: Root cause: LiteRtLmService.getMetadata reported the file-name family template, which the runtime renders only for text-only Qwen3 chats, and engine.chatTemplate() rendered that template. Format detection reads the same metadata key, so it could not simply be swapped. The fix splits the reported template from the parsing template. Regression tests are listed under Test Plan.
• Docs-only PR: N/A

Test Plan

• dart run tool/prepare_workspace.dart
• dart format --output=none --set-exit-if-changed . (0 changed)
• dart analyze (no issues); check_platform_boundaries.dart OK
• dart test -p vm -j 1 --exclude-tags local-only: CI Test Linux VM 1–3, Test Native macOS 1–3 and Windows pass; locally all touched suites (test/unit/backends/litert_lm/, test/unit/core/engine/, test/unit/core/template/, tooling and Markdown suites) pass
• dart test -p chrome --exclude-tags local-only: CI Test Web (Chrome) 1–3 pass; locally test/unit/core/template/, engine_test, chat_template_renderer_test, litert_lm_chat_templates_test and tool_result_media_public_api_test pass on Chrome
• coverage-lib matrix row when lib/ behavior changes: 93.64% line coverage (threshold 70). Touched files: chat_template_renderer.dart 45/48, chat_template_engine.dart 323/350, litert_lm_service.dart 1011/1031, engine.dart 1387/1430. See the matrix note.
• Other targeted/local-only validation: listed in the matrix below.

New tests:

• litert_lm_runtime_chat_template_test.dart (VM; fixtures are the templates embedded in the real bundles):
  • For Qwen3.5-0.8B, Qwen3.5-0.8B-VL, Gemma 4 E2B and a Qwen2.5 stand-in, across tools on/off × ToolChoice none/auto/required × thinking on/off: create()'s render with {tokenizer: bundle, parsing: family} equals the render with {tokenizer: family} in every field.
  • chatTemplate() shows the bundle prompt and token count with the parsing fields.
  • The Qwen3.5 prompt ends with </think> and has no tools.
  • ModelParams.chatTemplate keeps driving parsing.
• chat_template_renderer_test.dart: precedence, runtime prompt, media markers, fallback with warning, and source naming in errors.
• engine_test.dart: create() generates and parses tool calls with the parsing template; chatTemplate() shows the runtime prompt.
• tool_result_media_public_api_test.dart: toolResultMedia follows the parsing template, not the runtime one.
• litert_lm_service_test.dart:
  • Metadata for Qwen3.5, Qwen3.5-VL, Gemma 4 and Qwen2.5 names, overrides, unknown families, a missing embedded template, and Qwen3.
  • applyChatTemplate uses the parsing template.
  • JSON Schema admission follows generation routing.
• Chat app unit_test.dart: tools stay enabled when only the parsing key is Hermes.

Mutation checks (each applied alone and run against the five targeted test files; every one fails at least one test):

Mutation	Failing tests
Renderer ignores the parsing key	57
Parsing key beats ModelParams.chatTemplate	2
No fallback (only StateError caught)	1
chatTemplate() doesn't request the runtime prompt	1
create() requests the runtime prompt	1
_modelToolResultMedia reads tokenizer.chat_template	2
Service reports the family template (old behavior)	8
applyChatTemplate reads tokenizer.chat_template	1
JSON Schema admission uses the old routing	1
Runtime prompt ignores customTemplate	1
Chat app reads only tokenizer.chat_template	1 (chat app test)

After the audit, its 27-mutant script ran again at d6a62a80c: 25 killed. The newly killed ones are H (runtime prompt ignores enableThinking), I (drops chatTemplateKwargs), K (drops templateNow), N (plain-mode tool-result gate), O (plain-mode JSON-Schema-tool gate), S (plain-mode tool_use variant), U (source-name order), and T/T2 (fallback catches nothing / only Exception). L and L2 survive (plain-mode format detection). The audit judged both near-equivalent.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
root-vm	VM unit/integration	Linux, macOS, Windows CI	PASS	CI shards; touched suites also run locally
root-chrome	browser-compatible tests	Chrome	PASS	CI; locally the renderer, template and engine suites passed
coverage-lib	lib coverage	macOS arm64	PASS (93.64%)	format_lcov.dart aborts on 53 integration-test reports without a coverage key (the #1845 lane 2 issue), so those were dropped before formatting. Under full-parallel coverage, 10 tests failed locally: 6 windows_registry_qualification_test cases need pwsh, which is not installed here, and the 4 native-fixture/timing cases pass with -j 1. CI passes all of them
examples-tests	chat app	macOS arm64 VM + Chrome	PASS	flutter test test test_vm (690 passed); Chrome subset per AGENTS.md (133 passed)
template-parity / structured-output-adversarial	template detection/render/parse parity	macOS arm64	PASS	./tool/testing/run_template_parity_suites.sh exit 0. A first run timed out in the upstream full chat suite (15 min) while other suites shared the CPU; run alone, every suite passed
litert-lm-json-schema	native JSON Schema admission and enforcement	macOS arm64 CPU, Qwen3-0.6B c5f39b41…	PASS	run_local_e2e.dart --scenario litert-lm-json-schema exit 0
litert-lm-chat-features-smoke	chat, cutoff, tool loop, thinking, history, tools	macOS arm64 CPU	PASS ×4	Qwen3-0.6B, Qwen3.5-0.8B_int8 (thinking N/A), Qwen3.5-0.8B-VL_int8 (thinking N/A), gemma-4-E2B-it 181938… (tool call and native tool history pass)
docs-site	website build and links	macOS arm64	PASS	./tool/docs/build_site.sh: 3568 pages, no broken internal links. In the local sandbox the observability example test cannot reach localhost; it passes outside it
high-risk-exact-head-independent-qa	independent review	exact head vs 87aaec616	PASS at 40f9d1abb (no blocking findings); delta re-audit of d6a62a80c pending	The audit's non-blocking items are folded into d6a62a80c

Real bundles: chatTemplate().tokenCount compared with LiteRT-LM's native input tokens for the same create() request (system + user, CPU):

Bundle	Request	Before (main)	After
Qwen3.5-0.8B_int8	tools, thinking on/off	151 / 155 vs native 24	24 / 24 vs 24 (exact)
Qwen3.5-0.8B_int8	no tools, thinking on/off	20 / 24 vs 24	24 / 24 vs 24 (exact)
Qwen3.5-0.8B-VL_int8	same 4 requests	off by up to 131	all 4 exact
gemma-4-E2B-it	no tools / tools, thinking on	22 vs 24; 57 vs 60	24 vs 24; 60 vs 60 (exact)
gemma-4-E2B-it	no tools / tools, thinking off	24 vs 21; 59 vs 57	23 vs 21; 59 vs 57
Qwen3-0.6B	all 4	identical before and after (no parsing key)

Format is unchanged in every case (hermes, gemma4). With Gemma, the bundle template renders tool-response history from the runtime's content-part shape; a Dart string result renders differently. That is the documented fidelity caveat.

Review Notes

• Independent review status: an independent audit of 40f9d1abb passed with no blocking findings and accepted both author choices (JSON Schema routing; no template for unknown families). Its non-blocking items are in d6a62a80c: the fallback now also catches Errors, stale docs in chat-template-and-parsing.md and template-engine-internals.md are fixed, unknown-family behavior is documented, the changelog notes the JSON Schema change, and surviving mutants are closed. A delta re-audit of d6a62a80c is pending.
• CI status / head SHA: d6a62a80c, CI green: 35 pass, 9 skipped, 0 failed
• Correction to the design comment: the Gemma bundle template does render <|think|> when enable_thinking reaches it. My earlier probe called the handler without the kwargs.
• Pre-existing flake, not this PR: litert_lm_embedding_native_test "embedding deadline throws a typed error" failed once under parallel load and passed 3/3 alone and on base (0.19.2 release plan: audit follow-ups and correctness fixes #1845 lane 1).

High-risk regression review

• Classification: high-risk (backendRuntime, structuredOutput)
• Implementation task: lane 3 of 0.19.2 release plan: audit follow-ups and correctness fixes #1845 (this PR)
• Independent blocking QA task: independent audit PASS at 40f9d1abb; delta re-audit of d6a62a80c pending
• Exact head / current base: d6a62a80c / 87aaec616
• Production-branch deletion, bypass, or miswire proof: the mutation table above; the create() pin tests compare every result field before and after across four families, three tool choices and both thinking settings.
• Affected-family real-model/artifact evidence: Qwen3-0.6B, Qwen3.5-0.8B_int8, Qwen3.5-0.8B-VL_int8, gemma-4-E2B-it (non-web, downloaded for this run) on native CPU.
• Explicit unavailable-family or other N/A evidence: Qwen2.5 and Gemma 3/3n .litertlm were not run on real bundles. Qwen2.5 is covered by a ChatML stand-in fixture; Gemma 3/3n share the same code path.
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

[leehack]

Landed through the 0.19.2 batch integration PR #1870 (rebase merge, one commit per constituent; this PR's pinned head was audited/CI-gated there).

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-1854.