[leehack]

Summary

• Native LiteRT-LM engine.chatTemplate(...).tokenCount now equals the runtime's prefilled input for Gemma 4. Since Report the chat template native LiteRT-LM renders #1854 the shown prompt is a render of the bundle template and starts with the text <bos>; the renderer counted it with tokenize(prompt, addSpecial: false), which the LiteRT-LM tokenizer reads as three plain tokens, while native prefill strips that text and adds one start token (2 too many).
• A runtime-template prompt is now counted by the backend through an internal BackendRuntimePromptTokenization probe (registered like BackendGenerationLimitReporting, so the exported LiteRtLmBackend type does not expose it). LiteRtLmService.tokenizeRuntimePrompt reuses the start-token stripping raw generate() uses since Send the raw prompt unchanged from native LiteRT-LM generate (#1853) #1864 (_withoutLeadingStartToken) and tokenizes the rest with the start token added once. create() rendering and parsing are unchanged.

Fixes #1872
Refs #1845

Production-readiness scope

• User-facing scope: chatTemplate().tokenCount (and ChatSession trimming, which uses it) is exact for native Gemma 4 .litertlm, thinking on/off, with or without a system message.
• Supported platforms/paths: native LiteRT-LM prompts rendered from the runtime template (llamadart.chat_template.parsing present). Qwen3.5 and Qwen3 stay exact; llama.cpp/GGUF, Web, customTemplate and the parsing-template fallback keep the plain tokenize(prompt, addSpecial: false) count.
• Unsupported or intentionally unavailable paths: unchanged; media prompts still count <__media__> as text, as before.
• Out of scope / follow-ups: None.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable. (No new unsupported combinations.)
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (chatTemplate Dartdoc and both changelogs; no doc claimed exactness.)
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry: N/A, no public capability change.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Bugfix PR: root cause above; failing-first unit tests (fake tokenizer reading <bos> as [236820, 46757, 236813]) and a local-only real-model e2e; native LiteRT-LM only.
• Feature PR: N/A, bugfix.
• Docs-only PR: N/A.

Before / after (gemma-4-E2B-it.litertlm 181938105e0e, CPU)

Bundle	Request	main 99b8406 tokenCount / native input	This PR
Gemma 4 E2B	user, thinking on	18 / 16	16 / 16
Gemma 4 E2B	user, thinking off	12 / 10	10 / 10
Gemma 4 E2B	system + user, thinking on	22 / 20	20 / 20
Gemma 4 E2B	system + user, thinking off	21 / 19	19 / 19
Qwen3.5-0.8B int8	user, on / off	13 / 13, 13 / 13	unchanged
Qwen3.5-0.8B int8	system + user, on / off	22 / 22, 22 / 22	unchanged
Qwen3-0.6B (c5f39b41)	user, on / off	9 / 9, 13 / 13	unchanged
Qwen3-0.6B (c5f39b41)	system + user, on / off	18 / 18, 22 / 22	unchanged

Native input is getPerformanceContext().promptEvalTokens after create() with the same messages.

Test Plan

• dart run tool/prepare_workspace.dart
• dart format --output=none --set-exit-if-changed .
• dart analyze
• dart run tool/testing/check_platform_boundaries.dart
• dart test -p vm -j 1 --exclude-tags local-only on test/unit/core/engine test/unit/backends/litert_lm test/unit/backends/native test/integration/backends/litert_lm test/unit/tooling: all pass except 6 windows_registry_qualification_test.dart cases needing pwsh, absent on this Mac (unrelated). Full suite in CI.
• dart test -p chrome on chat_template_renderer_test.dart and engine_test.dart: pass.
• coverage-lib: full VM suite with coverage on b64aa59, LCOV line coverage 93.65% (threshold 70%). Only the 6 pwsh-dependent windows_registry_qualification_test.dart cases failed (no pwsh on this Mac).
• Other targeted/local-only validation: new test/e2e/backends/litert_lm_chat_template_token_count_e2e_test.dart with LITERT_LM_MODEL set to each of Gemma 4 E2B, Qwen3.5-0.8B int8 and Qwen3-0.6B: 4/4 each (Gemma 0/4 on main); litert-lm-chat-features-smoke on Gemma 4 CPU: pass.

Mutations (each reverted; every one fails the named tests)

Mutation	Caught by
Service counts the prompt as text (pre-fix behavior)	service test, real-model e2e
Service strips the start token but does not add it	service test
Renderer ignores the runtime tokenizer	renderer + engine tests
Renderer uses it for non-runtime / fallback prompts	renderer test
Engine passes no runtime tokenizer	engine test
Worker ignores runtimePrompt	worker test
LiteRtLmBackend sends a plain tokenize request	backend test
LiteRtLmBackend registers no tokenizer	backend test
NativeAutoBackend never delegates / counts with special tokens	native backend test

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
high-risk-exact-head-independent-qa	independent review of exact head	-	Pending	Required before mark-ready.
structured-output-adversarial	parser/grammar/streaming	-	N/A	Only the prompt token count changes; rendering, grammar, parsing and streaming are untouched (create() paths unchanged).
litert-lm-chat-features-smoke	real chat, thinking, tools, ChatSession	macOS arm64 / Gemma 4 E2B 181938105e0e / CPU	Pass	Local run on b64aa59.
coverage-lib	lib coverage	macOS arm64 VM	Pass	93.65% line coverage on b64aa59.
litert-lm-json-schema, litert-lm-v1-*	native schema / C API1 adoption	-	N/A	Not touched.

Review Notes

• Independent review status: pending (high-risk, needs an independent auditor before mark-ready).
• CI status / head SHA: green (34 pass, 9 skipped) / b64aa59

High-risk regression review

• Classification: high-risk (backendRuntime, structuredOutput)
• Implementation task: this PR's author session
• Independent blocking QA task: pending
• Exact head / current base: b64aa59 / 99b8406
• Production-branch deletion, bypass, or miswire proof: mutation table above
• Affected-family real-model/artifact evidence: Gemma 4 E2B 181938105e0e, Qwen3.5-0.8B int8, Qwen3-0.6B c5f39b41 (table above)
• Explicit unavailable-family or other N/A evidence: N/A
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

Independent audit

Exact head b64aa59a1f7c4b4234e3816cfb527c5b7b00cc68 against main 99b8406e3: PASS, 0 blocking, 0 PR-caused P1, 0 unresolved threads. Gemma 4 chatTemplate().tokenCount equals runtime prefill (22/22, tools 58/58; main was +2); Qwen3.5, Qwen3-0.6B c5f39b41, GGUF Qwen3-0.6B and customTemplate counts unchanged. Probe stays internal (not exported). 12/12 mutants killed. Readiness evidence v2: local run exits 2. Non-blocking P3s tracked on #1861.

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-1873.