[leehack]

Summary

• LiteRT-LM Web gets manual tool calling through LlamaBackend.withTextHistory() / LiteRtLmBackend.withTextHistory() on @litert-lm/[email protected].
• ToolDefinitions go to preface.tools; the runtime's parsed tool_calls come back from ChatSession.create / LlamaEngine.create as toolCalls, carried structurally (an internal BackendRuntimeToolCalls seam) so text the model streams before or after a call stays the reply's content, as on native; assistant calls and LlamaToolResultContent results are replayed in the 0.19.0 Message shape native LiteRT-LM sends (calls in tool_calls, results as tool_response parts), in order. A final tool-result message is sent as the request's Message object.
• The Web session budget counts tool declarations and assistant tool calls.
• A load-time probe (run only when the text-history probe passes) sets supportsToolCalling only when createConversation reads preface.tools; otherwise tools and tool history throw LlamaUnsupportedException naming the capability and @litert-lm/[email protected].
• ToolChoice.required, parallelToolCalls, tool-result content and media throw; ToolChoice.none declares no tools (as native). Automatic loops stay refused (Report LiteRT-LM Web token-limit truncation to the tool loop #919).
• The chat app keeps Function Calling off on LiteRT-LM Web because it runs calls automatically (Report LiteRT-LM Web token-limit truncation to the tool loop #919); new scenario litert-lm-web-tool-calling gives real-model evidence.

Fixes #1790
Refs #1874

Production-readiness scope

• User-facing scope: Web apps using the text-history factory can declare tools, get the model's tool calls from create, run them, add tool results and continue the conversation on LiteRT-LM Web.
• Supported platforms/paths: Web LiteRT-LM via withTextHistory() on runtimes that pass the preface.tools probe (@litert-lm/[email protected]). Checked in headless Chrome 157 with Qwen3 0.6B (WASM CPU) and Gemma 4 E2B -web.litertlm (WebGPU, GPU_ARTISAN).
• Unsupported or intentionally unavailable paths: default LlamaBackend() / LiteRtLmBackend() constructors (report supportsToolCalling: false, unchanged); runtimes that ignore preface.tools (typed LlamaUnsupportedException); sendWithTools / completeWithTools (unchanged typed refusal, no termination cause); ToolChoice.required, parallelToolCalls, tool-result content, media (typed refusal before inference).
• Out of scope / follow-ups: automatic loops and truncation reporting: Report LiteRT-LM Web token-limit truncation to the tool loop #919 (lane 2, stacked on this branch). Versioned Web validation profile: Keep the validation suite in step with public features #1196.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant.
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope).
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Feature PR: API Dartdoc (LlamaBackend.withTextHistory, LiteRtLmBackend, LlamaEngineCapabilities.supportsToolCalling/supportsMultiTurnChat), README, support matrix, tool-calling and backend-selection guides, chat app README, three skills, changelogs, capability coverage; happy and negative paths tested.
• Bugfix PR: N/A, feature.
• Docs-only PR: N/A.

Test Plan

• dart run tool/prepare_workspace.dart
• dart format --output=none --set-exit-if-changed .
• dart analyze
• dart test -p vm -j 1 --exclude-tags local-only — CI. Locally: test/unit/core/ + test/unit/backends/ (all pass), test/unit/tooling/ (all pass except windows_registry_qualification_test.dart, which needs pwsh, not installed here).
• dart test -p chrome --exclude-tags local-only — full run, 4676 pass locally at 7d4630f.
• coverage-lib — CI on main: the VM-measured changes (chat_completion_stream_parser.dart runtime-call merge, engine.dart hook, backend.dart registry) are covered by the new parser tests and the Chrome engine-path tests; the backend itself is browser-only.
• Other targeted/local-only validation: litert-lm-web-tool-calling and litert-lm-web-text-history (Qwen3 CPU, Gemma 4 WebGPU), chat app flutter test test test_vm plus the Chrome subset, validation_coverage_test.dart.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
static-format-analyze	format, analyzer, platform boundaries	macOS, Flutter 3.47.1	PASS
litert-lm-web-tool-calling	runtime tool_calls, result used, no-tools control, text before a call, cancel/reuse, reset	Chrome 157 headless, Qwen3-0.6B.litertlm c5f39b41… (613722264 B), WASM CPU	PASS (head 7d4630f)	direct runtime: get_weather {city: Paris} in tool_calls; adapter same call, answer "violet hail … -7°C" from the tool result; no tools: no call. Preamble case: the model streamed "One moment, checking." then the call; adapter returned that text as content plus the get_weather call
litert-lm-web-tool-calling	same	Chrome 157 headless, gemma-4-E2B-it-web.litertlm 3a08e8d9… (2008432640 B), WebGPU (GPU_ARTISAN)	PASS (head 7d4630f)	same; the preamble prompt did not elicit text before the call (call-only, matching direct); text-then-call, call-then-text and split chunks are covered by fake-runtime Chrome tests. Without tools the model writes a literal <|tool_call> that the runtime does not parse and the adapter leaves as text
litert-lm-web-text-history	history regression (preface/send path changed)	Chrome 157, same Qwen3 CPU and Gemma 4 WebGPU artifacts	PASS	direct/adapter parity, bypass controls, reset and cancel/reuse on both
docs-site	link check	CI	CI	local run blocked: port 8080 held by a system process
high-risk-exact-head-independent-qa	independent audit	exact head	PENDING	to be run by an independent auditor
structured-output-adversarial	runtime-call merge in ChatCompletionStreamParser	VM + Chrome unit	PASS	runtime calls append after text-parsed calls and re-index; ignored when tool parsing is off; text/call order matrix through router and direct backend

Mutation checks, all killed:

• Chrome manual tool calling group (first round, 7): probe ignores consumption, reasoning_content dropped, ToolChoice.none declaring tools, parallelToolCalls accepted, tool-result content accepted (SPI test), tools gate skipped, tool message sent as a string.
• Audit fixes (Chrome, 8): runtime calls always sent as a text envelope; calls not stored; reporter not registered; text-only session accounting; tool probe run without the history gate (legacy-probes-tools); chat-scope metadata unchanged; WebAutoBackend drops calls; engine ignores the reporter.
• Parser (VM, 4): runtime calls ignored; runtime calls replace text calls; runtime calls kept with parsing off; indices not renumbered.
• Driver (VM, 2): SUITES['tool-calling'] mapped to the text-history file; --suite ignored. Failing-first: 5 of the 6 original tests fail against origin/main's backend (the rejection test passes there by design, since main rejects all tools).

Review Notes

• Independent review status: audit of f2624a4 ACCEPTED with one P2 (tool call lost after preceding text), one P3 (session budget ignored tools) and test gaps/nits; all folded into 7d4630f. Re-audit of the new head pending.
• CI status / head SHA: green (CI, LiteRT-LM Smoke, Validation bundles, Chat App HF PR Preview, High-Risk Readiness Advisory) / 7d4630f

High-risk regression review

• Classification: high-risk (backendRuntime, regressionPolicy)
• Implementation task: lane 1 of 0.19.4 release plan: LiteRT-LM Web tool calling and parity fixes #1874
• Independent blocking QA task: f2624a4 audited (ACCEPTED, findings folded in); re-audit of 7d4630f pending
• Exact head / current base: 7d4630f / f78cafa
• Production-branch deletion, bypass, or miswire proof: mutation checks above; failing-first against origin/main
• Affected-family real-model/artifact evidence: Qwen3 0.6B (WASM CPU) and Gemma 4 E2B Web bundle (WebGPU), direct @litert-lm/[email protected] reference plus adapter
• Explicit unavailable-family or other N/A evidence: other browsers (Safari, Firefox) not run for tools; Chrome is the qualified browser
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

Seams for #919 (lane 2)

• generateChat returns the _generateWithHistory stream itself (no async* wrapper), so a BackendGenerationLimitReporting.generationLimitOf(stream) keyed on that stream sees the tool-calling path too. Runtime tool calls use the same pattern: BackendRuntimeToolCalls.runtimeToolCallsOf(stream), registered via registerRuntimeToolCallReporter and resolved with runtimeToolCallReporterOf(backend).
• WebAutoBackend forwards runtimeToolCallsOf to its delegate; its generationLimitOf still delegates via delegate is BackendGenerationLimitReporting, so lane 2 may need a registry lookup (generationLimitReporterOf) there.
• generationLimitUnsupportedReason is unchanged; clearing it is what lets the core tool loop run.
• The chat app's gate (unreportedTruncation in _updateToolTemplateSupport) keys on kIsWeb && runtime == liteRtLm; relax it when Report LiteRT-LM Web token-limit truncation to the tool loop #919 lands.
• generate() behavior is unchanged (it only passes prompt.toJS to the shared run).

[leehack]

Landed on main via batch integration PR #1895 (rebase merge, main ba4e0e1b; tree identical to the audited integration head 5ffd1224).