[leehack]

Summary

• Part of #174 (A1 and A3 of the scoping decision). The platform lift (A2) is the stacked PR-B.
• Native LiteRT-LM strict responseFormat now admits Gemma 4 E2B with its bundle template, alongside Qwen3 with its canonical text template, on the macOS arm64 CPU and GPU. NPU and other templates stay refused.
• The accepted JSON Schema is now the subset that both llama.cpp's GBNF converter and LiteRT-LM's LLGuidance 1.9.1 enforce. validateLlguidanceResponseSchema (lib/src/core/grammar/llguidance_schema_contract.dart) runs the GBNF validator and then refuses the features LLGuidance handles differently or fails on at send time. The refusal is a LlamaUnsupportedException that names the schema path, and it fires before generation. See the contract table below.
• Thinking with a schema is qualified per family. Gemma 4 can reason before its constrained reply: a C API ThinkingConfig keeps the constraint suspended until the reasoning channel closes. Qwen3 with thinking produced unconstrained text, so it is refused and the message names Gemma 4. The new backend SPI BackendJsonSchemaThinkingChatGeneration (exported from lib/backend.dart) carries this probe through NativeAutoBackend, the worker and the service.
• Runtime failures:
  • An INTERNAL send failure raised before any output becomes LlamaUnsupportedException ("runtime refused the JSON Schema request").
  • The service, not the worker, checks thinking after it re-admits the model, so Gemma 4 thinking still works after a client is recreated (an image turn, for example).
  • A failure after output has started becomes LlamaInferenceException, with no native text echoed.
  • A run that ends without stop or a maxTokens limit throws.
  • A reply cut off inside a constrained string reports length, and LlamaStructuredOutput.parse rejects it.
  • Admission also requires a runtime that reports finish reasons.
• Pins are unchanged (litert-lm-native v0.19.1-2). Only the C API 1.0 constraint and thinking setters are used, and only for JSON Schema; regex stays out of scope.

Schema contract

Feature	Native LiteRT-LM after this PR
type (incl. lists), properties, required, additionalProperties (bool or schema), items, min/maxItems, min/maxLength, anchored pattern, enum/const (incl. objects), anyOf, allOf, local $ref into $defs/definitions (incl. recursion through properties/items), integer minimum/maximum	Accepted (same as GBNF)
format	uuid and date only; others refused
Tuples (prefixItems, array-form items)	Closed only: minItems == maxItems == length
oneOf	Refused ("use anyOf"); LLGuidance has no referencing feature and fails at send time
uniqueItems, $dynamicAnchor, $recursiveAnchor, x-guidance	Refused
$schema	Root only, absent or draft 2020-12; nested $schema/$id refused
pattern	Refused: nested classes, &&/--/~~ class set operations, and nested counted repetition above 100000
enum/const integers beyond 2^53	Refused
allOf enum intersections above 20000 combinations, more than 400 optional properties at one value, $ref with ~0/~1 escapes, $ref cycles with no value between	Refused
Everything the GBNF subset already refuses	Refused (unchanged message)

When additionalProperties is absent, LiteRT-LM may emit extra keys, while the GBNF converter closes the object. The generation guide documents this.

Production-readiness scope

• User-facing scope: responseFormat: {'type': 'json_schema', ...} on the native LiteRT-LM backend is enforced for Qwen3 (canonical template) and Gemma 4 (bundle template). It runs on the macOS arm64 CPU or GPU and covers the shared schema contract. Gemma 4 can think first.

• Supported platforms/paths: macOS arm64, CPU and GPU (Metal), with the packaged v0.19.1-2 runtime and the C API 1.0 client.

• Unsupported or intentionally unavailable paths: Each throws LlamaUnsupportedException before generation:

  • other hosts (lifted per qualified host in PR-B)
  • NPU
  • other model families or templates
  • Qwen3 with thinking
  • thinking budgets
  • tools, media, speculation or caller stops with a schema
  • schemas outside the contract
  • runtimes without the constraint, thinking or finish-reason symbols

  Web keeps its existing refusal.

• Out of scope / follow-ups:

  • the platform lift (stacked PR-B)
  • regex constraints (follow-up on #174)
  • Web needs @litert-lm/core to expose constraints first; an upstream request is drafted, not filed

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (README, support matrix, generation and chat-template guides, doc/litert_lm_templates.md, doc/litert_lm_v1_adoption.md, doc/cross_platform_validation.md, doc/testing_matrix.md, llamadart-chat-streaming skill, llamadart_litert_json_example.dart, both changelogs)
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass. (capability_coverage.json structured-output entry; validation_coverage_test.dart passes)
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots. (native send diagnostics are not echoed; they can contain the schema or model path)
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Feature PR: API, docs, example and support-matrix updates; unsupported paths are listed above. The happy and negative paths have unit tests and real-model e2e runs.
• Bugfix PR / Docs-only PR: N/A.

Test Plan

At 6720dd22, with the targeted suites rerun at 280bf13d, macOS arm64, Flutter 3.47.1, isolated TMPDIR:

• dart run tool/prepare_workspace.dart (no tracked changes)
• dart format --output=none --set-exit-if-changed .
• dart analyze (no issues)
• dart run tool/testing/check_platform_boundaries.dart
• dart test -p vm -j 1 --exclude-tags local-only: 12866 pass, 129 skipped, 6 fail. All 6 are in windows_registry_qualification_test (needs pwsh, not installed locally; it fails on main the same way; CI runs it).
• dart test -p chrome: the contract test failed to compile on Chrome at 6720dd22 (literal 2^53+1 is not representable in JS); 4233707c uses 2^54. Contract, engine, planner and entrypoint tests pass locally on Chrome; full Chrome suite: CI.
• coverage-lib: 93.70% (33759/36029); llguidance_schema_contract.dart 151/153, litert_lm_service.dart 1061/1082
• dart test -p vm test/unit/tooling/ and test/unit/tooling/validation_coverage_test.dart
• Real-model litert-lm-json-schema e2e (below).

Mutation checks. Each mutant was applied alone against its targeted tests, then reverted. 22 of 22 were killed at 6720dd22. At 280bf13d, two more were killed: restoring the worker's pre-admission thinking guard fails worker_test, and dropping the maxTokens > 0 guard fails the stale-finish test. The worker-guard mutant row below is superseded, because the worker no longer checks thinking:

Mutant	Killed by
Thinking admitted for any family / without the bundle reasoning-channel probe / without the client thinking symbol	service tests
Drop the finish-reason admission requirement / the != stop end check	service tests
Map INTERNAL even after output started / never map INTERNAL	service tests
Service re-validation ignores thinking / service drops enableThinking before generation	service tests
Format allowlist adds time; oneOf allowed; open tuples allowed; $ref targets not walked; pattern repetition product ignored	contract tests
V1 conversation or turn ThinkingConfig always off	litert_lm_v1_json_schema_test
Worker admission reports no thinking / worker guard ignores thinking	worker_test
Backend guard ignores thinking	litert_lm_backend_test
Planner guard ignores thinking / engine pre-admission ignores the SPI / engine never routes to the thinking SPI	chat_completion_request_planner_test, engine_test

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
static-format-analyze	format, analyzer, boundaries	macOS arm64	PASS	at 6720dd22
coverage-lib	lib coverage >= 70%	macOS arm64	PASS	93.70% at 6720dd22; 4233707c changes one test literal only
litert-lm-json-schema	unconstrained control; two sentinels; scalar types; wide contract (pattern, uuid, bounds, $ref, anyOf, date, closed tuple) checked by the Dart validator; ja/ko Unicode; ordinary chat after schemas; typed oneOf refusal and recovery; thinking; history; cancel before/after content; reuse; length cutoff	macOS arm64, Qwen3-0.6B c5f39b41…, CPU and GPU	PASS 2/2	Qwen3 thinking refused naming Gemma 4
litert-lm-json-schema	same	macOS arm64, Gemma 4 E2B 18193810…, CPU and GPU	PASS 2/2	thinking non-empty (617 chars CPU, 502 GPU), then the JSON reply
structured-output-adversarial	compiled acceptance/rejection on the LiteRT path	macOS arm64	PASS	56-case contract test; each accepted case was compiled by both GBNF and the pinned runtime during design, and each refused case failed there or compiled differently. GBNF converter, templates and parsers are unchanged, so template-parity is N/A.
high-risk-exact-head-independent-qa	independent audit of the exact head	18e3993f	ACCEPT	Fresh auditor session; 0 P1/P2. Evidence in llamadart-evidence/pr-1907/. The evaluator exits 1 only on the two waived compiled_grammar_* axes (see the high-risk block).

Review Notes

• Independent review status: ACCEPT at 18e3993f (fresh auditor session, exact head; 0 P1/P2)
• CI status / head SHA: 18e3993f, CI pending; 280bf13d CI passed. 18e3993f adds causal tests (legacy runtime and facade schema-thinking refusal; matrix mode and registry note consistency) and merges current main (at 6720dd22: everything passed except Chrome shard 1/3, fixed by 4233707c)
• Independent audit at 4233707c: BLOCK on one P2, a stale worker thinking guard after client recreation. Fixed in 280bf13d together with P3s 1, 2 and 4 and the unreachable post-client backend check. Re-audit at 18e3993f: ACCEPT.
• Overlaps with parallel lanes:
  • packages/llamadart_validation/assets/capability_coverage.json: one description line (#1458 lane).
  • lib/src/backends/litert_lm/litert_lm_runtime.dart: a one-line supportsJsonSchemaThinking => false override on the legacy client (#1798 lane).
  • No edits to lib/src/core/template/** or the stream parser (#1898 lane).

High-risk regression review

• Classification: high-risk (backendRuntime, regressionPolicy, structuredOutput)
• Implementation task: llamadart 0.19.5 lane 6 (#1899)
• Independent blocking QA task: audit at 4233707c: BLOCK (P2, fixed in 280bf13d). Re-audit at 18e3993f: ACCEPT, 0 P1/P2.
• Maintainer waiver (one-time, 2026-11-10): the maintainer approved a one-time waiver of the compiled_grammar_acceptance and compiled_grammar_rejection axes, conditional on the independent audit confirming that this PR compiles no grammar. The re-audit confirmed it: llguidance_schema_contract.dart calls only the unchanged validateResponseSchema and JsonSchemaConverter.resolvePointer, then adds refusals. The grammarSchema minimum comes from the lib/src/core/grammar/ path, and the evaluator has no waiver flag (doc/high_risk_pre_merge_readiness.md), so it exits 1 (missingStructuredOutputEvidence) on exactly these two axes. All other axes are met. The admitted schema is compiled natively by LiteRT-LM LLGuidance and is exercised by the litert-lm-json-schema real-model scenario.
• Exact head / current base: 18e3993f / 33e00f5a (current origin/main merged in)
• Production-branch deletion, bypass, or miswire proof: 22/22 mutants killed (table above)
• Affected-family real-model/artifact evidence: Qwen3-0.6B and Gemma 4 E2B, macOS arm64 CPU and GPU, PASS
• Explicit unavailable-family or other N/A evidence: Other LiteRT-LM families are refused, not qualified
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

[leehack]

Landed via batch integration PR #1912 (rebase merge, main 2cfb936). The integration audit passed at a9c7772, and this PR's audited head is recorded in #1912's constituent table.

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-1907.