Support strict structured output for LiteRT-LM backends

Status 2026-11-11: Native strict JSON Schema for LiteRT-LM shipped in 0.19.5 (#1912). Remaining: L4 GPU and physical Android evidence. Main pins LiteRT-LM v0.19.1-4 (lib/src/hook/native_release_pins.dart); the 2026-11-11 comment's v0.19.1-3 is superseded.

Current scope — 2026-10-26

Native strict-output delivery is still open, but an absent C API is not the established blocker: the export/contract reconciliation found the constraint-provider and constraint setters in shipped native artifacts. Remaining work is an additive schema/regex capability contract, optional-symbol/version/platform gating, Dart wiring and real-model enforcement tests; GBNF must not be passed to an incompatible backend. Web support remains separate. The older C-API premise below is historical.

Problem

The new LiteRT-LM backends support Gemma 4 chat, thinking output, and tool-call parsing, but strict structured output is not supported yet. Today the LiteRT-LM native and web backends report supportsGrammarConstraints => false, the engine skips template grammars for those backends, and LiteRT-LM generation rejects GenerationParams.grammar, grammarLazy, grammarTriggers, and grammarRoot.

This means responseFormat / JSON schema requests can work with the llama.cpp backend through grammar-constrained decoding, but the LiteRT-LM backend can only rely on prompt/template behavior and post-generation parsing.

Current evidence

LiteRtLmBackend.supportsGrammarConstraints and LiteRtLmBackendWeb.supportsGrammarConstraints are false.
LiteRtLmService._validateGenerationParams and the web mirror reject grammar-related params.
The native FFI path currently only binds litert_lm_conversation_config_set_enable_constrained_decoding and sets it to false; there is no Dart API for passing JSON schema, Lark, or other constraint payloads.
Upstream LiteRT-LM has constrained decoding concepts in ConversationConfig, including LLGuidance-style JSON Schema and Lark constraint support, so this looks like a missing bridge/backend feature rather than an impossible runtime capability.

Relevant upstream references:

Proposed direction

Extend litert-lm-native to expose constrained decoding configuration through the C ABI.
Add Dart FFI bindings for constraint provider/configuration.
Map llamadart responseFormat / JSON schema requests to the LiteRT-LM constraint type when possible.
Decide how to handle existing llama.cpp GBNF grammar inputs, since they may not translate 1:1 to LiteRT-LM constraints.
Enable supportsGrammarConstraints only for LiteRT-LM runtimes/platforms where the constrained decoding path is actually available.
Add unit tests and real-model E2E/smoke coverage that proves invalid structured output is prevented, not only parsed after generation.

Acceptance criteria

responseFormat: { type: json_schema, ... } works with LiteRT-LM native where supported.
Web behavior is either implemented or explicitly documented as unsupported with a clear error.
Unsupported grammar modes produce clear backend-specific errors.
Tests cover native/web routing, error messages, and at least one real-model JSON-schema smoke.

Folding in the contents of doc/litert_lm_structured_output.md, which is being deleted as redundant — its user-facing half is already covered by the website (guides/backend-selection.md:77, guides/generation-and-streaming.md:184, guides/chat-template-and-parsing.md:60) and its analysis half duplicates this issue. Two details from it that were not already written down here:

The exact public C API surface. The pinned runtime exposes only these three:

litert_lm_conversation_config_set_enable_constrained_decoding
litert_lm_conversation_optional_args_set_max_output_tokens
litert_lm_conversation_optional_args_set_visual_token_budget

That is enough to toggle LiteRT-LM's built-in constrained-decoding path for runtime-managed features, but not enough to map an arbitrary responseFormat schema onto a native constraint. Upstream's C++ Conversation::OptionalArgs does have a decoding_constraint field and ConversationConfig::Builder accepts a constraint provider config — the gap is purely in what the C API re-exports.

Three runtime surfaces, any one of which unblocks this. Strict LiteRT-LM structured output needs one of:

A public C API setter accepting a JSON-schema or Lark constraint for Conversation::OptionalArgs.
A public C API setter accepting a ConstraintProviderConfig on ConversationConfig.
A higher-level conversation option that directly models OpenAI-style response_format.

Until one exists, LiteRtLmBackend.supportsGrammarConstraints stays false on both the native and web implementations.

Upstream references:

docs: delete the orphan LiteRT-LM structured-output note (#367)

The recorded blocker does not hold: the constraint setters are declared in the public C API and are exported by the pinned runtime on every platform bundle cached on this machine.

Upstream C API

c/conversation.h declares both setters, checked with gh api repos/google-ai-edge/LiteRT-LM/contents/c/conversation.h?ref=<ref>:

ref	`litert_lm_conversation_config_set_constraint_provider`	`litert_lm_conversation_optional_args_set_constraint`
v0.16.0	`:151`	`:346`
v0.17.0	`:155`	`:353`
main	`:155`	`:353`

At v0.17.0, LiteRtLmConstraintType is {None, Regex, JsonSchema} (:62-64) and LiteRtLmConstraintProviderType is {LlGuidance} (:71). v0.16.0 was published 2026-08-11, ten days before the comment above, so these setters were already declared then.

All three functions the comment above lists are declared in that same c/conversation.h at v0.17.0: set_enable_constrained_decoding at :146, optional_args_set_visual_token_budget at :320, optional_args_set_max_output_tokens at :332 — nine lines from set_constraint_provider. The other header the comment links, c/engine.h, contains no occurrence of "constrain" at all at v0.17.0 (grep -ci constrain -> 0).

Exports on the pinned v0.17.0-6, verified by execution

Pin: lib/src/hook/native_release_pins.dart:11. Every bundle below is a locally cached v0.17.0-6 artifact; both symbols are present in each.

platform	binary	tool	result
macOS arm64	`libLiteRtLm.dylib`, sha256 `42a1fa7c…`	`nm -gU`	both
Android arm64	`libLiteRtLm.so`, sha256 `502d017d…`	`llvm-nm -D --defined-only`	both
iOS arm64 and arm64-simulator	`LiteRtLm.framework/LiteRtLm` from `litert-lm-native-apple-LiteRtLm-xcframework-v0.17.0-6.zip` (SwiftPM checksum `3b9ed48c…`)	`nm -gU`	both, in each slice
Windows x64	`LiteRtLm.dll`, sha256 `09cef404…`	`llvm-readobj --coff-exports` (513 exports)	both
Linux x64	—	—	not checked; no v0.17.0-6 linux bundle cached locally

Symbols, verbatim: litert_lm_conversation_config_set_constraint_provider, litert_lm_conversation_optional_args_set_constraint.

llguidance is linked in, not merely declared: 44 exported symbols named llg_* in both the macOS dylib and the Android .so (llg_clone_constraint, llg_clone_matcher, …), and every bundle ships libGemmaModelConstraintProvider beside libLiteRtLm.

What is still missing in llamadart (read-only, main `5cb9798`)

Only litert_lm_conversation_config_set_enable_constrained_decoding is bound, via hand-written lookupFunction, at lib/src/backends/litert_lm/litert_lm_runtime.dart:3302.
supportsGrammarConstraints => false at litert_lm_backend.dart:85 and litert_lm_backend_web.dart:100.
LiteRtLmService._validateGenerationParams rejects grammar (litert_lm_service.dart:1045), grammarLazy, grammarTriggers and grammarRoot (:1060).
litert-lm-native tools/litert_lm_symbols.py requires five conversation symbols, none constraint-related, so no release gate depends on these yet.
The user-facing error text is now wrong for native: chat_completion_request_planner.dart:85-93 throws for strict responseFormat saying "LiteRT-LM native and web currently do not expose public runtime wiring for JSON-schema/Lark constraints" (:88-90). Needs rewording whichever way this issue goes.

Open design questions — these are the blocker now, not the C API

Capability shape. LiteRT-LM accepts JSON Schema or regex; the planner probes supportsGrammarConstraints (lib/src/core/engine/chat_completion_request_planner.dart:75), drops the template grammar when it is false (:78), and otherwise passes GBNF straight through (:111-114, :140). Flipping the existing flag would route GBNF at a backend that cannot read it, so this needs an additive capability (a separate JSON-schema-constraint interface probed with is), not a change in meaning of the current one.
Subset divergence. The JSON-Schema subset llguidance accepts differs from the GBNF subset llamadart validates against (lib/src/core/models/inference/structured_output.dart:233-263), so the same responseFormat would be accepted and enforced differently per backend. That contract needs deciding before any binding lands.
Web surface unknown. grep -rli constraint over litert-lm-native/web returns nothing, and the only hits in litert_lm_backend_web.dart are the BackendGrammarConstraintsSupport mixin (:33) and the false getter (:100) — no evidence llguidance is in the WASM build at all, so the acceptance criterion's "implemented" branch is out of reach for web until that is settled.
Per-platform gating. Linux x64 is unverified above, and an exported symbol is not proof the path works under each accelerator — GPU/WebGPU sampler paths may bypass constraint masks. Adding the two symbols to tools/litert_lm_symbols.py as required makes every bundle gate on them, so per-platform nm has to come first.
Real-model proof. Proposed direction item 6 asks for coverage that invalid structured output is prevented, not parsed afterwards, and the acceptance criteria require at least one real-model JSON-schema smoke. Not attempted here.

Smallest useful next slice, once the capability shape is agreed: bind the two functions beside litert_lm_runtime.dart:3302 with optional-symbol lookup, then spike a macOS arm64 JSON-schema constraint against a small Gemma model — before any capability flag changes.

2026-10-27 delivery reconciliation — retain P2 and the wider scope.

#1441 landed through integration #1450, commit 673c019. Native strict JSON is now source-implemented for canonical Qwen3 text on explicit macOS arm64 CPU, thinking off, using an actually constructed LLGuidance provider. The bounded object/scalar schema subset has enforcement, reset, history, cancellation, cutoff and missing-symbol controls. GBNF, wider schemas/models/hosts/delegates and Web retain explicit refusals; usage stays unavailable.

This uses already-published owner v0.18.1-4: no owner release or pin bump is required. Public core 0.16.4 does not contain this new route. It is selected source for 0.16.5, not published Dart support. Combined-main CI passes; constituent and integration PRs record separate independent acceptance.

Next: preserve the narrow capability/model/provider gate through release qualification and publication. Broader schema/platform coverage and Web termination remain separate; do not close this umbrella or repeat the implemented slice.

2026-10-28 publication correction — the bounded native schema route is now consumable.

Core0.16.5, published through #1455, contains#1441/#1450's native strict JSON implementation. Actual archive SHA25636eff30e2fa23134bd946e970e31f3d6ccb1d9ebb6221e3e5443c48c18b9e4a7 matches registry metadata and this scan's downloaded bytes. Owner v0.18.1-4 is already pinned; no further artifact or binding bump is needed for the delivered subset.

Keep P2/open for wider acceptance: canonical Qwen3 text, explicit macOS arm64 CPU, actual LLGuidance provider, thinking off and bounded object/scalar schemas are the qualified route. GBNF, unsupported schemas/models/hosts/delegates and Web still refuse; usage remains unavailable. Do not assign the already-published wiring again or infer raw0.19/other-platform support. Wider qualification remains a separately selected task.

2026-11-10 scoping for 0.19.5 (plan #1899 lane 6). No PRs, no pushes.

Outcome

Native: the pinned runtime already enforces JSON Schema well beyond what llamadart admits. With the shipped litert-lm-native v0.19.1-2 (upstream v0.19.1 7a03724c), it enforces on every bundle we can test and for every model tried: Gemma 4 E2B, Qwen3, Qwen3.5, on CPU and on macOS GPU (WebGPU). Nested, array, $ref, anyOf, pattern and min/max schemas all work. Widening llamadart needs no owner change, release or pin bump. The work is llamadart gating plus qualification.
Web: blocked upstream. @litert-lm/[email protected] compiles llguidance into the WASM but never exposes a provider or a per-message constraint, and upstream main doesn't either. The current typed refusal stays.
Option (b), llamadart-side logit masking: not viable. The public C API has no logit, sampler or mask hook.

Runtime surface (pinned)

Surface	Native v0.19.1-2	Web `@litert-lm/[email protected]`
JSON Schema / regex via llguidance	Public C API: `conversation_config_set_constraint_provider(LlGuidance)` + `optional_args_set_constraint({Regex, JsonSchema})` (`c/conversation.h:199,453`)	Not exposed. `ConversationConfig.createCustom` only takes `enableConstrainedDecoding`, and `sendMessage(json, blobs, visualTokenBudget)` takes no constraint. llguidance strings are present in all 4 `.wasm` files
Lark	C++ only (`LlgConstraintType::kLark`), not in the C API (also absent on `main`)	—
Gemma FST tool-call constraint	`set_enable_constrained_decoding`. Owner iOS builds compile it out (`docs/platform_strategy.md`)	Same boolean
`tool_choice` constraint	Only on upstream `main` (C API 1.1.0, 2026-11-05), not tagged	—
Logit / sampler hook	None. `ExternalConstraintArg` exists in C++ only	None
GPU path	`tasks.cc` uses `ConstrainedDecoder::Create(..., accelerator)` (LiteRT mask graph) on GPU, `CreateForHost` on CPU	—

Constraint exports plus linked llguidance were checked on every pinned archive (archive SHA-256 matches native_release_pins.dart): macOS arm64, Android arm64 and x64, iOS arm64 and arm64-sim (LiteRtLm.framework), Linux arm64 and x64, and Windows x64 (llvm-readobj --coff-exports).

Real-model probes

These are single greedy runs through a scratch C program that calls the public C API on the pinned libLiteRtLm (macOS sha 5054e7f2…, M4 Max), with thinking off. The adversarial prompt was "Reply with the single word BANANA". Without a constraint, every model answers BANANA.

Model	Backend / host	Schema	Result
Qwen3-0.6B `c5f39b41`	CPU macOS	flat object	`{ "name": "BANANA", "age": 10 }`
Qwen3-0.6B	GPU macOS (WebGPU)	flat object	enforced
Qwen3-0.6B	CPU macOS	nested + array + `pattern` + `min/max` + `anyOf`/`enum`/null	enforced
Qwen3-0.6B	CPU macOS	`$defs`/`$ref` + `maxLength`	enforced
Qwen3-0.6B	CPU macOS	regex `[0-9]{3}-[A-Z]{2}`	`100-BY`
Gemma 4 E2B `18193810`	CPU and GPU macOS	flat object, nested + array	enforced
Gemma 4 E2B	CPU macOS	Unicode enum + `maxLength` (ja/ko)	`{"city":"東京","note":"도쿄는 일본의 수도이며"}`
Qwen3.5-0.8B int8 `64ed396f`	CPU macOS	flat object	enforced
Qwen3-0.6B	CPU Linux arm64 (Docker, native arm64)	flat object	enforced
Qwen3-0.6B	CPU Linux x64 (Docker under Rosetta: pipeline evidence only)	flat object	enforced
Gemma 4 E2B	CPU, thinking on	flat object	enforced after the thought channel
Qwen3-0.6B	CPU, thinking on	flat object	not enforced: answer `"\n\nAlice and 31."`. Root cause not established

Failure modes llamadart must map:

Unsupported keywords fail closed at send time with INTERNAL (13), not at config time. Seen with uniqueItems, unknown format, invalid type, malformed JSON and a bad regex. The C API returns Failed to create LLGuidance constraint: ….
An accepted format: email let Qwen3 loop inside a string until max_output_tokens. The output was a valid prefix but incomplete JSON, so a cutoff has to surface as truncation, not as success.

Not run: Android, iOS and Windows devices, Linux x64 on a native host, and the Web pinned-asset path. Exports alone don't prove the path works on those platforms.

Options

Option	Platforms	Owner work	Risk	Validation	Size
A1. Widen admission to Gemma 4 (bundle template, no prompt override) and macOS arm64 GPU. Keep today's flat schema subset and the thinking/tools/media refusals	macOS arm64 CPU+GPU	None	High-risk (structured output, runtime routing): independent audit and readiness evidence	Parameterize `litert-lm-v1-json-schema` by model/backend. Real Gemma 4 E2B CPU+GPU and Qwen3 GPU runs. Unit tests for gate and refusal text. Capability coverage plus README/support matrix/skills/changelog	M (~300–500 LOC incl. tests)
A2. Lift the hard-coded macOS-only platform gate to Linux, Windows, Android and iOS, still behind the existing provider-admission probe	+Linux arm64/x64, Windows x64, Android arm64, iOS	None (optional: require the two constraint symbols in owner `tools/litert_lm_symbols.py`)	Medium per platform: GPU mask graphs on OpenCL/Metal are unverified	Native x64 Linux and Windows (GCE), Android device/FTL, iOS device or simulator, each CPU (+GPU where offered)	S code, L qualification
A3. Widen the schema subset (nested objects, arrays, `$ref`, `anyOf`, `pattern`, bounds)	Wherever A1/A2 apply	None	Contract divergence from llama.cpp GBNF (`structured_output.dart`). llguidance rejects at send with `INTERNAL`, so typed mapping needs a pre-validated keyword allowlist, not message parsing	Allowlist tests at the intersection of llguidance and the GBNF converter. Real-model nested/array/Unicode/cutoff cases	M
A4. Schema plus thinking	Gemma 4 only, per family	None	Qwen3 probe did not enforce. Must stay per-family	Per-family real-model control	S–M, after A1
B. llamadart-side logit masking	—	Owner C++ bridge export wrapping `ExternalConstraintArg` with a per-token callback	High: per-token Dart callback over a 150k–262k vocab across the worker boundary, and it duplicates llguidance	—	L. Rejected
C. Keep refusing	Web; GBNF, Lark, tools+schema, Qwen3 thinking+schema	Web needs an upstream `@litert-lm/core` change (`litertlm_web.cc` `MakeOptionalArgs`) or the owner-built WASM excluded in #1863	Low	Existing refusal tests (`chat_completion_request_planner.dart:343`)	Done today

Recommendation

0.19.5: A1 only. Gemma 4 E2B plus macOS arm64 GPU, on the existing bounded subset, with a real-model local scenario per model/backend. It is llamadart-only and qualifiable on the maintainer Mac.
Next minor: A2 and A3 as their own lanes, gated on device/GCE qualification and the schema-contract decision.
Kept as typed refusals: Web and option B.
Separate follow-ups: regex responseFormat and tool_choice (upstream C API 1.1.0 needs an upstream tag, then an owner release).

Decisions needed

Take A1 into 0.19.5, or defer all widening to the next minor.
Schema contract for A3: keep the flat subset, widen to the llguidance ∩ GBNF intersection (recommended: same request, same acceptance on both backends), or pass llguidance's full subset through.
Qualification bar for A2: approve GCE (native Linux x64, Windows) and Android/iOS device runs, and decide whether Docker Linux arm64 counts.
Web: accept the refusal until upstream exposes constraints, and say whether to raise an upstream feature request (maintainer-posted) or keep the LiteRT-LM Web generate() wraps the raw prompt in the bundle's user turn #1863 owner-WASM route excluded.
Thinking + schema: keep refusing for all families, or qualify Gemma 4 per family (A4).

Maintainer decisions (2026-11-10), for 0.19.5 (#1899):

Scope: "A1 + A2 + A3" — native Gemma 4 and macOS GPU, the gate lifted to Linux/Windows/Android/iOS (qualified on GCE and devices), and wider schemas.
Schema contract: "llguidance ∩ llama.cpp GBNF (Recommended)" — accept only features both enforce; others throw LlamaUnsupportedException before generation.
Thinking + schema: "Qualify per family (Recommended)" — Gemma 4 now; others refused.
Web: keep refusing; a feature request for @litert-lm/core will be drafted for the maintainer to post.

Admit native LiteRT-LM JSON Schema only where real-model runs passed

Let the service decide JSON Schema thinking after re-admission

Admit native LiteRT-LM JSON Schema on Linux x64/arm64 and Windows x64…

Gate native LiteRT-LM JSON Schema on the backend that actually loads

Status: native LiteRT-LM JSON Schema shipped to main in #1912 (constituents #1907 and #1909).

Qualified hosts: macOS arm64 CPU/GPU; iOS arm64 CPU/GPU (simulator evidence); Android arm64 CPU (emulator, Qwen3); Linux x64, Linux arm64 and Windows x64 CPU (GCE).
Gemma 4 thinking is supported; every other host and accelerator is refused.
Widen native LiteRT-LM JSON Schema to Gemma 4, macOS GPU and the shared schema contract #1907 carries a one-time maintainer waiver for the compiled-grammar evidence axes.
Still open: L4 GPU runs on Linux and Windows (stockout), physical Android and iPhone runs, the glibc >= 2.38 doc note for Linux arm64, regex constraints, and the Web upstream request. The P3s are on 0.19.x audit follow-ups (index) #1861.

2026-11-11 publication checkpoint: the widened native JSON Schema subset is first publicly consumable in 0.19.5 (pub.dev); #1907/#1909 via #1912 delivered it. Public/main/local now pin owner v0.19.1-3, upstream 7a03724c; the provider/thinking setters were already in C API 1.0.0. Widening required consumer gating/configuration/tests, not a new strict-output ABI.

Keep the qualified subset exact: Qwen3 canonical/Gemma4 bundle templates; shared LLGuidance/GBNF schema contract; Gemma4 thinking only. Recorded hosts are macOS arm64 CPU/GPU; iOS arm64 CPU/GPU with simulator evidence; Android arm64 CPU with Qwen emulator evidence; Linux x64/arm64 and Windows x64 CPU with GCE evidence. Gemma4 simulator GPU engine-creation failure is NOT a schema pass. Unsupported schemas/options/hosts retain typed pre-generation refusal; cut-off JSON reports length, not success.

Remaining acceptance is unchanged: Linux/Windows L4 GPU stockouts are NOT passes, physical Android/iPhone schema runs remain absent, and regex, Web constraints/upstream request and Linux-arm64 glibc guidance remain separate. The recorded one-time compiled-grammar waiver is not a reusable general exemption. No new owner release/pin change or device job is required merely to consume this published subset; do not close the wider platform/API work on publication alone.

Let Test Lab stage a LiteRT-LM model under its real file name (#174) (#…

Test automatic Android strict-output admission and close audit P3s (#174

2026-11-14 physical-device evidence/admission checkpoint: S24 and Pixel9Pro CPU/OpenCL-GPU schema controls, plus S24 WebGPU schema controls, are now recorded in plan#1995/repin#2003. The S24 WebGPU receipts use the corrected owner19.1-5 sampler route; they are separate from the full OpenCL qualifier and not CPU fallback.

Main/local6b18ff51 consumes published owner19.1-5 (same7a03724c/C API1.0 constraint setters), while public0.19.9 still pins-4. Draft#2006, inspected27b580e6, is the separate Android-arm64 GPU admission step: no public Android schema-GPU support claim until current-head tests/audit and core publication. No new constraint ABI/bindings are inherent to that widening; truthful runtime/probe/old-version refusal and exact model/backend/schema controls remain.

The earlier physical-Android absence is superseded only for the recorded devices/routes. Physical iPhone, Linux/Windows L4 schema-GPU, regex and Web constraints remain separate acceptance; other model/device/vendor combinations are not proven by one S24 success. Existing P2 scope and the one-time earlier waiver are unchanged; no device/paid run was started by this scan.

10:22 UTC delta: #2006 has now merged as ff6e028. Android-arm64 GPU schema admission is on GitHub main with the same owner19.1-5; this supersedes the draft-only checkpoint above. Public0.19.9 still lacks it, and core publication plus the remaining platform/schema acceptance stay separate.

Admit LiteRT-LM strict JSON Schema on the Android arm64 GPU (#174) (#…

Android arm64 GPU admission for strict JSON Schema merged in #2006 (ff6e028b). Set B on Test Lab passed at the PR head on LiteRT-LM v0.19.1-5: OpenCL on the S24 and Pixel 9 Pro with Gemma 4 and Qwen3, plus WebGPU on the Pixel. It ships in 0.19.10.

Still open: the Linux L4 GPU evidence.

Linux x64 GPU (NVIDIA L4, Vulkan) passes the strict JSON Schema scenario. GCE g2-standard-8, NVIDIA 615.71.09; vulkaninfo reports NVIDIA L4 as a discrete GPU. Source ff6e028b with the one-line Abi.linuxX64: {'cpu', 'gpu'} admission (diff SHA-256 1378573b…639e); run_local_e2e.dart --scenario litert-lm-json-schema.

Row	Cases	GPU memory peak	Result
qwen3-0.6b cpu	12	0	All tests passed
qwen3-0.6b gpu	12	1744 MiB	All tests passed
gemma-4-e2b cpu	13	0	All tests passed
gemma-4-e2b gpu	13	1127 MiB (85% util)	All tests passed

The GPU rows return the same outputs as the CPU control: sentinels, scalar types, wide contract, unicode, history, cancel/reuse, and a length cut inside a constrained string. Results tarball SHA-256: af95f86aea48a88f0be870c35484f090f6f6f0871129ba67bc1438b4ae742950. Next: a PR admitting linux_x64 GPU. Windows x64 (D3D12) is still unqualified.

Navigation Menu

Repository navigation

Support strict structured output for LiteRT-LM backends #174

Current scope — 2026-10-26

Problem

Current evidence

Proposed direction

Acceptance criteria

Related

Upstream C API

Exports on the pinned v0.17.0-6, verified by execution

What is still missing in llamadart (read-only, main `5cb9798`)

Open design questions — these are the blocker now, not the C API

Outcome

Runtime surface (pinned)

Real-model probes

Options

Recommendation

Decisions needed

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Support strict structured output for LiteRT-LM backends #174

Description

Current scope — 2026-10-26

Problem

Current evidence

Proposed direction

Acceptance criteria

Related

Activity

leehack commented on Aug 21, 2026

leehack commented on Sep 21, 2026

Upstream C API

Exports on the pinned v0.17.0-6, verified by execution

What is still missing in llamadart (read-only, main 5cb9798)

Open design questions — these are the blocker now, not the C API

leehack commented on Oct 27, 2026

leehack commented on Oct 28, 2026

leehack commented on Nov 10, 2026

Outcome

Runtime surface (pinned)

Real-model probes

Options

Recommendation

Decisions needed

leehack commented on Nov 10, 2026

leehack commented on Nov 10, 2026

leehack commented on Nov 11, 2026

leehack commented on Nov 14, 2026

leehack commented on Nov 14, 2026

leehack commented on Nov 14, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

What is still missing in llamadart (read-only, main `5cb9798`)