Keep response-format grammars applied through a budgeted thought on WebGPU (#1954)

CHANGELOG.md

@@ -5,6 +5,7 @@ Releases before 0.10.0 are in the
 
 - Fixed constrained JSON (`responseFormat`) ignoring the grammar after a `ThinkingBudget` closes a template-opened thought on native llama.cpp (Qwen3.5 and similar), and the same for required tool calls on templates whose grammar takes that thought.
 - Fixed a required tool call on Seed-OSS being unconstrained after a `ThinkingBudget` closes a thought the model opened, on native llama.cpp ([#1958](https://github.com/leehack/llamadart/issues/1958)).
+- Aligned the default WebGPU bridge assets to `v0.1.84`: constrained JSON (`responseFormat`) and Seed-OSS required tool calls stay constrained after a `ThinkingBudget` closes a thought on WebGPU, and older assets throw `LlamaUnsupportedException` for that combination ([#1954](https://github.com/leehack/llamadart/issues/1954)). Web keeps native llama.cpp `v0.9.0` parity (upstream `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`) and `@litert-lm/[email protected]`. Manifest SHA-256: `55abeb377597148156059a8ee5400f2314c6254ad91b57db65b709e304c0dea0`.
 
 ## 0.19.7
 

README.md

@@ -164,7 +164,7 @@ Current default runtime pins:
 | --- | --- |
 | Native llama.cpp / GGUF | `leehack/[email protected]` |
 | Native LiteRT-LM / `.litertlm` | `leehack/[email protected]` |
-| Web llama.cpp / GGUF | `leehack/[email protected].83` |
+| Web llama.cpp / GGUF | `leehack/[email protected].84` |
 | Web LiteRT-LM / `.litertlm` | `@litert-lm/[email protected]` |
 
 Native LiteRT-LM uses a lazy ABI-aware client for direct calls, services and

doc/multimodal_embeddings_web.md

@@ -39,7 +39,7 @@ Bridge source contract inspected read-only at
 restore and structured-clone ownership; those are bridge responsibilities.
 
 Qualification target: EmbeddingGemma-2 BF16 with its matching
-mmproj and pinned assets `v0.1.83` on Chrome CPU wasm memory64
+mmproj and pinned assets `v0.1.84` on Chrome CPU wasm memory64
 (`gpuLayers: 0`). The scenario below checks the bridge's production worker
 budget against references from the matching native `v0.9.0`.
 The table below retains the original `v0.1.66` lineage

doc/testing_matrix.md

@@ -1453,6 +1453,13 @@ prompt token, verifies unchanged ordinary grammar defaults, and verifies that
 explicit prompt penalties change the pick without activating a prompt-matching
 lazy trigger. Keep both memory64 assets beside the wasm32 files.
 
+The thinking-budget control adds `"reasoningModel"` (Qwen3.5-0.8B Q4_K_M) and,
+for the version-skew control, `"legacyAssets"` (an earlier bridge directory
+without `thinkingBudgetPauseGrammar`, such as `v0.1.83`) to the config. Qwen3.5
+`responseFormat` with a `ThinkingBudget` of 0, 1 and 16 tokens must still return
+the schema's JSON in both modes, and the legacy assets must throw
+`LlamaUnsupportedException` for a budget.
+
 This qualifies the Web grammar plumbing on the selected weights, not hardware
 WebGPU or LFM2 auto accuracy/enforcement (#1715). The latter still needs its
 separate template contract and false-probe restriction.

doc/webgpu_bridge.md

@@ -67,11 +67,11 @@ an earlier `<script>`:
 
 ## Pinned assets
 
-Default pinned tag in the example is `v0.1.83`.
+Default pinned tag in the example is `v0.1.84`.
 Vendor the pinned assets into the chat app with:
 
 ```bash
-WEBGPU_BRIDGE_ASSETS_TAG=v0.1.83 ./scripts/fetch_webgpu_bridge_assets.sh
+WEBGPU_BRIDGE_ASSETS_TAG=v0.1.84 ./scripts/fetch_webgpu_bridge_assets.sh
 ```
 
 `WEBGPU_BRIDGE_OUT_DIR` changes the destination. The script verifies
@@ -90,17 +90,17 @@ the chat app, set the bootstrap globals before the bootstrap runs:
 ```html
 <script>
   window.__llamadartBridgeAssetsRepo = 'leehack/llama-web-bridge-assets';
-  window.__llamadartBridgeAssetsTag = 'v0.1.83';
+  window.__llamadartBridgeAssetsTag = 'v0.1.84';
 </script>
 ```
 
 That release embeds llama.cpp `v0.9.0`, matching the `hook/build.dart` native pin
 (`v0.9.0`, both built from upstream llama.cpp `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`)
-even though the bridge asset tag `v0.1.83` differs from the native runtime tag
-`v0.9.0`. Provenance for this immutable consumer artifact: release `438242670`,
-tag commit `44aa84a8a51701dc9c4f82c21e6ee3e774919c20`, bridge source
-`cb6383fbe7e903764a89f5c80c7f81c219f99769`, and manifest SHA-256
-`ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5`.
+even though the bridge asset tag `v0.1.84` differs from the native runtime tag
+`v0.9.0`. Provenance for this immutable consumer artifact: release `443341300`,
+tag commit `0a3c45f24fd6293050e1772bed4a425b2b5a8493`, bridge source
+`dc359398bc6445ec15878ddb873555295f957f85`, and manifest SHA-256
+`55abeb377597148156059a8ee5400f2314c6254ad91b57db65b709e304c0dea0`.
 They expose the API-version-1 media embedding contract. State and extraction
 qualification covers wasm32 and memory64 through direct and worker runtimes. Existing decision,
 scoring, sampling, LoRA, speculative decoding and speech APIs remain available.
@@ -205,6 +205,15 @@ physical playback, intelligibility or speaker-reference fidelity.
   suppress tokens, so Qwen3-TTS no longer fails with `code out of range` when it
   samples a codec control token
   ([llama-web-bridge#211](https://github.com/leehack/llama-web-bridge/pull/211)).
+- `v0.1.84+`: `getCompletionCapabilities()` reports
+  `thinkingBudgetPauseGrammar`, and `thinkingBudget.pauseGrammar: false` keeps a
+  grammar applied while the model reasons
+  ([llama-web-bridge#213](https://github.com/leehack/llama-web-bridge/pull/213)).
+  llamadart sends it, as native does, only when `grammarLazy` is off and the
+  grammar takes the thought (`ToolCallGrammarUtils.keepsAppliedThroughThought`:
+  constrained JSON on Qwen3.5, required tool calls on Seed-OSS), and throws
+  `LlamaUnsupportedException` for that combination on assets without the flag.
+  Every other request omits `pauseGrammar` and is unchanged.
 
 ### Media embeddings
 

example/chat_app/web/index.html

@@ -157,7 +157,7 @@
       typeof configuredRepo === 'string' && configuredRepo.length > 0
         ? configuredRepo
         : 'leehack/llama-web-bridge-assets';
-    const defaultBridgeAssetsTag = 'v0.1.83';
+    const defaultBridgeAssetsTag = 'v0.1.84';
     const defaultBridgeLlamaCppTag = 'v0.9.0';
     const bridgeAssetsTag =
       typeof configuredTag === 'string' && configuredTag.length > 0

lib/src/backends/webgpu/interop.dart

@@ -299,6 +299,11 @@ extension type WebGpuThinkingBudgetOptions._(JSObject _) implements JSObject {
     @JS('endTag') required String endTag,
     @JS('forcedMessage') required String forcedMessage,
   });
+
+  /// `false` keeps the grammar applied inside a reasoning block; set only
+  /// when the loaded core reports `thinkingBudgetPauseGrammar`.
+  @JS('pauseGrammar')
+  external set pauseGrammar(bool value);
 }
 
 /// Completion options reported by `getCompletionCapabilities`.
@@ -317,6 +322,10 @@ extension type WebGpuCompletionCapabilities._(JSObject _) implements JSObject {
   @JS('thinkingBudget')
   external JSAny? get thinkingBudget;
 
+  /// Whether the loaded core applies `thinkingBudget.pauseGrammar`.
+  @JS('thinkingBudgetPauseGrammar')
+  external JSAny? get thinkingBudgetPauseGrammar;
+
   /// Whether the loaded core counts the prompt in its penalty window and
   /// applies `penaltyPromptTokens`.
   @JS('penaltyPromptTokens')

lib/src/backends/webgpu/webgpu_backend.dart

@@ -23,6 +23,7 @@ import '../../core/models/inference/model_params.dart';
 import '../../core/models/inference/model_params_runtime_support.dart';
 import '../../core/models/inference/next_token_scores.dart';
 import '../../core/template/media_placeholders.dart';
+import '../../core/template/tool_call_grammar_utils.dart';
 import '../backend.dart';
 import '../model_params_loras.dart';
 import '../web/webgpu_adapter_probe.dart';
@@ -43,6 +44,7 @@ typedef _WebGpuCompletionCapabilities = ({
   bool presencePenalty,
   bool minP,
   bool thinkingBudget,
+  bool thinkingBudgetPauseGrammar,
   bool penaltyPromptTokens,
   bool penaltyPromptTokensWithGrammar,
   bool lazyGrammar,
@@ -99,6 +101,7 @@ class WebGpuLlamaBackend
     presencePenalty: false,
     minP: false,
     thinkingBudget: false,
+    thinkingBudgetPauseGrammar: false,
     penaltyPromptTokens: false,
     penaltyPromptTokensWithGrammar: false,
     lazyGrammar: false,
@@ -1087,6 +1090,9 @@ class WebGpuLlamaBackend
         presencePenalty: reported(capabilities.presencePenalty),
         minP: reported(capabilities.minP),
         thinkingBudget: reported(capabilities.thinkingBudget),
+        thinkingBudgetPauseGrammar: reported(
+          capabilities.thinkingBudgetPauseGrammar,
+        ),
         penaltyPromptTokens: reported(capabilities.penaltyPromptTokens),
         penaltyPromptTokensWithGrammar: reported(
           capabilities.penaltyPromptTokensWithGrammar,
@@ -1941,6 +1947,22 @@ class WebGpuLlamaBackend
         'assets do not.',
       );
     }
+    final grammar = params.grammar;
+    final keepGrammarApplied =
+        thinkingBudget != null &&
+        grammar != null &&
+        !params.grammarLazy &&
+        ToolCallGrammarUtils.keepsAppliedThroughThought(grammar);
+    if (keepGrammarApplied && !capabilities.thinkingBudgetPauseGrammar) {
+      throw LlamaUnsupportedException(
+        'WebGPU thinking-budget control with a grammar that stays applied '
+        'through the thought (responseFormat on Qwen3.5, required tool calls '
+        'on Seed-OSS) needs bridge assets v0.1.84+ whose '
+        'getCompletionCapabilities() reports thinkingBudgetPauseGrammar; '
+        'the loaded core pauses every grammar while the model reasons, so '
+        'the reply would not be constrained.',
+      );
+    }
     final speculativeConfig = params.resolvedSpeculativeDecodingConfig;
     if (speculativeConfig != null) {
       requireWebGpuSpeculativeSupport(
@@ -1992,6 +2014,7 @@ class WebGpuLlamaBackend
         : _bridgeThinkingBudget(
             thinkingBudget,
             hasMediaParts: mediaParts != null,
+            pauseGrammar: !keepGrammarApplied,
           );
     final speculative = resolveWebGpuSpeculativeRequest(
       params,
@@ -2319,6 +2342,7 @@ class WebGpuLlamaBackend
   WebGpuThinkingBudgetOptions _bridgeThinkingBudget(
     ThinkingBudget budget, {
     required bool hasMediaParts,
+    required bool pauseGrammar,
   }) {
     if (budget.maxTokens < 0 || budget.maxTokens > _maxThinkingBudgetTokens) {
       throw RangeError.range(
@@ -2347,12 +2371,16 @@ class WebGpuLlamaBackend
         'selected chat template automatically.',
       );
     }
-    return WebGpuThinkingBudgetOptions(
+    final options = WebGpuThinkingBudgetOptions(
       maxTokens: budget.maxTokens,
       startTag: startTag,
       endTag: endTag,
       forcedMessage: budget.forcedMessage ?? '',
     );
+    // Sent only to keep a grammar applied, so every other request is
+    // unchanged on the wire.
+    if (!pauseGrammar) options.pauseGrammar = false;
+    return options;
   }
 
   @override

lib/src/core/models/inference/generation_params.dart

@@ -959,8 +959,10 @@ class GenerationParams {
   /// reasoning has ended. Native LiteRT-LM applies it to chat generation on
   /// bundles with a thinking channel, which it closes with the bundle's own
   /// end marker; it rejects a zero budget and [ThinkingBudget.forcedMessage].
-  /// WebGPU bridges that report `thinkingBudget` support it and pause every
-  /// grammar during reasoning.
+  /// WebGPU bridges that report `thinkingBudget` support it and pause grammars
+  /// the same way as native llama.cpp; keeping a grammar applied through the
+  /// thought needs assets that report `thinkingBudgetPauseGrammar` (v0.1.84+),
+  /// and older assets reject that combination.
   /// Other backends reject this setting rather than silently ignoring it.
   final ThinkingBudget? thinkingBudget;
 

lib/src/web_setup/web_bridge_setup.dart

@@ -15,14 +15,14 @@ import '../core/exceptions.dart';
 const String webBridgeAssetsRepository = 'leehack/llama-web-bridge-assets';
 
 /// The bridge asset release this llamadart version is qualified against.
-const String webBridgeAssetsTag = 'v0.1.83';
+const String webBridgeAssetsTag = 'v0.1.84';
 
 /// SHA-256 of the exact approved `manifest.json` of [webBridgeAssetsTag].
 ///
 /// The manifest lists the SHA-256 and size of every asset, so pinning its
 /// bytes pins every file the tool writes.
 const String webBridgeManifestSha256 =
-    'ae6f488408672f9ba28d8f9e664a2b85f4e12048f290e259e18dd0b750c038e5';
+    '55abeb377597148156059a8ee5400f2314c6254ad91b57db65b709e304c0dea0';
 
 /// Directory under the app's `web/` directory that receives the assets.
 const String webBridgeAssetsDirName = 'webgpu_bridge';

scripts/fetch_webgpu_bridge_assets.sh

@@ -4,7 +4,7 @@ set -euo pipefail
 ROOT_DIR="$(git rev-parse --show-toplevel)"
 OUT_DIR="${WEBGPU_BRIDGE_OUT_DIR:-$ROOT_DIR/example/chat_app/web/webgpu_bridge}"
 ASSETS_REPO="${WEBGPU_BRIDGE_ASSETS_REPO:-leehack/llama-web-bridge-assets}"
-ASSETS_TAG="${WEBGPU_BRIDGE_ASSETS_TAG:-v0.1.83}"
+ASSETS_TAG="${WEBGPU_BRIDGE_ASSETS_TAG:-v0.1.84}"
 CDN_BASE="${WEBGPU_BRIDGE_CDN_BASE:-https://cdn.jsdelivr.net/gh/${ASSETS_REPO}@${ASSETS_TAG}}"
 CURL_DOWNLOAD_ARGS=(
   --connect-timeout 15
@@ -19,7 +19,7 @@ if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
 Downloads prebuilt WebGPU bridge assets into the chat_app web directory.
 
 Default source:
-  https://cdn.jsdelivr.net/gh/leehack/[email protected].83
+  https://cdn.jsdelivr.net/gh/leehack/[email protected].84
 
 Environment variables:
   WEBGPU_BRIDGE_ASSETS_REPO   Asset repo in owner/repo format
@@ -31,7 +31,7 @@ Usage:
   ./scripts/fetch_webgpu_bridge_assets.sh
 
 Examples:
-  WEBGPU_BRIDGE_ASSETS_TAG=v0.1.83 ./scripts/fetch_webgpu_bridge_assets.sh
+  WEBGPU_BRIDGE_ASSETS_TAG=v0.1.84 ./scripts/fetch_webgpu_bridge_assets.sh
   WEBGPU_BRIDGE_ASSETS_REPO=acme/llama-web-bridge-assets WEBGPU_BRIDGE_ASSETS_TAG=v2 ./scripts/fetch_webgpu_bridge_assets.sh
 USAGE
   exit 0

skills/llamadart-web/SKILL.md

@@ -101,6 +101,10 @@ in the llamadart-getting-started and llamadart-chat-streaming skills.
     unless `withTextHistory()` is used and the loaded runtime supports them. Unreported
     controls throw `LlamaUnsupportedException`; `engine.runtime` names the
     runtime, so never sniff the `.litertlm` extension after loading.
+  - A `ThinkingBudget` with a grammar that stays applied through the thought
+    (`responseFormat` on Qwen3.5, required calls on Seed-OSS) needs assets that
+    report `thinkingBudgetPauseGrammar`; older assets throw
+    `LlamaUnsupportedException`.
   - `engine.supportsNextTokenScoring` and
     `engine.supportsStatePersistence`.
   - Runtime LoRA calls, and loads with `ModelParams.loras`, throw