Repin WebGPU bridge assets to v0.1.85 with isolation and GPU fallback errors (#1025)

CHANGELOG.md

@@ -4,6 +4,10 @@ Releases before 0.10.0 are in the
 ## Unreleased
 
 - Chat app microphone: a microphone that disconnects mid-recording now transcribes the audio captured so far with a warning, a missing input device on Web is reported as such instead of as a permission denial, and a failed Web start releases the microphone ([#329](https://github.com/leehack/llamadart/issues/329)).
+- Aligned the default WebGPU bridge assets to `v0.1.85`: a page without cross-origin isolation now fails at once with a `LlamaUnsupportedException` naming COOP/COEP `require-corp` instead of waiting minutes for a generic init timeout, and Firefox CPU runs (`gpuLayers: 0`) no longer create the WebGPU backend (stories15M, 32 tokens: 640 to 762 tok/s against 10 before) ([#1025](https://github.com/leehack/llamadart/issues/1025)).
+- `ComputeDevice.gpu` on Web now names why WebGPU was not used (for example `adapter_missing_shader_f16`: Chrome on Linux with NVIDIA needs driver 615.71 or newer); documented the measured Safari 27, Firefox 160 and NVIDIA L4 results ([#1025](https://github.com/leehack/llamadart/issues/1025)).
+
+Runtime baseline: native llama.cpp `v0.9.0` and Web use upstream `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`; native LiteRT-LM uses `v0.19.1-4` and stable-diffusion uses `v0.3.0-3`. Web LiteRT-LM uses `@litert-lm/[email protected]`. Manifest SHA-256: `6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3`.
 
 ## 0.19.9
 

README.md

@@ -164,7 +164,7 @@ Current default runtime pins:
 | --- | --- |
 | Native llama.cpp / GGUF | `leehack/[email protected]` |
 | Native LiteRT-LM / `.litertlm` | `leehack/[email protected]` |
-| Web llama.cpp / GGUF | `leehack/[email protected].84` |
+| Web llama.cpp / GGUF | `leehack/[email protected].85` |
 | Web LiteRT-LM / `.litertlm` | `@litert-lm/[email protected]` |
 
 Native LiteRT-LM uses a lazy ABI-aware client for direct calls, services and

doc/multimodal_embeddings_web.md

@@ -39,7 +39,7 @@ Bridge source contract inspected read-only at
 restore and structured-clone ownership; those are bridge responsibilities.
 
 Qualification target: EmbeddingGemma-2 BF16 with its matching
-mmproj and pinned assets `v0.1.84` on Chrome CPU wasm memory64
+mmproj and pinned assets `v0.1.85` on Chrome CPU wasm memory64
 (`gpuLayers: 0`), Chrome CPU wasm32 and Chrome WebGPU on an Apple GPU (Metal
 backend, `gpuLayers: 99`, memory64 core). The scenario below checks the bridge's production worker
 budget against references from the matching native `v0.9.0`.

doc/testing_matrix.md

@@ -305,8 +305,8 @@ compares that generation with its exact baseline before reusing TTS.
 These bounded controls do not qualify every output length or load setting.
 
 The `chrome-webgpu` test pages enable SharedArrayBuffer with browser flags but
-remain non-isolated (`crossOriginIsolated == false`), so the bridge caps native
-threads at one. In the [#1371 controls recorded in PR #1418](https://github.com/leehack/llamadart/pull/1418),
+remain non-isolated (`crossOriginIsolated == false`), so assets without the `v0.1.85+` isolation check
+capped native threads at one. In the [#1371 controls recorded in PR #1418](https://github.com/leehack/llamadart/pull/1418),
 with that run's staged Qwen3-TTS fixture and bridge manifest, the 4096/4096 batch
 profile aborted even on fresh one- and eight-frame synthesis, before
 cancellation. The 512/256 fresh controls succeeded. This historical diagnostic

doc/webgpu_bridge.md

@@ -60,18 +60,18 @@ an earlier `<script>`:
 | `__llamadartBridgeAssetsRepo`, `__llamadartBridgeAssetsTag` | CDN source; defaults to `leehack/llama-web-bridge-assets` at the pinned tag |
 | `__llamadartPreferLocalBridgeRuntime` | `true` loads local assets first off `localhost` |
 | `__llamadartBridgeEnableMem64` | `true` sets `__llamadartBridgePreferMemory64`; default off |
-| `__llamadartBridgeThreadPoolSize` | Thread hint; default 1 without cross-origin isolation, else `hardwareConcurrency` clamped to 2..4 |
+| `__llamadartBridgeThreadPoolSize` | Thread hint; chat app default 1 when the page is not cross-origin isolated, else `hardwareConcurrency` clamped to 2..4 |
 | `__llamadartBridgeSpeechToTextSupported` | Defaults to `true` only for the official repository at `v0.1.30` or newer |
 | `__llamadartLiteRtLmModuleUrl` | `@litert-lm/core` module; default `@litert-lm/[email protected]` from jsDelivr |
 | `__llamadartBridgeBootstrapVerbose` | `true` enables bootstrap `console.*` logs |
 
 ## Pinned assets
 
-Default pinned tag in the example is `v0.1.84`.
+Default pinned tag in the example is `v0.1.85`.
 Vendor the pinned assets into the chat app with:
 
 ```bash
-WEBGPU_BRIDGE_ASSETS_TAG=v0.1.84 ./scripts/fetch_webgpu_bridge_assets.sh
+WEBGPU_BRIDGE_ASSETS_TAG=v0.1.85 ./scripts/fetch_webgpu_bridge_assets.sh
 ```
 
 `WEBGPU_BRIDGE_OUT_DIR` changes the destination. The script verifies
@@ -90,17 +90,17 @@ the chat app, set the bootstrap globals before the bootstrap runs:
 ```html
 <script>
   window.__llamadartBridgeAssetsRepo = 'leehack/llama-web-bridge-assets';
-  window.__llamadartBridgeAssetsTag = 'v0.1.84';
+  window.__llamadartBridgeAssetsTag = 'v0.1.85';
 </script>
 ```
 
 That release embeds llama.cpp `v0.9.0`, matching the `hook/build.dart` native pin
 (`v0.9.0`, both built from upstream llama.cpp `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`)
-even though the bridge asset tag `v0.1.84` differs from the native runtime tag
-`v0.9.0`. Provenance for this immutable consumer artifact: release `443341300`,
-tag commit `0a3c45f24fd6293050e1772bed4a425b2b5a8493`, bridge source
-`dc359398bc6445ec15878ddb873555295f957f85`, and manifest SHA-256
-`55abeb377597148156059a8ee5400f2314c6254ad91b57db65b709e304c0dea0`.
+even though the bridge asset tag `v0.1.85` differs from the native runtime tag
+`v0.9.0`. Provenance for this immutable consumer artifact: release `444874209`,
+tag commit `28a6180925e74e1038ee4c22cde6b403ae21ed84`, bridge source
+`f1ac2fc1d4d4d3fda56c161ac6a629ae4c67a7d3`, and manifest SHA-256
+`6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3`.
 They expose the API-version-1 media embedding contract. State and extraction
 qualification covers wasm32 and memory64 through direct and worker runtimes. Existing decision,
 scoring, sampling, LoRA, speculative decoding and speech APIs remain available.
@@ -205,6 +205,10 @@ physical playback, intelligibility or speaker-reference fidelity.
   suppress tokens, so Qwen3-TTS no longer fails with `code out of range` when it
   samples a codec control token
   ([llama-web-bridge#211](https://github.com/leehack/llama-web-bridge/pull/211)).
+- `v0.1.85+`: a page without cross-origin isolation rejects the load at once
+  with `CrossOriginIsolationError` (`code: 'CROSS_ORIGIN_ISOLATION_REQUIRED'`);
+  `gpu_unavailable:<reason>` appears in the runtime notes when `nGpuLayers > 0`
+  and the GPU is not active; at 0 GPU layers the WebGPU backend is not created.
 - `v0.1.84+`: `getCompletionCapabilities()` reports
   `thinkingBudgetPauseGrammar`, and `thinkingBudget.pauseGrammar: false` keeps a
   grammar applied while the model reasons
@@ -336,8 +340,11 @@ isolated (`Cross-Origin-Opener-Policy: same-origin`,
 `Cross-Origin-Embedder-Policy: require-corp`; `credentialless` isolates only in
 Chromium-based browsers, see
 [#1025](https://github.com/leehack/llamadart/issues/1025)). Without it
-the bridge caps threads to one, and fetch-backed loading fails with
-thread-constructor errors.
+the web core cannot start: assets `v0.1.85+` reject the load at once
+with `CrossOriginIsolationError` (`code: 'CROSS_ORIGIN_ISOLATION_REQUIRED'`),
+which `WebGpuLlamaBackend` throws as `LlamaUnsupportedException` on the direct
+and worker paths; earlier assets waited minutes (the init timeout, retried: 180 s direct,
+360 s worker) before a generic `Bridge core init timeout` error. `credentialless` does not isolate Safari.
 
 For Hugging Face Spaces with `sdk: static`, set the headers in the Space
 README frontmatter; keys and values must be lowercase:
@@ -384,6 +391,14 @@ On macOS headless Chromium, keep the smoke script's default
 adapter can lack `shader-f16`, and llama.cpp may abort in `ggml-webgpu` even on
 `gpuLayers = 0` runs.
 
+When `nGpuLayers > 0` and the GPU is not active, assets `v0.1.85+` add
+the runtime note `gpu_unavailable:<reason>` (`no_webgpu`, `no_adapter`,
+`adapter_missing_shader_f16` or `unknown`), and the `ComputeDevice.gpu` error
+"loaded the model without GPU layers" names it. Chrome on Linux hides
+`shader-f16` from NVIDIA adapters with a driver older than 615.71. At 0 GPU
+layers the bridge no longer creates the WebGPU backend: on Firefox 160 (isolated
+page, CPU, stories15M, 32 tokens) that is 640 to 762 tok/s against 10 tok/s before.
+
 The speech smokes read the loaded runtime from the bridge, not from the
 request. `chat-app-web-speech-to-text-webgpu-smoke` requires `isGpuActive()` to
 be `true`, `-mem64-smoke` requires `llamadart.webgpu.core_variant` to be

example/chat_app/lib/providers/chat_provider.dart

@@ -2661,13 +2661,6 @@ class ChatProvider extends ChangeNotifier {
         .where((value) => value.isNotEmpty)
         .toSet();
 
-    if (runtimeNotes.contains('threads_capped_no_coi')) {
-      _addInfoMessage(
-        'Web runtime is not cross-origin isolated, so inference threads are capped to 1. '
-        'Enable COOP/COEP headers for better throughput.',
-      );
-    }
-
     if (runtimeNotes.contains('threads_capped_no_pthread')) {
       _addInfoMessage(
         'Loaded bridge core does not include pthread support, so runtime threads are capped to 1. '

example/chat_app/test/runtime_profile_service_test.dart

@@ -14,9 +14,9 @@ void main() {
           'llamadart.webgpu.thread_pool_size': '2',
           'llamadart.webgpu.execution': 'worker',
           'llamadart.webgpu.core_variant': 'wasm64',
-          'llamadart.webgpu.worker_fallback_reason': 'threads_capped_no_coi',
+          'llamadart.webgpu.worker_fallback_reason': 'worker_unavailable',
           'llamadart.webgpu.runtime_notes':
-              'threads_capped_no_coi;model_fetch_backend_attempt',
+              'core_pthreads:1;model_fetch_backend_attempt',
           'llamadart.webgpu.model_source': 'network-fetch',
           'llamadart.webgpu.model_cache_state': 'hit',
         },
@@ -27,10 +27,10 @@ void main() {
       expect(diagnostics.runtimeThreadPoolSize, 2);
       expect(diagnostics.runtimeExecution, 'worker');
       expect(diagnostics.runtimeCoreVariant, 'wasm64');
-      expect(diagnostics.runtimeWorkerFallbackReason, 'threads_capped_no_coi');
+      expect(diagnostics.runtimeWorkerFallbackReason, 'worker_unavailable');
       expect(
         diagnostics.runtimeNotes,
-        'threads_capped_no_coi;model_fetch_backend_attempt',
+        'core_pthreads:1;model_fetch_backend_attempt',
       );
       expect(diagnostics.runtimeModelSource, 'network-fetch');
       expect(diagnostics.runtimeModelCacheState, 'hit');

example/chat_app/web/index.html

@@ -157,7 +157,7 @@
       typeof configuredRepo === 'string' && configuredRepo.length > 0
         ? configuredRepo
         : 'leehack/llama-web-bridge-assets';
-    const defaultBridgeAssetsTag = 'v0.1.84';
+    const defaultBridgeAssetsTag = 'v0.1.85';
     const defaultBridgeLlamaCppTag = 'v0.9.0';
     const bridgeAssetsTag =
       typeof configuredTag === 'string' && configuredTag.length > 0

lib/src/backends/webgpu/webgpu_backend.dart

@@ -1332,9 +1332,18 @@ class WebGpuLlamaBackend
           await loadPromise.toDart;
         }
         if (setup.requireGpu && bridge.isGpuActive() != true) {
+          var gpuDetail = '';
+          try {
+            gpuDetail = webGpuUnavailableDetail(
+              _collectBridgeRuntimeHints(
+                    bridge,
+                  )['llamadart.webgpu.runtime_notes'] ??
+                  '',
+            );
+          } catch (_) {}
           throw gpuInactive = LlamaUnsupportedException(
             'ComputeDevice.gpu needs WebGPU for llama.cpp on the Web, but the '
-            'runtime loaded the model without GPU layers. Use '
+            'runtime loaded the model without GPU layers.$gpuDetail Use '
             'ComputeDevice.auto or cpu.',
           );
         }
@@ -1360,6 +1369,11 @@ class WebGpuLlamaBackend
           await _safeDisposeBridge();
           rethrow;
         }
+        final isolationError = webGpuCrossOriginIsolationError(e);
+        if (isolationError != null) {
+          await _safeDisposeBridge();
+          throw isolationError;
+        }
         lastError = e;
         Map<String, String> runtimeHints = const <String, String>{};
         if (bridgeForAttempt != null) {

lib/src/backends/webgpu/webgpu_errors.dart

@@ -103,3 +103,52 @@ LlamaException? webGpuBridgeRequestError(
   }
   return null;
 }
+
+/// The bridge's `CrossOriginIsolationError` code, which the worker path keeps
+/// on the error it rebuilds in the page.
+const String _crossOriginIsolationRequiredCode =
+    'CROSS_ORIGIN_ISOLATION_REQUIRED';
+
+/// Returns [LlamaUnsupportedException] when [error] is the bridge's
+/// `CrossOriginIsolationError` (assets `v0.1.85+`), otherwise null.
+LlamaUnsupportedException? webGpuCrossOriginIsolationError(Object error) {
+  if (error is LlamaException) return null;
+  try {
+    final object = error as JSObject;
+    final code = object.getProperty<JSAny?>('code'.toJS);
+    final name = object.getProperty<JSAny?>('name'.toJS);
+    final isIsolationError =
+        (code.isA<JSString>() &&
+            (code as JSString).toDart == _crossOriginIsolationRequiredCode) ||
+        (name.isA<JSString>() &&
+            (name as JSString).toDart == 'CrossOriginIsolationError');
+    if (!isIsolationError) return null;
+  } catch (_) {
+    return null;
+  }
+  return LlamaUnsupportedException(
+    'The Web runtime needs cross-origin isolation (COOP/COEP) for shared '
+    'memory, which this page does not have. Serve the page with '
+    '"Cross-Origin-Opener-Policy: same-origin" and '
+    '"Cross-Origin-Embedder-Policy: require-corp". '
+    '"credentialless" does not isolate Safari.',
+  );
+}
+
+final RegExp _gpuUnavailableNote = RegExp(r'(?:^|;)gpu_unavailable:([^;]+)');
+
+/// Why the bridge ran without GPU layers, from its `gpu_unavailable:<reason>`
+/// runtime note (assets `v0.1.85+`), or an empty string.
+String webGpuUnavailableDetail(String runtimeNotes) {
+  final reason = _gpuUnavailableNote.firstMatch(runtimeNotes)?.group(1);
+  return switch (reason) {
+    null => '',
+    'no_webgpu' => ' The browser exposes no WebGPU.',
+    'no_adapter' => ' The browser granted no WebGPU adapter.',
+    'adapter_missing_shader_f16' =>
+      ' WebGPU needs the shader-f16 feature, which this adapter does not '
+          'expose. On Linux with an NVIDIA GPU, Chrome exposes it only with '
+          'driver 615.71 or newer.',
+    _ => ' The bridge reported gpu_unavailable:$reason.',
+  };
+}

lib/src/web_setup/web_bridge_setup.dart

@@ -15,14 +15,14 @@ import '../core/exceptions.dart';
 const String webBridgeAssetsRepository = 'leehack/llama-web-bridge-assets';
 
 /// The bridge asset release this llamadart version is qualified against.
-const String webBridgeAssetsTag = 'v0.1.84';
+const String webBridgeAssetsTag = 'v0.1.85';
 
 /// SHA-256 of the exact approved `manifest.json` of [webBridgeAssetsTag].
 ///
 /// The manifest lists the SHA-256 and size of every asset, so pinning its
 /// bytes pins every file the tool writes.
 const String webBridgeManifestSha256 =
-    '55abeb377597148156059a8ee5400f2314c6254ad91b57db65b709e304c0dea0';
+    '6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3';
 
 /// Directory under the app's `web/` directory that receives the assets.
 const String webBridgeAssetsDirName = 'webgpu_bridge';

scripts/fetch_webgpu_bridge_assets.sh

@@ -4,7 +4,7 @@ set -euo pipefail
 ROOT_DIR="$(git rev-parse --show-toplevel)"
 OUT_DIR="${WEBGPU_BRIDGE_OUT_DIR:-$ROOT_DIR/example/chat_app/web/webgpu_bridge}"
 ASSETS_REPO="${WEBGPU_BRIDGE_ASSETS_REPO:-leehack/llama-web-bridge-assets}"
-ASSETS_TAG="${WEBGPU_BRIDGE_ASSETS_TAG:-v0.1.84}"
+ASSETS_TAG="${WEBGPU_BRIDGE_ASSETS_TAG:-v0.1.85}"
 CDN_BASE="${WEBGPU_BRIDGE_CDN_BASE:-https://cdn.jsdelivr.net/gh/${ASSETS_REPO}@${ASSETS_TAG}}"
 CURL_DOWNLOAD_ARGS=(
   --connect-timeout 15
@@ -19,7 +19,7 @@ if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
 Downloads prebuilt WebGPU bridge assets into the chat_app web directory.
 
 Default source:
-  https://cdn.jsdelivr.net/gh/leehack/[email protected].84
+  https://cdn.jsdelivr.net/gh/leehack/[email protected].85
 
 Environment variables:
   WEBGPU_BRIDGE_ASSETS_REPO   Asset repo in owner/repo format
@@ -31,7 +31,7 @@ Usage:
   ./scripts/fetch_webgpu_bridge_assets.sh
 
 Examples:
-  WEBGPU_BRIDGE_ASSETS_TAG=v0.1.84 ./scripts/fetch_webgpu_bridge_assets.sh
+  WEBGPU_BRIDGE_ASSETS_TAG=v0.1.85 ./scripts/fetch_webgpu_bridge_assets.sh
   WEBGPU_BRIDGE_ASSETS_REPO=acme/llama-web-bridge-assets WEBGPU_BRIDGE_ASSETS_TAG=v2 ./scripts/fetch_webgpu_bridge_assets.sh
 USAGE
   exit 0

skills/llamadart-web/SKILL.md

@@ -54,16 +54,18 @@ in the llamadart-getting-started and llamadart-chat-streaming skills.
 - Serve from a secure context (`https://`, `http://localhost` or
   `http://127.0.0.1`) and send `Cross-Origin-Opener-Policy: same-origin` with
   `Cross-Origin-Embedder-Policy: require-corp` (`credentialless` isolates only
-  in Chromium-based browsers). Without
-  isolation (`window.crossOriginIsolated == false`) the bridge caps inference
-  at one thread (`threads_capped_no_coi` in its notes) and large single-file
-  GGUF loads become unreliable.
+  in Chromium-based browsers; it does not isolate Safari). Without
+  isolation (`window.crossOriginIsolated == false`) assets `v0.1.85+`
+  reject the load at once and llamadart throws `LlamaUnsupportedException`
+  naming COOP/COEP; earlier assets wait minutes for the init timeout before a generic error.
 - Treat WebGPU as a runtime capability. A supported browser (Chrome/Edge 128+,
   Firefox 129+, Safari 17.4+) can still lack an adapter, features or memory.
   Prove a first load with a small quantized GGUF, `contextSize: 2048` or less
   and `gpuLayers: 0`, then raise GPU offload.
 - `device: ComputeDevice.gpu` requires a WebGPU adapter and active GPU layers
-  after load. The bridge ignores negative `mainGpu` values, including with
+  after load; with assets `v0.1.85+` the error names the bridge's
+  `gpu_unavailable:*` reason (for example `adapter_missing_shader_f16`: Chrome
+  on Linux with NVIDIA needs driver 615.71 or newer). The bridge ignores negative `mainGpu` values, including with
   `splitMode: ModelSplitMode.none`; use `device: ComputeDevice.cpu` to select
   WebAssembly CPU execution.
 - Expect automatic fallback: a failed GPU load retries on CPU, then at smaller