Fail fast without cross-origin isolation; keep WebGPU out of 0-layer loads

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ## Unreleased
 
+- A load on a page that is not cross-origin isolated rejects at once with `CrossOriginIsolationError`, which names `Cross-Origin-Embedder-Policy: require-corp` and that `credentialless` does not isolate Safari, instead of hanging until the core init timeout. The `threads_capped_no_coi` note now appears only for a core loaded with `allowSingleThreadCore`.
+- A load with `nGpuLayers: 0` keeps WebGPU out of the model and context backends, so CPU decoding on Firefox no longer waits on its 100 ms WebGPU queue ticks at every sync (about 50x faster on a small model). `splitMode: 3` at 0 layers still loads.
+- A load with `nGpuLayers > 0` that ends up on the CPU adds the runtime note `gpu_unavailable:<reason>` and one warning, such as `adapter_missing_shader_f16` for Chrome on NVIDIA drivers older than 615.71.
+- `allowSingleThreadCore: true` lets a `coreModuleUrl` core built without pthreads load on a page that is not cross-origin isolated, capped to one thread.
 - `CompletionOptions.thinkingBudget.pauseGrammar: false` keeps a grammar applied inside a reasoning block, so a grammar that takes the block and its end tag itself, such as one for a prompt that ends inside the block, sees the budget's forced `</think>`. It needs the new `thinkingBudgetPauseGrammar` capability; omitted, the grammar pauses as before.
 - WebGPU text-to-speech applies the GGUF's suppress tokens in the talker sampler, as upstream does.
   Qwen3-TTS no longer fails with `code out of range` when a codec control token is sampled.

docs/api.md

@@ -154,9 +154,10 @@ it.
 | `coreModuleUrlMem64` | `string` | URL for the optional wasm64 Emscripten JS loader. |
 | `wasmUrl` | `string` | URL for the wasm32 `llama_webgpu_core.wasm` binary. |
 | `wasmUrlMem64` | `string` | URL for the optional wasm64 wasm binary. |
-| `coreModuleFactory` | `function \| Promise<function>` | Preloaded Emscripten factory. Supplying this disables worker mode because factories cannot be transferred to the worker. |
+| `coreModuleFactory` | `LlamaCoreModuleFactory \| Promise<LlamaCoreModuleFactory>` | Preloaded Emscripten factory. Supplying this disables worker mode because factories cannot be transferred to the worker. It also skips the cross-origin isolation check, so a custom pthread core on a page that is not isolated still waits for `coreInitTimeoutMs`. |
 | `workerUrl` | `string` | URL for `llama_webgpu_bridge_worker.js`. Defaults to a sibling of the bridge module when possible. |
 | `disableWorker` | `boolean` | Force direct runtime mode. |
+| `allowSingleThreadCore` | `boolean` | Set for a `coreModuleUrl` core built without pthreads (`WEBGPU_BRIDGE_PTHREADS=0`). It then loads on a page that is not cross-origin isolated, capped to one thread (`threads_capped_no_coi`), instead of rejecting with `CrossOriginIsolationError`. |
 | `preferMemory64` | `boolean` | Prefer the wasm64 core when available. Defaults to `true`; the runtime can still fall back to wasm32. |
 | `workerRequestTimeoutMs` | `number` | Default timeout for worker RPC calls, 2 minutes unless a method has its own default. It also sets the `embed`/`embedBatch` budget, which defaults to 10 minutes when an input has image or audio parts. |
 | `workerInitTimeoutMs` | `number` | Timeout for worker initialization. |
@@ -167,7 +168,7 @@ it.
 | `workerGenerationStallTimeoutMs` | `number` | Stall timeout for worker generation after no token events arrive; clamped by the bridge. |
 | `coreInitTimeoutMs` | `number` | Timeout while initializing the Emscripten core module. |
 | `cacheName` | `string` | Cache Storage name used by model prefetch/load helpers. |
-| `threads` | `number` | Requested llama.cpp thread count. The bridge caps this to the compiled pthread pool and runtime isolation support. |
+| `threads` | `number` | Requested llama.cpp thread count. The bridge caps this to the compiled pthread pool, and to one thread for a single-thread core on a page that is not isolated. |
 | `threadsBatch` | `number` | Requested batch thread count; defaults to `threads`. |
 | `threadPoolSize` | `number` | Hint for the compiled pthread pool size, used before the core can report it. |
 | `nBatch` | `number` | llama.cpp batch size override. |
@@ -222,7 +223,7 @@ preserves the model's resolved native load settings, including sequence count,
 batch sizes, memory mapping/locking, KV cache types, flash attention, RoPE and
 device split settings. It changes GPU placement to CPU and retains the existing
 memory32 preference and transfer safeguards for recovery. Thread counts remain
-subject to the replacement core's pool and browser isolation limits. Recovery
+subject to the replacement core's pool. Recovery
 continues to refuse active LoRA adapters and speculative generation.
 
 ### `loadModelFromUrl(url, options?)`
@@ -1153,7 +1154,7 @@ include:
 | `llamadart.webgpu.core_variant` | `wasm32`, `wasm64`, or initialization state. |
 | `llamadart.webgpu.model_source` | Model load source, such as network/cache/remote fetch. |
 | `llamadart.webgpu.model_cache_state` | Cache Storage state for the loaded model. |
-| `llamadart.webgpu.runtime_notes` | Semicolon-separated bridge notes such as thread caps or fallback reasons. |
+| `llamadart.webgpu.runtime_notes` | Semicolon-separated bridge notes such as thread caps or fallback reasons. `gpu_unavailable:<reason>` (`no_webgpu`, `no_adapter`, `adapter_missing_shader_f16`, `unknown`) says why a load with `nGpuLayers > 0` runs on the CPU; the bridge also logs one warning. Chrome hides `shader-f16` on NVIDIA drivers older than 615.71, and ggml-webgpu then registers no device. |
 | `llamadart.webgpu.mmproj_loaded` | `1` when a projector is loaded. |
 | `llamadart.webgpu.supports_vision` | `1` when the active projector supports vision. |
 | `llamadart.webgpu.supports_audio` | `1` when the active projector supports audio. |
@@ -1242,7 +1243,10 @@ this function themselves.
 - Large single-file model loads and pthread-backed runtime paths require
   cross-origin isolation (`Cross-Origin-Opener-Policy: same-origin` and
   `Cross-Origin-Embedder-Policy: require-corp`) so `SharedArrayBuffer` is
-  available.
+  available. `credentialless` does not isolate Safari. On a page that cannot
+  share memory with workers, a load rejects immediately with
+  `CrossOriginIsolationError` (`code: 'CROSS_ORIGIN_ISOLATION_REQUIRED'`)
+  instead of waiting for the core init timeout.
 - Serve `llama_webgpu_core.js` and `llama_webgpu_core.wasm` from URLs reachable
   by both the page and worker. If using wasm64, also serve the `_mem64` files.
 - Keep `llama_webgpu_bridge.js`, `llama_webgpu_bridge_worker.js`, the core JS,

js/llama_webgpu_bridge.d.ts

@@ -7,7 +7,10 @@ export interface LlamaWebGpuBridgeConfig {
   coreModuleUrlMem64?: string;
   wasmUrl?: string;
   wasmUrlMem64?: string;
+  /** A custom factory skips the cross-origin isolation check, so a custom pthread core on a page that is not isolated still waits for `coreInitTimeoutMs`. */
   coreModuleFactory?: LlamaCoreModuleFactory | Promise<LlamaCoreModuleFactory>;
+  /** Set for a `coreModuleUrl` core built without pthreads (`WEBGPU_BRIDGE_PTHREADS=0`): it loads on a page that is not cross-origin isolated, capped to one thread, instead of rejecting with `CrossOriginIsolationError`. */
+  allowSingleThreadCore?: boolean;
   workerUrl?: string;
   disableWorker?: boolean;
   preferMemory64?: boolean;
@@ -298,6 +301,13 @@ export class EmbeddingError extends Error {
   constructor(code: EmbeddingErrorCode, message: string);
 }
 
+export type CrossOriginIsolationErrorCode = 'CROSS_ORIGIN_ISOLATION_REQUIRED';
+/** Thrown by a load or core init on a page that cannot share memory with workers, before the core starts. */
+export class CrossOriginIsolationError extends Error {
+  code: CrossOriginIsolationErrorCode;
+  constructor(message?: string);
+}
+
 export interface NextTokenScoreOptions {
   /** Token ids to score, in the order the result lists them. */
   candidates?: ArrayLike<number>;

js/src/bridge.ts

@@ -1460,6 +1460,7 @@ export class LlamaWebGpuBridge implements PublicLlamaWebGpuBridge {
       preferMemory64: typeof config.preferMemory64 === 'boolean'
         ? config.preferMemory64
         : undefined,
+      allowSingleThreadCore: config.allowSingleThreadCore === true ? true : undefined,
       threads: Number(config.threads) > 0 ? Number(config.threads) : undefined,
       threadsBatch: Number(config.threadsBatch) > 0
         ? Number(config.threadsBatch)

js/src/internal/environment.ts

@@ -10,15 +10,55 @@ export function isSafariUserAgent(userAgent: unknown): boolean {
   return hasSafariToken && !hasOtherBrowserToken;
 }
 
-export function isCrossOriginIsolatedRuntime(): boolean {
+// A pthread core preallocates workers and posts its shared memory to them.
+// Browsers refuse that post (DataCloneError) unless the page is cross-origin
+// isolated, and core init then never settles. Probing the post itself, rather
+// than `crossOriginIsolated`, also accepts pages that enable shared memory
+// another way, such as Chrome's SharedArrayBufferUnrestrictedAccessAllowed.
+export function canPostSharedMemory(): boolean {
+  if (typeof MessageChannel !== 'function' || typeof WebAssembly === 'undefined') {
+    return true;
+  }
+
+  let channel: MessageChannel | null = null;
+  try {
+    channel = new MessageChannel();
+    channel.port1.postMessage(new WebAssembly.Memory({ initial: 1, maximum: 1, shared: true }));
+    return true;
+  } catch (_) {
+    return false;
+  } finally {
+    channel?.port1.close();
+    channel?.port2.close();
+  }
+}
+
+export type WebGpuUnavailable = {
+  reason: 'no_webgpu' | 'no_adapter' | 'adapter_missing_shader_f16' | 'unknown';
+  adapter: string;
+};
+
+// ggml-webgpu registers its device only for an adapter with shader-f16 and
+// logs nothing otherwise, so the page-side adapter is the only place that
+// can say why a GPU load runs on the CPU.
+export async function describeWebGpuUnavailable(): Promise<WebGpuUnavailable> {
+  const gpu = (globalThis.navigator as { gpu?: { requestAdapter?: () => Promise<any> } } | undefined)?.gpu;
+  if (!gpu || typeof gpu.requestAdapter !== 'function') {
+    return { reason: 'no_webgpu', adapter: '' };
+  }
+
+  let adapter = null;
   try {
-    if (typeof globalThis.crossOriginIsolated === 'boolean') {
-      return globalThis.crossOriginIsolated;
-    }
+    adapter = await gpu.requestAdapter();
   } catch (_) {
-    // ignore environment probing failures
+    adapter = null;
+  }
+  if (!adapter) {
+    return { reason: 'no_adapter', adapter: '' };
   }
 
-  // Assume isolated in runtimes that do not expose the signal.
-  return true;
+  return {
+    reason: adapter.features?.has?.('shader-f16') ? 'unknown' : 'adapter_missing_shader_f16',
+    adapter: [adapter.info?.vendor, adapter.info?.architecture].filter(Boolean).join('/'),
+  };
 }

js/src/internal/isolation.ts

@@ -0,0 +1,16 @@
+// Raised before core init when the page cannot share memory with workers.
+export const CROSS_ORIGIN_ISOLATION_REQUIRED = 'CROSS_ORIGIN_ISOLATION_REQUIRED';
+
+export class CrossOriginIsolationError extends Error {
+  declare code: typeof CROSS_ORIGIN_ISOLATION_REQUIRED;
+  constructor(message: string = CROSS_ORIGIN_ISOLATION_MESSAGE) {
+    super(message);
+    this.name = 'CrossOriginIsolationError';
+    this.code = CROSS_ORIGIN_ISOLATION_REQUIRED;
+  }
+}
+
+export const CROSS_ORIGIN_ISOLATION_MESSAGE =
+  'The llama.cpp web core needs shared memory, which this page cannot use because it is not '
+  + 'cross-origin isolated. Serve the page with "Cross-Origin-Opener-Policy: same-origin" and '
+  + '"Cross-Origin-Embedder-Policy: require-corp". "credentialless" does not isolate Safari.';

js/src/llama_webgpu_bridge.d.ts

@@ -5,7 +5,10 @@ export interface LlamaWebGpuBridgeConfig {
   coreModuleUrlMem64?: string;
   wasmUrl?: string;
   wasmUrlMem64?: string;
+  /** A custom factory skips the cross-origin isolation check, so a custom pthread core on a page that is not isolated still waits for `coreInitTimeoutMs`. */
   coreModuleFactory?: LlamaCoreModuleFactory | Promise<LlamaCoreModuleFactory>;
+  /** Set for a `coreModuleUrl` core built without pthreads (`WEBGPU_BRIDGE_PTHREADS=0`): it loads on a page that is not cross-origin isolated, capped to one thread, instead of rejecting with `CrossOriginIsolationError`. */
+  allowSingleThreadCore?: boolean;
   workerUrl?: string;
   disableWorker?: boolean;
   preferMemory64?: boolean;
@@ -296,6 +299,13 @@ export class EmbeddingError extends Error {
   constructor(code: EmbeddingErrorCode, message: string);
 }
 
+export type CrossOriginIsolationErrorCode = 'CROSS_ORIGIN_ISOLATION_REQUIRED';
+/** Thrown by a load or core init on a page that cannot share memory with workers, before the core starts. */
+export class CrossOriginIsolationError extends Error {
+  code: CrossOriginIsolationErrorCode;
+  constructor(message?: string);
+}
+
 export interface NextTokenScoreOptions {
   /** Token ids to score, in the order the result lists them. */
   candidates?: ArrayLike<number>;

js/src/llama_webgpu_bridge.js

@@ -21,3 +21,4 @@ if (typeof window !== 'undefined') {
 
 export { LlamaWebGpuBridge, enableBridgeWorkerHost };
 export { EmbeddingError } from './internal/embedding.ts';
+export { CrossOriginIsolationError } from './internal/isolation.ts';