Link the WebGPU sampler into the Android runtime

README.md

@@ -326,7 +326,9 @@ v0.18 this includes the iOS `LiteRtMetalAccelerator` and
 `LiteRtTopKMetalSampler` modules; their framework-relative loader paths avoid
 flat dylibs that App Store bundles cannot ship. v0.19 `LiteRtLm` links its GPU
 accelerator and sampler, so neither is needed; releases after `v0.19.1-2` ship
-no Metal module. Releases after `v0.19.1-3` also drop the macOS `libLiteRt` and
+no Metal module. Android v0.19 runtimes after `v0.19.1-4` link both the OpenCL
+and WebGPU samplers, because the linked Android GPU accelerator can serve either
+backend. Releases after `v0.19.1-3` also drop the macOS `libLiteRt` and
 `libLiteRtWebGpuAccelerator` dylibs, the iOS loose `libLiteRt` and the matching
 `LiteRt` and `LiteRtWebGpuAccelerator` XCFramework zips: v0.19 `LiteRtLm`
 embeds LiteRT and never opens them.

docs/android_gpu_qualification.md

@@ -55,6 +55,18 @@ immutable wrapper version, then qualify its exact packaged artifact before
 updating downstream pins. The local single-library candidate does not qualify
 a future release or authorize consumer PR #503 to merge unchanged.
 
+## WebGPU sampler (v0.19.0 through v0.19.1-4)
+
+Android runtimes `v0.19.0` through `v0.19.1-4` define the OpenCL sampler
+functions but not the WebGPU ones. On a device or app without loadable OpenCL,
+the linked accelerator selects WebGPU, the engine loads, and the first
+`SampleLogits` returns `Can not find OpenCL library on this device` because the
+OpenCL sampler is the only one linked and does not report itself unavailable
+(Galaxy S24, Gemma 4 E2B,
+[litert-lm-native#120](https://github.com/leehack/litert-lm-native/issues/120)).
+Later runtimes link both samplers and the build rejects a candidate missing
+either. Device result on the fixed runtime: pending.
+
 ## GPU environment teardown patch
 
 LiteRT `9fe5be45`, pinned by LiteRT-LM 0.17.x, loads OpenCL before it records

docs/v019_qualification.md

@@ -33,6 +33,7 @@ The exact-tag verification gate remains enabled.
 | ASR | `omni/asr` and the legacy engine are gone; `omni/speech2text` factories route through `OmniRegistry` and require a container | C ASR ABI v1 is unchanged. Containers use the registry; a `.tflite` plus tokenizer uses a bridge factory (`litert_lm_asr_omni_engine.cc`). Container metadata and Parakeet sections pick the registry name, so file names, free-form `model_name`s and `decoder_type` do not change routing, and a `model_name` naming another family fails at engine creation. Other LLM containers lost the generic LLM route and need a registered name. Start tokens are dropped for LM-backed families, whose decoders reject them. |
 | Protos | `Asr*` became `Speech2Text*` | Internal to the adapter. |
 | Packaging | No `LiteRtTopK*Sampler` library; `libLiteRtLm` links Dawn | Validators and manifests key this on `has_prebuilt_topk_samplers`. |
+| Android GPU samplers | `libLiteRtLm` links `ml_drift_gpu_accelerator`, which serves OpenCL and WebGPU, but upstream links only the OpenCL sampler unless built with `--copt=-DLITERT_USE_WEBGPU_ACCELERATOR`. Without OpenCL the WebGPU route loads, then fails at the first token with `Can not find OpenCL library on this device` ([litert-lm-native#120](https://github.com/leehack/litert-lm-native/issues/120), [llamadart#1998](https://github.com/leehack/llamadart/issues/1998)) | The bridge links `//runtime/components:top_k_webgpu_sampler_static` on Android (`--define=litert_lm_android_webgpu_sampler=true`). Build and package validators compare exact dynamic exports and reject a candidate missing either sampler set (`android_static_gpu_sampler_symbols`). Re-audit on the next upstream bump: upstream `main` already splits the Android accelerator by `gpu_accelerator`. |
 | Apple GPU | `LiteRtLm` links its GPU accelerator: ml_drift Metal on iOS, WebGPU on macOS | LiteRT registers it before searching for a shared one, so the prebuilt Metal accelerator is never opened ([llamadart#1798](https://github.com/leehack/llamadart/issues/1798)). Packaging after `v0.19.1-2` skips it on iOS and macOS, requires the static accelerator symbol in `LiteRtLm`, and rejects a candidate that still ships it (`has_prebuilt_apple_metal_accelerator`). |
 | Apple LiteRT companions | `LiteRtLm` embeds LiteRT and its GPU accelerator | dyld traces on macOS (CPU, GPU, vision, audio, MTP, embeddings, ASR) and an iOS simulator probe load neither `libLiteRt` nor `libLiteRtWebGpuAccelerator` ([llamadart#1911](https://github.com/leehack/llamadart/issues/1911)). Packaging after `v0.19.1-3` skips the macOS `libLiteRt` and `libLiteRtWebGpuAccelerator` and the iOS loose `libLiteRt`, and rejects a candidate that still ships them (`has_prebuilt_apple_litert_companions`). |
 

native/bridge/BUILD.bazel

@@ -89,6 +89,14 @@ config_setting(
     define_values = {"litert_lm_c_api_v1": "true"},
 )
 
+# Upstream links only the OpenCL TopK sampler on Android, but its GPU
+# accelerator runs on WebGPU when OpenCL cannot be loaded.
+config_setting(
+    name = "android_webgpu_sampler",
+    constraint_values = ["@platforms//os:android"],
+    define_values = {"litert_lm_android_webgpu_sampler": "true"},
+)
+
 # v0.17 moved the capabilities C implementation into //c:engine.
 config_setting(
     name = "capabilities_in_c_engine",
@@ -278,7 +286,10 @@ cc_binary(
         ":litert_lm_bridge",
         ":litert_lm_asr_bridge",
         "//c:engine",
-    ] + CAPABILITIES_DEPS,
+    ] + CAPABILITIES_DEPS + select({
+        ":android_webgpu_sampler": ["//runtime/components:top_k_webgpu_sampler_static"],
+        "//conditions:default": [],
+    }),
 )
 
 cc_binary(

tools/build_upstream_runtime.py

@@ -26,8 +26,10 @@
     verify_dynamic_runtime,
 )
 from litert_lm_symbols import (
+    android_static_gpu_sampler_symbols,
     has_asr_bridge,
     has_omni_asr_engine,
+    has_prebuilt_topk_samplers,
     in_process_litert_abi_version,
     is_at_least,
     uses_c_api_v1,
@@ -39,6 +41,7 @@
 from macos_runtime_policy import UNSUPPORTED_MACOS_ARCH_REASON
 from package_upstream_prebuilts import apply_prebuilt_overrides
 from runtime_dependency_utils import (
+    elf_exported_symbols,
     elf_has_global_flag,
     elf_needed_libraries,
     is_elf,
@@ -585,6 +588,8 @@ def build_runtime(
         command.append("--define=litert_lm_c_api_v1=true")
     if in_process_litert_abi_version(upstream_tag) == 1:
         command.append("--define=litert_lm_in_process_litert=true")
+    if platform == "android" and not has_prebuilt_topk_samplers(upstream_tag):
+        command.append("--define=litert_lm_android_webgpu_sampler=true")
     if require_finish_reason_patch_decision(upstream_tag):
         command.append(FINISH_REASON_DEFINE)
     if platform == "macos":
@@ -899,6 +904,19 @@ def validate_android_global_visibility(output: Path, platform: str) -> None:
     print(f"Validated Android global symbol visibility in {output}", flush=True)
 
 
+def validate_android_gpu_samplers(output: Path, platform: str, upstream_tag: str) -> None:
+    if platform != "android":
+        return
+    missing = sorted(
+        set(android_static_gpu_sampler_symbols(upstream_tag)) - elf_exported_symbols(output))
+    if missing:
+        raise RuntimeError(
+            f"{output} does not define the GPU sampler functions "
+            f"{', '.join(missing)}; generation fails on the GPU backend they "
+            "serve (#120)")
+    print(f"Validated linked OpenCL and WebGPU samplers in {output}", flush=True)
+
+
 def validate_android_gpu_environment_teardown(output: Path, platform: str) -> None:
     if platform != "android":
         return
@@ -911,6 +929,7 @@ def validate_runtime(output: Path, platform: str, upstream_tag: str) -> None:
     validate_in_process_litert_linkage(output, platform, upstream_tag)
     validate_android_global_visibility(output, platform)
     validate_android_protobuf_isolation(output, platform, upstream_tag)
+    validate_android_gpu_samplers(output, platform, upstream_tag)
     validate_android_gpu_environment_teardown(output, platform)
 
 

tools/litert_lm_symbols.py

@@ -81,6 +81,16 @@
     "LiteRtTopKOpenClSampler_SetInferenceFuncAndInputTensors",
 ]
 
+ANDROID_WEBGPU_SAMPLER_SYMBOLS = [
+    "LiteRtTopKWebGpuSampler_Create",
+    "LiteRtTopKWebGpuSampler_Destroy",
+    "LiteRtTopKWebGpuSampler_SampleToIdAndScoreBuffer",
+    "LiteRtTopKWebGpuSampler_UpdateConfig",
+    "LiteRtTopKWebGpuSampler_CanHandleInput",
+    "LiteRtTopKWebGpuSampler_HandlesInput",
+    "LiteRtTopKWebGpuSampler_SetInferenceFuncAndInputTensors",
+]
+
 APPLE_METAL_SAMPLER_SYMBOLS = [
     "LiteRtTopKMetalSampler_Create",
     "LiteRtTopKMetalSampler_Destroy",
@@ -158,6 +168,17 @@ def has_prebuilt_topk_samplers(upstream_tag: str) -> bool:
     return not is_at_least(upstream_tag, (0, 19, 0))
 
 
+def android_static_gpu_sampler_symbols(upstream_tag: str) -> list[str]:
+    """Sampler functions an Android libLiteRtLm.so must define itself.
+
+    The v0.19 Android GPU accelerator runs on OpenCL, or on WebGPU when OpenCL
+    cannot be loaded, and the sampler is created for whichever it chose (#120).
+    """
+    if has_prebuilt_topk_samplers(upstream_tag):
+        return []
+    return ANDROID_OPENCL_SAMPLER_SYMBOLS + ANDROID_WEBGPU_SAMPLER_SYMBOLS
+
+
 def has_prebuilt_apple_metal_accelerator(upstream_tag: str) -> bool:
     """v0.19 Apple runtimes link their GPU accelerator and never open the prebuilt one."""
     return not is_at_least(upstream_tag, (0, 19, 0))

tools/test_android_protobuf_policy.py

@@ -12,7 +12,7 @@
 
 import android_protobuf_policy as policy
 from litert_lm_symbols import (
-    ANDROID_OPENCL_SAMPLER_SYMBOLS,
+    android_static_gpu_sampler_symbols,
     required_bridge_symbols,
     required_c_api_symbols,
 )
@@ -151,7 +151,7 @@ def setUpClass(cls):
     def compile(self, directory, arch, *, private=True, global_flag=True, missing=None, alignment=16384, dependency=None):
         directory.mkdir(parents=True, exist_ok=True)
         required = {symbol.decode() for symbol in required_c_api_symbols('v0.19.0') + required_bridge_symbols('v0.19.0')}
-        public = required | set(ANDROID_OPENCL_SAMPLER_SYMBOLS) | {'LiteRtAcceleratorImpl'}
+        public = required | set(android_static_gpu_sampler_symbols('v0.19.0')) | {'LiteRtAcceleratorImpl'}
         self.assertFalse(any('protobuf' in symbol for symbol in public))
         source = directory / 'fixture.cc'
         source.write_text('''namespace google { namespace protobuf { namespace internal {

tools/test_build_upstream_runtime.py

@@ -11,6 +11,8 @@
 
 import build_upstream_runtime
 import package_upstream_prebuilts
+from litert_lm_symbols import ANDROID_OPENCL_SAMPLER_SYMBOLS, ANDROID_WEBGPU_SAMPLER_SYMBOLS
+from test_runtime_dependency_utils import write_elf_with_exports
 
 
 class BuildUpstreamRuntimeTest(unittest.TestCase):
@@ -161,6 +163,43 @@ def test_desktop_build_selects_shared_runtime_only_for_v016_to_v018(self) -> Non
                     self.assertEqual("--define=litert_lm_in_process_litert=true" in command,
                                      tag in {"v0.19.0", "v0.19.1"})
                     self.assertIn("--define=resolve_symbols_in_exec=false", command)
+                    self.assertEqual("--define=litert_lm_android_webgpu_sampler=true" in command,
+                                     platform == "android" and tag in {"v0.19.0", "v0.19.1"})
+
+    def test_android_webgpu_sampler_define_links_the_static_sampler(self) -> None:
+        bazel = (build_upstream_runtime.REPO_ROOT / "native/bridge/BUILD.bazel").read_text()
+        setting = re.search(
+            r'name = "android_webgpu_sampler",\s*constraint_values = \["@platforms//os:android"\],'
+            r'\s*define_values = \{"([^"]+)": "true"\}', bazel)
+        self.assertIsNotNone(setting)
+        script = Path(build_upstream_runtime.__file__).read_text()
+        self.assertIn(f'"--define={setting.group(1)}=true"', script)
+        self.assertEqual(len(re.findall(
+            r'":android_webgpu_sampler": \["//runtime/components:top_k_webgpu_sampler_static"\]',
+            bazel)), 1)
+
+    def test_android_v019_build_rejects_a_runtime_without_both_gpu_samplers(self) -> None:
+        both = ANDROID_OPENCL_SAMPLER_SYMBOLS + ANDROID_WEBGPU_SAMPLER_SYMBOLS
+        published = ANDROID_OPENCL_SAMPLER_SYMBOLS + [
+            f"{symbol}_Static" for symbol in ANDROID_WEBGPU_SAMPLER_SYMBOLS]
+        with tempfile.TemporaryDirectory() as temp:
+            output = Path(temp) / "libLiteRtLm.so"
+            write_elf_with_exports(output, published)
+            with self.assertRaisesRegex(RuntimeError, r"LiteRtTopKWebGpuSampler_Create,.*#120"):
+                build_upstream_runtime.validate_android_gpu_samplers(output, "android", "v0.19.1")
+            build_upstream_runtime.validate_android_gpu_samplers(output, "android", "v0.18.1")
+            build_upstream_runtime.validate_android_gpu_samplers(output, "linux", "v0.19.1")
+            write_elf_with_exports(output, both)
+            build_upstream_runtime.validate_android_gpu_samplers(output, "android", "v0.19.1")
+            with patch.object(build_upstream_runtime, "validate_exported_symbols"), \
+                 patch.object(build_upstream_runtime, "validate_in_process_litert_linkage"), \
+                 patch.object(build_upstream_runtime, "validate_android_global_visibility"), \
+                 patch.object(build_upstream_runtime, "validate_android_protobuf_isolation"), \
+                 patch.object(build_upstream_runtime, "validate_android_gpu_environment_teardown"):
+                build_upstream_runtime.validate_runtime(output, "android", "v0.19.1")
+                write_elf_with_exports(output, published)
+                with self.assertRaisesRegex(RuntimeError, "#120"):
+                    build_upstream_runtime.validate_runtime(output, "android", "v0.19.1")
 
     def test_in_process_litert_define_matches_bazel_config_setting(self) -> None:
         bazel = (build_upstream_runtime.REPO_ROOT / "native/bridge/BUILD.bazel").read_text()

tools/test_runtime_dependency_utils.py

@@ -10,7 +10,9 @@
 import runtime_dependency_utils
 from litert_lm_symbols import (
     ANDROID_OPENCL_SAMPLER_SYMBOLS,
+    ANDROID_WEBGPU_SAMPLER_SYMBOLS,
     APPLE_METAL_SAMPLER_SYMBOLS,
+    android_static_gpu_sampler_symbols,
 )
 from validate_runtime_dependencies import (
     newest_glibc_requirement,
@@ -225,6 +227,45 @@ def test_android_opencl_sampler_accepts_complete_plugin_api(self) -> None:
 
             self.assertEqual(validate_elf_dependencies(root, "v0.18.0"), 1)
 
+    def test_android_v019_runtime_requires_both_linked_gpu_samplers(self) -> None:
+        # sampler_factory.cc resolves the same seven functions for each family.
+        self.assertEqual(
+            ANDROID_WEBGPU_SAMPLER_SYMBOLS,
+            [symbol.replace("OpenCl", "WebGpu") for symbol in ANDROID_OPENCL_SAMPLER_SYMBOLS],
+        )
+        self.assertEqual(android_static_gpu_sampler_symbols("v0.18.1"), [])
+        both = ANDROID_OPENCL_SAMPLER_SYMBOLS + ANDROID_WEBGPU_SAMPLER_SYMBOLS
+        for tag in ("v0.19.0", "v0.19.1", "v0.20.0"):
+            self.assertEqual(android_static_gpu_sampler_symbols(tag), both)
+        # The published v0.19.1-4 shape: OpenCL functions plus the WebGPU
+        # function pointers sampler_factory.cc always defines.
+        published = ANDROID_OPENCL_SAMPLER_SYMBOLS + [
+            f"{symbol}_Static" for symbol in ANDROID_WEBGPU_SAMPLER_SYMBOLS
+        ]
+        for arch in ("arm64", "x64"):
+            with self.subTest(arch=arch), tempfile.TemporaryDirectory() as temp:
+                root = Path(temp)
+                library = root / "bin" / "android" / arch / "libLiteRtLm.so"
+                library.parent.mkdir(parents=True)
+                write_elf_with_exports(library, published)
+                with self.assertRaises(SystemExit) as raised:
+                    validate_elf_dependencies(root, "v0.19.1")
+                message = str(raised.exception)
+                for symbol in ANDROID_WEBGPU_SAMPLER_SYMBOLS:
+                    self.assertRegex(message, rf"{symbol}(,|$|\n)")
+                self.assertNotIn("LiteRtTopKOpenClSampler", message)
+                write_elf_with_exports(library, both)
+                with self.assertRaises(SystemExit) as raised:
+                    validate_elf_dependencies(root, "v0.19.1")
+                self.assertNotIn("Sampler", str(raised.exception))
+                write_elf_with_exports(library, ANDROID_WEBGPU_SAMPLER_SYMBOLS)
+                with self.assertRaisesRegex(SystemExit, "LiteRtTopKOpenClSampler_Create"):
+                    validate_elf_dependencies(root, "v0.19.1")
+                write_elf_with_exports(library, published)
+                with self.assertRaises(SystemExit) as raised:
+                    validate_elf_dependencies(root, "v0.18.1")
+                self.assertNotIn("Sampler", str(raised.exception))
+
     def test_ios_metal_sampler_requires_complete_plugin_api(self) -> None:
         with tempfile.TemporaryDirectory() as temp:
             root = Path(temp)

tools/validate_runtime_dependencies.py

@@ -20,6 +20,7 @@
 )
 from litert_lm_symbols import (
     ANDROID_OPENCL_SAMPLER_SYMBOLS,
+    android_static_gpu_sampler_symbols,
     APPLE_METAL_ACCELERATOR_SYMBOLS,
     APPLE_METAL_SAMPLER_SYMBOLS,
 )
@@ -118,6 +119,15 @@ def validate_elf_dependencies(root: Path, upstream_tag: str) -> int:
                     validate_android_protobuf_isolation(library, platform, upstream_tag)
                 except RuntimeError as error:
                     errors.append(str(error))
+                missing = sorted(
+                    set(android_static_gpu_sampler_symbols(upstream_tag))
+                    - elf_exported_symbols(library)
+                )
+                if missing:
+                    errors.append(
+                        f"{library.relative_to(root).as_posix()} does not "
+                        "define the GPU sampler functions: " + ", ".join(missing)
+                    )
             if library.name == "libLiteRtTopKOpenClSampler.so":
                 exported = elf_exported_symbols(library)
                 missing = sorted(set(ANDROID_OPENCL_SAMPLER_SYMBOLS) - exported)