Stop packaging the unused Apple Metal accelerator from v0.19

README.md

@@ -321,10 +321,12 @@ XCFramework contains the primary iOS runtime and macOS framework wrapper.
 `CLiteRTLMMac` is retained as a macOS compatibility re-export target. Upstream
 `v0.14.0` uses the official consolidated Apple XCFrameworks and does not require
 a separate iOS `GemmaModelConstraintProvider` target. Source-built Apple
-releases also publish any required companion XCFrameworks. For v0.16 this
-includes the iOS `LiteRtMetalAccelerator` and `LiteRtTopKMetalSampler` modules;
-their framework-relative loader paths avoid flat dylibs that App Store bundles
-cannot ship.
+releases also publish any required companion XCFrameworks. For v0.16 through
+v0.18 this includes the iOS `LiteRtMetalAccelerator` and
+`LiteRtTopKMetalSampler` modules; their framework-relative loader paths avoid
+flat dylibs that App Store bundles cannot ship. v0.19 `LiteRtLm` links its GPU
+accelerator and sampler, so neither is needed; releases after `v0.19.1-2` ship
+no Metal module.
 
 The macOS packager embeds the primary framework's private dylib dependencies
 inside `LiteRtLm.framework/Versions/A/Dependencies` and rewrites their loader

docs/apple_privacy_manifest.md

@@ -36,6 +36,9 @@ No framework tracks, contacts tracking domains, or collects data.
 | `LiteRtTopKMetalSampler` | iOS | User Defaults | `CA92.1` |
 | `LiteRtTopKMetalSampler` | macOS | none | |
 
+`LiteRtTopKMetalSampler` ships only before v0.19, and `LiteRtMetalAccelerator`
+only up to `v0.19.1-2`.
+
 Reason texts are in Apple's
 [NSPrivacyAccessedAPITypeReasons](https://developer.apple.com/documentation/bundleresources/app-privacy-configuration/nsprivacyaccessedapitypes/nsprivacyaccessedapitypereasons)
 reference.

docs/platform_strategy.md

@@ -88,7 +88,9 @@ support. Source-built Apple releases can publish additional companion binary
 targets when the primary runtime links or dynamically loads them. The v0.16 iOS
 package includes `GemmaModelConstraintProvider`, `LiteRtMetalAccelerator`, and
 `LiteRtTopKMetalSampler`; the Metal modules use framework-relative loader paths
-that are compatible with App Store bundle layout.
+that are compatible with App Store bundle layout. v0.19 `LiteRtLm` links its
+GPU accelerator and sampler on iOS and macOS; releases after `v0.19.1-2` ship
+no Metal module.
 
 Each SPM framework slice embeds a privacy manifest generated from an audited
 per-framework declaration, and packaging rejects an archive whose declarations

docs/v019_qualification.md

@@ -33,6 +33,7 @@ The exact-tag verification gate remains enabled.
 | ASR | `omni/asr` and the legacy engine are gone; `omni/speech2text` factories route through `OmniRegistry` and require a container | C ASR ABI v1 is unchanged. Containers use the registry; a `.tflite` plus tokenizer uses a bridge factory (`litert_lm_asr_omni_engine.cc`). Container metadata and Parakeet sections pick the registry name, so file names, free-form `model_name`s and `decoder_type` do not change routing, and a `model_name` naming another family fails at engine creation. Other LLM containers lost the generic LLM route and need a registered name. Start tokens are dropped for LM-backed families, whose decoders reject them. |
 | Protos | `Asr*` became `Speech2Text*` | Internal to the adapter. |
 | Packaging | No `LiteRtTopK*Sampler` library; `libLiteRtLm` links Dawn | Validators and manifests key this on `has_prebuilt_topk_samplers`. |
+| Apple GPU | `LiteRtLm` links its GPU accelerator: ml_drift Metal on iOS, WebGPU on macOS | LiteRT registers it before searching for a shared one, so the prebuilt Metal accelerator is never opened ([llamadart#1798](https://github.com/leehack/llamadart/issues/1798)). Packaging after `v0.19.1-2` skips it on iOS and macOS, requires the static accelerator symbol in `LiteRtLm`, and rejects a candidate that still ships it (`has_prebuilt_apple_metal_accelerator`). |
 
 `ubuntu-latest` moved to 26.04, so a Linux x64 build there needs glibc 2.43 and
 fails to load on Ubuntu 24.04. Linux x64 now builds on `ubuntu-24.04` and

tools/litert_lm_symbols.py

@@ -95,6 +95,16 @@
     "LiteRtAcceleratorImpl",
 ]
 
+APPLE_METAL_ACCELERATOR_LIBRARY = "libLiteRtMetalAccelerator.dylib"
+
+# LiteRT registers a statically linked GPU accelerator before it searches for a
+# shared one (gpu_registry.cc), so these classes make the prebuilt Metal
+# accelerator unreachable.
+APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS = {
+    "ios": b"_ZN19GpuMetalAccelerator14CreateDelegate",
+    "macos": b"_ZN20GpuWebGpuAccelerator14CreateDelegate",
+}
+
 
 def _version_tuple(tag: str) -> tuple[int, int, int] | None:
     match = re.search(r"v?(\d+)\.(\d+)\.(\d+)", tag)
@@ -146,6 +156,11 @@ def has_prebuilt_topk_samplers(upstream_tag: str) -> bool:
     return not is_at_least(upstream_tag, (0, 19, 0))
 
 
+def has_prebuilt_apple_metal_accelerator(upstream_tag: str) -> bool:
+    """v0.19 Apple runtimes link their GPU accelerator and never open the prebuilt one."""
+    return not is_at_least(upstream_tag, (0, 19, 0))
+
+
 def required_bridge_symbols(upstream_tag: str) -> list[bytes]:
     symbols = list(BRIDGE_SYMBOLS)
     if has_asr_bridge(upstream_tag):

tools/package_ios_runtime.py

@@ -15,6 +15,9 @@
 
 from ios_framework_metadata import framework_minimum_os, validate_framework_metadata
 from litert_lm_symbols import (
+    APPLE_METAL_ACCELERATOR_LIBRARY,
+    APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS,
+    has_prebuilt_apple_metal_accelerator,
     has_prebuilt_topk_samplers,
     BRIDGE_SYMBOLS,
     has_asr_bridge,
@@ -40,9 +43,11 @@
     b"LiteRtTopKMetalSampler_SampleToIdAndScoreBuffer_Static",
 )
 def ios_dlopen_dependencies(upstream_tag: str) -> tuple[str, ...]:
+    if not has_prebuilt_apple_metal_accelerator(upstream_tag):
+        return ()
     if has_prebuilt_topk_samplers(upstream_tag):
-        return ("libLiteRtMetalAccelerator.dylib", "libLiteRtTopKMetalSampler.dylib")
-    return ("libLiteRtMetalAccelerator.dylib",)
+        return (APPLE_METAL_ACCELERATOR_LIBRARY, "libLiteRtTopKMetalSampler.dylib")
+    return (APPLE_METAL_ACCELERATOR_LIBRARY,)
 EXPECTED_OFFICIAL_ARCHIVE_SHA256 = {
     "v0.14.0": "dddac2f6713ed65eaf01c18e115d9fec22184adf575cc7856a21387e8ba937e1",
 }
@@ -156,6 +161,8 @@ def validate_source_built_symbols(output: Path, upstream_tag: str) -> None:
         required_c_api_symbols(upstream_tag)
         + required_bridge_symbols(upstream_tag)
     )
+    if not has_prebuilt_apple_metal_accelerator(upstream_tag):
+        required_symbols.append(APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS["ios"])
     missing = [
         symbol.decode("ascii")
         for symbol in required_symbols

tools/package_macos_runtime.py

@@ -11,7 +11,9 @@
 from pathlib import Path
 
 from litert_lm_symbols import (
+    APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS,
     BRIDGE_SYMBOLS,
+    has_prebuilt_apple_metal_accelerator,
     has_asr_bridge,
     required_bridge_symbols,
     required_c_api_symbols,
@@ -126,6 +128,8 @@ def validate_source_built_symbols(output: Path, upstream_tag: str) -> None:
         required_c_api_symbols(upstream_tag)
         + required_bridge_symbols(upstream_tag)
     )
+    if not has_prebuilt_apple_metal_accelerator(upstream_tag):
+        required_symbols.append(APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS["macos"])
     missing = [
         symbol.decode("ascii")
         for symbol in required_symbols

tools/package_upstream_prebuilts.py

@@ -10,6 +10,10 @@
 
 from download_utils import download_to_path
 from git_lfs_utils import materialize_git_lfs_libraries
+from litert_lm_symbols import (
+    APPLE_METAL_ACCELERATOR_LIBRARY,
+    has_prebuilt_apple_metal_accelerator,
+)
 from prebuilt_overrides import (
     UPSTREAM_MEDIA_BASE_URL,
     PrebuiltOverride,
@@ -61,6 +65,14 @@ def extract_source(archive: Path, output_dir: Path) -> Path:
     return roots[0]
 
 
+def unused_prebuilts(upstream_tag: str, platform: str) -> frozenset[str]:
+    if platform in {"ios", "macos"} and not has_prebuilt_apple_metal_accelerator(
+        upstream_tag
+    ):
+        return frozenset({APPLE_METAL_ACCELERATOR_LIBRARY})
+    return frozenset()
+
+
 def copy_prebuilts(
     source_root: Path,
     upstream_tag: str,
@@ -84,9 +96,13 @@ def copy_prebuilts(
             shutil.rmtree(target_dir)
         target_dir.mkdir(parents=True, exist_ok=True)
 
+        unused = unused_prebuilts(upstream_tag, platform)
         for file in sorted(source_dir.iterdir()):
             if not file.is_file() or not file.name.endswith(LIB_SUFFIXES):
                 continue
+            if file.name in unused:
+                print(f"Skipped unused {upstream_name}/{file.name}", flush=True)
+                continue
             shutil.copy2(file, target_dir / file.name)
             copied += 1
         print(f"Packaged {upstream_name} -> {target_dir}", flush=True)

tools/test_package_upstream_prebuilts.py

@@ -67,6 +67,36 @@ def materialize(
                 b"\x7fELF materialized",
             )
 
+    def test_skips_apple_metal_accelerator_from_v019(self) -> None:
+        targets = {
+            "macos_arm64": ("macos", "arm64"),
+            "ios_arm64": ("ios", "arm64"),
+            "android_arm64": ("android", "arm64"),
+        }
+        for tag, metal_copied in (("v0.18.1", True), ("v0.19.1", False)):
+            with self.subTest(tag=tag), tempfile.TemporaryDirectory() as temp:
+                root = Path(temp)
+                for upstream_name in targets:
+                    source_dir = root / "source" / "prebuilt" / upstream_name
+                    source_dir.mkdir(parents=True)
+                    for name in ("libLiteRt.dylib", "libLiteRtMetalAccelerator.dylib"):
+                        (source_dir / name).write_bytes(b"binary")
+                output = root / "bin"
+                with patch.object(package_upstream_prebuilts, "BIN_DIR", output), \
+                        patch.object(package_upstream_prebuilts, "PREBUILT_TARGETS", targets), \
+                        patch.object(package_upstream_prebuilts,
+                                     "materialize_git_lfs_libraries", return_value=0):
+                    package_upstream_prebuilts.copy_prebuilts(
+                        root / "source", upstream_tag=tag, clean=True)
+
+                for platform in ("macos", "ios", "android"):
+                    staged = output / platform / "arm64"
+                    self.assertTrue((staged / "libLiteRt.dylib").is_file())
+                    self.assertEqual(
+                        (staged / "libLiteRtMetalAccelerator.dylib").is_file(),
+                        metal_copied or platform == "android",
+                    )
+
     def test_overrides_only_does_not_download_or_copy_upstream_source(self) -> None:
         with patch(
             "sys.argv",

tools/test_v019_decisions.py

@@ -9,12 +9,21 @@
 from asr_overlap_patch import V019_MERGER_SOURCE, source_patch_manifest
 from litert_cpu_options_identifier_patch import require_cpu_options_identifier_patch_decision
 from ios_runtime_policy import verify_upstream_fst_gate
-from litert_lm_symbols import has_prebuilt_topk_samplers, uses_c_api_v1
+from litert_lm_symbols import (
+    APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS,
+    has_prebuilt_apple_metal_accelerator,
+    has_prebuilt_topk_samplers,
+    required_bridge_symbols,
+    required_c_api_symbols,
+    uses_c_api_v1,
+)
 from package_ios_runtime import ios_dlopen_dependencies
+from package_ios_runtime import validate_source_built_symbols as ios_source_built_symbols
+from package_macos_runtime import validate_source_built_symbols as macos_source_built_symbols
 from prebuilt_overrides import prebuilt_overrides
 from stream_finish_reason_patch import require_finish_reason_patch_decision
 from validate_release_manifest import allowed_spm_assets, required_spm_assets
-from validate_runtime_artifacts import required_runtime_artifacts
+from validate_runtime_artifacts import forbidden_runtime_artifacts, required_runtime_artifacts
 from validate_runtime_dependencies import newest_glibc_requirement
 from workspace_build_patch import require_workspace_build_patch_decision
 from xnnpack_pf32_sme2_patch import commit_for as xnnpack_commit_for
@@ -45,13 +54,48 @@ def test_topk_sampler_packaging_stops_at_v019(self) -> None:
         self.assertFalse(uses_c_api_v1("v0.18.1"))
         self.assertIn("libLiteRtTopKMetalSampler.dylib",
                       ios_dlopen_dependencies("v0.18.1"))
-        self.assertEqual(ios_dlopen_dependencies("v0.19.0"),
-                         ("libLiteRtMetalAccelerator.dylib",))
+        self.assertEqual(ios_dlopen_dependencies("v0.19.0"), ())
         for assets in (required_spm_assets("v0.19.0"), allowed_spm_assets("v0.19.0")):
             self.assertFalse([name for name in assets if "TopK" in name])
         self.assertTrue([name for name in required_spm_assets("v0.18.1") if "TopK" in name])
         self.assertFalse([path for path in required_runtime_artifacts("v0.19.0") if "TopK" in str(path)])
 
+    def test_apple_metal_accelerator_packaging_stops_at_v019(self) -> None:
+        metal_zip = "litert-lm-native-apple-LiteRtMetalAccelerator-xcframework-{tag}.zip"
+        self.assertTrue(has_prebuilt_apple_metal_accelerator("v0.18.1"))
+        self.assertFalse(has_prebuilt_apple_metal_accelerator("v0.19.1"))
+        self.assertIn("libLiteRtMetalAccelerator.dylib", ios_dlopen_dependencies("v0.18.1"))
+        self.assertIn(metal_zip, required_spm_assets("v0.18.1"))
+        self.assertNotIn(metal_zip, allowed_spm_assets("v0.19.1"))
+        self.assertTrue([p for p in required_runtime_artifacts("v0.18.1") if "Metal" in str(p)])
+        self.assertFalse([p for p in required_runtime_artifacts("v0.19.1") if "Metal" in str(p)])
+        self.assertEqual(forbidden_runtime_artifacts("v0.18.1"), [])
+        self.assertEqual(
+            {p.as_posix() for p in forbidden_runtime_artifacts("v0.19.1")},
+            {
+                "bin/ios/arm64/LiteRtMetalAccelerator.framework/LiteRtMetalAccelerator",
+                "bin/ios/arm64-sim/LiteRtMetalAccelerator.framework/LiteRtMetalAccelerator",
+                "bin/ios/arm64/libLiteRtMetalAccelerator.dylib",
+                "bin/ios/arm64-sim/libLiteRtMetalAccelerator.dylib",
+                "bin/macos/arm64/libLiteRtMetalAccelerator.dylib",
+            },
+        )
+
+    def test_v019_apple_runtimes_require_a_static_gpu_accelerator(self) -> None:
+        required = (required_c_api_symbols("v0.19.1")
+                    + required_bridge_symbols("v0.19.1"))
+        for platform, validate in (("ios", ios_source_built_symbols),
+                                   ("macos", macos_source_built_symbols)):
+            static = APPLE_STATIC_GPU_ACCELERATOR_SYMBOLS[platform]
+            with self.subTest(platform=platform), tempfile.TemporaryDirectory() as temp:
+                binary = Path(temp) / "LiteRtLm"
+                binary.write_bytes(b"\0".join(required))
+                with self.assertRaisesRegex(RuntimeError, static.decode()):
+                    validate(binary, "v0.19.1")
+                validate(binary, "v0.18.1")
+                binary.write_bytes(b"\0".join([*required, static]))
+                validate(binary, "v0.19.1")
+
     def test_ios_fst_gate_follows_the_build_config_layout(self) -> None:
         provider = '"//runtime/components/constrained_decoding:gemma_model_constraint_provider_lib",\n'
         processors = (

tools/validate_release_manifest.py

@@ -11,6 +11,7 @@
 from fetch_litert_lm_asr_smoke_assets import ASSETS
 from litert_lm_symbols import (
     has_asr_bridge,
+    has_prebuilt_apple_metal_accelerator,
     has_prebuilt_topk_samplers,
     is_at_least,
     stream_proxy_callback_abi_version,
@@ -622,7 +623,8 @@ def validate_schema_2_payload(
 def required_spm_assets(compatibility_tag: str) -> list[str]:
     assets = list(REQUIRED_SPM_XCFRAMEWORKS)
     if is_at_least(compatibility_tag, (0, 16, 0)):
-        assets.extend(V0_16_IOS_GPU_SPM_XCFRAMEWORKS)
+        if has_prebuilt_apple_metal_accelerator(compatibility_tag):
+            assets.extend(V0_16_IOS_GPU_SPM_XCFRAMEWORKS)
         if has_prebuilt_topk_samplers(compatibility_tag):
             assets.extend(PREBUILT_SAMPLER_SPM_XCFRAMEWORKS)
     return assets

tools/validate_runtime_artifacts.py

@@ -4,7 +4,11 @@
 import argparse
 from pathlib import Path
 
-from litert_lm_symbols import has_prebuilt_topk_samplers, is_at_least
+from litert_lm_symbols import (
+    has_prebuilt_apple_metal_accelerator,
+    has_prebuilt_topk_samplers,
+    is_at_least,
+)
 from windows_dxc import dxc_license_paths, dxc_runtime_paths, requires_dxc
 
 REPO_ROOT = Path(__file__).resolve().parents[1]
@@ -28,6 +32,12 @@
     Path("bin/ios/arm64-sim/LiteRtMetalAccelerator.framework/LiteRtMetalAccelerator"),
 ]
 
+APPLE_METAL_ACCELERATOR_ARTIFACTS = V0_16_IOS_GPU_ARTIFACTS + [
+    Path("bin/ios/arm64/libLiteRtMetalAccelerator.dylib"),
+    Path("bin/ios/arm64-sim/libLiteRtMetalAccelerator.dylib"),
+    Path("bin/macos/arm64/libLiteRtMetalAccelerator.dylib"),
+]
+
 PREBUILT_SAMPLER_IOS_ARTIFACTS = [
     Path("bin/ios/arm64/LiteRtTopKMetalSampler.framework/LiteRtTopKMetalSampler"),
     Path("bin/ios/arm64-sim/LiteRtTopKMetalSampler.framework/LiteRtTopKMetalSampler"),
@@ -44,7 +54,8 @@ def required_runtime_artifacts(
 ) -> list[Path]:
     required = list(REQUIRED_RUNTIME_ARTIFACTS)
     if is_at_least(upstream_tag, (0, 16, 0)):
-        required.extend(V0_16_IOS_GPU_ARTIFACTS)
+        if has_prebuilt_apple_metal_accelerator(upstream_tag):
+            required.extend(V0_16_IOS_GPU_ARTIFACTS)
         if has_prebuilt_topk_samplers(upstream_tag):
             required.extend(PREBUILT_SAMPLER_IOS_ARTIFACTS)
     if requires_dxc(upstream_tag):
@@ -57,6 +68,12 @@ def required_runtime_artifacts(
     return required
 
 
+def forbidden_runtime_artifacts(upstream_tag: str) -> list[Path]:
+    if has_prebuilt_apple_metal_accelerator(upstream_tag):
+        return []
+    return list(APPLE_METAL_ACCELERATOR_ARTIFACTS)
+
+
 def main() -> int:
     parser = argparse.ArgumentParser(
         description="Validate that release packaging contains upstream runtime libraries."
@@ -85,6 +102,14 @@ def main() -> int:
     if missing:
         formatted = "\n".join(f"- {path.as_posix()}" for path in missing)
         raise SystemExit(f"Missing required runtime artifacts:\n{formatted}")
+    unused = [
+        path
+        for path in forbidden_runtime_artifacts(args.upstream_tag)
+        if (REPO_ROOT / path).exists()
+    ]
+    if unused:
+        formatted = "\n".join(f"- {path.as_posix()}" for path in unused)
+        raise SystemExit(f"Runtime packaging contains unused artifacts:\n{formatted}")
 
     print(f"Validated {len(required_with_licences)} required runtime artifacts")
     return 0