diff --git a/CHANGELOG.md b/CHANGELOG.md index 658d1cac0..9ad247145 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ Releases before 0.10.0 are in the ## Unreleased +- Stop shipping the unused LiteRT-LM Metal accelerator on iOS and macOS: native LiteRT-LM moves to `v0.19.1-3`, whose `LiteRtLm` has its GPU accelerator built in, so GPU generation is unchanged while the macOS hook bundle drops 8.7 MB and iOS apps 8.1 MB, and the iOS LiteRT-LM frameworks no longer use the User Defaults required-reason API ([#1798](https://github.com/leehack/llamadart/issues/1798)). - `engine.getContextSize()` on LiteRT-LM Web `withTextHistory()` loads returns the context size the runtime's executor enforces when its load probe finds one below the requested size, such as Qwen3 0.6B's 4096 on WASM CPU, instead of the requested `ModelParams.contextSize` ([#1861](https://github.com/leehack/llamadart/issues/1861)). - Parse Qwen3-Coder XML tool calls as llama.cpp does: a call needs a line break after each tag and around each value (a CRLF or compact `` not followed by a line break. Nemotron 3 Nano now parses as Qwen3-Coder XML, so a `` also ends its thought, and the tool-call grammar of these templates now requires that shape and starts at `` ([#1898](https://github.com/leehack/llamadart/issues/1898)). - Parse Seed-OSS tool calls closer to llama.cpp: a finished reply whose last `` scope is never closed keeps a call complete through `` and drops one cut off before it, an argument declared as a string keeps its value as generated instead of being read as JSON (`\n42\n` stays a string), and with `ToolChoice.required` the grammar lets a thought and text come before the calls ([#1898](https://github.com/leehack/llamadart/issues/1898)). diff --git a/README.md b/README.md index b6c4b05dc..41753a510 100644 --- a/README.md +++ b/README.md @@ -163,7 +163,7 @@ Current default runtime pins: | Runtime | Pin | | --- | --- | | Native llama.cpp / GGUF | `leehack/llamadart-native@v0.9.0` | -| Native LiteRT-LM / `.litertlm` | `leehack/litert-lm-native@v0.19.1-2` | +| Native LiteRT-LM / `.litertlm` | `leehack/litert-lm-native@v0.19.1-3` | | Web llama.cpp / GGUF | `leehack/llama-web-bridge-assets@v0.1.83` | | Web LiteRT-LM / `.litertlm` | `@litert-lm/core@0.19.0` | @@ -171,8 +171,8 @@ Native LiteRT-LM uses a lazy ABI-aware client for direct calls, services and speech. The public selector admits legacy runtimes and C API 1.0.x; unknown or future versions are refused before engine calls. The pinned runtime is published native -[v0.19.1-2](https://github.com/leehack/litert-lm-native/releases/tag/v0.19.1-2), -owner source `0b15fb34` / upstream `7a03724c`. The selected loader requires +[v0.19.1-3](https://github.com/leehack/litert-lm-native/releases/tag/v0.19.1-3), +owner source `ed8ffe33` / upstream `7a03724c`. The selected loader requires macOS 14+ before owned library loads, including legacy libraries and custom paths. Web LiteRT-LM uses `@litert-lm/core@0.19.0`; through `LlamaBackend.withTextHistory()` it forwards chat history, supports tool diff --git a/hook/build.dart b/hook/build.dart index 53a01be25..50bd1f77f 100644 --- a/hook/build.dart +++ b/hook/build.dart @@ -43,7 +43,7 @@ const _liteRtLmFlutterPackageName = 'llamadart_litert_lm_flutter'; // normalized. Only this manifest links the complete macOS runtime, so any other // resolved companion keeps macOS LiteRT-LM on the hook cache. const _liteRtLmCompanionManifestTemplateSha256 = - 'df03e4e53c34d1d7c7039e221092d30fb72b2a71d93b4f6227299af41aff26cb'; + '3d1654fa98dd747c4c9f0e642f3fa27d7657f7e8e25cdab4591e5eacd0de9b17'; const _stableDiffusionFlutterPackageName = 'llamadart_stable_diffusion_flutter'; const _stableDiffusionCompanionManifestTemplateSha256 = '829188afd63eec24b16259b05c0d884725818558fed30b41fae40fa39eeef66c'; diff --git a/lib/src/backends/litert_lm/litert_lm_runtime.dart b/lib/src/backends/litert_lm/litert_lm_runtime.dart index ad6771aa0..980ca1d1b 100644 --- a/lib/src/backends/litert_lm/litert_lm_runtime.dart +++ b/lib/src/backends/litert_lm/litert_lm_runtime.dart @@ -33,8 +33,8 @@ export 'litert_lm_native_settlement.dart'; part 'litert_lm_embedding_native.dart'; part 'litert_lm_runtime_facade.dart'; -const _litertLmReleaseTag = 'v0.19.1-2'; -const _litertLmVersion = '0.19.1-2'; +const _litertLmReleaseTag = 'v0.19.1-3'; +const _litertLmVersion = '0.19.1-3'; const _litertLmLibDirEnv = 'LLAMADART_LITERT_LM_LIB_DIR'; const _liteRtLmIosNativeAsset = 'package:llamadart/litert_lm_LiteRtLm'; const _processLibraryCandidate = ''; @@ -467,7 +467,6 @@ List _liteRtLmV1RequiredLibrariesForAbi(Abi abi) { 'libGemmaModelConstraintProvider.dylib', 'libLiteRt.dylib', 'libLiteRtLm.dylib', - 'libLiteRtMetalAccelerator.dylib', 'libLiteRtWebGpuAccelerator.dylib', 'libwebgpu_dawn.dylib', ], @@ -690,7 +689,6 @@ List _liteRtLmV1MacOsRequiredFrameworksForAbi(Abi abi) { 'GemmaModelConstraintProvider.framework/Versions/A/GemmaModelConstraintProvider', 'LiteRt.framework/Versions/A/LiteRt', 'LiteRtLm.framework/Versions/A/LiteRtLm', - 'LiteRtMetalAccelerator.framework/Versions/A/LiteRtMetalAccelerator', 'LiteRtWebGpuAccelerator.framework/Versions/A/LiteRtWebGpuAccelerator', 'webgpu_dawn.framework/Versions/A/webgpu_dawn', ], diff --git a/lib/src/hook/native_release_pins.dart b/lib/src/hook/native_release_pins.dart index f0dd4b18a..78ed4e433 100644 --- a/lib/src/hook/native_release_pins.dart +++ b/lib/src/hook/native_release_pins.dart @@ -106,10 +106,10 @@ const llamaCppBackendSidecarSpecs = [ ]; /// `leehack/litert-lm-native` release tag downloaded for the LiteRT-LM runtime. -const liteRtLmReleaseTag = 'v0.19.1-2'; +const liteRtLmReleaseTag = 'v0.19.1-3'; /// LiteRT-LM cache directory version derived from [liteRtLmReleaseTag]. -const liteRtLmVersion = '0.19.1-2'; +const liteRtLmVersion = '0.19.1-3'; /// A published LiteRT-LM runtime archive and the libraries it must contain. class LiteRtLmBundleSpec { @@ -134,7 +134,7 @@ class LiteRtLmBundleSpec { const liteRtLmBundleSpecs = [ LiteRtLmBundleSpec( 'android-arm64', - sha256: '1b72d7f6b144b3c7514e7c8a1573abf190e57dfc9808037302c113d1713c1bf5', + sha256: 'c79e5d825537f0c914cefc6a94001cbb5aa55dab1d9f44aa4d76c61c86e1a069', requiredLibraries: { 'libGemmaModelConstraintProvider.so', 'libLiteRtGpuAccelerator.so', @@ -146,7 +146,7 @@ const liteRtLmBundleSpecs = [ ), LiteRtLmBundleSpec( 'android-x64', - sha256: 'd6a1601750a4ebe4d994971673d8fc6d05c31a1baaf1462ca0db2f249b66af16', + sha256: '24f8fd512b538f7b7e99c56113aeb43b34f075c7ec99fc6a908de2624934564d', requiredLibraries: { 'libGemmaModelConstraintProvider.so', 'libLiteRtGpuAccelerator.so', @@ -158,30 +158,29 @@ const liteRtLmBundleSpecs = [ ), LiteRtLmBundleSpec( 'ios-arm64', - sha256: '26dc6e031d5a55f98c9a4b7183829e941e69fa4399ecc8fa0c4853acf0fe2743', - requiredLibraries: {'CLiteRTLM', 'LiteRtLm', 'LiteRtMetalAccelerator'}, + sha256: '685ae904656b18bac91a552349202f4a10bc783085464e2dcd2f6d74ffe5078f', + requiredLibraries: {'CLiteRTLM', 'LiteRtLm'}, ), LiteRtLmBundleSpec( 'ios-arm64-sim', - sha256: '90348dfa05b4e7f5e2a5dc5731a3490cce5794d1b8bb586a9f7d597e888b3474', - requiredLibraries: {'CLiteRTLM', 'LiteRtLm', 'LiteRtMetalAccelerator'}, + sha256: '381e03858be852d34693bbd69b341931aa8ada878101f49b0c9930090869a69c', + requiredLibraries: {'CLiteRTLM', 'LiteRtLm'}, ), LiteRtLmBundleSpec( 'macos-arm64', - sha256: '84bc2e7fc00ac9bc3135f5a7e05da6af0bb2b813cf5d996ba8749b86dde37b23', + sha256: 'a33371e94876eb453667cc37a75316b3532abf63a5783cdedac1b5ae15868541', requiredLibraries: { 'libCLiteRTLM_mac.dylib', 'libGemmaModelConstraintProvider.dylib', 'libLiteRt.dylib', 'libLiteRtLm.dylib', - 'libLiteRtMetalAccelerator.dylib', 'libLiteRtWebGpuAccelerator.dylib', 'libwebgpu_dawn.dylib', }, ), LiteRtLmBundleSpec( 'linux-arm64', - sha256: 'c01d608cc1f332e090f96295c32019dd238b16cc194f29264aec950e92e511b5', + sha256: 'bc71965265bd683939395bd0585dc7a3f277e9d13e6255955015468abe5274d6', requiredLibraries: { 'libGemmaModelConstraintProvider.so', 'libLiteRtLm.so', @@ -191,7 +190,7 @@ const liteRtLmBundleSpecs = [ ), LiteRtLmBundleSpec( 'linux-x64', - sha256: '360171860fac20057e107dbbe6fd1b1376d4011efe1bcf6ad7b8045ff13b4d5d', + sha256: '03f6aba1da9b6f76b794928ce8fdaedce357ce18aab344ae8494116c0b63d0d0', requiredLibraries: { 'libGemmaModelConstraintProvider.so', 'libLiteRtLm.so', @@ -201,7 +200,7 @@ const liteRtLmBundleSpecs = [ ), LiteRtLmBundleSpec( 'windows-x64', - sha256: 'dd25067751cb9f42434e4f2b8dcdae725dd72b94f36ca5c359bb9b3616750f83', + sha256: '2c1fbd3973a22b186d2ffb114779eecdfae0445bd1724df4edf3991bae17ad92', requiredLibraries: { 'LiteRtLm.dll', 'dxcompiler.dll', diff --git a/packages/llamadart_litert_lm_flutter/CHANGELOG.md b/packages/llamadart_litert_lm_flutter/CHANGELOG.md index c131dd514..0a6f3a6f0 100644 --- a/packages/llamadart_litert_lm_flutter/CHANGELOG.md +++ b/packages/llamadart_litert_lm_flutter/CHANGELOG.md @@ -1,3 +1,7 @@ +## Unreleased + +- Pin `leehack/litert-lm-native@v0.19.1-3` and drop the unused `LiteRtMetalAccelerator` target, which saves 8.1 MB per iOS app ([#1798](https://github.com/leehack/llamadart/issues/1798)). + ## 0.19.4 - Refresh README installation guidance for core `0.19.4`; Apple SwiftPM stays pinned to `leehack/litert-lm-native@v0.19.1-2`. diff --git a/packages/llamadart_litert_lm_flutter/README.md b/packages/llamadart_litert_lm_flutter/README.md index 8739e11f6..b4e4fab30 100644 --- a/packages/llamadart_litert_lm_flutter/README.md +++ b/packages/llamadart_litert_lm_flutter/README.md @@ -7,12 +7,12 @@ support. Add the package to a Flutter iOS or macOS app so it links the prebuilt LiteRT-LM Apple XCFrameworks through SwiftPM instead of relying on the core package's native-assets fallback. On macOS it also links the `LiteRt`, WebGPU -accelerator and sampler, and Dawn XCFrameworks, so the app bundle holds the +accelerator and Dawn XCFrameworks, so the app bundle holds the complete runtime and a sandboxed or distributed app loads LiteRT-LM on CPU and GPU. Every macOS XCFramework is arm64 only; a universal build prints an `ignoring file ... required architecture 'x86_64'` linker warning for each. -The Metal accelerator and sampler are iOS-only; the macOS GPU path uses the -WebGPU accelerator. +`LiteRtLm` has its GPU accelerator built in (Metal on iOS, WebGPU on macOS), so +the package links no separate Metal accelerator. LiteRT-LM is not available on x86_64 macOS: `litert-lm-native` publishes arm64-only macOS runtimes. A universal app still launches on an Intel Mac or @@ -34,7 +34,7 @@ dependencies: This package has no runtime Dart API of its own. Import `package:llamadart` normally from the core package and use `LlamaBackend()` / `LlamaEngine` there. -The Apple SwiftPM manifest pins `leehack/litert-lm-native@v0.19.1-2`. +The Apple SwiftPM manifest pins `leehack/litert-lm-native@v0.19.1-3`. That runtime loads dedicated ASR `.litertlm` containers such as Whisper tiny. Use the companion that shares the core's `MAJOR.MINOR`: companion `0.19.x` pairs @@ -60,11 +60,9 @@ its iOS frameworks carry Apple privacy manifests for the [required-reason API](https://developer.apple.com/documentation/bundleresources/describing-use-of-required-reason-api) they reference, and the dylibs the core package's native-assets fallback bundles carry none. `LiteRtLm` declares File Timestamp (`C617.1`, `3B52.1`) -and System Boot Time (`35F9.1`), `LiteRtMetalAccelerator` File Timestamp -(`C617.1`, `3B52.1`) and User Defaults (`CA92.1`), `LiteRtTopKMetalSampler` -User Defaults (`CA92.1`), and `CLiteRTLM` no API. If the embedded LiteRT-LM -frameworks in the built app have no `PrivacyInfo.xcprivacy`, or the app uses -the fallback, declare those three categories and reasons in the app's own +and System Boot Time (`35F9.1`), and `CLiteRTLM` no API. If the embedded +LiteRT-LM frameworks in the built app have no `PrivacyInfo.xcprivacy`, or the +app uses the fallback, declare those two categories and reasons in the app's own `PrivacyInfo.xcprivacy`. The reasons come from the runtime owner's audit ([litert-lm-native#65](https://github.com/leehack/litert-lm-native/issues/65)) and have not been verified with an App Store Connect upload. See diff --git a/packages/llamadart_litert_lm_flutter/darwin/llamadart_litert_lm_flutter/Package.swift b/packages/llamadart_litert_lm_flutter/darwin/llamadart_litert_lm_flutter/Package.swift index fc61b8429..510e506ce 100644 --- a/packages/llamadart_litert_lm_flutter/darwin/llamadart_litert_lm_flutter/Package.swift +++ b/packages/llamadart_litert_lm_flutter/darwin/llamadart_litert_lm_flutter/Package.swift @@ -4,7 +4,7 @@ import PackageDescription let packageRoot = URL(fileURLWithPath: #filePath).deletingLastPathComponent() let artifactsRoot = packageRoot.appendingPathComponent("Artifacts") -let liteRtLmTag = "v0.19.1-2" +let liteRtLmTag = "v0.19.1-3" func localArtifactPath(_ name: String) -> String? { let path = artifactsRoot.appendingPathComponent(name).path @@ -47,49 +47,42 @@ let package = Package( repository: "leehack/litert-lm-native", artifactName: "litert-lm-native-apple-LiteRtLm-xcframework-\(liteRtLmTag).zip", tag: liteRtLmTag, - checksum: "66ee469310ee7aec7bcc438cf0ab5bed6f8ab376f49516839bb18ed70e4e40e0" + checksum: "685ff2f959ed07d9753d5e1022770ff20c981c7acea42a9c27a43139e3e94e00" ), nativeRepoBinaryTarget( name: "CLiteRTLM", repository: "leehack/litert-lm-native", artifactName: "litert-lm-native-apple-CLiteRTLM-xcframework-\(liteRtLmTag).zip", tag: liteRtLmTag, - checksum: "ef10e320169b741552b99dad122d0f7c6b8929332e53c86213e9754c3191c9f4" + checksum: "c1573984374c7b56da1b9314f948d54300fb4b6e0988fd83f31fbd5101ace254" ), nativeRepoBinaryTarget( name: "CLiteRTLMMac", repository: "leehack/litert-lm-native", artifactName: "litert-lm-native-apple-CLiteRTLMMac-xcframework-\(liteRtLmTag).zip", tag: liteRtLmTag, - checksum: "94083e21752430320f3ff210712efcfc090719dd6f63968a7e7fdf5ce57042af" - ), - nativeRepoBinaryTarget( - name: "LiteRtMetalAccelerator", - repository: "leehack/litert-lm-native", - artifactName: "litert-lm-native-apple-LiteRtMetalAccelerator-xcframework-\(liteRtLmTag).zip", - tag: liteRtLmTag, - checksum: "12a5315dd958956724fc7a1fa42e179a69575d6fc9be18c22dc4db4bc3019fff" + checksum: "cd06e43b5b08d436be076da25e5e14f354409d8d97f246009331b5e5546fbace" ), nativeRepoBinaryTarget( name: "LiteRt", repository: "leehack/litert-lm-native", artifactName: "litert-lm-native-apple-LiteRt-xcframework-\(liteRtLmTag).zip", tag: liteRtLmTag, - checksum: "3a57e1b43a341629dd7934863d2c02d97d17bf935fe8908a24aea757d93a8461" + checksum: "254f07840b2c3dac8c575615086540f8fd6cd6174a395542379406012e58a42a" ), nativeRepoBinaryTarget( name: "LiteRtWebGpuAccelerator", repository: "leehack/litert-lm-native", artifactName: "litert-lm-native-apple-LiteRtWebGpuAccelerator-xcframework-\(liteRtLmTag).zip", tag: liteRtLmTag, - checksum: "22c737793e329dd7c05fce3d928572a1679906feceee199c02918960225ca225" + checksum: "88ec8b0531eab8267015f6821d7c8732ffa61b4fed9e7e220e1f31b751567b84" ), nativeRepoBinaryTarget( name: "WebgpuDawn", repository: "leehack/litert-lm-native", artifactName: "litert-lm-native-apple-WebgpuDawn-xcframework-\(liteRtLmTag).zip", tag: liteRtLmTag, - checksum: "9b943bed89b9dd21178d4b0f3f2dab5273c941ce045ba28940e5e494ece75036" + checksum: "dda09bae837e3859497d643daa02ef6328374d1509b116ec60027111e909af31" ), .target( name: "llamadart_litert_lm_flutter", @@ -97,7 +90,6 @@ let package = Package( .target(name: "LiteRtLm", condition: .when(platforms: [.iOS, .macOS])), .target(name: "CLiteRTLM", condition: .when(platforms: [.iOS])), .target(name: "CLiteRTLMMac", condition: .when(platforms: [.macOS])), - .target(name: "LiteRtMetalAccelerator", condition: .when(platforms: [.iOS])), .target(name: "LiteRt", condition: .when(platforms: [.macOS])), .target(name: "LiteRtWebGpuAccelerator", condition: .when(platforms: [.macOS])), .target(name: "WebgpuDawn", condition: .when(platforms: [.macOS])) diff --git a/packages/llamadart_litert_lm_flutter/test/llamadart_litert_lm_flutter_test.dart b/packages/llamadart_litert_lm_flutter/test/llamadart_litert_lm_flutter_test.dart index 21e737f62..5cd0e5896 100644 --- a/packages/llamadart_litert_lm_flutter/test/llamadart_litert_lm_flutter_test.dart +++ b/packages/llamadart_litert_lm_flutter/test/llamadart_litert_lm_flutter_test.dart @@ -15,7 +15,7 @@ void main() { expect(manifest, contains('name: "llamadart-litert-lm-flutter"')); expect(manifest, isNot(contains('GemmaModelConstraintProvider'))); - expect(manifest, contains('name: "LiteRtMetalAccelerator"')); + expect(manifest, isNot(contains('LiteRtMetalAccelerator'))); expect( manifest, isNot( @@ -39,13 +39,6 @@ void main() { manifest, contains('name: "CLiteRTLM", condition: .when(platforms: [.iOS])'), ); - // macOS uses WebGPU; the Metal accelerator remains iOS-only. - expect( - manifest, - contains( - 'name: "LiteRtMetalAccelerator", condition: .when(platforms: [.iOS])', - ), - ); }); test('links the complete macOS arm64 runtime', () { @@ -68,7 +61,7 @@ void main() { } }); - test('declares the seven published v0.19 runtime targets', () { + test('declares the six published v0.19 runtime targets', () { final manifest = File( 'darwin/llamadart_litert_lm_flutter/Package.swift', ).readAsStringSync(); @@ -82,7 +75,6 @@ void main() { 'LiteRtLm', 'CLiteRTLM', 'CLiteRTLMMac', - 'LiteRtMetalAccelerator', 'LiteRt', 'LiteRtWebGpuAccelerator', 'WebgpuDawn', diff --git a/packages/llamadart_validation/lib/android_litert_qualification.dart b/packages/llamadart_validation/lib/android_litert_qualification.dart index 3d441d0ae..b2236c376 100644 --- a/packages/llamadart_validation/lib/android_litert_qualification.dart +++ b/packages/llamadart_validation/lib/android_litert_qualification.dart @@ -28,8 +28,9 @@ const s24LiteRtNativeLibraries = { /// Linux and Windows hosts; its Android binaries are byte-identical to /// `v0.19.1` (#1808). `v0.19.1-2` changes only the Android `libLiteRtLm.so` /// and passed the full S24 graph again (Test Lab `matrix-wmalbq6nz437a`, -/// #1830). -const s24LiteRtRuntimeTags = {'v0.19.1', 'v0.19.1-1', 'v0.19.1-2'}; +/// #1830). `v0.19.1-3` changes only the Apple bundles; its Android arm64 +/// archive is byte-identical to `v0.19.1-2` (#1798). +const s24LiteRtRuntimeTags = {'v0.19.1', 'v0.19.1-1', 'v0.19.1-2', 'v0.19.1-3'}; /// Fixed prospective stage caps sum to 1,050s; FTL still caps the run at 1,200s. const s24LiteRtStageSeconds = { diff --git a/packages/llamadart_validation/native/android_litert/README.md b/packages/llamadart_validation/native/android_litert/README.md index cdc1758fa..8034667f4 100644 --- a/packages/llamadart_validation/native/android_litert/README.md +++ b/packages/llamadart_validation/native/android_litert/README.md @@ -221,8 +221,8 @@ needed for this Dart/diagnostic-only change. Freeze clean source and independent review before preparing an APK. The final binding must contain exact source commit/tree/inventory, runtime tag `v0.19.1`, -`v0.19.1-1` (same Android binaries) or `v0.19.1-2` (requalified on S24 in -`matrix-wmalbq6nz437a`, see +`v0.19.1-1` or `v0.19.1-3` (same Android binaries) or `v0.19.1-2` +(requalified on S24 in `matrix-wmalbq6nz437a`, see [the adoption notes](../../../../doc/litert_lm_v1_adoption.md)) and its owner commit, runtime archive and manifest hashes, complete owner archive libraries, complete installed APK libraries, child executable and canonical Qwen template diff --git a/packages/llamadart_validation/test/android_litert_qualification_test.dart b/packages/llamadart_validation/test/android_litert_qualification_test.dart index ed558cff1..36073470d 100644 --- a/packages/llamadart_validation/test/android_litert_qualification_test.dart +++ b/packages/llamadart_validation/test/android_litert_qualification_test.dart @@ -137,7 +137,7 @@ void main() { } for (final tag in [ 'v0.19.0', - 'v0.19.1-3', + 'v0.19.1-4', 'v0.19.2', 'v0.20.0', '0.19.1-1', diff --git a/skills/llamadart-flutter-app/SKILL.md b/skills/llamadart-flutter-app/SKILL.md index 8266589bc..045b0e577 100644 --- a/skills/llamadart-flutter-app/SKILL.md +++ b/skills/llamadart-flutter-app/SKILL.md @@ -160,8 +160,8 @@ or shipping runtimes the app does not use. the app's own `PrivacyInfo.xcprivacy` with reason `C617.1`, plus `3B52.1` only when it ships stable_diffusion. When LiteRT-LM runs on the hook path, or its embedded frameworks have no `PrivacyInfo.xcprivacy`, add File Timestamp - (`C617.1`, `3B52.1`), System Boot Time (`35F9.1`) and User Defaults - (`CA92.1`) to the app's own manifest. + (`C617.1`, `3B52.1`) and System Boot Time (`35F9.1`) to the app's own + manifest. - When the llama.cpp or LiteRT-LM companion is present, the installed companions pick those Apple runtime families and `llamadart_native_runtimes` is otherwise ignored with a warning; the tag, repository, path and backend diff --git a/skills/llamadart-getting-started/SKILL.md b/skills/llamadart-getting-started/SKILL.md index b909c398c..ca037103f 100644 --- a/skills/llamadart-getting-started/SKILL.md +++ b/skills/llamadart-getting-started/SKILL.md @@ -24,10 +24,10 @@ interface does not establish backend/model support; read `await engine.capabilit ## Guidelines -- Use the pinned native LiteRT-LM v0.19.1-2 runtime. Direct calls, service +- Use the pinned native LiteRT-LM v0.19.1-3 runtime. Direct calls, service probes and speech share ABI-aware discovery. Public selection admits legacy runtimes and C API 1.0.x; unknown/future versions are refused before engine - calls. The pin is published native v0.19.1-2 (owner `0b15fb34`, upstream + calls. The pin is published native v0.19.1-3 (owner `ed8ffe33`, upstream `7a03724c`). The selected loader requires macOS 14+ even for legacy libraries and custom paths. Web uses `@litert-lm/core@0.19.0`. diff --git a/skills/llamadart-speech/SKILL.md b/skills/llamadart-speech/SKILL.md index 2095ac7c6..4efa8d9ee 100644 --- a/skills/llamadart-speech/SKILL.md +++ b/skills/llamadart-speech/SKILL.md @@ -15,7 +15,7 @@ description: >- - Native LiteRT-LM speech probes and workers share the text client's ABI-aware discovery. Public selection admits legacy runtimes and C API 1.0.x; the - pinned runtime is published native v0.19.1-2. The selector checks macOS 14+ + pinned runtime is published native v0.19.1-3. The selector checks macOS 14+ before owned loads, including legacy libraries and custom paths. ASR also requires its independent bridge version, layout and complete export group; a text C API version proves none of those capabilities. diff --git a/test/unit/backends/litert_lm/litert_lm_discovery_inventory_test.dart b/test/unit/backends/litert_lm/litert_lm_discovery_inventory_test.dart index 908d885b7..037acea57 100644 --- a/test/unit/backends/litert_lm/litert_lm_discovery_inventory_test.dart +++ b/test/unit/backends/litert_lm/litert_lm_discovery_inventory_test.dart @@ -23,19 +23,25 @@ void main() { Abi.linuxX64: 'libLiteRt.so', Abi.windowsX64: 'libLiteRt.dll', }; + // v0.19.1-3 stopped shipping the Metal accelerator that the linked-in GPU + // accelerator makes unreachable. + const unusedMetalAccelerator = 'libLiteRtMetalAccelerator.dylib'; for (final abi in desktopAbis) { - test('$abi v1 keeps all owner companions except retired TopK libraries ' - 'and a linked-in LiteRT', () { + test('$abi v1 keeps all owner companions except retired TopK libraries, ' + 'the unused Metal accelerator and a linked-in LiteRT', () { final legacy = liteRtLmRequiredLibrariesForAbi(abi); final modern = liteRtLmRequiredLibrariesForAbi(abi, cApiV1: true); expect(modern.length, switch (abi) { - Abi.macosArm64 => 7, + Abi.macosArm64 => 6, Abi.windowsX64 => 11, _ => 4, }); final retired = legacy.toSet().difference(modern.toSet()); expect( - retired.where((file) => file != linkedInLiteRt[abi]), + retired.where( + (file) => + file != linkedInLiteRt[abi] && file != unusedMetalAccelerator, + ), everyElement(contains('TopK')), ); if (linkedInLiteRt[abi] case final liteRt?) { @@ -85,7 +91,9 @@ void main() { ..sort(); final linked = linkedInLiteRt[entry.key]; if (linked != null) expect(owner, contains(linked)); - final expected = owner.where((file) => file != linked).toList(); + final expected = owner + .where((file) => file != linked && file != unusedMetalAccelerator) + .toList(); expect( liteRtLmRequiredLibrariesForAbi(entry.key, cApiV1: true), expected, diff --git a/test/unit/hook/build_hook_litert_lm_integration_test.dart b/test/unit/hook/build_hook_litert_lm_integration_test.dart index ecd0327f4..c4286d8e9 100644 --- a/test/unit/hook/build_hook_litert_lm_integration_test.dart +++ b/test/unit/hook/build_hook_litert_lm_integration_test.dart @@ -2761,16 +2761,11 @@ const List _androidLiteRtLibraries = [ 'libwebgpu_dawn.so', ]; -const List _iosLiteRtLibraries = [ - 'LiteRtLm', - 'CLiteRTLM', - 'LiteRtMetalAccelerator', -]; +const List _iosLiteRtLibraries = ['LiteRtLm', 'CLiteRTLM']; const List _iosLiteRtAssetNames = [ 'litert_lm_LiteRtLm', 'litert_lm_CLiteRTLM', - 'litert_lm_LiteRtMetalAccelerator', ]; const List _macosArm64LiteRtLibraries = [ @@ -2778,7 +2773,6 @@ const List _macosArm64LiteRtLibraries = [ 'libGemmaModelConstraintProvider.dylib', 'libLiteRt.dylib', 'libLiteRtLm.dylib', - 'libLiteRtMetalAccelerator.dylib', 'libLiteRtWebGpuAccelerator.dylib', 'libwebgpu_dawn.dylib', ]; diff --git a/test/unit/hook/native_release_pins_test.dart b/test/unit/hook/native_release_pins_test.dart index f8d266028..0a6483b7d 100644 --- a/test/unit/hook/native_release_pins_test.dart +++ b/test/unit/hook/native_release_pins_test.dart @@ -143,18 +143,14 @@ void main() { ); }); - test('iOS inventory excludes the provider and retains CAPI and Metal', () { + test('iOS inventory excludes the provider and the Metal accelerator', () { for (final bundle in const ['ios-arm64', 'ios-arm64-sim']) { final spec = liteRtLmBundleSpecs.singleWhere( (spec) => spec.bundle == bundle, ); expect( spec.requiredLibraries, - unorderedEquals(const [ - 'CLiteRTLM', - 'LiteRtLm', - 'LiteRtMetalAccelerator', - ]), + unorderedEquals(const ['CLiteRTLM', 'LiteRtLm']), reason: bundle, ); } @@ -186,35 +182,43 @@ void main() { } }); - test('legacy overrides retain their prebuilt sampler requirements', () { - const samplers = { - Abi.macosArm64: [ - 'libLiteRtTopKMetalSampler.dylib', - 'libLiteRtTopKWebGpuSampler.dylib', - ], - Abi.linuxArm64: ['libLiteRtTopKWebGpuSampler.so'], - Abi.linuxX64: ['libLiteRtTopKWebGpuSampler.so'], - Abi.windowsX64: ['libLiteRtTopKWebGpuSampler.dll'], - }; - // v0.18.x Linux and Windows hosts import the prebuilt libLiteRt that - // C API 1 hosts from v0.19.1-1 link in. - const liteRt = { - Abi.linuxArm64: ['libLiteRt.so'], - Abi.linuxX64: ['libLiteRt.so'], - Abi.windowsX64: ['libLiteRt.dll'], - }; - for (final entry in samplers.entries) { - final modern = liteRtLmRequiredLibrariesForAbi(entry.key, cApiV1: true); - expect( - liteRtLmRequiredLibrariesForAbi(entry.key), - unorderedEquals([...modern, ...entry.value, ...?liteRt[entry.key]]), - reason: entry.key.toString(), - ); - for (final sampler in entry.value) { - expect(modern, isNot(contains(sampler)), reason: entry.key.toString()); + test( + 'legacy overrides retain their prebuilt accelerator and sampler requirements', + () { + const samplers = { + Abi.macosArm64: [ + 'libLiteRtMetalAccelerator.dylib', + 'libLiteRtTopKMetalSampler.dylib', + 'libLiteRtTopKWebGpuSampler.dylib', + ], + Abi.linuxArm64: ['libLiteRtTopKWebGpuSampler.so'], + Abi.linuxX64: ['libLiteRtTopKWebGpuSampler.so'], + Abi.windowsX64: ['libLiteRtTopKWebGpuSampler.dll'], + }; + // v0.18.x Linux and Windows hosts import the prebuilt libLiteRt that + // C API 1 hosts from v0.19.1-1 link in. + const liteRt = { + Abi.linuxArm64: ['libLiteRt.so'], + Abi.linuxX64: ['libLiteRt.so'], + Abi.windowsX64: ['libLiteRt.dll'], + }; + for (final entry in samplers.entries) { + final modern = liteRtLmRequiredLibrariesForAbi(entry.key, cApiV1: true); + expect( + liteRtLmRequiredLibrariesForAbi(entry.key), + unorderedEquals([...modern, ...entry.value, ...?liteRt[entry.key]]), + reason: entry.key.toString(), + ); + for (final sampler in entry.value) { + expect( + modern, + isNot(contains(sampler)), + reason: entry.key.toString(), + ); + } } - } - }); + }, + ); test('stable_diffusion pins the qualified v0.3.0-3 release artifacts', () { // Exact published owner manifest, independently checked against GitHub diff --git a/test/unit/platform/io/native_diagnostics_child_test.dart b/test/unit/platform/io/native_diagnostics_child_test.dart index 41946532d..c48323257 100644 --- a/test/unit/platform/io/native_diagnostics_child_test.dart +++ b/test/unit/platform/io/native_diagnostics_child_test.dart @@ -140,7 +140,6 @@ void main() { 'libGemmaModelConstraintProvider.dylib', 'libLiteRt.dylib', 'libLiteRtLm.dylib', - 'libLiteRtMetalAccelerator.dylib', 'libLiteRtWebGpuAccelerator.dylib', 'libwebgpu_dawn.dylib', ]) { diff --git a/test/unit/platform/io/native_diagnostics_hook_cache_test.dart b/test/unit/platform/io/native_diagnostics_hook_cache_test.dart index 7d2e98d29..1c5ff32b3 100644 --- a/test/unit/platform/io/native_diagnostics_hook_cache_test.dart +++ b/test/unit/platform/io/native_diagnostics_hook_cache_test.dart @@ -14,7 +14,6 @@ const _v1Libraries = [ 'libGemmaModelConstraintProvider.dylib', 'libLiteRt.dylib', 'libLiteRtLm.dylib', - 'libLiteRtMetalAccelerator.dylib', 'libLiteRtWebGpuAccelerator.dylib', 'libwebgpu_dawn.dylib', ]; diff --git a/test/unit/platform/io/runtime_setup_report_io_test.dart b/test/unit/platform/io/runtime_setup_report_io_test.dart index 4da14b727..dac9928f1 100644 --- a/test/unit/platform/io/runtime_setup_report_io_test.dart +++ b/test/unit/platform/io/runtime_setup_report_io_test.dart @@ -57,7 +57,7 @@ void main() { ]); expect(report.runtimes.map((runtime) => runtime.expectedReleaseTag), [ 'v0.9.0', - 'v0.19.1-2', + 'v0.19.1-3', 'v0.3.0-3', ]); expect(report.dartVersion, '3.13.1'); diff --git a/test/unit/tooling/check_webgpu_bridge_tag_test.dart b/test/unit/tooling/check_webgpu_bridge_tag_test.dart index 819e77ae6..5de339336 100644 --- a/test/unit/tooling/check_webgpu_bridge_tag_test.dart +++ b/test/unit/tooling/check_webgpu_bridge_tag_test.dart @@ -313,7 +313,7 @@ void main() { ); expect( File(nativeLlamaCppTagPath).readAsStringSync(), - contains("const liteRtLmReleaseTag = 'v0.19.1-2';"), + contains("const liteRtLmReleaseTag = 'v0.19.1-3';"), ); }, ); diff --git a/test/unit/tooling/litert_lm_library_smoke_test.dart b/test/unit/tooling/litert_lm_library_smoke_test.dart index abd06f420..df449b1df 100644 --- a/test/unit/tooling/litert_lm_library_smoke_test.dart +++ b/test/unit/tooling/litert_lm_library_smoke_test.dart @@ -103,6 +103,7 @@ void main() { expect(liteRtLmSmokeUsesCApiV1('v0.19.1'), isTrue); expect(liteRtLmSmokeUsesCApiV1('v0.19.1-1'), isTrue); expect(liteRtLmSmokeUsesCApiV1('v0.19.1-2'), isTrue); + expect(liteRtLmSmokeUsesCApiV1('v0.19.1-3'), isTrue); expect(liteRtLmSmokeUsesCApiV1('v0.18.1-4'), isFalse); for (final tag in ['v0.19.0-1', 'v0.19.2', 'v0.20.0', 'g0123456789ab']) { expect( @@ -122,6 +123,7 @@ void main() { ('v0.19.1-2', Abi.linuxX64), ('v0.19.1-2', Abi.macosArm64), ('v0.19.1-2', Abi.windowsX64), + ('v0.19.1-3', Abi.macosArm64), ('v0.18.1-4', Abi.linuxX64), ('v0.18.1-4', Abi.macosArm64), ('v0.18.1-4', Abi.windowsX64), @@ -258,7 +260,7 @@ void main() { runLiteRtLmLibrarySmoke( abi: abi, directory: null, - releaseTag: 'v0.19.1-2', + releaseTag: 'v0.19.1-3', openLibrary: (_) => libraries[fixture]!, ); diff --git a/test/unit/tooling/macos_litert_lm_prepare_app_script_test.dart b/test/unit/tooling/macos_litert_lm_prepare_app_script_test.dart index 903b7a957..b69814f22 100644 --- a/test/unit/tooling/macos_litert_lm_prepare_app_script_test.dart +++ b/test/unit/tooling/macos_litert_lm_prepare_app_script_test.dart @@ -583,7 +583,6 @@ const List _arm64Libraries = [ 'libGemmaModelConstraintProvider.dylib', 'libLiteRt.dylib', 'libLiteRtLm.dylib', - 'libLiteRtMetalAccelerator.dylib', 'libLiteRtWebGpuAccelerator.dylib', 'libwebgpu_dawn.dylib', ]; diff --git a/test/unit/tooling/sync_native_release_pins_script_test.dart b/test/unit/tooling/sync_native_release_pins_script_test.dart index 843d9b204..bc0b8b8ae 100644 --- a/test/unit/tooling/sync_native_release_pins_script_test.dart +++ b/test/unit/tooling/sync_native_release_pins_script_test.dart @@ -1602,7 +1602,7 @@ printf '%s\\n' '{"tag_name":"v0.2.0-1","assets":[]}' test('keeps LiteRT release identity separate from cache version', () { final pins = File(_pinsPath).readAsStringSync(); - expect(pins, contains("const liteRtLmReleaseTag = 'v0.19.1-2';")); + expect(pins, contains("const liteRtLmReleaseTag = 'v0.19.1-3';")); final hook = File('hook/build.dart').readAsStringSync(); expect(hook, contains(r"'$liteRtLmReleaseTag'")); expect(hook, isNot(contains(r"v$liteRtLmVersion"))); diff --git a/tool/litert_lm_library_smoke.dart b/tool/litert_lm_library_smoke.dart index 34796edb3..b2106f373 100644 --- a/tool/litert_lm_library_smoke.dart +++ b/tool/litert_lm_library_smoke.dart @@ -91,7 +91,7 @@ bool _isSupportedHost() { /// Selects only a reviewed release's required smoke inventory. bool liteRtLmSmokeUsesCApiV1(String releaseTag) { return switch (releaseTag) { - 'v0.19.0' || 'v0.19.1' || 'v0.19.1-1' || 'v0.19.1-2' => true, + 'v0.19.0' || 'v0.19.1' || 'v0.19.1-1' || 'v0.19.1-2' || 'v0.19.1-3' => true, 'v0.18.1-4' => false, _ => throw UnsupportedError( 'LiteRT-LM library smoke has no reviewed C API inventory for $releaseTag.', diff --git a/tool/macos_litert_lm_prepare_app.sh b/tool/macos_litert_lm_prepare_app.sh index 4b7d0fd42..d1a726efa 100755 --- a/tool/macos_litert_lm_prepare_app.sh +++ b/tool/macos_litert_lm_prepare_app.sh @@ -47,7 +47,6 @@ required_libraries() { "libGemmaModelConstraintProvider.dylib" \ "libLiteRt.dylib" \ "libLiteRtLm.dylib" \ - "libLiteRtMetalAccelerator.dylib" \ "libLiteRtWebGpuAccelerator.dylib" \ "libwebgpu_dawn.dylib" ;; @@ -112,8 +111,8 @@ resolve_litert_dir() { fi local candidates=( - "$ROOT_DIR/.dart_tool/llamadart/litert_lm/0.19.1-2/macos_$LITERT_ARCH" - "$ROOT_DIR/.dart_tool/llamadart/litert_lm/0.19.1-2/macos/$LITERT_ARCH" + "$ROOT_DIR/.dart_tool/llamadart/litert_lm/0.19.1-3/macos_$LITERT_ARCH" + "$ROOT_DIR/.dart_tool/llamadart/litert_lm/0.19.1-3/macos/$LITERT_ARCH" ) local candidate for candidate in "${candidates[@]}"; do diff --git a/tool/native/sync_native_release_pins.py b/tool/native/sync_native_release_pins.py index 198a166c6..e8a28f404 100755 --- a/tool/native/sync_native_release_pins.py +++ b/tool/native/sync_native_release_pins.py @@ -225,8 +225,11 @@ "litert-lm-native-apple-CLiteRTLMMac-xcframework-{tag}.zip", "litert-lm-native-apple-LiteRtLm-xcframework-{tag}.zip", ) +LITERT_METAL_ACCELERATOR_SPM_ASSET_PATTERN = ( + "litert-lm-native-apple-LiteRtMetalAccelerator-xcframework-{tag}.zip" +) LITERT_V0_16_SPM_ASSET_PATTERNS = ( - "litert-lm-native-apple-LiteRtMetalAccelerator-xcframework-{tag}.zip", + LITERT_METAL_ACCELERATOR_SPM_ASSET_PATTERN, "litert-lm-native-apple-LiteRtTopKMetalSampler-xcframework-{tag}.zip", ) LITERT_V0_16_ALLOWED_SPM_COMPANION_ASSET_PATTERNS = ( @@ -435,6 +438,10 @@ def litert_spm_asset_patterns( "litert-lm-native-apple-LiteRtTopKMetalSampler-xcframework-{tag}.zip", "litert-lm-native-apple-LiteRtTopKWebGpuSampler-xcframework-{tag}.zip", } + if not allowed: + # v0.19 Apple runtimes link their GPU accelerator, so owner + # releases after v0.19.1-2 drop the unused Metal one (#1798). + obsolete.add(LITERT_METAL_ACCELERATOR_SPM_ASSET_PATTERN) patterns = [pattern for pattern in patterns if pattern not in obsolete] return tuple(patterns) @@ -557,7 +564,6 @@ def required_litert_manifest_paths( required.update( path for path in LITERT_V0_16_IOS_GPU_PATHS if version not in LITERT_V1_COMPATIBILITY_VERSIONS - or "TopKMetalSampler" not in path ) if official_assets and version >= (0, 14, 0): required.update( @@ -665,6 +671,18 @@ def litert_schema2_apple_targets( for bundle, libraries in bundles.items() if bundle.startswith("ios-") ): targets.add("GemmaModelConstraintProvider") + if _litert_is_v1_compatibility_tag(compatibility_tag): + ios_metal = { + "LiteRtMetalAccelerator" in libraries + for bundle, libraries in bundles.items() if bundle.startswith("ios-") + } + if ios_metal == {True}: + targets.add("LiteRtMetalAccelerator") + elif ios_metal != {False}: + raise ReleaseError( + f"LiteRT-LM {compatibility_tag} owner inventory ships the iOS " + "Metal accelerator for only some iOS bundles" + ) allowed_asset_names = { Path(path).name for path in litert_spm_artifact_paths( @@ -4137,6 +4155,7 @@ def replace_swift_binary_target_checksum( def _validate_litert_lm_v019_swift_topology( swift_text: str, *, current_tag: str, include_obsolete_samplers: bool, + include_metal_accelerator: bool = True, ) -> dict[str, str]: platforms = { "LiteRtLm": ".iOS, .macOS", @@ -4152,6 +4171,8 @@ def _validate_litert_lm_v019_swift_topology( if not include_obsolete_samplers: platforms.pop("LiteRtTopKMetalSampler") platforms.pop("LiteRtTopKWebGpuSampler") + if not include_metal_accelerator: + platforms.pop("LiteRtMetalAccelerator") label = "LiteRT-LM v0.19 Swift topology" assets = swift_native_repo_binary_targets( swift_text, tag_variable="liteRtLmTag", current_tag=current_tag, @@ -4231,6 +4252,26 @@ def _remove_litert_lm_obsolete_topk_samplers( return swift_text +def _remove_litert_lm_unused_metal_accelerator( + swift_text: str, *, current_tag: str, +) -> str: + blocks = _validate_litert_lm_v019_swift_topology( + swift_text, current_tag=current_tag, include_obsolete_samplers=False, + ) + if swift_text.count("LiteRtMetalAccelerator") != 3: + raise ReleaseError( + "LiteRT-LM v0.19 removal has ambiguous Metal accelerator references" + ) + swift_text = swift_text.replace(blocks["LiteRtMetalAccelerator"], "", 1) + return replace_one( + swift_text, + r'(?m)^[ \t]*\.target\(name: "LiteRtMetalAccelerator", ' + r'condition: \.when\(platforms: \[\.iOS\]\)\),\n', + "", + "LiteRT-LM unused Metal accelerator must have exactly one iOS dependency", + ) + + def prepare_litert_lm_package_swift( original_swift_text: str, *, @@ -4263,23 +4304,29 @@ def prepare_litert_lm_package_swift( "LiteRtLm", "CLiteRTLM", "CLiteRTLMMac", "LiteRtMetalAccelerator", "LiteRt", "LiteRtWebGpuAccelerator", "WebgpuDawn", } - if ( - _litert_is_v1_compatibility_tag(manifest["upstream"]["compatibilityTag"]) - and expected_targets == sampler_free_targets - and current_targets == expected_targets | obsolete_samplers - ): - swift_text = _remove_litert_lm_obsolete_topk_samplers( - swift_text, current_tag=original_tag, - ) - current_targets -= obsolete_samplers - if ( - _litert_is_v1_compatibility_tag(manifest["upstream"]["compatibilityTag"]) - and expected_targets == sampler_free_targets - and current_targets == expected_targets + metal = "LiteRtMetalAccelerator" + if _litert_is_v1_compatibility_tag( + manifest["upstream"]["compatibilityTag"] + ) and expected_targets in ( + sampler_free_targets, sampler_free_targets - {metal}, ): - _validate_litert_lm_v019_swift_topology( - swift_text, current_tag=original_tag, include_obsolete_samplers=False, - ) + if current_targets == expected_targets | {metal} | obsolete_samplers: + swift_text = _remove_litert_lm_obsolete_topk_samplers( + swift_text, current_tag=original_tag, + ) + current_targets -= obsolete_samplers + if metal not in expected_targets and current_targets == sampler_free_targets: + swift_text = _remove_litert_lm_unused_metal_accelerator( + swift_text, current_tag=original_tag, + ) + current_targets.discard(metal) + if current_targets == expected_targets: + _validate_litert_lm_v019_swift_topology( + swift_text, + current_tag=original_tag, + include_obsolete_samplers=False, + include_metal_accelerator=metal in expected_targets, + ) if current_targets == legacy_targets: if swift_text.count("GemmaModelConstraintProvider") != 3: raise ReleaseError( diff --git a/tool/native/test_sync_native_release_pins.py b/tool/native/test_sync_native_release_pins.py index d40275771..c5537e996 100644 --- a/tool/native/test_sync_native_release_pins.py +++ b/tool/native/test_sync_native_release_pins.py @@ -1,5 +1,6 @@ from __future__ import annotations +import copy import hashlib import http.client import io @@ -682,9 +683,18 @@ def _run_dart_inventory_checks( encoding="utf-8" ) ) + platforms = copy.deepcopy(projection["platforms"]) + for platform in platforms: + if platform["platform"] == "macos": + # The v0.19.0 projection predates the pin that stopped shipping it. + platform["artifactPaths"] = [ + path + for path in platform["artifactPaths"] + if "LiteRtMetalAccelerator" not in path + ] manifest = { "upstream": {"compatibilityTag": projection["compatibilityTag"]}, - "platforms": projection["platforms"], + "platforms": platforms, # The pinned release decides which owner files its hosts load. "release": {"tag": pinned_tag}, } @@ -1206,6 +1216,68 @@ def test_v0191_reuses_the_reviewed_sampler_free_topology(self) -> None: "v0.19.1", self.tag, official_assets=False, ))) + def drop_metal_accelerator(self, *bundles: str) -> None: + for item in self.manifest["platforms"]: + if f'{item["platform"]}-{item["arch"]}' in bundles: + item["artifactPaths"] = [ + path for path in item["artifactPaths"] if "MetalAccelerator" not in path + ] + self.release["assets"] = [item for item in self.release["assets"] + if item["name"] != self.asset("LiteRtMetalAccelerator")["name"]] + + def test_metal_free_owner_removes_the_metal_target_and_is_idempotent(self) -> None: + with_metal = self.prepare(self.original) + self.drop_metal_accelerator("ios-arm64", "ios-arm64-sim", "macos-arm64") + for label, source in (("nine-target", self.original), ("seven-target", with_metal)): + with self.subTest(source=label): + prepared = self.prepare(source) + self.assertNotIn("LiteRtMetalAccelerator", prepared) + self.assertNotIn("TopK", prepared) + self.assertEqual({name for name, _ in pins.swift_native_repo_binary_targets( + prepared, tag_variable="liteRtLmTag", current_tag=self.tag, + )}, self.targets - {"LiteRtMetalAccelerator"}) + block = re.search( + r'(?m)^[ \t]*nativeRepoBinaryTarget\(\s*name: "LiteRtMetalAccelerator",.*?^[ \t]*\),\n', + with_metal, re.DOTALL, + ).group() + self.assertEqual(prepared, with_metal.replace(block, "").replace( + ' .target(name: "LiteRtMetalAccelerator", ' + 'condition: .when(platforms: [.iOS])),\n', "")) + self.assertEqual(self.prepare(prepared), prepared) + + def test_metal_accelerator_is_optional_from_v019(self) -> None: + for compatibility in ("v0.19.0", "v0.19.1"): + with self.subTest(version=compatibility): + self.assertFalse(any("Metal" in path for path in pins.required_litert_manifest_paths( + compatibility, self.tag, official_assets=False, + ))) + spm = {item["path"] for item in self.projection["spmArtifacts"]} + for paths in (spm, {path for path in spm if "MetalAccelerator" not in path}): + pins.validate_litert_spm_artifact_inventory( + paths, compatibility_tag=compatibility, release_tag=self.tag, + ) + required = pins.required_litert_manifest_paths("v0.18.1", self.tag, official_assets=False) + self.assertIn("bin/ios/arm64/LiteRtMetalAccelerator.framework/LiteRtMetalAccelerator", required) + self.assertIn(f"dist/spm/{self.tag}/" + self.asset("LiteRtMetalAccelerator")["name"], required) + + def test_metal_accelerator_on_only_some_ios_bundles_refuses(self) -> None: + self.drop_metal_accelerator("ios-arm64-sim") + with self.assertRaisesRegex(ReleaseError, "only some iOS bundles"): + self.prepare(self.original) + + def test_metal_removal_refuses_ambiguous_metal_references(self) -> None: + with_metal = self.prepare(self.original) + self.drop_metal_accelerator("ios-arm64", "ios-arm64-sim", "macos-arm64") + dependency = '.target(name: "LiteRtMetalAccelerator", condition: .when(platforms: [.iOS])),' + for label, source in ( + ("extra-reference", with_metal + "\n// LiteRtMetalAccelerator"), + ("duplicate-dependency", with_metal.replace(dependency, dependency * 2)), + ("missing-dependency", with_metal.replace(dependency, "")), + ("wrong-platform", with_metal.replace(dependency, dependency.replace(".iOS", ".macOS"))), + ): + with self.subTest(source=label), self.assertRaises(ReleaseError): + self.prepare(source) + def test_malformed_or_ambiguous_topology_refuses(self) -> None: for sampler in self.samplers: block, dependency = self.binary_block(sampler), self.dependency(sampler) diff --git a/website/docs/changelog/recent-releases.md b/website/docs/changelog/recent-releases.md index 98fbca742..ede8caae0 100644 --- a/website/docs/changelog/recent-releases.md +++ b/website/docs/changelog/recent-releases.md @@ -13,6 +13,7 @@ validate against the exact release tag notes. ## Unreleased +- Stop shipping the unused LiteRT-LM Metal accelerator on iOS and macOS: native LiteRT-LM moves to `v0.19.1-3`, whose `LiteRtLm` has its GPU accelerator built in, so GPU generation is unchanged while the macOS hook bundle drops 8.7 MB and iOS apps 8.1 MB, and the iOS LiteRT-LM frameworks no longer use the User Defaults required-reason API ([#1798](https://github.com/leehack/llamadart/issues/1798)). - `engine.getContextSize()` on LiteRT-LM Web `withTextHistory()` loads returns the context size the runtime's executor enforces when its load probe finds one below the requested size, such as Qwen3 0.6B's 4096 on WASM CPU, instead of the requested `ModelParams.contextSize` ([#1861](https://github.com/leehack/llamadart/issues/1861)). - Parse Qwen3-Coder XML tool calls as llama.cpp does: a call needs a line break after each tag and around each value (a CRLF or compact `` not followed by a line break. Nemotron 3 Nano now parses as Qwen3-Coder XML, so a `` also ends its thought, and the tool-call grammar of these templates now requires that shape and starts at `` ([#1898](https://github.com/leehack/llamadart/issues/1898)). - Parse Seed-OSS tool calls closer to llama.cpp: a finished reply whose last `` scope is never closed keeps a call complete through `` and drops one cut off before it, an argument declared as a string keeps its value as generated instead of being read as JSON (`\n42\n` stays a string), and with `ToolChoice.required` the grammar lets a thought and text come before the calls ([#1898](https://github.com/leehack/llamadart/issues/1898)). diff --git a/website/docs/maintainers/native-and-web-sync.md b/website/docs/maintainers/native-and-web-sync.md index 6013400ce..001bf0f01 100644 --- a/website/docs/maintainers/native-and-web-sync.md +++ b/website/docs/maintainers/native-and-web-sync.md @@ -167,6 +167,13 @@ Metal accelerator and other runtime targets. Versions 0.16 through 0.18 retain their prebuilt sampler requirements. This removal does not apply to other versions or layouts, which need a reviewed transition before pin files change. +v0.19 Apple `LiteRtLm` also links its GPU accelerator, so LiteRT never opens +the prebuilt Metal one, and owner releases after `v0.19.1-2` stop shipping it +([#1798](https://github.com/leehack/llamadart/issues/1798)). For v0.19 the +iOS bundles decide: sync keeps the `LiteRtMetalAccelerator` target while every +iOS bundle ships it, removes it from the seven-target layout once none does, +and refuses a release that ships it for only some iOS bundles. + LiteRT-LM sync also stages the hook's accepted full-manifest template hash from the same rendered `Package.swift`. The original manifest must already match the hook's reviewed template; a stale hash or a hand-edited manifest fails before any @@ -393,8 +400,8 @@ parameters. The public lazy facade, service metadata/finish probes and speech worker share discovery and ABI selection. `_liteRtLmV1ProductionSelection` in `litert_lm_runtime_facade.dart` enables public C API 1.0.x dispatch. The native hook and Apple companion source now pin published -[v0.19.1-2](https://github.com/leehack/litert-lm-native/releases/tag/v0.19.1-2), -owner `0b15fb34` / upstream `7a03724c`, which keeps C API 1.0.0 and the +[v0.19.1-3](https://github.com/leehack/litert-lm-native/releases/tag/v0.19.1-3), +owner `ed8ffe33` / upstream `7a03724c`, which keeps C API 1.0.0 and the `v0.19.0` exports and adds the in-process LiteRT marker `litert_lm_native_in_process_litert_abi_version` (1 on every platform); the `LiteRT-LM Smoke` library step fails when the bundled Linux x64 or Windows @@ -441,7 +448,7 @@ blocks unload and dispose. On Linux ext4, releasing a freshly written CPU XNNPACK weight cache flushes it to disk, about 4 s for Qwen3 0.6B's 600 MB cache on CI disks ([#1814](https://github.com/leehack/llamadart/issues/1814)). -Public ABI-aware selection and the v0.19.1-2 source pin are enabled; remaining +Public ABI-aware selection and the v0.19.1-3 source pin are enabled; remaining consumer, feature and platform gates are recorded in `doc/litert_lm_v1_adoption.md`. This transport contract alone does not qualify a model or device. The owner proxy's diff --git a/website/docs/platforms/apple-setup.md b/website/docs/platforms/apple-setup.md index 5e3d6415b..465a2f42d 100644 --- a/website/docs/platforms/apple-setup.md +++ b/website/docs/platforms/apple-setup.md @@ -30,9 +30,9 @@ Add the companions you need next to `llamadart`; the dependency snippet is in - `llamadart_llama_cpp_flutter` links the llama.cpp XCFrameworks (GGUF). - `llamadart_litert_lm_flutter` links the LiteRT-LM iOS and macOS - XCFrameworks (`.litertlm` models), including the macOS WebGPU accelerator, - sampler and Dawn. The Metal accelerator and sampler stay iOS-only; the macOS - GPU path uses WebGPU. + XCFrameworks (`.litertlm` models), including the macOS WebGPU accelerator + and Dawn. `LiteRtLm` has its GPU accelerator built in (Metal on iOS, WebGPU + on macOS), so the companion links no separate Metal accelerator. - `llamadart_stable_diffusion_flutter` links the stable-diffusion.cpp XCFramework for [image generation](../guides/image-generation). It is opt-in: adding it opts iOS and macOS builds into the image generation runtime (about @@ -197,10 +197,9 @@ Apple's requirement covers iOS, iPadOS, tvOS, visionOS and watchOS apps. The app-level declaration is a workaround for a runtime without its own manifest; llamadart has not verified it with an App Store Connect upload. -The LiteRT-LM iOS frameworks reference required-reason API in three -categories: File Timestamp (`stat`, `fstat`, `lstat`), System Boot Time -(`mach_absolute_time`) and User Defaults (`NSUserDefaults`, in the Metal -accelerator and sampler). Each framework +The LiteRT-LM iOS frameworks reference required-reason API in two +categories: File Timestamp (`stat`, `fstat`, `lstat`) and System Boot Time +(`mach_absolute_time`). Each framework the companion links on iOS carries a manifest for what that framework references: @@ -208,18 +207,19 @@ references: | --- | --- | --- | | `LiteRtLm` | `NSPrivacyAccessedAPICategoryFileTimestamp` | `C617.1`, `3B52.1` | | `LiteRtLm` | `NSPrivacyAccessedAPICategorySystemBootTime` | `35F9.1` | -| `LiteRtMetalAccelerator` | `NSPrivacyAccessedAPICategoryFileTimestamp` | `C617.1`, `3B52.1` | -| `LiteRtMetalAccelerator` | `NSPrivacyAccessedAPICategoryUserDefaults` | `CA92.1` | -| `LiteRtTopKMetalSampler` | `NSPrivacyAccessedAPICategoryUserDefaults` | `CA92.1` | | `CLiteRTLM` | None: its manifest declares no API | | -An app declares those three categories, with the same reasons, in its own +LiteRT-LM runtimes up to `v0.19.1-2` also ship a `LiteRtMetalAccelerator` +framework that declares File Timestamp and User Defaults (`CA92.1`); apps that +pin one of them declare User Defaults too. + +An app declares those categories, with the same reasons, in its own `PrivacyInfo.xcprivacy` when it runs LiteRT-LM on the hook path, or when the built app has no `PrivacyInfo.xcprivacy` inside the embedded LiteRT-LM frameworks. On macOS, which Apple's requirement does not cover, the companion's `LiteRtLm.framework` carries a File Timestamp manifest, while its flat dylibs -(`libCLiteRTLM_mac`, `libLiteRt`, the WebGPU accelerator and sampler, and +(`libCLiteRTLM_mac`, `libLiteRt`, the WebGPU accelerator and `libwebgpu_dawn`) and the hook-managed runtime dylibs are bare libraries without one. The reasons come from the runtime owner's [audit of the binaries](https://github.com/leehack/litert-lm-native/blob/v0.17.0-8/docs/apple_privacy_manifest.md) diff --git a/website/docs/platforms/native-build-hooks.md b/website/docs/platforms/native-build-hooks.md index 213444945..53e5721f2 100644 --- a/website/docs/platforms/native-build-hooks.md +++ b/website/docs/platforms/native-build-hooks.md @@ -101,7 +101,7 @@ hooks: ABI checks and requires macOS 14+ before loading even an explicit path or legacy library; public selection admits C API 1.0.x while unknown/future versions remain unsupported. The hook and Apple companion pin published - native `v0.19.1-2`. + native `v0.19.1-3`. - An empty or all-unknown selection falls back to `llama_cpp` and `litert_lm`; one that leaves no runtime, such as `none`, fails the build. diff --git a/website/docs/platforms/support-matrix.md b/website/docs/platforms/support-matrix.md index f2e7ef375..a7cdca9f8 100644 --- a/website/docs/platforms/support-matrix.md +++ b/website/docs/platforms/support-matrix.md @@ -218,20 +218,20 @@ web are on [Known platform issues](./known-platform-issues). | Runtime | Pinned release | | --- | --- | | llama.cpp native | `leehack/llamadart-native@v0.9.0` | -| LiteRT-LM native | `leehack/litert-lm-native@v0.19.1-2` | +| LiteRT-LM native | `leehack/litert-lm-native@v0.19.1-3` | | stable-diffusion.cpp native (opt-in, Preview) | `leehack/stable-diffusion-native@v0.3.0-3`, for [image generation](../guides/image-generation); see [Opt-in stable_diffusion runtime](./native-build-hooks#opt-in-stable_diffusion-runtime-experimental). Flutter iOS/macOS apps link its XCFramework through `llamadart_stable_diffusion_flutter` | | WebGPU bridge assets | `leehack/llama-web-bridge-assets`; see [Pinned bridge assets](./webgpu-bridge#pinned-bridge-assets) | The native-assets hook currently pins `llamadart-native` tag `v0.9.0` and -`litert-lm-native` release `v0.19.1-2` (`lib/src/hook/native_release_pins.dart`). +`litert-lm-native` release `v0.19.1-3` (`lib/src/hook/native_release_pins.dart`). The direct native client, service probes and speech worker share ABI-aware discovery. Public selection admits legacy runtimes and C API 1.0.x; unknown or future versions are refused before engine calls. The pinned runtime is published native -[v0.19.1-2](https://github.com/leehack/litert-lm-native/releases/tag/v0.19.1-2) -(owner `0b15fb34`, upstream `7a03724c`). The selected loader requires macOS 14+ +[v0.19.1-3](https://github.com/leehack/litert-lm-native/releases/tag/v0.19.1-3) +(owner `ed8ffe33`, upstream `7a03724c`). The selected loader requires macOS 14+ before loading any core or companion library, including legacy libraries and custom paths. Web uses `@litert-lm/core@0.19.0`. The Flutter companion's iOS 16.4 floor remains separate.