diff --git a/CHANGELOG.md b/CHANGELOG.md index 471dfe6a0..43f80ad4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,12 +2,13 @@ Releases before 0.10.0 are in the [changelog archive](https://github.com/leehack/llamadart/blob/main/doc/changelog_archive.md). ## Unreleased +- Adopt stable-diffusion `v0.3.0-4`: on iOS Metal the runtime splits graphs, so a full VAE decodes without tripping the GPU watchdog and `ImageGenerationEngine.load` no longer refuses it there; a runtime without the split keeps the refusal. Run on an iPhone 16 Pro and an iPhone 17 Pro (30 of 30 soak decodes). On iOS 26 the second engine dispose can leave about 2 GB resident ([#1047](https://github.com/leehack/llamadart/issues/1047), [#2004](https://github.com/leehack/llamadart/issues/2004)). - Chat app microphone: a microphone that disconnects mid-recording now transcribes the audio captured so far with a warning, a missing input device on Web is reported as such instead of as a permission denial, and a failed Web start releases the microphone ([#329](https://github.com/leehack/llamadart/issues/329)). - Aligned the default WebGPU bridge assets to `v0.1.85`: a page without cross-origin isolation now fails at once with a `LlamaUnsupportedException` naming COOP/COEP `require-corp` instead of waiting minutes for a generic init timeout, and Firefox CPU runs (`gpuLayers: 0`) no longer create the WebGPU backend (stories15M, 32 tokens: 640 to 762 tok/s against 10 before) ([#1025](https://github.com/leehack/llamadart/issues/1025)). - `ComputeDevice.gpu` on Web now names why WebGPU was not used (for example `adapter_missing_shader_f16`: Chrome on Linux with NVIDIA needs driver 615.71 or newer); documented the measured Safari 27, Firefox 160 and NVIDIA L4 results ([#1025](https://github.com/leehack/llamadart/issues/1025)). -Runtime baseline: native llama.cpp `v0.9.0` and Web use upstream `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`; native LiteRT-LM uses `v0.19.1-4` and stable-diffusion uses `v0.3.0-3`. Web LiteRT-LM uses `@litert-lm/core@0.19.0`. Manifest SHA-256: `6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3`. +Runtime baseline: native llama.cpp `v0.9.0` and Web use upstream `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`; native LiteRT-LM uses `v0.19.1-4` and stable-diffusion uses `v0.3.0-4`. Web LiteRT-LM uses `@litert-lm/core@0.19.0`. Manifest SHA-256: `6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3`. ## 0.19.9 diff --git a/README.md b/README.md index 565deb916..d9e011dd0 100644 --- a/README.md +++ b/README.md @@ -146,10 +146,9 @@ creation fails, `ComputeDevice.auto` retries on the CPU with a warning; Image generation is not available on the web. On iOS, use `ImageModelPlacement(vaeOnCpu: true)` to keep diffusion on Metal, or CPU -generation. Metal VAE decoding needs a native context policy that preserves -an intrinsic SDXS tiny decoder through model refresh. The pinned -`v0.3.0-3` runtime provides that policy; full VAE and external-decoder -Metal decoding remain refused on iOS. This is not new physical-device qualification. +generation. The pinned `v0.3.0-4` runtime splits Metal graphs, so a full VAE +decodes on iOS Metal; a runtime without the split keeps the intrinsic SDXS +tiny-decoder policy and refuses full-VAE Metal decoding. Per-target support, device limitations and runtime options: - [Platform and backend support matrix](https://llamadart.leehack.com/docs/platforms/support-matrix), including diff --git a/doc/cross_platform_validation.md b/doc/cross_platform_validation.md index 8ea0852bd..b07a5c4ac 100644 --- a/doc/cross_platform_validation.md +++ b/doc/cross_platform_validation.md @@ -919,9 +919,11 @@ a follow-up. Do not insert requested bundle metadata as observed runtime evidenc checks: `image-sdxs-cpu` (Android, CPU) and `image-sdxs-metal` (iOS, Metal). The stable-diffusion-native `v0.3.0-2` pin provides the intrinsic tiny-decoder v1 context policy required by the iOS Metal SDXS profile. Its physical-device -qualification must still use this exact runtime and consumer build. Full-VAE -and external-decoder Metal paths remain refused; use CPU VAE placement for -those models ([#1047](https://github.com/leehack/llamadart/issues/1047)). +qualification must still use this exact runtime and consumer build. The +`v0.3.0-4` graph split lifts the full-VAE refusal on iOS Metal (see +[iOS full-VAE graph split](#ios-full-vae-graph-split-1047-2026-11-13)); a +runtime without it still refuses, and needs CPU VAE placement +([#1047](https://github.com/leehack/llamadart/issues/1047)). They run the chat-app test `integration_test/image_generation_e2e_test.dart` (SDXS download through the image screen, one seeded 512x512 image, and a @@ -2341,3 +2343,29 @@ unqualified, as do accelerators, other models and physical devices. This covers one device, model and quantization and qualifies no other GPU. See [Known platform issues](../website/docs/platforms/known-platform-issues.md). + +## iOS full-VAE graph split (#1047, 2026-11-13) + +stable-diffusion-native `v0.3.0-4` (native commit `4a84ac24`, manifest SHA-256 +`86c674911194b5b59f2e78f1906052af2df899ca60de56881690efcf0a6833a1`) splits +Metal graphs on iOS through upstream's eval callback at 2^26 elements and +exports `sd_dart_graph_split_set_threshold`, `sd_dart_graph_split_threshold` +and `sd_dart_graph_split_cuts`. The core lifts its full-VAE iOS Metal refusal +when the threshold getter exists and returns above 0. + +DDP ran the XCFramework (zip SHA-256 `0d022638f54be80aa75fd8e73a94d139ad8887f4f3c73084501369ca5e7ac354`) +with the SD-Turbo full-VAE harness: + +| Run | Devices | Result | +| --- | --- | --- | +| C1, split arms | iPhone 16 Pro, iOS 18.3 | every arm passed 3 of 3 decodes | +| C1, split off | iPhone 16 Pro, iOS 18.3 | third decode failed with the GPU command-buffer error (watchdog) | +| C2b soak | iPhone 16 Pro, iPhone 17 Pro | 30 of 30 decodes passed, one image hash, flat footprint | + +Known limit ([#2004](https://github.com/leehack/llamadart/issues/2004)): on +the iPhone 17 Pro (iOS 26.5.1) the second engine dispose kept about 2 GB +resident and later loads failed the memory check; the iPhone 16 Pro returned to +its baseline. Whether it is a leak or delayed reclaim is undecided. + +Final qualification of the public API at the PR head and the published pin is a +separate DDP run (`r1047-final`). diff --git a/example/chat_app/README.md b/example/chat_app/README.md index 0a2d0c39a..aa544acb5 100644 --- a/example/chat_app/README.md +++ b/example/chat_app/README.md @@ -306,7 +306,7 @@ layouts) opens a separate screen for the experimental `ImageGenerationEngine` runtime…** meanwhile. With an empty Metal shader cache the first check compiles ggml's Metal library, about 16 s on an M4 Max; with the cache warm it takes under 0.5 s. -- On iOS, the pinned SD runtime preserves the intrinsic SDXS tiny decoder across model refresh. Full VAE and external-decoder recipes require CPU VAE placement or CPU generation; this update does not add physical-iOS qualification. +- On iOS, the pinned SD runtime (`v0.3.0-4`) splits Metal graphs, so full-VAE recipes decode on the GPU; the app has no iOS-specific gating. A runtime without the split refuses them and needs CPU VAE placement or CPU generation. On iOS 26 the second engine dispose can leave about 2 GB resident ([#2004](https://github.com/leehack/llamadart/issues/2004)). - Models: **SDXS-512** (683 MB, recommended, fits most phones) and **SD-Turbo + TAESD** (2.0 GB). Both download into the app's model cache from pinned Hugging Face revisions with exact sizes and SHA-256 checks; an diff --git a/lib/src/backends/stable_diffusion/stable_diffusion_bindings.dart b/lib/src/backends/stable_diffusion/stable_diffusion_bindings.dart index 79226ed6c..5c34eb817 100644 --- a/lib/src/backends/stable_diffusion/stable_diffusion_bindings.dart +++ b/lib/src/backends/stable_diffusion/stable_diffusion_bindings.dart @@ -358,6 +358,15 @@ external int sd_dart_gpu_device_memory( ffi.Pointer out, ); +@ffi.Native() +external int sd_dart_graph_split_cuts(); + +@ffi.Native() +external void sd_dart_graph_split_set_threshold(int elements); + +@ffi.Native() +external int sd_dart_graph_split_threshold(); + @ffi.Native, ffi.Size)>() external int sd_dart_last_error(ffi.Pointer text, int capacity); diff --git a/lib/src/backends/stable_diffusion/stable_diffusion_calls.dart b/lib/src/backends/stable_diffusion/stable_diffusion_calls.dart index d6aec229f..ae3b126c4 100644 --- a/lib/src/backends/stable_diffusion/stable_diffusion_calls.dart +++ b/lib/src/backends/stable_diffusion/stable_diffusion_calls.dart @@ -27,6 +27,7 @@ typedef _CancelGenerationNative = Void Function(Pointer, UnsignedInt); typedef _ExitFreeNative = Void Function(Pointer); typedef _GpuDeviceCountNative = Int32 Function(); +typedef _GraphSplitThresholdNative = Uint64 Function(); typedef _GpuDeviceMemoryNative = Int32 Function(Int32, Pointer); typedef _LogEnableNative = Void Function(); @@ -65,7 +66,8 @@ typedef _FreeImageGenerationResultNative = /// added. Each group is bound only when the runtime exports all of it, and is /// `null` otherwise: a runtime between the two releases still generates /// images, without the runtime's log and its reason for a failed load, and -/// without the memory check of a Vulkan GPU. +/// without the memory check of a Vulkan GPU. [graphSplit] is the same for +/// [graphSplitNativeRelease]'s Metal graph splitting. /// /// `sd_dart_exit_teardown`, `sd_dart_exit_call_begin`, /// `sd_dart_exit_call_end`, `sd_dart_exit_track` and `sd_dart_exit_untrack` @@ -94,6 +96,7 @@ final class StableDiffusionCalls { required this.freeImages, this.log, this.gpu, + this.graphSplit, this.loadsFp8AsF16OnMetal = false, this.newContextIntrinsicTinyV1, }); @@ -131,6 +134,16 @@ final class StableDiffusionCalls { 'sd_dart_gpu_device_memory', ]; + /// The `stable-diffusion-native` release that added [graphSplitSymbols]. + static const String graphSplitNativeRelease = 'v0.3.0-4'; + + /// The `sd_dart_` functions of [graphSplit] that [tryResolve] binds. The + /// release also exports `sd_dart_graph_split_set_threshold` and + /// `sd_dart_graph_split_cuts`, which Dart does not call. + static const List graphSplitSymbols = [ + 'sd_dart_graph_split_threshold', + ]; + /// The first `stable-diffusion-native` release that loads FP8 weights a /// Metal backend computes with as F16 instead of aborting the process /// (leehack/stable-diffusion-native#29). @@ -225,6 +238,10 @@ final class StableDiffusionCalls { Native.addressOf>( sd.sd_dart_gpu_device_memory, ), + 'sd_dart_graph_split_threshold' => + Native.addressOf>( + sd.sd_dart_graph_split_threshold, + ), 'free_sd_image_generation_result' => Native.addressOf>( sd.free_sd_image_generation_result, @@ -307,6 +324,13 @@ final class StableDiffusionCalls { ).asFunction(), ), ), + graphSplit: optional( + () => StableDiffusionGraphSplitCalls( + threshold: function<_GraphSplitThresholdNative>( + 'sd_dart_graph_split_threshold', + ).asFunction(), + ), + ), loadsFp8AsF16OnMetal: _releaseAtLeast(releaseTag, fp8MetalNativeRelease) && (optional(() => fp8MetalSymbols.map(symbol).toList()) != null), @@ -398,6 +422,10 @@ final class StableDiffusionCalls { /// [optionalNativeRelease]. final StableDiffusionGpuCalls? gpu; + /// The runtime's Metal graph-split query, or `null` on a runtime older than + /// [graphSplitNativeRelease]. + final StableDiffusionGraphSplitCalls? graphSplit; + /// Whether the runtime loads FP8 weights a Metal backend computes with as /// F16, so FP8 weights can reach a Metal context without aborting the /// process. @@ -435,6 +463,22 @@ final class StableDiffusionGpuCalls { deviceMemory; } +/// The Metal graph-split query of the stable_diffusion runtime. +/// +/// On iOS Metal the runtime commits a graph in several command buffers, so +/// that no single one of a full-VAE decode outlasts the GPU watchdog. +final class StableDiffusionGraphSplitCalls { + /// Creates the calls from functions. + const StableDiffusionGraphSplitCalls({required this.threshold}); + + /// `sd_dart_graph_split_threshold`: the number of tensor elements after + /// which a graph is split, or 0 when it is not split. Returns at once. + final int Function() threshold; + + /// Whether the runtime splits graphs. + bool get isActive => threshold() > 0; +} + /// The log recorder of the stable_diffusion runtime: the library copies each /// message of stable-diffusion.cpp and ggml into a buffer of its own and Dart /// reads them, so Dart gives the runtime no log callback either. diff --git a/lib/src/backends/stable_diffusion/stable_diffusion_runtime_io.dart b/lib/src/backends/stable_diffusion/stable_diffusion_runtime_io.dart index 55883b1ea..3820f536c 100644 --- a/lib/src/backends/stable_diffusion/stable_diffusion_runtime_io.dart +++ b/lib/src/backends/stable_diffusion/stable_diffusion_runtime_io.dart @@ -34,6 +34,10 @@ abstract interface class StableDiffusionNativeApi { /// [StableDiffusionCalls.loadsFp8AsF16OnMetal] of the runtime. bool loadsFp8AsF16OnMetal(); + /// Whether the runtime exports [StableDiffusionCalls.graphSplit] and + /// reports a threshold above 0. + bool splitsGraphsOnMetal(); + /// Whether the runtime enforces the intrinsic tiny-decoder v1 contract. bool supportsIntrinsicTinyDecoderV1(); } @@ -182,6 +186,7 @@ StableDiffusionRuntimeStatus probeStableDiffusionRuntime({ commit: commit, devices: devices, loadsFp8AsF16OnMetal: api.loadsFp8AsF16OnMetal(), + splitsGraphsOnMetal: api.splitsGraphsOnMetal(), supportsIntrinsicTinyDecoderV1: api.supportsIntrinsicTinyDecoderV1(), ); } on ArgumentError catch (error) { @@ -404,6 +409,9 @@ final class BundledStableDiffusionApi implements StableDiffusionNativeApi { @override bool loadsFp8AsF16OnMetal() => resolveCalls()?.loadsFp8AsF16OnMetal ?? false; + @override + bool splitsGraphsOnMetal() => resolveCalls()?.graphSplit?.isActive ?? false; + @override bool supportsIntrinsicTinyDecoderV1() => resolveCalls()?.newContextIntrinsicTinyV1 != null; diff --git a/lib/src/backends/stable_diffusion/stable_diffusion_runtime_status.dart b/lib/src/backends/stable_diffusion/stable_diffusion_runtime_status.dart index 24f82492f..848e19812 100644 --- a/lib/src/backends/stable_diffusion/stable_diffusion_runtime_status.dart +++ b/lib/src/backends/stable_diffusion/stable_diffusion_runtime_status.dart @@ -31,6 +31,11 @@ final class StableDiffusionRuntimeStatus { /// weights abort the process when a Metal context is created. final bool loadsFp8AsF16OnMetal; + /// Whether the runtime splits large graphs on Metal, as + /// `StableDiffusionCalls.graphSplit` reports: a full-VAE decode then stays + /// under the iOS GPU watchdog. + final bool splitsGraphsOnMetal; + /// Whether a native context can enforce the intrinsic tiny-decoder v1 /// policy through every automatic model refresh. final bool supportsIntrinsicTinyDecoderV1; @@ -41,6 +46,7 @@ final class StableDiffusionRuntimeStatus { required String this.commit, required this.devices, this.loadsFp8AsF16OnMetal = false, + this.splitsGraphsOnMetal = false, this.supportsIntrinsicTinyDecoderV1 = false, }) : unavailableReason = null; @@ -52,6 +58,7 @@ final class StableDiffusionRuntimeStatus { commit = null, devices = const [], loadsFp8AsF16OnMetal = false, + splitsGraphsOnMetal = false, supportsIntrinsicTinyDecoderV1 = false; /// Whether the runtime loaded and answered the probe. diff --git a/lib/src/core/image/image_generation_engine.dart b/lib/src/core/image/image_generation_engine.dart index 91cba3295..eb5b16c1a 100644 --- a/lib/src/core/image/image_generation_engine.dart +++ b/lib/src/core/image/image_generation_engine.dart @@ -417,8 +417,9 @@ class ImageGenerationEngine /// unused here. Also when /// [ImageModelParams.vaeDirectConvolution] is `true` on iOS with the /// Metal backend, before any file is resolved or loaded. Full-VAE Metal - /// decoding on iOS is also refused. Known full-VAEs are rejected before - /// native creation; tiny candidates require a native context policy that + /// decoding on iOS is also refused unless the runtime splits graphs + /// (stable-diffusion-native v0.3.0-4 or later). Known full-VAEs are + /// rejected before native creation; tiny candidates require a native context policy that /// preserves the intrinsic decoder through automatic source refreshes. Use /// [ImageModelPlacement.vaeOnCpu] or [ComputeDevice.cpu] for other recipes, /// including external tiny decoders. Tiling does not bypass this restriction. @@ -645,12 +646,15 @@ class ImageGenerationEngine final requiresIosTinyDecoder = driver.operatingSystem == 'ios' && _isMetal(backendName) && - !placement.vaeOnCpu; + !placement.vaeOnCpu && + !runtime.splitsGraphsOnMetal; LlamaUnsupportedException unsupportedIosDecoder() => LlamaUnsupportedException( 'Full-VAE decoding on iOS with the Metal backend ($backendName) is ' - 'not supported: repeated decodes can trip the GPU watchdog ' - '(https://github.com/leehack/llamadart/issues/1047). GPU decoding ' + 'not supported on this runtime: repeated decodes can trip the GPU ' + 'watchdog (https://github.com/leehack/llamadart/issues/1047). ' + 'stable-diffusion-native v0.3.0-4 or later splits the graph and ' + 'decodes a full VAE; this runtime does not. Otherwise GPU decoding ' 'requires an intrinsic SDXS tiny decoder and a selected runtime with ' 'the intrinsic tiny-decoder v1 context policy, so model refresh cannot ' 'switch to a full VAE. External tiny decoders can ' diff --git a/lib/src/hook/native_release_pins.dart b/lib/src/hook/native_release_pins.dart index 9a5c40d26..96e532968 100644 --- a/lib/src/hook/native_release_pins.dart +++ b/lib/src/hook/native_release_pins.dart @@ -217,10 +217,10 @@ const liteRtLmBundleSpecs = [ /// `leehack/stable-diffusion-native` release tag downloaded for the opt-in /// stable-diffusion.cpp runtime. -const stableDiffusionReleaseTag = 'v0.3.0-3'; +const stableDiffusionReleaseTag = 'v0.3.0-4'; /// First Apple companion release containing [stableDiffusionReleaseTag]. -const stableDiffusionAppleCompanionVersion = '0.18.0'; +const stableDiffusionAppleCompanionVersion = '0.19.10'; /// Explicitly qualified older Apple SD pins that preserve existing app locks. /// @@ -232,13 +232,13 @@ const stableDiffusionCompatibleAppleCompanionTags = {}; /// Checksums of the exact published XCFrameworks accepted by the SD hook. const stableDiffusionAppleCompanionChecksums = { - 'v0.3.0-3': - '8f35c67f72d1d4bf86d31f20f3437d04baaf3b6ac7389908000fd166e387783f', + 'v0.3.0-4': + '0d022638f54be80aa75fd8e73a94d139ad8887f4f3c73084501369ca5e7ac354', }; /// stable_diffusion cache directory version derived from /// [stableDiffusionReleaseTag]. -const stableDiffusionVersion = '0.3.0-3'; +const stableDiffusionVersion = '0.3.0-4'; /// A published stable-diffusion-native runtime archive and the library it must /// contain under `lib/`. @@ -265,62 +265,62 @@ class StableDiffusionBundleSpec { const stableDiffusionBundleSpecs = [ StableDiffusionBundleSpec( 'android-arm64', - sha256: '0ba2a126bed223e2783736791ddcacaec2545c34d55b6358eea4afc7827889db', + sha256: 'fdc1a614bc7b50d838dfee4416a8ffea04eb01a0042f2d65821e80225e4000cd', requiredLibraries: {'libstable-diffusion.so'}, ), StableDiffusionBundleSpec( 'ios-arm64', - sha256: 'f91ad7b00592385e6a6e1a1af3989255886a08f0f22ea3c242da2de66a68a38a', + sha256: 'c24c7df0a7c21370d026668cd12a98baf646d7951829088e6957f25ad13cdce9', requiredLibraries: {'libstable-diffusion.dylib'}, ), StableDiffusionBundleSpec( 'ios-arm64-sim', - sha256: '6f4badad4a56ac3c6bfa696a8184cd0d4f4a12f2f185fa2ddc84183665532d61', + sha256: 'be5ab833a7a902ba75bce16c3020565372bef672106e5b91a2d9bcb7e85d0fb2', requiredLibraries: {'libstable-diffusion.dylib'}, ), StableDiffusionBundleSpec( 'ios-x64-sim', - sha256: 'a969b7ed0e27f9099e5abfc46d2136f0c956382cd638332d2b207380221692bf', + sha256: 'eae74517da056d3cd44964a5ada097b4ec58158be4978721efdb9801d90e7540', requiredLibraries: {'libstable-diffusion.dylib'}, ), StableDiffusionBundleSpec( 'macos-arm64', - sha256: '2a198413707f8a6a70cd11dfc97d5fb8e6979f7fd389efdc9afa2e6c0bf0f6b5', + sha256: 'd34dbaf3c9ecfbbaa97e1c89687e27cefdae824ed27a1a8cf694a9c9489d3eae', requiredLibraries: {'libstable-diffusion.dylib'}, ), StableDiffusionBundleSpec( 'macos-x64', - sha256: '0b72b8c5a6dbf87ce630e1c3a7af9129c770481e3813d7623c23ae8afdce1d1b', + sha256: '7f5954264d0e904a5049822e320eb6b06696c38a59428868537915b439eec478', requiredLibraries: {'libstable-diffusion.dylib'}, ), StableDiffusionBundleSpec( 'linux-arm64', - sha256: '71f59f5ba02d3ef56c7d82bd475efa2a68de5f0641ed142bf5b6f42ab9d30bab', + sha256: '71277d3a6f2c3a500b69fae829318c19ef00ed035a86cef09215865167b354a4', requiredLibraries: {'libstable-diffusion.so'}, ), StableDiffusionBundleSpec( 'linux-arm64-vulkan', - sha256: '1a2a089f229204e94a61f19d1ffe977aecd561d40f6d05ec9a740bd23c06506d', + sha256: '58b362b189473334d3ca92f40eb998228ef723b740efc1f1d9b885c7ad711be6', requiredLibraries: {'libstable-diffusion.so'}, ), StableDiffusionBundleSpec( 'linux-x64', - sha256: 'df062e77cd8f1d365946d436f4c5e795ec22b7ef49f2ccfeb4506c9548e0cccc', + sha256: '4b5405c319f500d1190e2e363386d561484ebcc88fe41cdfb6fa67fe863e68f4', requiredLibraries: {'libstable-diffusion.so'}, ), StableDiffusionBundleSpec( 'linux-x64-vulkan', - sha256: '8b58ffed2a2719080d70cdac884dc3769ad245824cf1a1e5480472abc406e0fc', + sha256: '43c3331afc6c72dd5ac16eef6625abdbc8fbed29544b442d3299e11c36d1caba', requiredLibraries: {'libstable-diffusion.so'}, ), StableDiffusionBundleSpec( 'windows-x64', - sha256: '4185409411f1b1a23988da232bf6c4fa044b46cd48fccb30b379ea915de57384', + sha256: '5eff9c03d5a8b13412df02816d451b9f5e53aba689d8b65118cc2e4862dd1848', requiredLibraries: {'stable-diffusion.dll'}, ), StableDiffusionBundleSpec( 'windows-x64-vulkan', - sha256: '05d4c2545cac4e5c96b14c9440584c9c634572329183334a55a84c113aeb0aef', + sha256: '19351af58dc09ee6d66b2b09b3206518a4920e28377c86a5f1ebf2fb8dfac8c4', requiredLibraries: {'stable-diffusion.dll'}, ), ]; diff --git a/packages/llamadart_stable_diffusion_flutter/CHANGELOG.md b/packages/llamadart_stable_diffusion_flutter/CHANGELOG.md index 22c061a82..e243dabc3 100644 --- a/packages/llamadart_stable_diffusion_flutter/CHANGELOG.md +++ b/packages/llamadart_stable_diffusion_flutter/CHANGELOG.md @@ -1,3 +1,7 @@ +## Unreleased + +- Updated Apple SwiftPM native pin to `leehack/stable-diffusion-native@v0.3.0-4`. + ## 0.19.9 - Refresh README installation guidance for core `0.19.9`; Apple SwiftPM stays pinned to `leehack/stable-diffusion-native@v0.3.0-3`. diff --git a/packages/llamadart_stable_diffusion_flutter/README.md b/packages/llamadart_stable_diffusion_flutter/README.md index c30bb6bb2..0a959ca60 100644 --- a/packages/llamadart_stable_diffusion_flutter/README.md +++ b/packages/llamadart_stable_diffusion_flutter/README.md @@ -34,7 +34,7 @@ Flutter iOS and macOS it does so only with this package. This package has no runtime Dart API of its own. Import `package:llamadart` normally from the core package and use `ImageGenerationEngine` there. -The Apple SwiftPM manifest pins `leehack/stable-diffusion-native@v0.3.0-3`. +The Apple SwiftPM manifest pins `leehack/stable-diffusion-native@v0.3.0-4`. Apps bound for the App Store should use this package: the Apple privacy manifest for the stable-diffusion.cpp runtime ships inside this XCFramework, diff --git a/packages/llamadart_stable_diffusion_flutter/darwin/llamadart_stable_diffusion_flutter/Package.swift b/packages/llamadart_stable_diffusion_flutter/darwin/llamadart_stable_diffusion_flutter/Package.swift index 99309c7f0..402ce12de 100644 --- a/packages/llamadart_stable_diffusion_flutter/darwin/llamadart_stable_diffusion_flutter/Package.swift +++ b/packages/llamadart_stable_diffusion_flutter/darwin/llamadart_stable_diffusion_flutter/Package.swift @@ -4,7 +4,7 @@ import PackageDescription let packageRoot = URL(fileURLWithPath: #filePath).deletingLastPathComponent() let artifactsRoot = packageRoot.appendingPathComponent("Artifacts") -let stableDiffusionTag = "v0.3.0-3" +let stableDiffusionTag = "v0.3.0-4" func localArtifactPath(_ name: String) -> String? { let path = artifactsRoot.appendingPathComponent(name).path @@ -47,7 +47,7 @@ let package = Package( repository: "leehack/stable-diffusion-native", artifactName: "stable-diffusion-native-apple-xcframework-\(stableDiffusionTag).zip", tag: stableDiffusionTag, - checksum: "8f35c67f72d1d4bf86d31f20f3437d04baaf3b6ac7389908000fd166e387783f" + checksum: "0d022638f54be80aa75fd8e73a94d139ad8887f4f3c73084501369ca5e7ac354" ), .target( name: "llamadart_stable_diffusion_flutter", diff --git a/packages/llamadart_validation/assets/capability_coverage.json b/packages/llamadart_validation/assets/capability_coverage.json index 4d4e509a8..ea507f0f4 100644 --- a/packages/llamadart_validation/assets/capability_coverage.json +++ b/packages/llamadart_validation/assets/capability_coverage.json @@ -335,7 +335,7 @@ ], "exemption": { "id": "image-capabilities", - "reason": "Dedicated image profiles and device evidence exist, but reported image capability fields are not yet bound to versioned validation feature/case IDs. This is a coverage-map gap, not a new runtime support claim. ImageConditioningCacheParams.capacity has maintained engine/worker tests and image-generation-smoke qualification on SDXS/SD-Turbo CPU and Metal; it has no versioned validation-pack case yet. SeamlessImageGenerationRequest circularX/circularY has maintained API/ABI refusal tests and image-seamless-textures CPU/Metal quality and cancellation qualification for SDXS; versioned pack case IDs remain open in https://github.com/leehack/llamadart/issues/1234. TiledImageGenerationRequest has maintained request/legacy-interface, engine/worker/ABI guards and image-generation-smoke SD-Turbo full-VAE/TAESD CPU/Metal qualification; it has no versioned validation-pack case yet (https://github.com/leehack/llamadart/issues/1231). FP8 weights load on Metal with the pinned stable-diffusion-native v0.3.0-1, which converts them to F16, decided by StableDiffusionCalls.loadsFp8AsF16OnMetal; engine/worker/probe tests cover both that path and the refusal on a runtime without the fix, and local-only SD-Turbo FP8 re-encodings generate on macOS arm64 CPU and Metal. A versioned FP8 validation-pack case remains open in https://github.com/leehack/llamadart/issues/1446. Approximation caching remains https://github.com/leehack/llamadart/issues/1233. ImageModelPlacement (text encoder or VAE on the CPU, maxGpuMemoryGiB, autoFit) has maintained engine, parameter-mapping and CPU/Web refusal tests and an image-generation-smoke check on SD-Turbo with Metal that reads the runtime's weight placement and compares pixels; Vulkan and multi-GPU hosts have no evidence yet (https://github.com/leehack/llamadart/issues/806). The iOS Metal VAE restriction has maintained known-full-VAE and old-runtime pre-native rejection, versioned native decoder-policy admission and worker routing, actual intrinsic SDXS family verification, rejected-session disposal, external-TAESD fallback refusal and CPU preservation tests; local native CPU/Metal controls exercise unchanged/refreshed SDXS and refuse SDXS-to-SD-Turbo source replacement before runner probes; it does not qualify a full-VAE iPhone watchdog mitigation recipe or a new device pack (https://github.com/leehack/llamadart/issues/1047).", + "reason": "Dedicated image profiles and device evidence exist, but reported image capability fields are not yet bound to versioned validation feature/case IDs. This is a coverage-map gap, not a new runtime support claim. ImageConditioningCacheParams.capacity has maintained engine/worker tests and image-generation-smoke qualification on SDXS/SD-Turbo CPU and Metal; it has no versioned validation-pack case yet. SeamlessImageGenerationRequest circularX/circularY has maintained API/ABI refusal tests and image-seamless-textures CPU/Metal quality and cancellation qualification for SDXS; versioned pack case IDs remain open in https://github.com/leehack/llamadart/issues/1234. TiledImageGenerationRequest has maintained request/legacy-interface, engine/worker/ABI guards and image-generation-smoke SD-Turbo full-VAE/TAESD CPU/Metal qualification; it has no versioned validation-pack case yet (https://github.com/leehack/llamadart/issues/1231). FP8 weights load on Metal with the pinned stable-diffusion-native v0.3.0-1, which converts them to F16, decided by StableDiffusionCalls.loadsFp8AsF16OnMetal; engine/worker/probe tests cover both that path and the refusal on a runtime without the fix, and local-only SD-Turbo FP8 re-encodings generate on macOS arm64 CPU and Metal. A versioned FP8 validation-pack case remains open in https://github.com/leehack/llamadart/issues/1446. Approximation caching remains https://github.com/leehack/llamadart/issues/1233. ImageModelPlacement (text encoder or VAE on the CPU, maxGpuMemoryGiB, autoFit) has maintained engine, parameter-mapping and CPU/Web refusal tests and an image-generation-smoke check on SD-Turbo with Metal that reads the runtime's weight placement and compares pixels; Vulkan and multi-GPU hosts have no evidence yet (https://github.com/leehack/llamadart/issues/806). The iOS Metal VAE restriction applies only to a runtime without the graph split (StableDiffusionCalls.graphSplit, stable-diffusion-native v0.3.0-4); engine, probe and calls tests cover both the split-runtime load and the old-runtime refusal, and DDP ran the split full-VAE decode on an iPhone 16 Pro and an iPhone 17 Pro (see doc/cross_platform_validation.md); the final public-API qualification run is tracked in https://github.com/leehack/llamadart/issues/1995. It has maintained known-full-VAE and old-runtime pre-native rejection, versioned native decoder-policy admission and worker routing, actual intrinsic SDXS family verification, rejected-session disposal, external-TAESD fallback refusal and CPU preservation tests; local native CPU/Metal controls exercise unchanged/refreshed SDXS and refuse SDXS-to-SD-Turbo source replacement before runner probes; it adds no new device pack (https://github.com/leehack/llamadart/issues/1047).", "issue": "https://github.com/leehack/llamadart/issues/1196" } }, diff --git a/skills/llamadart-image-generation/SKILL.md b/skills/llamadart-image-generation/SKILL.md index 64b9e44d2..c82bec7bc 100644 --- a/skills/llamadart-image-generation/SKILL.md +++ b/skills/llamadart-image-generation/SKILL.md @@ -53,7 +53,7 @@ description: >- `xcodebuild` show both. Use the companion with the core's `MAJOR.MINOR`. Adding it opts the app into the runtime (about 37 MB per Apple target). The core accepts only an Apple SD companion pinning its own runtime -(`v0.3.0-3`): runtimes before `v0.3.0-1` abort on FP8 weights on Metal, and those before +(`v0.3.0-4`): runtimes before `v0.3.0-1` abort on FP8 weights on Metal, and those before `v0.3.0` have a smaller `sd_ctx_params_t`. Upgrade the core and `llamadart_stable_diffusion_flutter` together. @@ -283,10 +283,10 @@ core and `llamadart_stable_diffusion_flutter` together. true` on iOS Metal throws `LlamaUnsupportedException` at load: every decode tripped the GPU watchdog and returned a corrupt image ([#1044](https://github.com/leehack/llamadart/issues/1044)). Full-VAE Metal - decoding on iOS is also refused, even with direct convolutions off or tiling. - GPU decoding needs the intrinsic tiny-decoder v1 policy across refresh. - The pinned `v0.3.0-3` provides that policy for intrinsic SDXS tiny decoders; other recipes require `ImageModelPlacement(vaeOnCpu: true)` or `ComputeDevice.cpu`. Desktop checks do not add physical-iOS qualification. - External TAESD can fall back to a full VAE; no iPhone full-VAE fix is claimed (#1047). + decoding on iOS is refused only on a runtime without the graph split; the + pinned `v0.3.0-4` has it (#1047). An older runtime needs an intrinsic SDXS + decoder, `ImageModelPlacement(vaeOnCpu: true)` or `ComputeDevice.cpu`. On + iOS 26 the second engine dispose can leave about 2 GB resident (#2004). - `ImageConditioningCacheParams(capacity: ...)`: ordinary `ImageModelParams` preserve the native default of 4 prompt-conditioning entries. `0` disables reuse; positive values set an LRU entry limit up to `2147483647` (the signed 32-bit ABI diff --git a/test/fixtures/native/stable_diffusion_v0_3_0_4_manifest.json b/test/fixtures/native/stable_diffusion_v0_3_0_4_manifest.json new file mode 100644 index 000000000..d3acabc83 --- /dev/null +++ b/test/fixtures/native/stable_diffusion_v0_3_0_4_manifest.json @@ -0,0 +1,323 @@ +{ + "schemaVersion": 1, + "package": "stable-diffusion-native", + "tag": "v0.3.0-4", + "source": { + "repository": "leehack/stable-diffusion-native", + "commit": "4a84ac24bbbe4260308a4d80fae9b15900db456c" + }, + "upstream": { + "repository": "leejet/stable-diffusion.cpp", + "commit": "be486498eadc29dc3cd234d944c2180ed42b1415", + "patches": [ + "fp8_metal_f16_load.patch", + "intrinsic_tiny_decoder_policy.patch", + "lora_immediate_fallback.patch", + "pickle_line_span.patch", + "torch_zip_entry_lookup.patch" + ] + }, + "artifacts": [ + { + "target": "macos-arm64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-macos-arm64-v0.3.0-4.tar.gz", + "sha256": "d34dbaf3c9ecfbbaa97e1c89687e27cefdae824ed27a1a8cf694a9c9489d3eae", + "size": 16469767, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "macos-arm64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-macos-arm64-v0.3.0-4.tar.gz", + "sha256": "055a3aa4705998d573e75f4fbe5d0f2cfd9edef77feff797ee408c2afb270faa", + "size": 16777170, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "macos-x64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-macos-x64-v0.3.0-4.tar.gz", + "sha256": "7f5954264d0e904a5049822e320eb6b06696c38a59428868537915b439eec478", + "size": 16450728, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "macos-x64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-macos-x64-v0.3.0-4.tar.gz", + "sha256": "e6483b3c20d63cd810273a82163a3fc7c792d99c91bd329d39ab106a81d72caa", + "size": 16736220, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "ios-arm64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-ios-arm64-v0.3.0-4.tar.gz", + "sha256": "c24c7df0a7c21370d026668cd12a98baf646d7951829088e6957f25ad13cdce9", + "size": 16071357, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "ios-arm64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-ios-arm64-v0.3.0-4.tar.gz", + "sha256": "99320280749355803360b81a0b71dafa99d55df98322f97637b767b9437c4284", + "size": 16359773, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "ios-arm64-sim", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-ios-arm64-sim-v0.3.0-4.tar.gz", + "sha256": "be5ab833a7a902ba75bce16c3020565372bef672106e5b91a2d9bcb7e85d0fb2", + "size": 16469409, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "ios-arm64-sim", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-ios-arm64-sim-v0.3.0-4.tar.gz", + "sha256": "6eb81568a6a14884eb18cecf8d38584fc8d9e7fa2ae23672c5a233319d38cdfb", + "size": 16776773, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "ios-x64-sim", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-ios-x64-sim-v0.3.0-4.tar.gz", + "sha256": "eae74517da056d3cd44964a5ada097b4ec58158be4978721efdb9801d90e7540", + "size": 16467026, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "ios-x64-sim", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-ios-x64-sim-v0.3.0-4.tar.gz", + "sha256": "4f7db0b095cc4c4cb022cf998a19e0051081cc23ce559e0a9baf4c17ac167c16", + "size": 16748062, + "library": "libstable-diffusion.dylib", + "accelerators": [ + "metal", + "cpu" + ] + }, + { + "target": "android-arm64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-android-arm64-v0.3.0-4.tar.gz", + "sha256": "fdc1a614bc7b50d838dfee4416a8ffea04eb01a0042f2d65821e80225e4000cd", + "size": 15482177, + "library": "libstable-diffusion.so", + "accelerators": [ + "cpu" + ] + }, + { + "target": "android-arm64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-android-arm64-v0.3.0-4.tar.gz", + "sha256": "93bea993d23751cd6f034cce299490cbb66bce2aa486be3a072c579a0e582cd3", + "size": 61629947, + "library": "libstable-diffusion.so", + "accelerators": [ + "cpu" + ] + }, + { + "target": "linux-x64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-linux-x64-v0.3.0-4.tar.gz", + "sha256": "4b5405c319f500d1190e2e363386d561484ebcc88fe41cdfb6fa67fe863e68f4", + "size": 16456342, + "library": "libstable-diffusion.so", + "accelerators": [ + "cpu" + ] + }, + { + "target": "linux-x64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-linux-x64-v0.3.0-4.tar.gz", + "sha256": "b8f943f62dd67fa03d69eccd19ffe50e4779854602e5761d027dd8480ab4f6b7", + "size": 16780400, + "library": "libstable-diffusion.so", + "accelerators": [ + "cpu" + ] + }, + { + "target": "linux-x64-vulkan", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-linux-x64-vulkan-v0.3.0-4.tar.gz", + "sha256": "43c3331afc6c72dd5ac16eef6625abdbc8fbed29544b442d3299e11c36d1caba", + "size": 26517624, + "library": "libstable-diffusion.so", + "accelerators": [ + "vulkan", + "cpu" + ] + }, + { + "target": "linux-x64-vulkan", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-linux-x64-vulkan-v0.3.0-4.tar.gz", + "sha256": "34aefcd550a9ad5381ed3b25a13f20ff5367b72c1ba38cf7c1c6a1ce72ccbd54", + "size": 26925689, + "library": "libstable-diffusion.so", + "accelerators": [ + "vulkan", + "cpu" + ] + }, + { + "target": "linux-arm64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-linux-arm64-v0.3.0-4.tar.gz", + "sha256": "71277d3a6f2c3a500b69fae829318c19ef00ed035a86cef09215865167b354a4", + "size": 16022453, + "library": "libstable-diffusion.so", + "accelerators": [ + "cpu" + ] + }, + { + "target": "linux-arm64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-linux-arm64-v0.3.0-4.tar.gz", + "sha256": "867c41626c25faf3c9ef856787cad5d0ac4f7f2bdae4c328cd5b9c7470195675", + "size": 16368033, + "library": "libstable-diffusion.so", + "accelerators": [ + "cpu" + ] + }, + { + "target": "linux-arm64-vulkan", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-linux-arm64-vulkan-v0.3.0-4.tar.gz", + "sha256": "58b362b189473334d3ca92f40eb998228ef723b740efc1f1d9b885c7ad711be6", + "size": 26054081, + "library": "libstable-diffusion.so", + "accelerators": [ + "vulkan", + "cpu" + ] + }, + { + "target": "linux-arm64-vulkan", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-linux-arm64-vulkan-v0.3.0-4.tar.gz", + "sha256": "31fe3d52bb677f98220230581cbd92c4bb5b1c43a7f8aeb896e43c629a7b3595", + "size": 26484034, + "library": "libstable-diffusion.so", + "accelerators": [ + "vulkan", + "cpu" + ] + }, + { + "target": "windows-x64", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-windows-x64-v0.3.0-4.tar.gz", + "sha256": "5eff9c03d5a8b13412df02816d451b9f5e53aba689d8b65118cc2e4862dd1848", + "size": 15014025, + "library": "stable-diffusion.dll", + "accelerators": [ + "cpu" + ] + }, + { + "target": "windows-x64", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-windows-x64-v0.3.0-4.tar.gz", + "sha256": "83514fb2c495c49cd3f2a7b8825603fa28990dfaaf00c19d53b653969296e24d", + "size": 14991345, + "library": "stable-diffusion.dll", + "accelerators": [ + "cpu" + ] + }, + { + "target": "windows-x64-vulkan", + "kind": "runtime", + "file": "stable-diffusion-native-runtime-windows-x64-vulkan-v0.3.0-4.tar.gz", + "sha256": "19351af58dc09ee6d66b2b09b3206518a4920e28377c86a5f1ebf2fb8dfac8c4", + "size": 27188408, + "library": "stable-diffusion.dll", + "accelerators": [ + "vulkan", + "cpu" + ] + }, + { + "target": "windows-x64-vulkan", + "kind": "symbols", + "file": "stable-diffusion-native-symbols-windows-x64-vulkan-v0.3.0-4.tar.gz", + "sha256": "16b1a7d56055a22a575d6be98bfd57212c54e6780059be9b84cf164279a38a4f", + "size": 27167664, + "library": "stable-diffusion.dll", + "accelerators": [ + "vulkan", + "cpu" + ] + }, + { + "target": "apple", + "kind": "xcframework", + "file": "stable-diffusion-native-apple-xcframework-v0.3.0-4.zip", + "sha256": "0d022638f54be80aa75fd8e73a94d139ad8887f4f3c73084501369ca5e7ac354", + "size": 82460841, + "library": "stable_diffusion.framework", + "accelerators": [ + "metal", + "cpu" + ], + "slices": [ + "ios-arm64", + "ios-arm64_x86_64-simulator", + "macos-arm64_x86_64" + ], + "minimumOS": { + "ios": "16.4", + "macos": "13.3" + } + } + ] +} diff --git a/test/unit/backends/stable_diffusion/stable_diffusion_calls_test.dart b/test/unit/backends/stable_diffusion/stable_diffusion_calls_test.dart index 826a17df5..026e4149d 100644 --- a/test/unit/backends/stable_diffusion/stable_diffusion_calls_test.dart +++ b/test/unit/backends/stable_diffusion/stable_diffusion_calls_test.dart @@ -39,6 +39,9 @@ const _deviceMemorySymbols = [ // Upstream master-1017's, which v0.3.0-1 first ships. const _fp8MetalSymbols = ['free_sd_image_generation_result']; +// v0.3.0-4's Metal graph split; Dart reads only the threshold. +const _graphSplitSymbols = ['sd_dart_graph_split_threshold']; + // The root package does not bundle the stable_diffusion runtime, so these // tests resolve the functions from stand-ins. That each name resolves to the // runtime's own export is checked against the real runtime by @@ -60,6 +63,7 @@ void main() { ..._logSymbols, ..._deviceMemorySymbols, ..._fp8MetalSymbols, + ..._graphSplitSymbols, 'sd_dart_new_sd_ctx_intrinsic_tiny_v1', ]), ); @@ -136,6 +140,36 @@ void main() { }, ); + test('the graph split query is bound only when the runtime exports it, and ' + 'is active only above a zero threshold', () { + expect( + StableDiffusionCalls.graphSplitSymbols, + unorderedEquals(_graphSplitSymbols), + ); + expect(StableDiffusionCalls.graphSplitNativeRelease, 'v0.3.0-4'); + final absent = StableDiffusionCalls.tryResolve( + symbol: (name) => _graphSplitSymbols.contains(name) + ? throw ArgumentError("Couldn't resolve native function '$name'") + : Pointer.fromAddress(0x1000), + ); + expect(absent, isNotNull); + expect(absent!.graphSplit, isNull); + expect( + StableDiffusionCalls.tryResolve( + symbol: (_) => Pointer.fromAddress(0x1000), + )!.graphSplit, + isNotNull, + ); + expect( + StableDiffusionGraphSplitCalls(threshold: () => 0).isActive, + isFalse, + ); + expect( + StableDiffusionGraphSplitCalls(threshold: () => 1 << 26).isActive, + isTrue, + ); + }); + group('loadsFp8AsF16OnMetal', () { StableDiffusionCalls resolve({ String? releaseTag, diff --git a/test/unit/backends/stable_diffusion/stable_diffusion_runtime_io_test.dart b/test/unit/backends/stable_diffusion/stable_diffusion_runtime_io_test.dart index dcb8f141a..037f70a01 100644 --- a/test/unit/backends/stable_diffusion/stable_diffusion_runtime_io_test.dart +++ b/test/unit/backends/stable_diffusion/stable_diffusion_runtime_io_test.dart @@ -131,6 +131,34 @@ void main() { expect(absent.supportsIntrinsicTinyDecoderV1(), isFalse); }); + test('reports whether the runtime splits graphs on Metal', () { + for (final splits in [false, true]) { + final status = probeStableDiffusionRuntime( + abi: Abi.macosArm64, + api: _FakeApi(graphSplit: splits), + ); + expect(status.splitsGraphsOnMetal, splits); + } + expect( + BundledStableDiffusionApi( + resolveCalls: () => null, + ).splitsGraphsOnMetal(), + isFalse, + ); + for (final (graphSplit, expected) in [ + (null, false), + (const StableDiffusionGraphSplitCalls(threshold: _noSplit), false), + (const StableDiffusionGraphSplitCalls(threshold: _split), true), + ]) { + expect( + BundledStableDiffusionApi( + resolveCalls: () => _callsWith(graphSplit), + ).splitsGraphsOnMetal(), + expected, + ); + } + }); + test('reports whether the runtime loads FP8 weights as F16 on Metal', () { for (final fp8Metal in [true, false]) { final status = probeStableDiffusionRuntime( @@ -715,6 +743,27 @@ void main() { }); } +int _noSplit() => 0; +int _split() => 1 << 26; + +StableDiffusionCalls _callsWith(StableDiffusionGraphSplitCalls? graphSplit) => + StableDiffusionCalls( + progressEnable: () {}, + progressRead: (after, reports, capacity, latest) => 0, + newContext: (params) => nullptr, + generateImage: (context, params, result, failure) => false, + cancelGeneration: (context) {}, + exitFree: (context) {}, + exitFreeAddress: nullptr, + contextParamsInit: (params) {}, + supportsImageGeneration: (context) => true, + hasControlNet: (context) => false, + modelVersionName: (context) => nullptr, + imageGenerationParamsInit: (params) {}, + freeImages: (images, count) {}, + graphSplit: graphSplit, + ); + final class _FakeApi implements StableDiffusionNativeApi { _FakeApi({ this.devices = 'CPU\tHost CPU\n', @@ -722,6 +771,7 @@ final class _FakeApi implements StableDiffusionNativeApi { this.wrapperCalls = true, this.fp8Metal = false, this.intrinsicTinyV1 = false, + this.graphSplit = false, }); final String devices; @@ -729,6 +779,7 @@ final class _FakeApi implements StableDiffusionNativeApi { final bool wrapperCalls; final bool fp8Metal; final bool intrinsicTinyV1; + final bool graphSplit; int calls = 0; final List order = []; @@ -765,6 +816,9 @@ final class _FakeApi implements StableDiffusionNativeApi { @override bool loadsFp8AsF16OnMetal() => _call(fp8Metal); + @override + bool splitsGraphsOnMetal() => _call(graphSplit); + @override bool supportsIntrinsicTinyDecoderV1() => _call(intrinsicTinyV1); } diff --git a/test/unit/core/image/image_generation_engine_test.dart b/test/unit/core/image/image_generation_engine_test.dart index a9622c86c..f5702033c 100644 --- a/test/unit/core/image/image_generation_engine_test.dart +++ b/test/unit/core/image/image_generation_engine_test.dart @@ -2243,6 +2243,55 @@ void main() { } }); + test('a runtime that splits graphs loads a full VAE on Metal without ' + 'the intrinsic tiny-decoder policy', () async { + driver.status = _available(iphone, splitsGraphsOnMetal: true); + for (final device in [ComputeDevice.auto, ComputeDevice.gpu]) { + driver.started.clear(); + await load(checkpoint, params: ImageModelParams(device: device)); + final config = driver.started.single; + expect(config.requiresIntrinsicTinyDecoderV1, isFalse); + expect(config.backend, isNot(contains('vae=cpu'))); + } + }); + + test('a runtime without the graph split refuses a full VAE and names ' + 'the release that adds it', () async { + driver.status = _available( + iphone, + supportsIntrinsicTinyDecoderV1: true, + ); + await expectLater( + load(checkpoint), + throwsA( + isA().having( + (error) => error.message, + 'message', + contains('v0.3.0-4'), + ), + ), + ); + expect(driver.started, isEmpty); + }); + + test('splitting graphs does not lift the direct VAE convolution ' + 'refusal', () async { + driver.status = _available(iphone, splitsGraphsOnMetal: true); + await expectLater( + load( + checkpoint, + params: const ImageModelParams(vaeDirectConvolution: true), + ), + throwsA( + isA().having( + (error) => error.message, + 'message', + contains('/issues/1044'), + ), + ), + ); + }); + test('a full VAE can explicitly compute on the CPU while diffusion ' 'uses Metal', () async { driver.status = _available(iphone); @@ -4313,12 +4362,14 @@ StableDiffusionRuntimeStatus _available( String devices, { bool loadsFp8AsF16OnMetal = false, bool supportsIntrinsicTinyDecoderV1 = true, + bool splitsGraphsOnMetal = false, }) => StableDiffusionRuntimeStatus.available( version: 'master-929', commit: 'be48649', devices: parseStableDiffusionDeviceList(devices), loadsFp8AsF16OnMetal: loadsFp8AsF16OnMetal, supportsIntrinsicTinyDecoderV1: supportsIntrinsicTinyDecoderV1, + splitsGraphsOnMetal: splitsGraphsOnMetal, ); final class _FakeDriver implements ImageGenerationDriver { diff --git a/test/unit/hook/native_release_pins_test.dart b/test/unit/hook/native_release_pins_test.dart index 413b708c9..20eb7adc1 100644 --- a/test/unit/hook/native_release_pins_test.dart +++ b/test/unit/hook/native_release_pins_test.dart @@ -222,13 +222,13 @@ void main() { }, ); - test('stable_diffusion pins the qualified v0.3.0-3 release artifacts', () { + test('stable_diffusion pins the qualified v0.3.0-4 release artifacts', () { // Exact published owner manifest, independently checked against GitHub // asset digests before adoption. This also binds the Apple binary target. final manifest = jsonDecode( File( - 'test/fixtures/native/stable_diffusion_v0_3_0_3_manifest.json', + 'test/fixtures/native/stable_diffusion_v0_3_0_4_manifest.json', ).readAsStringSync(), ) as Map; @@ -256,7 +256,7 @@ void main() { final apple = artifacts.singleWhere( (artifact) => artifact['file'] == - 'stable-diffusion-native-apple-xcframework-v0.3.0-3.zip', + 'stable-diffusion-native-apple-xcframework-v0.3.0-4.zip', ); final swift = File( 'packages/llamadart_stable_diffusion_flutter/darwin/' @@ -272,7 +272,7 @@ void main() { final manifest = jsonDecode( File( - 'test/fixtures/native/stable_diffusion_v0_3_0_3_manifest.json', + 'test/fixtures/native/stable_diffusion_v0_3_0_4_manifest.json', ).readAsStringSync(), ) as Map; diff --git a/test/unit/platform/io/runtime_setup_report_io_test.dart b/test/unit/platform/io/runtime_setup_report_io_test.dart index 87e1cc9fb..fe7468dda 100644 --- a/test/unit/platform/io/runtime_setup_report_io_test.dart +++ b/test/unit/platform/io/runtime_setup_report_io_test.dart @@ -58,7 +58,7 @@ void main() { expect(report.runtimes.map((runtime) => runtime.expectedReleaseTag), [ 'v0.9.0', 'v0.19.1-4', - 'v0.3.0-3', + 'v0.3.0-4', ]); expect(report.dartVersion, '3.13.1'); expect(jsonEncode(report.toJson()), isNot(contains('private-user'))); diff --git a/website/docs/changelog/recent-releases.md b/website/docs/changelog/recent-releases.md index 21a838fb4..76f23fe19 100644 --- a/website/docs/changelog/recent-releases.md +++ b/website/docs/changelog/recent-releases.md @@ -12,12 +12,13 @@ For upgrade planning, combine this page with the validate against the exact release tag notes. ## Unreleased +- Adopt stable-diffusion `v0.3.0-4`: on iOS Metal the runtime splits graphs, so a full VAE decodes without tripping the GPU watchdog and `ImageGenerationEngine.load` no longer refuses it there; a runtime without the split keeps the refusal. Run on an iPhone 16 Pro and an iPhone 17 Pro (30 of 30 soak decodes). On iOS 26 the second engine dispose can leave about 2 GB resident ([#1047](https://github.com/leehack/llamadart/issues/1047), [#2004](https://github.com/leehack/llamadart/issues/2004)). - Chat app microphone: a microphone that disconnects mid-recording now transcribes the audio captured so far with a warning, a missing input device on Web is reported as such instead of as a permission denial, and a failed Web start releases the microphone ([#329](https://github.com/leehack/llamadart/issues/329)). - Aligned default WebGPU bridge assets to `v0.1.85`: a page without cross-origin isolation now fails at once with a `LlamaUnsupportedException` naming COOP/COEP `require-corp` instead of waiting minutes for a generic init timeout, and Firefox CPU runs (`gpuLayers: 0`) no longer create the WebGPU backend (stories15M, 32 tokens: 640 to 762 tok/s against 10 before) ([#1025](https://github.com/leehack/llamadart/issues/1025)). - `ComputeDevice.gpu` on Web now names why WebGPU was not used (for example `adapter_missing_shader_f16`: Chrome on Linux with NVIDIA needs driver 615.71 or newer); documented the measured Safari 27, Firefox 160 and NVIDIA L4 results ([#1025](https://github.com/leehack/llamadart/issues/1025)). -Runtime baseline: native llama.cpp `v0.9.0` and Web use upstream `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`; native LiteRT-LM uses `v0.19.1-4` and stable-diffusion uses `v0.3.0-3`. Web LiteRT-LM uses `@litert-lm/core@0.19.0`. Manifest SHA-256: `6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3`. +Runtime baseline: native llama.cpp `v0.9.0` and Web use upstream `v0.9.0@cd959bae99ff4bed22bef2604f45241c09bbb8c8`; native LiteRT-LM uses `v0.19.1-4` and stable-diffusion uses `v0.3.0-4`. Web LiteRT-LM uses `@litert-lm/core@0.19.0`. Manifest SHA-256: `6767783edd85e14853a8a79cd6ce53d53f4eb2d99aeb62601014bd68c1b6b7b3`. ## 0.19.9 diff --git a/website/docs/guides/image-generation-settings.md b/website/docs/guides/image-generation-settings.md index ce06ab102..4aaf322d5 100644 --- a/website/docs/guides/image-generation-settings.md +++ b/website/docs/guides/image-generation-settings.md @@ -81,20 +81,20 @@ Peak memory is the `phys_footprint` interval peak on the iPhone and a sampled has no SD-Turbo figures, and its SDXS figures are an observation, not verified evidence. The iPhone full-VAE arm has not been measured, because its decode trips the GPU watchdog ([#1047](https://github.com/leehack/llamadart/issues/1047)). -On iOS, Metal VAE decoding requires the intrinsic tiny-decoder v1 native -context policy. It preserves an intrinsic `SDXS (512-DS)` or `SDXS (09)` -decoder across automatic source refresh and rejects a replacement before -runner construction, backend probes and decoding. A header or initial native -family query alone cannot enforce that invariant. - -The pinned stable-diffusion-native `v0.3.0-3` provides this policy for -intrinsic SDXS tiny decoders. For other models, use -`ImageModelPlacement(vaeOnCpu: true)` to keep diffusion on Metal, or -`ComputeDevice.cpu`. Recipes with an external TAESD -also need CPU VAE placement: an external decoder can fail and fall back to a -full VAE. Setting `vaeDirectConvolution: false` or requesting VAE tiling -does not bypass the restriction. These guards refuse the unqualified GPU -decode path; they do not qualify a repaired full-VAE iPhone recipe. +On iOS, a full-VAE Metal decode can outlast the GPU watchdog. The pinned +stable-diffusion-native `v0.3.0-4` splits the graph into several command +buffers (`sd_dart_graph_split_threshold`, 2^26 elements), so full-VAE and +external-decoder recipes decode on Metal there. + +A runtime without the split refuses them. Its Metal decoding requires the +intrinsic tiny-decoder v1 native context policy, which preserves an intrinsic +`SDXS (512-DS)` or `SDXS (09)` decoder across automatic source refresh and +rejects a replacement before runner construction, backend probes and decoding. +For other models, use `ImageModelPlacement(vaeOnCpu: true)` to keep diffusion on +Metal, or `ComputeDevice.cpu`. `vaeDirectConvolution: true` stays refused on iOS +Metal with either runtime +([#1044](https://github.com/leehack/llamadart/issues/1044)), and VAE tiling does +not bypass the refusal. With a tiny autoencoder, direct convolutions cost time for little memory: on an M4 Max CPU they took 1.4 to 1.7 s per 512x512 SDXS image against 1.2 diff --git a/website/docs/guides/image-generation.md b/website/docs/guides/image-generation.md index f5683108d..b7cf0e291 100644 --- a/website/docs/guides/image-generation.md +++ b/website/docs/guides/image-generation.md @@ -189,14 +189,17 @@ Use trusted model files. There is no public conversion API or load-time quantization/mmap option. Weights load eagerly; see the [memory check](./image-generation-memory-and-limits#memory-check) before substituting larger unquantized files. -On iOS, full VAE and external-decoder recipes require -`ImageModelPlacement(vaeOnCpu: true)` for Metal diffusion, or -`ComputeDevice.cpu`. The pinned `v0.3.0-3` runtime provides the intrinsic -tiny-decoder v1 context policy for SDXS Metal decoding: it rejects any -refreshed model that could select a full VAE before runner probes and -decoding. This restriction is tested on desktop; it does not establish new -physical-iOS qualification or repair the full-VAE watchdog failure -([#1047](https://github.com/leehack/llamadart/issues/1047)). +On iOS, the pinned `v0.3.0-4` runtime splits Metal graphs, so full-VAE +recipes decode on the GPU without tripping the watchdog +([#1047](https://github.com/leehack/llamadart/issues/1047)); it was run on an +iPhone 16 Pro and an iPhone 17 Pro. A runtime without the split refuses full VAE +and external-decoder recipes: use `ImageModelPlacement(vaeOnCpu: true)` for +Metal diffusion, or `ComputeDevice.cpu`. Its intrinsic tiny-decoder v1 context +policy still serves SDXS Metal decoding. + +Known limit: on iOS 26 the second `ImageGenerationEngine` dispose can leave +about 2 GB resident, so later loads may fail the memory check +([#2004](https://github.com/leehack/llamadart/issues/2004)). #### Pinned-runtime format checks diff --git a/website/docs/maintainers/feature-validation.md b/website/docs/maintainers/feature-validation.md index dc2fce9d8..3d6256d59 100644 --- a/website/docs/maintainers/feature-validation.md +++ b/website/docs/maintainers/feature-validation.md @@ -115,10 +115,22 @@ on Android, where Test Lab refused the `VmHWM` reset on all three phones. The Galaxy A53's memory check refused SD-Turbo (about 2.9 GiB needed, 2.5 GiB available), so it has no SD-Turbo figures, and its SDXS figures come from a run that does not verify because of that refusal: they are an observation, not -verified evidence. The iPhone full-VAE arm has not been measured, because its -decode trips the GPU watchdog +verified evidence. The iPhone full-VAE arm was not measured in those runs because the +`v0.3.0-3` decode trips the GPU watchdog; the `v0.3.0-4` graph split is +qualified separately below ([#1047](https://github.com/leehack/llamadart/issues/1047)). +### iOS full-VAE graph split + +The `v0.3.0-4` XCFramework splits Metal graphs through upstream's eval callback +at 2^26 elements. On 2026-11-13, DDP ran these exact bytes on an iPhone 16 Pro +(iOS 18.3) and an iPhone 17 Pro. Every split arm passed 3 of 3 decodes, the +unsplit control failed its third decode with the GPU command-buffer error (the +watchdog), and a 30-decode soak passed 30 of 30 with one image hash and a flat +footprint. The iPhone 17 Pro (iOS 26.5) showed the dispose limit in +[#2004](https://github.com/leehack/llamadart/issues/2004). The dated record is in +[cross-platform validation](https://github.com/leehack/llamadart/blob/main/doc/cross_platform_validation.md#ios-full-vae-graph-split-1047-2026-11-13). + ### Full-checkpoint container checks Earlier `v0.2.0-4` checks rejected the 2.6-GB legacy file and timed out on full diff --git a/website/docs/platforms/support-matrix.md b/website/docs/platforms/support-matrix.md index 3931a6f03..a27166283 100644 --- a/website/docs/platforms/support-matrix.md +++ b/website/docs/platforms/support-matrix.md @@ -55,15 +55,15 @@ the opt-in `stable_diffusion` runtime. Per-platform requirements, validated models, memory figures and the bounded recipes for tiling, editing, ControlNet, hires fix, reference editing, seamless textures, module placement, the conditioning cache, FP8 weights and local checkpoint sets are in -[Image generation](../guides/image-generation#support). On iOS, Metal VAE -decoding requires an intrinsic SDXS tiny decoder and a selected runtime with -the intrinsic tiny-decoder v1 context policy, which keeps automatic model -refresh from selecting a full VAE. The pinned `v0.3.0-3` runtime provides this -policy. Full VAE and external-decoder Metal decoding remain refused; use -`ImageModelPlacement(vaeOnCpu: true)` for Metal diffusion or `ComputeDevice.cpu` -for other recipes. Tiling does not bypass this restriction; iPhone full-VAE -watchdog avoidance remains unqualified -([#1047](https://github.com/leehack/llamadart/issues/1047)). +[Image generation](../guides/image-generation#support). On iOS, the pinned +`v0.3.0-4` runtime splits Metal graphs, so full-VAE decoding stays under the GPU +watchdog (qualified on an iPhone 16 Pro and an iPhone 17 Pro, +[#1047](https://github.com/leehack/llamadart/issues/1047)). A runtime without +the split refuses full-VAE Metal decoding; it then needs an intrinsic SDXS tiny +decoder and the intrinsic tiny-decoder v1 context policy, or +`ImageModelPlacement(vaeOnCpu: true)` or `ComputeDevice.cpu`. On iOS 26 the +second engine dispose can leave about 2 GB resident, so later loads may fail the +memory check ([#2004](https://github.com/leehack/llamadart/issues/2004)). GGUF Qwen3-ASR accepts WAV, MP3 and FLAC; real-model checks cover all three on native macOS and in headless Chromium. LiteRT-LM ASR is CPU-only streaming @@ -219,7 +219,7 @@ web are on [Known platform issues](./known-platform-issues). | --- | --- | | llama.cpp native | `leehack/llamadart-native@v0.9.0` | | LiteRT-LM native | `leehack/litert-lm-native@v0.19.1-4` | -| stable-diffusion.cpp native (opt-in, Preview) | `leehack/stable-diffusion-native@v0.3.0-3`, for [image generation](../guides/image-generation); see [Opt-in stable_diffusion runtime](./native-build-hooks#opt-in-stable_diffusion-runtime-experimental). Flutter iOS/macOS apps link its XCFramework through `llamadart_stable_diffusion_flutter` | +| stable-diffusion.cpp native (opt-in, Preview) | `leehack/stable-diffusion-native@v0.3.0-4`, for [image generation](../guides/image-generation); see [Opt-in stable_diffusion runtime](./native-build-hooks#opt-in-stable_diffusion-runtime-experimental). Flutter iOS/macOS apps link its XCFramework through `llamadart_stable_diffusion_flutter` | | WebGPU bridge assets | `leehack/llama-web-bridge-assets`; see [Pinned bridge assets](./webgpu-bridge#pinned-bridge-assets) | The native-assets hook currently pins `llamadart-native` tag