[leehack]

Summary

• Lift the native Qwen3.5 Android Vulkan restriction (#948) on llama.cpp runtimes with ggml 0.28.0 or newer. The small-matmul tile defect behind Android llama.cpp Vulkan on Pixel 9 Pro (Mali-G715): wrong output for prompts over ~32 tokens, then abort under a grammar #948 was fixed upstream in ggml-org/llama.cpp#30106; every bundled runtime from llamadart-native v0.8.0 (8642541b, ggml 0.28.0) on has it, and the pinned v0.9.0 reports ggml 0.29.0.
• Older runtime decision: the service reads ggml_version() once and keeps the full restriction (refusals, capability probes, fail-closed missing architecture) when the version is below 0.28.0, unparsable, or not exported. 0.27.0 spans commits before and after #30106, so it counts as unfixed. The ggml version is the release number of the component that carries the fix; a llama.cpp commit hash in llama_dart_build_identity cannot be ordered, and no native capability flag exists.
• Qwen3.5 on Android Vulkan keeps its actual-placement context policy on fixed runtimes: KQV stays on the GPU even when the context prefers the CPU, and microBatchSize unset keeps the 8-token prompt cap. The cap is now documented as a speed default (Pixel 9 Pro: about 44 tok/s capped against 27-29 at 32 or 512); explicit sizes run uncapped.
• The image encoder stays off Vulkan: on Android Vulkan with the same small-tile device facts (subgroup below 32 other than 8, or unknown), every Qwen3.5 projector now loads on the CPU on every runtime. The rule reads the loaded model's general.architecture (an unreadable one counts as Qwen3.5), not the file name. The Qwen3.5-0.8B file-name rule (shouldForceCpuProjectorForAndroid) is unchanged, so the device-tested 0.8B path behaves as before.
• Unchanged: the generic cap's device-fact gate, Mali-G68 (#782) and Adreno handling.
• The Pixel guard pack (pixel_qwen35_guard_test.dart, PIXEL_QWEN35_GUARD.md) is marked retired: its bound native v0.8.0-2 has ggml 0.28.0, so its refusal cells now fail by design.

Production-readiness scope

• User-facing scope: With the bundled runtime, native Qwen3.5 loaded on Android Vulkan accepts microBatchSize above 8, image/audio prompts (image encoder on the CPU on small-tile GPUs), every speculative strategy and embeddings; capability probes report them.
• Supported platforms/paths: Native llama.cpp on Android Vulkan with the pinned runtime. Device evidence: Pixel 9 Pro (Mali-G715, Android 15) and Pixel 7 (Mali-G710, Android 13) on this PR's build without any bypass, plus the earlier bypass runs on Pixel 9 Pro, Pixel 9 and Pixel 7 recorded on Android llama.cpp Vulkan on Pixel 9 Pro (Mali-G715): wrong output for prompts over ~32 tokens, then abort under a grammar #948.
• Unsupported or intentionally unavailable paths: A runtime older than ggml 0.28.0 (native release override or custom build) keeps the restriction: microBatchSize > 8 throws LlamaUnsupportedException before context creation, and media, speculation and embeddings throw before evaluation, naming ggml 0.28.0 / llamadart-native v0.8.0 and Android llama.cpp Vulkan on Pixel 9 Pro (Mali-G715): wrong output for prompts over ~32 tokens, then abort under a grammar #948.
• Out of scope / follow-ups: Qwen3.5 image encoder on small-tile Mali Vulkan (kept on the CPU); Qwen3.5 sizes and quantizations other than 0.8B Q4_0; Mali-G68 #782; Adreno. These stay unclaimed in the docs; no new follow-up issue.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant.
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope).
• Security/privacy review completed: no secrets, bearer tokens, signed URLs, or raw secret-bearing paths leak through logs, cache keys, metadata, errors, or snapshots.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Feature PR: capability lift with docs (README, known issues, support matrix, upgrade checklist, guides, runtime parameters, benchmarks), skills, changelog, coverage registry, and tests for the lifted path and the older-runtime path.
• Bugfix PR: N/A, the root cause was fixed upstream (#30106); this removes the mitigation where the fix is present.
• Docs-only PR: N/A.

Test Plan

• dart run tool/prepare_workspace.dart
• dart format --output=none --set-exit-if-changed .
• dart analyze
• dart run tool/testing/check_platform_boundaries.dart
• dart test -p vm -j 1 --exclude-tags local-only — 11544 pass, 129 skipped, 6 env-only failures (windows_registry_qualification_test, needs pwsh) on 990a36aaa
• dart test -p chrome --exclude-tags local-only — 4630 pass, 1 skipped (on 0921339ed; the later commits are VM-only service code, tests and docs)
• coverage-lib — 93.51% (32944/35231) on 0921339ed; vulkan_device_probe.dart 80/85
• Other targeted/local-only validation: chat app flutter test test test_vm and the Chrome chat-app set (pass), flutter analyze in example/chat_app, validation_coverage_test, package_skills_test, check_released_changelog_sections_test, docs site build (pass), and the FTL device runs below. Rerun on the new head: format, analyze, platform boundaries, qwen35_vulkan_restriction_test and vulkan_device_probe_test (82 pass), the tooling tests, the chat app VM and Chrome sets and flutter analyze.

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
root-vm	full VM suite on 990a36aaa	macOS arm64, pinned v0.9.0	PASS except 6 env failures	6 windows_registry_qualification_test failures need pwsh (env-only, same as main)
root-chrome	browser suite	Chrome, macOS	PASS (4630)
coverage-lib	lib/ line coverage	macOS arm64	PASS (93.51%)	same 6 env failures plus one LiteRT-LM embedding deadline timing failure under coverage load that passes alone; empty {} reports from model-less local-only files removed before format_lcov
android-arm64-device-smoke (with model load/generation)	production path without bypass: CPU then Vulkan in one process; media, ngram speculation with verify batches of 41/49 tokens, embeddings, cancel/reuse/reload	FTL Pixel 9 Pro caiman API 35 (Mali-G715, subgroup 16), Qwen3.5-0.8B Q4_0 + mmproj F16, Vulkan 25/25 layers	PASS	matrix-20gdqfruugqqn; journal bypass:false; Dart log ggml 0.29.0 has the Vulkan small matmul tile fix; 10/10 text outputs identical to the CPU (raw strings), embedding cosine 0.99987/0.99984, no typed errors
android-arm64-device-smoke (same harness)	same	FTL Pixel 7 panther API 33 (Mali-G710, subgroup 16)	PASS	matrix-3qsokm5qfa3ha; bypass:false; ggml 0.29.0 has the Vulkan small matmul tile fix; 10/10 text outputs identical to the CPU (raw strings), embedding cosine 0.99982/0.99977, CLIP on CPU, no typed errors

Device runs used the build of 0921339ed. The later projector rule does not change the 0.8B path they exercised, since the file-name rule already kept that projector on the CPU, so they were not repeated.
| chat-app-micro-batch-sweep | sweep default/explicit sizes | N/A | N/A | already run with the restriction bypassed on Pixel 9 Pro (v0.9.0 and v0.8.0-3, #948 comment); this PR's production path is the bypass's path, and the follow-up harness above covers it without the bypass |
| high-risk-exact-head-independent-qa | exact-head audit | fresh Claude Opus auditor session | ACCEPT | full audit at fe3c4c575/ce8ddf674, delta re-audit at 990a36aaa/d53971eef; see below |
| structured-output-adversarial | parsers/grammar | N/A | N/A | no parser, grammar or template change; grammar/tool calls on Vulkan covered by the #948 device runs |
| release-metadata-verification | release docs pins | N/A | N/A | no pins or install snippets changed |
| litert-lm-*, web-*, webgpu-*, gemma4-*, iOS/macOS/Linux/Windows runtime rows | other runtimes | N/A | N/A | behavior changes only on Android with actual Vulkan placement. generationCapabilities, supportsVision and supportsAudio read the ggml version once per service on every platform (it returns null on Windows, where the default asset lacks ggml_version), but the result only matters for an Android Vulkan model |

Review Notes

• Independent review status: ACCEPT, no blocking findings (see below).
• CI status / head SHA: 34 pass, 9 skipped, 0 fail on 990a36aaa. Previous head: 34 pass, 9 skipped, 0 fail on fe3c4c575 (first attempt: one Linux VM failure in model_download_manager_io_concurrency_test "two processes sharing a cache root", unrelated to this PR, passing locally; the failed jobs passed on rerun)

High-risk regression review

• Classification: high-risk (backendRuntime)
• Implementation task: this PR
• Independent blocking QA task: ACCEPT from a fresh Claude Opus auditor session, uninvolved in the implementation (operator-owned). Full audit at fe3c4c575 / ce8ddf674, delta re-audit at exact head 990a36aaa / base d53971eef. Threshold: fix 459a1ba5 is in ggml 0.27.0 and an ancestor of the 0.28.0 bump (v0.7.0=0.27.0, v0.8.0=0.28.0, v0.9.0=0.29.0); unknown or unparsable versions fail closed. Older-runtime refusals are unchanged apart from wording, now pinned by tests; the projector rule is the only mtmd path. 31 mutants: survivors are equivalent or more conservative, plus M35, now killed (M16, M19, M23, M35b-d re-killed at the delta). Delta: lib logic and PR diff identical against the new base (remerge-diff empty), Speak with Chatterbox and Pocket TTS, with predefined voices #1792 TTS paths unaffected (166 tests), doc fixes accurate. Suites: VM 11535 pass (6 env-only pwsh failures), coverage-lib 93.54%, chat app 689 VM / 132 Chrome. FTL matrix-226337fgaz7n6 (Pixel 9 Pro, Mali-G715, no bypass): 10/10 CPU-vs-Vulkan text exact, embedding cosine 0.99987/0.99984, projector on CPU. Readiness CLI (high_risk_readiness.dart, exact head and base): exit 2 unverifiedPrerequisites, evidence internally consistent.
• Exact head / current base: 990a36aaa / d53971eef. origin/main was merged twice with merge commits (c21d46e7f for Speak with LFM3-Audio and Soprano through the native model-type opt-in #1791, c4b3404cd for Speak with Chatterbox and Pocket TTS, with predefined voices #1792), both without textual conflicts. Speak with LFM3-Audio and Soprano through the native model-type opt-in #1791 and Speak with Chatterbox and Pocket TTS, with predefined voices #1792 change TTS code in llama_cpp_service.dart and docs in separate hunks; Speak with Chatterbox and Pocket TTS, with predefined voices #1792 adds no projector path, and TTS projectors keep their placement because the CPU rule matches only qwen35 or an unreadable architecture on small-tile Android Vulkan.
• Production-branch deletion, bypass, or miswire proof: forcing the ggml probe to unfixed fails the lifted-path tests; forcing it to fixed fails the older-runtime tests; dropping the unset-size cap or the actual-placement KQV branch on the lifted path fails qwen35_vulkan_restriction_test.dart; disabling the projector rule, applying it to every model, or dropping the unreadable-architecture case fails its projector placement tests; changing the older-runtime refusal wording fails the exact-message test. The device runs used no bypass define (journal bypass:false) and logged the probe result.
• Affected-family real-model/artifact evidence: Qwen3.5-0.8B Q4_0 + unsloth mmproj F16 on Mali-G715 and Mali-G710 via FTL (above), plus Android llama.cpp Vulkan on Pixel 9 Pro (Mali-G715): wrong output for prompts over ~32 tokens, then abort under a grammar #948's bypass runs on Pixel 9 Pro, Pixel 9 and Pixel 7.
• Explicit unavailable-family or other N/A evidence: No older-runtime device run (the older path is the code that shipped in 0.18.0, unchanged except for the probe gate, and is covered by unit tests with injected ggml versions). Other Qwen3.5 sizes, CLIP on Vulkan (now kept on the CPU on these GPUs), Mali-G68 and Adreno not run.
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-1800.