[leehack]

Summary

• Before a LiteRT-LM GPU load on Linux, list the Vulkan physical devices through the system loader (libvulkan.so.1, Dart FFI, Vulkan 1.0 entry points). If every device is VK_PHYSICAL_DEVICE_TYPE_CPU (Mesa llvmpipe/lavapipe, SwiftShader), the GPU backend could only pick a software adapter. An explicit GPU load (ComputeDevice.gpu, LiteRtLmBackendPreference.gpu, the deprecated preferredBackend: 'gpu') now throws LlamaUnsupportedException, and a GPU hint under ComputeDevice.auto loads on the CPU with a warning.
• Root cause (LiteRT-LM GPU accepts a Mesa llvmpipe software adapter on Linux (no crash on v0.19.x; small models can return wrong text) #572): LiteRT-LM's Linux GPU backend runs Dawn on Vulkan, and Dawn falls back to the CPU adapter when no hardware one exists (Selected adapter: llvmpipe ... adapterType=CPU / Software). The C API does not expose the selected adapter, and llamadart did not check one, so small models ran on llvmpipe and could return wrong text.
• Layer choice: the system Vulkan loader is the same one Dawn uses in-process, so the device list is an explicit probe that needs no native change. Dawn ranks CPU adapters last, so "every device is software" is a sound refusal condition. Any hardware device, an empty list, a VK_INCOMPLETE (partial) list, or a loader that can't be opened or queried keeps the old behavior: the runtime reports No adapters found as a typed engine-create error itself. The llama.cpp backend's VulkanDeviceProbe reads through libllamadart, which a LiteRT-LM-only app may not load, so it isn't reused here.
• The probe runs once per isolate, only on Linux (liteRtLmGateAbi()), and only when a GPU engine is about to be used. Service tests inject it; a service built with an injected client factory skips it. debugLiteRtLmSystemVulkanAdaptersOverride replaces the system probe so the default selection is tested without probing the host.

Fixes #572

Production-readiness scope

• User-facing scope: On Linux hosts whose only Vulkan devices are software rasterizers, LiteRT-LM GPU fails at load with a typed error that names the devices, instead of silently running on llvmpipe; auto with a GPU hint runs on the CPU.
• Supported platforms/paths: Native LiteRT-LM on Linux x64 (the only Linux ABI with a LiteRT-LM GPU backend). macOS, iOS, Android and Windows are unchanged.
• Unsupported or intentionally unavailable paths: Explicit GPU on a software-only Vulkan host throws LlamaUnsupportedException ("LiteRT-LM GPU on linux_x64 needs a hardware Vulkan device, but every Vulkan device here is a software rasterizer (VK_PHYSICAL_DEVICE_TYPE_CPU: "llvmpipe ...") ..."). checkLoad opens no driver, so it still reports such a load as supported (documented in known issues).
• Out of scope / follow-ups: Windows D3D12 WARP (Microsoft Basic Render Driver) is not probed: there is no Windows evidence of a software-adapter selection, and the Vulkan list would not describe D3D12. Exact selected-adapter reporting would need a litert-lm-native C API change; the device-list rule doesn't depend on it.

Completeness checklist

• Declared scope is fully implemented, or explicitly reduced above.
• Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable.
• Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (support matrix, known issues, troubleshooting, getting-started skill, both changelogs; no public API added)
• Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
• Public capability changes update the validation coverage registry with a pack or a reasoned, issue-linked exemption; inventory checks pass (or N/A with scope). N/A: no new public API, capability report or matrix row; validation_coverage_test.dart passes.
• Security/privacy review completed: the error and warning carry only Vulkan device names.
• Follow-up work that is useful but not required for this PR is tracked in GitHub Issues, or explicitly marked "None" above.

PR type guidance

• Bugfix PR: root cause and layer above; regression tests: litert_lm_gpu_adapter_test.dart (FFI struct reading through fake loader functions, software-only rule, unavailable loader) and the GPU loads on software Vulkan adapters group in litert_lm_service_test.dart (refused for 3 explicit selectors, auto falls back to CPU with warning, hardware beside llvmpipe / empty list / unavailable probe start a GPU engine, CPU load never probes). The default probe selection tests cover the production wiring: a Linux service with the production client probes, while macOS, Windows, or an injected legacy or runtime client factory does not. A VK_INCOMPLETE list (a spec-conformant fake fills its capacity and returns 5) is unavailable, so the load proceeds. Mutations killed: the gate result replaced by null; false && / true && in the default selection; dropping the clientFactory == null check; accepting VK_INCOMPLETE as a complete list.

Test Plan

• dart run tool/prepare_workspace.dart
• dart format --output=none --set-exit-if-changed .
• dart analyze
• dart test -p vm -j 1 --exclude-tags local-only (CI; locally ran test/unit/backends/litert_lm/, test/unit/tooling/, test/unit/test_structure/)
• dart test -p chrome --exclude-tags local-only N/A: native-only change (dart:ffi), no Web path touched; CI runs it.
• coverage-lib matrix row: 93.62% lib line coverage (threshold 70); litert_lm_gpu_adapter.dart 52/56 lines. The local macOS run also failed windows_registry_qualification_test.dart and four /var vs /private/var tmpdir assertions in model_download_manager_io_concurrency_test.dart; the Windows test also fails with origin/main's lib/ and test/.
• Other targeted/local-only validation: real llvmpipe host on GCE (below).

Matrix Evidence

Matrix row	Scope covered	Platform / model / backend	Result	Evidence / notes
coverage-lib	lib line coverage	macOS arm64 local	PASS	93.62% (33267/35534)
Real llvmpipe host	probe + public LlamaEngine(LlamaBackend()) load/generate	GCE n2-standard-8, Ubuntu 24.04, Mesa llvmpipe only (PHYSICAL_DEVICE_TYPE_CPU, LLVM 20.1.2), LiteRT-LM v0.19.1-1, SmolLM2-135M .litertlm ccdc5c85…, head 1d0231af3 (the head only adds @TestOn('vm') to the new test after that)	PASS	gpu: LlamaUnsupportedException naming "llvmpipe (LLVM 20.1.2, 256 bits)"; auto + GpuBackend.vulkan: warning, LiteRT-LM cpu, "The capital of France is Paris."; cpu: same text
Real hardware Vulkan host	GPU still allowed beside llvmpipe	GCE g2-standard-8 L4, Ubuntu 24.04, NVIDIA 580.178.04 + Mesa llvmpipe, LiteRT-LM v0.19.1-1 (1c93ead8… = pin), Qwen3-0.6B 555579ff…, head e51e76cb	PASS	Explicit gpu 2/2 and auto+GpuBackend.vulkan run on LiteRT-LM gpu (NVIDIA L4 … backend=Vulkan). The probe gives softwareOnly=false. With the loader limited to lvp_icd.json, explicit gpu throws LlamaUnsupportedException and auto falls back to CPU. results
high-risk-exact-head-independent-qa	independent blocking-only review	head e51e76cb vs base ec4bf4f8	PASS	fresh auditor session: 0 P1/P2; mutations killed; main merge clean; P3s on #1837
release-metadata-verification			N/A	not a release-metadata PR
structured-output-adversarial			N/A	no parser, grammar, streaming or template change
litert-lm-v1-public-direct / -service / -speech			N/A	engine creation and C API calls are unchanged; only backend selection before load changes, and the llvmpipe run covers the public service path
litert-lm-json-schema			N/A	no JSON Schema change

Review Notes

• Independent review status: exact-head audit PASSED at e51e76cb (earlier passes at 4b542ac1 and d2baf301); draft until the maintainer merges
• CI status / head SHA: 34 pass, 9 skipped, 0 failed on e51e76cbdef304939837c0e640e8d05372538302 (merge of origin/main ec4bf4f89 into b4347c5a1; no conflicts, Refuse LiteRT-LM GPU on Linux when every Vulkan device is software #1831's own diff unchanged)

High-risk regression review

• Classification: high-risk (backendRuntime)
• Implementation task: this PR's author session
• Independent blocking QA task: a fresh auditor session (no implementation work) PASSED the exact head e51e76cb against base ec4bf4f8: no P1/P2. These mutations were all killed: accepting VK_INCOMPLETE, false && / true && on the Linux check, dropping clientFactory == null, dropping runtimeClientFactory == null. The main merge is clean, and the changelogs only gain the LiteRT-LM GPU accepts a Mesa llvmpipe software adapter on Linux (no crash on v0.19.x; small models can return wrong text) #572 bullet under Unreleased. P3s are filed on #1837. Earlier: the audit PASSED at 4b542ac1 (struct layouts checked against vulkan_core.h, real-loader container runs). Its follow-ups (M1 and M4 mutation gaps, support-matrix wording) are fixed at d2baf301c. The re-audit PASSED at d2baf301c (no P1/P2). Its P3 is fixed at b4347c5a1: a VK_INCOMPLETE list is now unavailable instead of a possibly truncated software-only list, and the refusal names the gate ABI
• Exact head / current base: e51e76cbdef304939837c0e640e8d05372538302 / ec4bf4f89dc224296ec0c8e6201ff13a3339e51e (base is an ancestor of head)
• Production-branch deletion, bypass, or miswire proof: replacing the probe result with null in _gpuBackendForAdapters fails the GPU loads on software Vulkan adapters group. The default-wiring mutations (false &&, true &&, dropping the clientFactory == null check) each fail at least one test (audit M1, fixed at d2baf301c). Accepting VK_INCOMPLETE as a complete list fails is unavailable when the loader reports VK_INCOMPLETE (re-audit P3, fixed at b4347c5a1)
• Affected-family real-model/artifact evidence: L4 hardware-Vulkan run at e51e76cb (results): with NVIDIA L4 + llvmpipe installed, softwareOnly=false and explicit gpu runs 2/2 on Vulkan through the public LlamaEngine with Qwen3-0.6B. With llvmpipe only (VK_ICD_FILENAMES=lvp_icd.json), explicit gpu throws LlamaUnsupportedException naming llvmpipe and auto falls back to the CPU. Also the llvmpipe-only GCE n2 run at 1d0231af (SmolLM2-135M .litertlm, LiteRT-LM v0.19.1-1)
• Explicit unavailable-family or other N/A evidence: none. The L4 run covers the hardware-Vulkan family at the final head
• Readiness evaluator: tool/testing/high_risk_readiness.dart on head e51e76cb / base ec4bf4f8 exits 2 with unverifiedPrerequisites / externalPrerequisitesUnavailable (repository-local evidence internally consistent; external auditor/App/ruleset prerequisites unavailable, as designed)
• Known PR-caused P1 regressions: 0
• Unresolved review threads: 0

[leehack]

L4 hardware Vulkan check: PASS (head e51e76cbdef304939837c0e640e8d05372538302)

The host checked out this head and ran prepare_workspace.dart. The runtime is the hook-fetched v0.19.1-1 linux-x64 bundle, sha256 1c93ead8d8977b0b1e69d0054555881af16e2b6e46a584870ab86cd51c28589c, which equals the pin. The probe uses the public LlamaEngine(LlamaBackend()), loadModel and create, prints probeLiteRtLmVulkanAdapters(), and generates with temp 0 and enableThinking: false.

Platform	Backend	Model (sha256)	Head	Result
Linux x64 + L4, g2-standard-8, Ubuntu 24.04, NVIDIA 580.178.04. Default ICD discovery: Vulkan sees NVIDIA L4 (type 2, discrete) and Mesa llvmpipe (type 4, CPU)	explicit ComputeDevice.gpu, ×2	Qwen3-0.6B 555579ff2f4fd13379abe69c1c3ab5200f7338bc92471557f1d6614a6e5ab0b4	e51e76cb	PASS: probe softwareOnly=false, Selected adapter: NVIDIA L4 … backend=Vulkan, adapterType=Discrete GPU, backend LiteRT-LM gpu, text "France's capital is Paris." (2/2)
same host	preferredBackend: GpuBackend.vulkan (auto) and ComputeDevice.cpu	same	e51e76cb	PASS: auto runs on LiteRT-LM gpu (NVIDIA L4 Vulkan); CPU runs on LiteRT-LM cpu; same text
same host, VK_ICD_FILENAMES/VK_DRIVER_FILES = Mesa lvp_icd.json (llvmpipe only)	explicit ComputeDevice.gpu	same	e51e76cb	Rejected as designed: probe softwareOnly=true; LlamaUnsupportedException "LiteRT-LM GPU on linux_x64 needs a hardware Vulkan device, but every Vulkan device here is a software rasterizer (VK_PHYSICAL_DEVICE_TYPE_CPU: "llvmpipe (LLVM 20.1.2, 256 bits)") …"
same lavapipe-only environment	preferredBackend: GpuBackend.vulkan (auto)	same	e51e76cb	PASS: falls back to LiteRT-LM cpu and logs the "found only software Vulkan devices … running on the CPU" warning; same text

The probe does not reject a host with hardware Vulkan, even with llvmpipe also installed, and it still rejects a llvmpipe-only loader environment. No crash or Dawn error on any run. Cleanup: the VM and the job bucket were deleted, and gcloud compute instances list is empty.

[leehack]

Landed via batch integration PR #1843 (rebase merge), one commit per constituent.

[github-actions]

Chat app preview removed for leehack/llamadart-chat-pr-1831.