Follow-ups from #112 audit (desktop in-process LiteRT) #113

Activity

1. leehack

   added

   enhancementNew feature or requestNew feature or request

   priority:P3Watch or strategic work blocked by upstream/runtime/design dependenciesWatch or strategic work blocked by upstream/runtime/design dependencies

   on Nov 8, 2026
2. leehack commented on Nov 8, 2026

   

   leehack

   on Nov 8, 2026

   OwnerAuthor

   More actions

   Triage: enhancement, P3. None of these block v0.19.1-1. Item 3, the Windows timeout margin, is worth checking just before the dispatch.
3. leehack commented on Nov 8, 2026

   

   leehack

   on Nov 8, 2026

   OwnerAuthor

   More actions

   From the llamadart#1808 re-audit: llamadart cannot tell custom C API 1.0 runtimes v0.19.0/v0.19.1 (crash-prone desktop embeddings, #1367/llamadart#1795) from v0.19.1-1 by API version, so a custom LLAMADART_LITERT_LM_LIB_DIR pointing at them reaches native embedding creation on Linux/Windows (documented in llamadart known issues). Incidental binary differences exist (v0.19.1-1 Windows LiteRtLm.dll has no libLiteRt.dll import and exports LiteRtDispatchGetApiVersion; Linux libLiteRtLm.so has no DT_NEEDED libLiteRt.so), but a durable probe needs an exported capability marker here (e.g. an in-process-LiteRT / embeddings-qualified flag or a bumped C API minor) that llamadart can check before creating an embedding engine.
4. leehack commented on Nov 8, 2026

   

   leehack

   on Nov 8, 2026

   OwnerAuthor

   More actions

   Status after #115:

   Item	Status
   Latest comment: runtime capability marker	#115 adds litert_lm_native_in_process_litert_abi_version(). It returns 1 on v0.19+ builds on every platform, and the build verifies their linkage; it returns 0 before v0.19. Treat a missing export as 0. Contract: docs/platform_strategy.md.
   1. Interposition hazard	Deferred. Hiding the LiteRt* exports or linking with -Bsymbolic changes the export surface and symbol binding of every desktop host. The prebuilt-accelerator fallback (item 2) still resolves LiteRt* from the host, and the change needs real-model CPU/GPU requalification. Dropping the shipped libLiteRt/accelerator also needs a llamadart hook change. llamadart is not affected because Dart loads with RTLD_LOCAL.
   2. GPU fallback dlopen	Deferred. Removing the fallback means carrying a new upstream LiteRT gpu_registry.cc source patch, which needs L4 Vulkan/D3D12 GPU qualification. The fallback never triggered in any run.
   3. Windows CI time	Done in #114 (timeout raised to 150 minutes).
   4. Linkage-check test gap	Done in #115. main() calls the new validate_runtime(), and a new test fails on both build paths if the linkage check is dropped (mutation-checked).
   5. L4 bundle hashes	Deferred. This is missing evidence from a past manual GCE run, so no code change fixes it. It needs a fresh L4 run that records bundle SHA-256s, best done when qualifying the next release that carries #115.
   6. Docs nit	Done in #115. The decision message no longer points future tags at the prebuilt libLiteRt.
5. leehack

   added a commit that references this issue on Nov 8, 2026

   Export an in-process LiteRT marker from the runtime bridge (#115)

   ...

   3e40b53
6. leehack commented on Nov 8, 2026

   

   leehack

   on Nov 8, 2026

   OwnerAuthor

   More actions

   #115 merged (3e40b53): litert_lm_native_in_process_litert_abi_version() returns 1 on all 8 targets (verified by disassembling CI artifacts). Not yet in a release; v0.19.1-1 lacks it.

   Follow-ups from the #115 audit (non-blocking):

   • tools/litert_lm_finish_reason_smoke.py main() ignoring the marker result survives all tests; add a test of main()'s exit code on mismatch.
   • Renaming the define only in native/bridge/BUILD.bazel passes unit tests (only the desktop --abi-only real-binary check catches it; fails safe to 0). Pin the define name across BUILD.bazel and build_upstream_runtime.py in a test.
   • Docs: state in docs/platform_strategy.md that marker 1 covers the LiteRT-LM→LiteRT weight handoff only, not GPU qualification (the gpu_registry.cc prebuilt-accelerator fallback is still the Diagnose NVIDIA L4 desktop WebGPU initialization crashes before enabling consumer GPU #47 crash class), and that an RTLD_GLOBAL preload of a separate libLiteRt on Linux recreates Linux/Windows: external-weight handoff crosses the libLiteRt C++ ABI boundary (embeddings fail/crash) #111 while the marker still reads 1. Mention the marker in README's bridge ABI section.
7. leehack

   added a commit that references this issue on Nov 8, 2026

   Pin the in-process LiteRT marker contract with tests and docs (#117)

   ...

   0b15fb3
8. leehack commented on Nov 9, 2026

   

   leehack

   on Nov 9, 2026

   Last edited by leehack

   OwnerAuthor

   More actions

   2026-11-09 material follow-through:

   • The marker is now released in v0.19.1-2, owner 0b15fb3 / upstream7a03724c. Publish42546469976 succeeded by validating the prepared candidate. The prior “not yet released” status is superseded.
   • #117 supplies the marker-result main() exit-code test, define-name pin test and linkage/GPU/global-preload caveats. Those focused test/doc follow-ups are source-delivered and included in -2, not still unimplemented.
   • Core#1830's receipts now bind bundle hashes to L4 Linux Vulkan/Windows D3D12 GPU3/3 and CPU2/2 results, plus three-host embedding controls. This addresses the previously missing Linux L4 bundle-hash evidence for that candidate, not arbitrary future artifacts.
   • Core#1819 stops bundling/preloading unused desktop libLiteRt for fixed-runtime layouts. Historical explicit override compatibility remains scoped.

   Retain external global-library interposition and prebuilt-accelerator fallback limitations. Marker1 describes the ABI-valid weight handoff, not universal GPU safety. Concurrent10:03UTC merge of batch#1843 as1fdc222f source-delivers core marker admission/pin on GitHub main; it is still not in public0.19. No issue-state, artifact or setting change by this scan.