Skip to content

Fail the LiteRT-LM smoke when a desktop runtime lacks the in-process LiteRT marker - #1849

Closed
leehack wants to merge 3 commits into
mainfrom
l2-litert-abi-marker-ci
Closed

leehack wants to merge 3 commits into
mainfrom
l2-litert-abi-marker-ci

Conversation

@leehack

@leehack leehack commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • tool/litert_lm_library_smoke.dart (the LiteRT-LM Smoke library step) now fails when the pinned Linux or Windows LiteRT-LM runtime does not export litert_lm_native_in_process_litert_abi_version or does not return 1, so a repin that drops the marker fails CI instead of only hardware runs.
  • Covers Linux x64 and Windows x64 (the existing smoke matrix). Linux arm64 is not covered: an ubuntu-24.04-arm entry failed in flutter-action ("Unable to determine Flutter version ... 3.47.1 architecture: arm64"), so it was dropped.
  • The smoke body is now runLiteRtLmLibrarySmoke, which enforces the marker and puts the value in the RESULT JSON as inProcessLiteRtAbi; main() only prints it.
  • Maintainer note in native-and-web-sync.md, including that rolling the pin back to a pre-marker tag (v0.18.1-4, v0.19.0, v0.19.1, v0.19.1-1) fails the smoke on Linux/Windows by design.

Refs #1837 #1845

Production-readiness scope

  • User-facing scope: None directly; CI guard for the embedding gate from #1830.
  • Supported platforms/paths: CI (Linux x64, Windows x64), tooling.
  • Unsupported or intentionally unavailable paths: Apple/Android runtimes are not required to export the marker (helper returns early).
  • Out of scope / follow-ups: Linux arm64 marker check (needs an arm64 runner with Dart, or a static ELF export probe of the pinned archive); to be filed under #1845.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Unsupported platform/option combinations fail loudly with actionable diagnostics or are clearly documented as unavailable. (N/A: tooling/docs only)
  • Public API docs, README/website docs, examples, support matrices, and changelog entries are updated where relevant. (changelog N/A: not user-visible)
  • Regression coverage covers the original issue plus key negative/version-skew paths where relevant.
  • Public capability changes update the validation coverage registry. (N/A: no capability change)
  • Security/privacy review completed: no secrets, tokens, signed URLs or secret-bearing paths added.
  • Follow-up work is tracked in GitHub Issues, or marked "None" above.

PR type guidance

  • Bugfix PR (CI gap): no runtime code; negative path covered by tests below.

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • dart test -p vm -j 1 --exclude-tags local-only test/unit/tooling/
  • dart test -p chrome -- N/A: no web code
  • coverage-lib -- N/A: no lib/ change
  • Docs: ./tool/docs/build_site.sh --port 4817 passes

Failing-first / mutation evidence

  • Tests compile c_api_version.c three ways (symbol absent, returns 0, returns 1). Absent and 0 throw StateError for linuxX64/linuxArm64/windowsX64; 1 passes; macOS/Android skip the check.
  • runLiteRtLmLibrarySmoke tests (injected library) assert inProcessLiteRtAbi == 1 on Linux/Windows and StateError for absent/0.
  • Mutants killed: (1) replacing the requireInProcessLiteRtMarker call with a constant null; (2) wrapping it in try/on StateError; (3) tolerating a missing export.

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
static-format-analyze format, analyze macOS, Flutter 3.47.1 Pass clean
tooling suite test/unit/tooling/ macOS VM Pass litert_lm_library_smoke_test.dart 65/65; only windows_registry_qualification_test.dart fails locally (no pwsh)
LiteRT-LM Smoke pinned v0.19.1-2 desktop bundles CI linux x64, windows x64 Pass (run 42636717551) prior head dc058a2a: run 42610329205 passed on both

Review Notes

  • Independent review status: pending (standard-risk)
  • CI status / head SHA: green / b068961a2

Risk classification

  • Classification: standard (classify_high_risk_changes.dart on the current diff)
  • Implementation task: lane 2 of #1845
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

leehack added a commit that referenced this pull request Nov 9, 2026
…LiteRT marker (#1849)

Constituent of the 0.19.2 batch integration (#1845). Squash of #1849 at pinned head b068961 (git diff main...b068961).
@leehack

leehack commented Nov 9, 2026

Copy link
Copy Markdown
Owner Author

Landed through the 0.19.2 batch integration PR #1870 (rebase merge, one commit per constituent; this PR's pinned head was audited/CI-gated there).

@leehack leehack closed this Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1849.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant