Skip to content

Pin readiness-tool and public thought-parse test gaps (#1861) - #1878

Closed
leehack wants to merge 2 commits into
mainfrom
chore/1861-policy-test-gaps
Closed

leehack wants to merge 2 commits into
mainfrom
chore/1861-policy-test-gaps

Conversation

@leehack

@leehack leehack commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Tests only, from #1861 (0.19.3 plan #1874, lane 6). Classified high-risk only by path (readiness policy and structured-output test files); no production code changes.
  • high_risk_readiness_test.dart: N2 (a PR that removes a generated header and edits a comment is rejected, both revision orders), N3/N3b (every Dart file the pin tooling reads is in rawTextConsumedSources: the DEFAULT_* Dart paths in tool/native/sync_native_release_pins.py, which include hook/build.dart and litert_lm_runtime.dart, plus nativeLlamaCppTagPath and the bridge-tag pin paths), N4 (a comment edit must end its line in both revisions).
  • public_parse_unopened_thought_test.dart: M11, ChatTemplateEngine.parse ends a Hermes / DeepSeek V3 / Qwen3-Coder thought at </think> before any <think>.
  • The lib/llama_cpp_bindings.dart Dartdoc reword was dropped: it is not comment-only at base and would need a separate PR with a recorded maintainer waiver.

Production-readiness scope

  • User-facing scope: None.
  • Supported platforms/paths: VM tests.
  • Unsupported or intentionally unavailable paths: N/A
  • Out of scope / follow-ups: #1861.

Completeness checklist

  • Declared scope is fully implemented, or explicitly reduced above.
  • Regression coverage covers the original issue.
  • Security/privacy review completed: none needed.

PR type guidance

  • Docs-only PR: N/A (tests only; no runtime behavior changes).

Test Plan

  • dart run tool/prepare_workspace.dart
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • high_risk_readiness_test.dart, public_parse_unopened_thought_test.dart, test/unit/core/template, test/unit/test_structure, VM, isolated TMPDIR
  • coverage-lib N/A: no lib/ behavior change

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
N/A tests only macOS VM pass mutants below

Mutants killed: N2 (read only head comments), N3 (drop each of hook/build.dart, litert_lm_runtime.dart, web_bridge_setup.dart, native_release_pins.dart from the set), N4 (check only the head gap; only the base gap), M11 (parse passes promptOpensNoThought: true).

Review Notes

  • Independent review status: pending (high-risk by path; needs an independent auditor before ready)
  • CI status / head SHA: 691a2a3

High-risk regression review

  • Classification: high-risk (surfaces: backendRuntime, regressionPolicy, structuredOutput; all by path, tests only)
  • Implementation task: this lane (l193-sweep)
  • Independent blocking QA task: pending
  • Exact head / current base: 691a2a3 / f78cafa
  • Production-branch deletion, bypass, or miswire proof: N/A (no production code)
  • Local evaluator check: a draft v2 evidence file (tests-only change, cited compiled-grammar test) run through tool/testing/high_risk_readiness.dart exits 2 (unverifiedPrerequisites); the independent auditor builds the real one.
  • Affected-family real-model/artifact evidence: N/A (no production code)
  • Explicit unavailable-family or other N/A evidence: N/A
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

Refs #1861, Refs #1874

@leehack

leehack commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Landed on main via batch integration PR #1895 (rebase merge, main ba4e0e1b; tree identical to the audited integration head 5ffd1224).

@leehack leehack closed this Nov 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1878.

@leehack
leehack deleted the chore/1861-policy-test-gaps branch November 10, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant