Repository navigation
Reuse a cached native bundle only while it matches its archive; pin Seed-OSS ASCII trim - #1572
Merged
Merged
Conversation
A Seed-OSS reply that starts with a no-break space before <seed:think> and ends a thought parses as after a template that opens the thought, as llama.cpp does; a Unicode trim there (mutant B12 of #1569) now fails.
The build hook writes extracted.json beside each llama.cpp bundle's extracted/ directory, recording the archive's SHA-256, size and modification time and every extracted file's size. A later build reuses extracted/ only while the record matches the cached archive and the files; otherwise it extracts the archive again or downloads it. A cache without a record is extracted again once from its cached archive, offline: verified for the pinned release or a cached SHA256SUMS, as is for an override cached before the hook kept SHA256SUMS. Hook tests that seed fake extracted/ dirs now write a matching archive and marker; without them the hook would download the real bundles.
Contributor
|
Chat app preview removed for |
15 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Clears two items on tracker #1565:
.dart_tool/llamadart/native_bundles/<tag>/<bundle>/extracted/directory whose files are not what the cached archive beside it extracted to. After each extraction it writesextracted.jsonbesideextracted/(archive SHA-256, size, modification time; every extracted file's relative path and size). Reuse needs that record to match the archive in hand and the directory's files; the archive is re-hashed only when its modification time changed. Otherwise the hook extracts the cached archive again (digest-verified as before) or downloads it if the archive is gone._endsUnopenedSeedOssThought.SHA256SUMSSHA256SUMSDesign choice: re-extraction instead of native-header validation for legacy caches, because it proves
extracted/equals the archive (headers would accept a real-but-wrong library) and needs no network. File sizes rather than per-file hashes keep reuse cheap; the archive hash is skipped on an unchanged mtime because hashing the 688 MiB CUDA sidecar costs about 4 s at the measured ~180 MB/s.Production-readiness scope
llamadart_native_pathbundles are unchanged (never reused fromextracted/).extracted/whose archive was deleted is no longer reused offline; the build downloads it again. A same-size in-place edit of a library is not detected (sizes, not hashes).build_hook_stable_diffusion_integration_test.dart("the stable_diffusion companion selects and links it in process" downloads a real LiteRT-LM macOS bundle onmaintoo).Completeness checklist
native-build-hooks.md, both changelogs; no API change)PR type guidance
_acquireBundleDirectoryreused anyextracted/holding complete-looking library names, so a text stub there was copied into.dart_tool/lib. Regression tests:build_hook_archive_digest_test.dartgroup "a cached extraction" (7 tests, all fail onmain) andnative_extraction_marker_test.dart. Platforms: all.Test Plan
dart run tool/prepare_workspace.dartdart format --output=none --set-exit-if-changed .dart analyzedart run tool/testing/check_platform_boundaries.dartdart test -p vm -j 1 --exclude-tags local-only(macOS arm64, isolated TMPDIR): +9658 ~126, all passeddart test -p chrome --exclude-tags local-onlytest/unit/hooktwice at default concurrency (isolated TMPDIR each): +555, all passed both timestest/unit/hookwith outbound HTTP(S) blocked (sandbox-exec): all pass except one test that also needs the network onmain(see Out of scope)main'shook/build.dart; the B12 test fails with UnicodetrimLeft()Matrix Evidence
static-format-analyzeroot-vmnative-hook-bundlesTest Linux VM (tooling),Test Native (windows-latest),Test Native (macos-latest)(link test skipped on Windows by design)root-chromedocs-sitehigh-risk-exact-head-independent-qa/opt/UnitySrc/personal/llama/llamadart-evidence/pr-1572/evidence-1572-04c3576-base-04821e0ab.jsonstructured-output-adversarialMutation check (
llamadart-evidence/sweep-1565-final/mut.py, each mutant alone againstnative_extraction_marker_test.dart+build_hook_archive_digest_test.dart):trimLeft()in_endsUnopenedSeedOssThought"Header check accepting anything" does not apply: the design uses no header check.
Review Notes
High-risk regression review
artifactConsumer)