Repository navigation
feat(localmodel): classify non-blessed entries by origin and plan fit with PlanFit - #638
Open
ion-alpha-dev wants to merge 7 commits into
Open
ion-alpha-dev wants to merge 7 commits into
ion-alpha-dev wants to merge 7 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
The fit judged models against the card's total memory, so a model that fit an empty card could fail to load while the desktop held part of it. Free memory now comes from the same nvidia-smi query Detect already runs, as Box.VRAMFreeBytes, so the fit no longer needs a second probe. The separate free-VRAM probe and its hook in the models command are removed; a fit with no free figure falls back to the total. Drivers 610 and later print the CUDA version as "CUDA UMD Version:" instead of "CUDA Version:". The banner parser accepts both labels, so the detected CUDA version is not lost on those drivers. Signed-off-by: Ion Alpha <[email protected]> Signed-off-by: Ion Alpha <[email protected]>
The table in TestPolicyRejectsMutatedClaims was not gofmt-aligned, so the repository format gate printed this file and failed. The change is whitespace only: the case names and mutations are untouched. Signed-off-by: Ion Alpha <[email protected]> Signed-off-by: Ion Alpha <[email protected]>
The earlier gofmt-only commit on this file is outside this change and undoes gofumpt alignment that CI's lint enforces. Putting main's content back leaves the branch diff limited to the hardware probe. Signed-off-by: Ion Alpha <[email protected]>
… with PlanFit A model in Config.Catalog whose trust tier is not blessed is now classified by the origin it names (file, URL or Hub repository) and never as the pinned catalog source Serve runs. A user's entry from the catalog overlay therefore cannot reach the trusted class by its id, and the publisher it names no longer vouches for other Hub models. Fit now goes through ModelSpec.PlanFit against the machine the fit view uses: free VRAM rather than the card's total, system RAM after the OS reserve, and the cache counted at the chosen window. Host.PlanFit and Host.Rank expose the settings and the ranking, and the Trust, KVType and Placement constants are re-exported so a host can build and read them. A host that supplies its own Config.Catalog must now set Trust to TrustBlessed on the entries it wants served. Signed-off-by: Ion Alpha <[email protected]>
The patch coverage report showed the multi-file branches of the entry origin check and Host.Hardware unexercised. Cover a directory model whose files all come from the claimed Hub repository, one that mixes in another host, one fetched wholly elsewhere and one that names no download, and read the box back through the same hook Machine uses. Signed-off-by: Ion Alpha <[email protected]>
…origin needs Serve refused every entry that was not a vetted catalog source with ErrNotServable. The flynn binary hands overlay entries (models add, catalog.user.json) to Host.Serve, so none of them could run. Serve now runs any entry in the host's catalog. An entry below TrustBlessed is classified by the origin it names and is refused with ErrNotAdmitted where the sandbox gives less isolation than that origin needs. A reference that names no catalog entry is still ErrNotServable. Admission reads the isolation through a hook the tests set, so an entry that needs a microvm can reach the admitted path on a machine that has none. entrySource decided whether a Hub claim was fetched from its repository with a prefix test on the URL text, which accepted "/acme/tiny/../other/x.gguf". The path is now cleaned first, an encoded parent segment included. Free VRAM comes from Box.VRAMFreeBytes, which replaces hardware.FreeVRAM. This depends on #640 and must merge after it. Tests assert the exact source kind per origin (file, URL, Hub repository) and serve a TrustLocal overlay entry end to end. Signed-off-by: Ion Alpha <[email protected]>
ion-alpha-dev
force-pushed
the
feat/localmodel-planfit-untrusted-overlay
branch
from
October 31, 2026 10:34
bfaf20e to
525296d
Compare
The repository fails a declaration comment of 15 lines or more. The Serve comment is cut to the admission rule, the refusal for a reference outside the catalog, and the adoption and handshake rules. Signed-off-by: Ion Alpha <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A
Config.Catalogentry belowTrustBlessedis classified by the origin it names (a local file, a direct URL, a Hub repository), never as a pinned catalog entry.Host.Serveruns such an entry under the isolation that origin needs, and refuses it withErrNotAdmittedwhere the sandbox gives less. A reference that names no catalog entry staysErrNotServable. The host's fit judgement now usesModelSpec.PlanFit(Machine, FitSettings), with free VRAM read fromBox.VRAMFreeBytes.Why
The flynn binary passes overlay entries (
flynn models add,catalog.user.json) toHost.Serve. An earlier revision of this PR refused every entry that was not a vetted catalog source, so none of them could run. They are now served, as the untrusted source their origin implies, so admission applies the stricter isolation. The Hub-claim check also accepted/acme/tiny/../other/x.ggufas a file ofacme/tiny; the path is now cleaned before it is compared.How to verify
./dev/test ./localmodel/... ./cmd/flynn/...and./dev/lint.TestOverlayEntryIsServedUnderTheIsolationItsOriginNeedsserves aTrustLocalentry end to end: refused withErrNotAdmittedat no isolation and at kernel isolation, served at microvm.TestEntrySourceNamesTheOriginItClaimsasserts the exact kind per origin, plus the.., encoded.., plain http and lookalike host cases. Each new test fails when its cause is removed: restoring the catalog-only gate inServe, classifying everything as a file, comparing the URL text without cleaning the path, admitting against the real sandbox instead of the hook, and treating a zero free VRAM as free.Notes for reviewers
This branch is rebased onto #640 and uses
Box.VRAMFreeBytes, which replaceshardware.FreeVRAM. #640 must merge first; until then this PR's diff against main also shows its commits. No exportedlocalmodelsignature changed:SetMachineandSetProvidedlive inexport_test.go../dev/apidiffreports onlyencoding/jsontojsontextlines for other packages, from the local toolchain, and nothing underlocalmodel. Admission reads the isolation through an unexported hook so a test can reach the admitted path for an entry that needs a microvm on a machine that has none.