Repository navigation
feat(init): add flynn init --local, a first-run wizard for a local model - #642
Merged
Merged
Conversation
…model The wizard takes a machine with no model configured to a working local default. It detects the GPU, free VRAM, RAM, platform, the llama.cpp runtime and any running Ollama or LM Studio server, then offers up to three catalog models ranked by the same fit verdicts as `flynn models --local --fit`, plus the models a running server already holds (adopted with no download). Every download shows its size and asks first; --yes accepts the defaults and still prints the size. The chosen model is probed, and the default is written only after the probe: a model scoring under 90% on tool calls or schemas is set only on an explicit yes, never under --yes. A declined prompt, Ctrl-C or a failed step leaves the previous default as it was. It reuses the models commands for fit, runtime install, fetch, probe and `models use`, and the onboarding prompt reader, so there is one prompt path. Also list the servers named in providers.json in first-run onboarding, point onboarding at `flynn init --local`, and document `models add`, `models remove` and the user overlay in the usage text. Signed-off-by: Ion Alpha <[email protected]> Signed-off-by: Ion Alpha <[email protected]>
Document the first-run wizard beside the runtime it installs: what it reports, that every download asks first, and that the default is only set after the probe passes. Signed-off-by: Ion Alpha <[email protected]>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
The adopted-model probe had no test. This one reaches a fake server through the provider's base URL reference, probes it into a temp data dir store and asserts the profile is recorded under the provider:model spec and not under the bare model id. Signed-off-by: Ion Alpha <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
flynn init --local, which takes a fresh install to a working local default model by composing what theflynn modelssubcommands already do. It also lists providers.json servers in the cloud onboarding prompt, and brings the usage text up to date (models add,models remove,blesssaving to the user overlay).Why
A new user with a GPU has to know the order of
models --local --fit,models install llama.cpp,models fetch,models probeandmodels use. The wizard detects GPU, free VRAM, RAM, OS and runtimes, recommends up to three models using the same fit verdicts asmodels --local --fit, lists models a running Ollama or LM Studio already holds as an alternative to downloading, installs the runtime if none is present, fetches or adopts, probes, and only then sets the default. Every download asks first and shows its size. Ctrl-C or a declined prompt leaves the default unchanged.--yes,--model <id>and--data-dirare honoured, and there is one prompt path shared with the other prompts.How to verify
Run
go test ./cmd/flynn -run 'TestInit|TestServerModelSpec|TestOnboardListsConfiguredServers|TestLiveInitEnv|TestProbeServerModel'; the wizard is driven through an injectable environment, so no GPU is needed. By hand:flynn --data-dir <dir> init --local --yes --model <id>(the global flag goes before the subcommand).Notes for reviewers
Run on Windows 11 with an RTX 4070 Ti (12 GB): it detected 11.0G free of 12.9G VRAM, installed the cuda-13.3 runtime, and fetched and digest-verified gemma4:12b. The probe then failed with a client timeout while checking the launched listener, so the default was correctly left unset.
flynn models probe gemma4:12bfails the same way without the wizard, ininternal/localmodel, which this PR does not touch; a follow-up is filed.Recommendations follow the catalog ranking (largest first), so the Enter default under
--yescan be a split GPU+RAM placement. I kept the ranking so the verdicts matchmodels --fit; whether the default pick should change is left to the maintainer.probeServerModelis covered byTestProbeServerModelStoresProfileUnderServerSpec: a fake OpenAI-compatible server reached throughOLLAMA_BASE_URL, a temp data dir store, and an assertion that the profile is stored under theprovider:modelspec and not the bare model id. Replacing the spec passed toprobeAndStorewith a bare-id entry makes it fail.Uncovered lines (codecov/patch)
The remaining uncovered changed lines are live wiring that needs a machine CI does not have.
runInitbuilds a signal context and the live environment fromos.Stdinandos.Stdout, so a test would have to take over the process terminal and interrupt signal. The closures inliveInitEnvopen a host, install a runtime, download weights or start a model server; each one only forwards to a function theflynn modelscommands already use and test (installRuntime,fetchWeights,modelProbe,runModelUse,openHost). The wizard's own logic runs against an injected environment in theTestInit*tests, andTestLiveInitEnvReadsThisInstallruns the detection closures (catalog, profiles, servers, discovery) against an empty data directory. The live path was exercised by hand on the RTX 4070 Ti run above.The context ladder from #646 measures up to the context the server was started with. Ollama and LM Studio do not report that, so for a model adopted from either the served context is unknown and only the overflow error from the server ends the ladder.