Repository navigation
Release Arc Llama 0.9.1.2 - #90
Merged
Merged
Conversation
Plugins can now target arc_llama.plugin_api instead of private modules. It exposes admin auth, GPU leases, model and GPU status, and model lifecycle events. Plugins declare requires_api and incompatible ones are reported instead of loaded. Adds plugin pages on the dashboard, 'arc-llama plugin new' and 'plugin list', and plugin health in doctor and the support bundle. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016cZHDeCUwZMGxphYnZ62H8
Pair mmproj projectors with their models during discovery and Hugging Face downloads, launch with --mmproj, count projector VRAM, and reject image requests to text-only models before loading anything. Register split GGUFs as one model, size and fit them across every shard, download all shards, and fail fast on missing ones. Add /v1/rerank for models with reranking enabled, advertise capabilities in /v1/models, and accept the new recipe fields in the model edit API. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016cZHDeCUwZMGxphYnZ62H8
'arc-llama runtime update' installs a llama.cpp release beside the current one, checks that it supports every flag the registered recipes use, starts the smallest model on it and requests a short reply, and only then switches paths.llama_server. Failures keep the current runtime. 'arc-llama runtime rollback' restores the previous binary and backend. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_016cZHDeCUwZMGxphYnZ62H8
Remote callers of /v1 and /api need a key once one exists; loopback stays open. Keys are stored hashed in the state dir, managed with 'arc-llama keys' or /admin/api-keys, and reloaded on change. 'serve --lan' confirms exposure, creates a key if needed, and prints LAN URLs. The chat page asks remote browsers for a key. llama-server backends now always bind 127.0.0.1. They used to follow server.host, which exposed every raw backend port when serving on the network.
The send button stops a reply and keeps what arrived. The latest turn gets Regenerate and Edit. The context meter estimates the next request while typing, starting from the last measured prompt size, and warns before it overflows. Per-model system prompt and temperature presets are saved in the browser. Images can be attached for vision models and are sent as image_url parts. /v1/models now reports each model's ctx.
- 'benchmark --depths' measures decode speed after real prefill sizes. - The tune fingerprint now includes the kernel, compute runtime, Level Zero, IGC, Mesa Vulkan and oneAPI versions, so driver upgrades retune. - 'speculative --auto' measures every fitting draft and n-gram against one baseline and keeps the fastest. - tensor_split + split_gpus spread one model across several GPUs: device selectors list every card and the fit check uses their combined VRAM. - Hourly generation and TTFT medians persist to the state dir, are served at /admin/metrics/history, and show as a trend line on the dashboard.
… use The dashboard can search GGUF repos, list each quant (split files folded into one) with a fits / tight / too-big badge for the enabled GPU, and download one in the background; it registers automatically, projector included. A disk panel lists model sizes and last use, and removes models. Deletion only touches files inside models_dir that no other model shares; anything else is just unregistered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Arc Llama 0.9.1.2 brings the local model workflow onto main: Hugging Face discovery beside the library, automatic download registration, model review before chat, publisher avatars, and explicit runtime architecture checks shown separately from memory fit. The README now announces this release and includes the screenshot walkthrough.
The branch also includes versioned plugin APIs, vision projectors and split GGUFs, rerankers, chat controls and presets, API keys/LAN mode, canary-checked runtime updates with rollback, and tuning/history improvements. Compatibility work has bounded reads and deadlines, limited concurrency, cancellation on disconnect/shutdown, and protection against stale UI results. Browser regressions are organized into separate runner, fixture, and case modules.
Validation:
twine checkpass.The initial duplicate branch run exposed a Windows timestamp-resolution assumption in a file-replacement regression. The test now performs an actual atomic replacement while preserving size and modification time; 41 targeted tests pass locally. No application code changed in that correction. Windows skips include 22 platform-specific cases (two new POSIX runtime fixtures) and the separately validated live smoke.
GitHub's Linux/Windows Python matrix, browser job, and bare-wheel jobs must pass before merging. Physical Windows, other Arc GPUs, and the separate vision companion's image rendering were not rerun for this release. Runtime architecture recognition alone does not certify full model inference. Multi-GPU tensor split remains experimental.
The requested version is
0.9.1.2, a valid PEP 440 release version; package metadata, lockfile, README, and changelog agree.