palan

Distribute GGUF and safetensors weights through the OCI registry you already run, with signatures verified before a byte moves. Daemonless, runs offline, one static binary. Serves GGUF locally too.

palan treats local LLMs the way docker treats images: models are CNCF ModelPack artifacts in any OCI 1.1 registry (zot, Harbor, distribution, GHCR, ...), served locally through managed llama.cpp llama-server processes behind one OpenAI-compatible endpoint.

palan is used two ways:

• Edge and on-prem operators serving models on modest hardware, often offline. palan does the whole job: pull, verify, serve through llama.cpp.
• Platform teams distributing weights into an inference stack they already run, such as vLLM under KServe or KAITO. palan pulls, verifies and places the weights; that runtime serves them.

A palan is the French block-and-tackle hoist for lifting heavy loads. This one pulls weights.

Artifacts are plain ModelPack. They round-trip against modctl, oras, and cosign in CI, so whatever packs a model for an OCI registry can feed palan, and whatever reads OCI artifacts can read what palan writes. Weights stay raw and mmap-ready, the inference engine arrives from the same registry as the models, and one static binary runs them behind an OpenAI endpoint, in places where no container engine exists and no internet ever will.

Highlights

• Transfer: pull/push/cp against any OCI registry, concurrent and digest-verified; interrupted pulls resume across process restarts (HTTP Range). Blobs dedup across tags and repositories (cross-repo mount).
• Reproducible packing: the same weights in ⇒ the same digest out. Metadata goes into the ModelPack config from wherever the format publishes it: a GGUF header states architecture, quantization, context length and license; a safetensors model states architecture, context length, precision and quantization in config.json, and its parameter count comes from the shard headers unless its weights are packed several to an element.
• Serving: palan run for a REPL; palan serve for an OpenAI-compatible router on :11500: lazy load, idle unload, memory-budget LRU eviction that frees device memory rather than overcommitting it, SSE streaming, Prometheus metrics.
• Runtime distribution: llama-server builds travel as OCI artifacts through the same registry as the weights: version-pinned per release, signable like any other artifact, and swappable without rebuilding palan. Offline hosts get engine upgrades through the channel they already have.
• Zero-copy: weight layers are raw, so the blob in the store is the file the runtime opens, whether that is llama-server mmapping a GGUF or another engine reading safetensors shards. No unpack step, no double storage.
• Air gap: save/load tar bundles (standard OCI layout) and direct registry-to-registry cp; models and runtimes travel the same two paths.
• Supply chain: cosign-compatible key-based signing that works fully offline; palan pull --verify refuses unsigned or foreign-signed models before a single weight byte moves.
• Import provenance: palan pack hf://org/repo fetches a published model whole, resolving its shard index, and checks each file against the digest the repository publishes for it, which it publishes for the LFS-stored files the weights and shards live in. --oms-key verifies the repository's OpenSSF model-signing signature and refuses any downloaded file that signature does not cover, including the small files served inline with no digest of their own.
• Kubernetes: init-container puller image and image volumes (K8s ≥ 1.36) via the car profile, both exercised on real clusters. A KServe modelcar manifest ships alongside them and has not been run yet. See deploy/k8s-examples/.

Install

Each release carries static binaries for linux amd64, linux arm64 and darwin arm64, an SBOM per archive, a checksum file signed with cosign, and SLSA provenance over every artifact.

A signed release archive. Verify the signature on the checksum file first, then check the archive against the checksums it now vouches for. Doing it in that order is the point: a checksum you have not authenticated only tells you the download was not corrupted.

V=0.4.0; OS=linux; ARCH=amd64          # or OS=darwin ARCH=arm64
B=https://github.com/aimd54/palan/releases/download/v$V
curl -sSLO $B/palan_${V}_${OS}_${ARCH}.tar.gz
curl -sSLO $B/checksums.txt
curl -sSLO $B/checksums.txt.sigstore.json

cosign verify-blob --bundle checksums.txt.sigstore.json \
  --certificate-identity-regexp 'github.com/aimd54/palan' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  checksums.txt
sha256sum --ignore-missing -c checksums.txt

tar xzf palan_${V}_${OS}_${ARCH}.tar.gz palan
install -m 0755 palan ~/.local/bin/palan

Container image, distroless, with palan as the entrypoint:

docker run --rm ghcr.io/aimd54/palan:0.4.0 version

From source, needing Go ≥ 1.26. A binary built this way reports no commit or build date, since those are stamped when a release is built:

go install github.com/aimd54/palan/cmd/palan@latest

Or from a clone, which is also how you would develop against it:

git clone https://github.com/aimd54/palan && cd palan && make build

Quickstart

# A throwaway local registry
docker run -d --rm -p 5000:5000 ghcr.io/project-zot/zot-linux-amd64:v2.1.18

# Pack a GGUF you already have, push it
palan pack qwen3-8b-instruct-q4_k_m.gguf -t localhost:5000/llm/qwen3:8b-q4 \
  --plain-http --ctx 8192 --push

# Or pack a safetensors model, which is published as a directory. Naming the
# directory packs every shard the index lists, plus config.json, the
# tokenizer files and the chat template. These travel and verify like any
# other artifact; serving them is another runtime's job.
palan pack ./Qwen3-8B/ -t localhost:5000/llm/qwen3:8b-bf16 \
  --plain-http --license Apache-2.0 --push

# Anywhere else: pull and chat (llama-server in PATH, or `palan runtime pull`)
palan pull localhost:5000/llm/qwen3:8b-q4 --plain-http
palan run localhost:5000/llm/qwen3:8b-q4 --prompt "Say hi"

# Or serve everything you have behind one OpenAI endpoint
palan serve
curl localhost:11500/v1/chat/completions -d '{
  "model": "localhost:5000/llm/qwen3:8b-q4",
  "messages": [{"role": "user", "content": "Say hi"}]
}'

Full walkthrough: docs/quickstart.md.

At a terminal palan run renders replies as markdown and keeps the conversation in scrollback, and naming no model opens a filterable list of what is in the store. Piped or redirected, every command produces the same plain text it always has, --json included, because that is what scripts parse; NO_COLOR and --no-color turn colour off at a terminal too (ADR-0011).

Documentation

Document	What it covers
Quickstart	zero to served model in ~5 minutes
Architecture	how the pieces fit together
Comparison	how palan relates to neighbouring tools, and where they overlap
Air-gap guide	offline bundles, mirroring, offline verification
Kubernetes guide	init puller, image volumes, KServe
Security guide	signing, verification policy, TLS, auth
CLI reference	generated from the command tree (make docs)
Configuration	config file, env vars, precedence
Registry deployment	zot + object storage + OIDC reference setup
ADRs	decisions and their reasoning
Roadmap	shipped vs. planned

Status

Pre-1.0, under active development. Serving is deliberately scoped to GGUF through llama.cpp. Distribution is format-neutral: weight layers are raw and the media type does not name a format, so any weight format travels and is verified the same way. palan is the project's release name; early ADRs predate it and use the working codename moci (ADR-0006). CI enforces interoperability on every commit: artifacts round-trip against modctl and oras, palan's own signatures against cosign, and a publisher's signature is read from a bundle the OpenSSF model_signing tool wrote.

Contributing

See CONTRIBUTING.md; DCO sign-off and Conventional Commits are required. Security reports: SECURITY.md.

License

Apache-2.0. Third-party notices are in NOTICE.

Trademarks

Other project and product names appearing in this documentation, and in the published image labels, identify the software palan interoperates with. They belong to their respective owners. Their use here is descriptive and implies neither endorsement nor affiliation.