Skip to content

feat: add spinloop gateway, a fleet's OpenAI-compatible front door - #170

Merged
outofcoffee merged 4 commits into
mainfrom
gateway
Sep 13, 2026
Merged

outofcoffee merged 4 commits into
mainfrom
gateway

Conversation

@outofcoffee

Copy link
Copy Markdown
Collaborator

Serves a fleet under one OpenAI-compatible endpoint: an agent's Spinloop names the gateway's address in its FLEET, and the agent needs only the gateway's token.

Summary

  • spinloop gateway command and internal/gateway: one OpenAI-compatible endpoint in front of a fleet.yaml. /v1/models lists what the fleet is running; completion requests are routed with the fleet's own selector; a node is woken from its own Spinloop source when nothing is serving, and the request is held until the engine answers; the caller's authorisation is swapped for the engine key the node's fleet entry names
  • A new top-level wake: on|off setting in the fleet file decides whether routing may start an engine; off refuses without starting anything, naming the node and the spinloop fleet start command that would
  • A Spinloop's FLEET may now name an endpoint (a URL) as well as a fleet file: the launch is pointed at it, with the OpenAI-compatible prefix added where needed, and authenticates with the endpoint's token through the launch's existing key chain. spinloop fleet route answers an endpoint without querying or starting a node
  • A node wake now carries the Spinloop's BASEURL into the pushed deploy config, translated to the engine's bind exactly as a local serve does. Without it a node with no preset woke onto the engine's own default bind — loopback for llama.cpp — which nothing but the node itself could reach
  • examples/gateway-docker/: two daemon nodes and two gateways over them (one waking, one wake: off) in containers. It is the gateway's integration test, run per PR by a new CI job
  • Docs: docs/commands/gateway.md, the endpoint form of FLEET in docs/spinloop-file.md, the wake setting in docs/commands/fleet.md

Implementation details

  • The gateway is the fleet client wearing a server: selection, waking, readiness and key handling all come from internal/fleet; the handler reuses a fan-out reading taken within the last two seconds rather than fanning out per request
  • The wake config's bind travels in the deploy config's serve args, as a final override layer — the preset's host/port were already dropped for a node target. The Spinloop's bind therefore wins over the preset's exactly as it does for a local serve, and the cloud path is untouched: the cloud still assigns its own port
  • A running engine bound to loopback, without an engine: override taking responsibility for reachability, is never selected — and when it is the only candidate, the failure says so
  • Concurrent requests for the same model wake at most one engine: a request that loses the start to the daemon's 409 takes the node the other one started

@outofcoffee outofcoffee added documentation Improvements or additions to documentation enhancement New feature or request go Pull requests that update go code labels Sep 6, 2026
@outofcoffee
outofcoffee force-pushed the gateway branch 3 times, most recently from 1816d40 to a76cd11 Compare September 12, 2026 20:54
@outofcoffee
outofcoffee marked this pull request as ready for review September 13, 2026 00:00
@outofcoffee
outofcoffee merged commit b25a86c into main Sep 13, 2026
4 checks passed
@outofcoffee
outofcoffee deleted the gateway branch September 13, 2026 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant