Skip to content

feat(compute): opt-in shared bearer token for the worker - #70

Merged
obvious-autobuild[bot] merged 3 commits into
mainfrom
feat/worker-auth-token
Oct 12, 2026
Merged

obvious-autobuild[bot] merged 3 commits into
mainfrom
feat/worker-auth-token

Conversation

@obvious-autobuild

Copy link
Copy Markdown
Contributor

Why

Today the remote worker trusts every byte that reaches it through the SSH tunnel: all 8 of its
routes are unauthenticated, by a documented design that assumes the worker is loopback-only. Once a
worker host is shared, reachable from a lab network, or simply misconfigured, the only thing
between a stranger and free GPU compute is the tunnel's existence. The backlog spec (BACKLOG-2,
issue #45) adds an opt-in shared token that changes nothing until an operator sets one — and when
the two ends disagree, says so plainly instead of reporting a generic "unreachable".

Fixes #45

What

Opt-in by absence. With FUSIONLAB_WORKER_TOKEN unset (the default), behavior is byte-for-byte
today's: no middleware effect, loopback-first, every existing test untouched, pinned by a new
no-op test. With it set on both ends:

  • Worker (fusionlab/worker.py): one @app.middleware("http") gate — every /v1/* route
    demands Authorization: Bearer <token>, compared with hmac.compare_digest (constant time),
    answered 401 + WWW-Authenticate: Bearer before any body parsing or queue interaction.
    GET /health stays outside the check: a token-gated probe would be misread as "worker absent"
    and trigger a doomed autostart. All 8 route decorators untouched.
  • Handshake: GET /v1/capabilities advertises "auth": "token" | "open" so both sides can see
    a disagreement coming.
  • Provider (fusionlab/compute/ssh.py): the header rides all three httpx clients — the
    long-lived client, the SSE stream client, the health-probe client (9 call sites). Any 401 maps
    through one shared helper (_raise_for_auth) to RemoteComputeError("auth", …) whose message
    the status layer already renders verbatim as the fallback banner: "worker rejected the token —
    FUSIONLAB_WORKER_TOKEN differs between this machine and the worker host (or is set on one side
    only)". No router changes; the local fallback still answers.
  • Config, env-only: ssh_options() gains a loud guard — a credential-shaped key
    (token/worker_token/password/secret/api_key) in fusionlab.toml raises a ValueError
    naming FUSIONLAB_WORKER_TOKEN. This enforces, rather than re-states, the twice-documented
    no-secrets-in-config rule.
  • Docs: .env.example (the token's documented home, with the no-secrets comment scoped to SSH
    credentials), fusionlab.toml.example, docs/COMPUTE.md (token covered in the security list;
    "auth" removed from "Not in this version" — per-user accounts stay out), README's SSH section,
    and the worker module docstring, which gains its missing 8th route line (GET /v1/jobs/{id}).

Rejected alternative: a token field in fusionlab.toml — it would contradict the standing
no-secrets rule (test_example_config_parses_and_holds_no_credentials); per-host tokens, if a real
user appears, use an env-var indirection key, not a secret in the file.

How to Review

  • The policy lives in one place per side: the middleware in worker.py (right after the app
    object) and _worker_auth_headers / _raise_for_auth in compute/ssh.py.
  • /health exemption is deliberate and tested — see the middleware docstring for the
    autostart-probe reasoning.
  • stream() builds a fresh AsyncClient per call, so its 401 test injects the mock transport at
    that construction site (same patch the probe test uses).

Test Evidence

  • Full suite: 505 passed, 1 skipped (baseline 476 + 29 new), uv run ruff check and
    uv run mypy fusionlab clean; pre-push hook re-ran the suite on push.
  • New coverage: unset-token no-op pin; 401 matrix (missing + wrong Bearer) over all seven /v1
    routes asserting 401 precedes routing/validation (ghost-id and valid-body rows would answer
    404/201/422 if the gate leaked); rejected submit leaves the depth-1 queue empty; /health open
    with the gate on; loopback end-to-end with the token (handshake, upload, submit, stream, result,
    cancel) through the real worker; mismatched-env test asserting the auth error and banner
    wording; config guard tests for both toml spellings and for FUSIONLAB_WORKER_TOKEN never
    riding ssh_options().
  • Backend-only change: no rendered UI surface changes — the operator-facing wording lives in
    the fallback banner and is asserted in tests, so visual dogfood evidence does not apply.

Human author: Tapan Ganatma Nakkina ([email protected])

🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact

ObviousApp and others added 3 commits October 12, 2026 16:27
FUSIONLAB_WORKER_TOKEN set on both ends locks the worker's whole /v1
surface: one FastAPI middleware demands Authorization: Bearer (constant-
time hmac.compare_digest) and answers 401 + WWW-Authenticate before any
body parsing or queue interaction, while /health stays open so the
tunnel probe and autostart keep working. Unset — the default — the
worker is byte-for-byte today's behavior.

Provider side: the token rides all three httpx clients in compute/ssh.py
(long-lived client, SSE stream client, health probe — 9 call sites), and
any 401 maps through one shared helper to RemoteComputeError("auth", …)
whose message the fallback banner renders: the two ends disagree about
the token, not "unreachable". The handshake advertises the auth mode.

ssh_options() gains a loud guard: a credential-shaped key in
fusionlab.toml is a ValueError naming FUSIONLAB_WORKER_TOKEN — the
no-secrets-in-config rule is enforced, not just documented. The worker
docstring gains its missing 8th route line (GET /v1/jobs/{id}).

Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
- unset-token no-op pin: /v1 answers unauthenticated requests exactly as
  before the token existed
- 401 matrix over all seven /v1 routes (missing and wrong Bearer), each
  asserting 401 + WWW-Authenticate arrives before routing and validation
  — ghost-id and valid-body rows would answer 404/201/422 if the gate
  ever leaked past the middleware
- a rejected submit leaves the depth-1 queue empty; /health stays open
- loopback end-to-end with the token set: handshake, weight upload,
  submit, SSE stream, result and cooperative cancel all green through
  the real worker; a mismatched env re-point asserts the actionable
  "auth" error and its banner wording
- config guard: a token/worker_token key in fusionlab.toml (either
  spelling, base or host profile) is a loud ValueError, and
  FUSIONLAB_WORKER_TOKEN never rides ssh_options()

Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
.env.example scopes its no-secrets comment to SSH credentials and gains
the commented FUSIONLAB_WORKER_TOKEN line (same home as ANTHROPIC_API_KEY);
fusionlab.toml.example notes the token is env-only and a credential key
there is a loud error. docs/COMPUTE.md covers the token in the security
list and drops auth from "Not in this version" (multi-user accounts stay
out — one shared secret is not an account system); README's SSH section
gains the two-line when-to-set-it note.

Co-authored-by: Tapan Ganatma Nakkina <[email protected]>
@obvious-autobuild
obvious-autobuild Bot marked this pull request as ready for review October 12, 2026 16:29
@obvious-autobuild
obvious-autobuild Bot merged commit a7a6e87 into main Oct 12, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BACKLOG-2: Opt-in worker auth token

2 participants