diff --git a/CLAUDE.md b/CLAUDE.md index a4b2785e..b2f8bca0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -85,7 +85,7 @@ npx wrangler dev ### Auth Flow - Frontend never talks directly to sandbox; all traffic goes through the control plane - Cloudflare control plane uses dev auth via headers/query params when `DEV_AUTH_ENABLED=true` -- Internal sandbox auth uses `SANDBOX_INTERNAL_TOKEN` (do not reuse Cloudflare API keys) +- Control plane → sandbox requests are **signed per machine** (Ed25519, `SANDBOX_AUTH_SIGNING_KEY` in the control plane; VMs get only `SANDBOX_AUTH_PUBLIC_KEY`). The legacy static `SANDBOX_INTERNAL_TOKEN` is accepted only while configured — do not reuse Cloudflare API keys. Desktop generates the key pair per boot and passes the public key on the VM's kernel command line; its VM holds no static token and no `INTERNAL_API_TOKEN` - **Personal access tokens (PATs)** authenticate the `orcabot` CLI / scripts against a remote control plane: prefix `orca_pat_`, sent as `Authorization: Bearer`, only the SHA-256 hash is stored (`api_tokens` table). Middleware is prefix-gated (no collision with PTY JWT bearers) and **method-gated** — a PAT cannot mint/list/revoke other PATs (`rejectPatAuth`). See `controlplane/CLAUDE.md`. ### Secrets Protection (Security-Critical) @@ -108,6 +108,12 @@ Security invariants (non-negotiable): - Broker URLs include session ID: `/broker/{sessionID}/{provider}/...` - Approved domains keyed by `sessionID:secretName` — never just secret name - Empty `INTERNAL_API_TOKEN` = all PTY tokens rejected (fail-closed) +- PTY + dashboard tokens are signed with `PTY_TOKEN_SECRET`, a control-plane-only secret — never `INTERNAL_API_TOKEN`, which every sandbox VM holds (root can read it from `/proc`). Without `PTY_TOKEN_SECRET` set the control plane falls back to `INTERNAL_API_TOKEN` and logs a SECURITY WARNING +- Sandbox → control plane `/internal/*` calls authenticate with the VM's **dashboard-scoped token** (`X-Dashboard-Token`, `sandbox/internal/cpauth`), and every VM-reachable route checks the resource belongs to that dashboard (`controlplane/src/auth/sandbox-auth.ts`). `X-Internal-Token` is full trust for control-plane-side callers only; with `WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX=true` new VMs never receive `INTERNAL_API_TOKEN`. New VM-called routes must use `authorizeSandboxForDashboard`, never `requireInternalAuth` +- Every control plane → sandbox request goes through `sandboxFetch` / `sandboxHeaders` / `SandboxClient` so it carries `X-Sandbox-Auth`: an Ed25519 signature over target machine (`FLY_MACHINE_ID`), method and path+query, valid 30s plus the VM's 30s skew allowance (60s replay window at most), plus `fly-force-instance-id` so no other VM is the first hop. Never call a sandbox with a hand-built `fetch`. A bad signature never falls back to the static token +- The dashboard browser runs unprivileged, never root: Chromium as `orcabot-browser`, Xvfb/x11vnc/websockify as `orcabot-vnc`, on loopback ports 6100-6299 that pool UIDs **and `orcabot-browser`** (page JS) are blocked from, so one dashboard's page can't reach another's VNC/DevTools; the X display requires its auth cookie. See `sandbox/CLAUDE.md` (Browser Block) +- A dashboard VM only ever receives its **owner's** secrets (`dashboardSecretsOwner`, `controlplane/src/secrets/handler.ts`): one VM is shared by every member's terminals, so pushing whoever acted last would put their keys behind every terminal's broker URL +- Pool UIDs (2000-2099) are egress default-deny at the kernel: only loopback (incl. the 80/443 REDIRECT), established replies, DNS to resolv.conf nameservers, and `EGRESS_EXTRA_TCP_PORTS` (opt-in, e.g. `22`) are allowed - Localhost MCP/event endpoints require valid `X-MCP-Secret` (no fail-open) - Egress proxy fail-closed: 60s timeout = deny; proxy crash = connections fail - Localhost always bypasses egress proxy (NO_PROXY + server-side check) diff --git a/controlplane/CLAUDE.md b/controlplane/CLAUDE.md index acebcfe2..06542c42 100644 --- a/controlplane/CLAUDE.md +++ b/controlplane/CLAUDE.md @@ -97,7 +97,10 @@ Fly.io (execution plane – separate repo) - Domain approval workflow for custom secrets - **Integration policy enforcement** (gateway execute, rate limits, audit) - **OAuth token lifecycle** (connect, refresh, encrypt at rest) -- **PTY token issuance** (HMAC-SHA256 JWT for terminal-level auth — **fail-closed**: empty `INTERNAL_API_TOKEN` rejects all tokens) +- **Control plane → sandbox auth** (`src/sandbox/request-signing.ts`): `sandboxFetch`, `sandboxHeaders(env, headers, machineId, method, url)` (async) and `SandboxClient` (3rd ctor arg `env.SANDBOX_AUTH_SIGNING_KEY`) sign each request for its target machine. `sandboxHeaders` strips client credentials (Cookie, Authorization, X-Dashboard-Token, Cf-Access-Jwt-Assertion, dev-auth `X-User-*` / `X-Orcabot-Surface`) from proxied browser headers — the VM is shared by every member and must never see them. New VMs get `sandboxAuthEnv(env)`: the derived public key, plus `SANDBOX_INTERNAL_TOKEN` only while that secret exists. Generate the key with `openssl genpkey -algorithm ed25519 -outform DER | base64` +- **Sandbox `/internal/*` auth** (`src/auth/sandbox-auth.ts`): routes the sandbox calls accept `X-Internal-Token` (full trust) or the VM's dashboard token (`X-Dashboard-Token`) and scope-check the resource's dashboard via `authorizeSandboxForDashboard` / `requireSandboxDashboard` (sessions resolve through `dashboardIdForSandboxSession`). The sandbox refreshes its 7-day token at `POST /internal/sandbox/dashboard-token/refresh` (30-day grace for VMs that were suspended), only while the token's `sbx_gen` matches the dashboard's current `dashboard_sandboxes` row (its `created_at`): retiring or replacing the sandbox ends refresh for every token minted for the old one. Mint VM tokens with `sandboxGeneration(now)` and claim the row with that same `now`. `WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX=true` stops new VMs receiving `INTERNAL_API_TOKEN` (`sandboxInternalTokenEnv`) +- **PTY token issuance** (HMAC-SHA256 JWT for terminal-level auth, signed with the control-plane-only `PTY_TOKEN_SECRET` via `tokenSigningSecret(env)` — never with `INTERNAL_API_TOKEN`, which sandboxes hold — **fail-closed**: an empty secret rejects all tokens) +- **`PTY_TOKEN_SECRET` rollout**: set the secret *before or together with* the control-plane deploy that relies on it (`wrangler secret put PTY_TOKEN_SECRET -c wrangler.production.toml`, then deploy). Once it is set, tokens signed with the old `INTERNAL_API_TOKEN` fallback are rejected — by design, since VMs can read that key — so live VMs' dashboard tokens stop working (refresh → E79395, internal calls → E79393). The sandbox drops a refused token, and the control plane sends a fresh one (bound to the current `dashboard_sandboxes` generation, `mintCurrentSandboxDashboardToken`) as `mcp_token` with every new PTY, so the next terminal opened on that dashboard restores it; running terminals' MCP/integration calls fail until then. Rotating the secret later has the same effect. Never re-add an `INTERNAL_API_TOKEN` verification fallback - **Personal access token (PAT) issuance** (CLI / scripted auth — only the SHA-256 hash is stored; **method-gated** so a PAT cannot mint/list/revoke other PATs) - **File proxy** (read/write/import a session workspace, proxied to the sandbox; write + import are owner-only) - **Response filtering** (policy-based filtering before LLM sees data) @@ -208,7 +211,7 @@ curl "http://localhost:8787/internal/sessions/{sessionId}/approved-domains" \ ``` ### Auto-Apply Mechanism -When secrets change, `autoApplySecretsToSessions()` pushes updates to all active sandbox sessions for that dashboard via the sandbox's `updateEnv()` endpoint. +When secrets change, `autoApplySecretsToSessions()` pushes updates to the dashboard's **current** sandbox session (`dashboard_sandboxes`, via `pushOwnerSecretsToCurrentSandbox` — never a `sessions` row, which can point at a session a reprovision replaced) through the sandbox's `updateEnv()` endpoint. Secret names must be env var names (`^[A-Z_][A-Z0-9_]*$`, E79732): the sandbox rejects a whole update over one bad name, so stored names that don't match are skipped. Only the dashboard owner's secrets are applied. Before starting a terminal, `applyDashboardOwnerSecrets()` reconciles previously applied names, unsetting legacy collaborator secrets (and their `_BROKER` variables), even if the owner has no secrets. Cleanup must succeed before terminal startup and before replacing the tracked names (a stale/unreachable sandbox instead takes the normal reprovision path; the fresh VM's pre-apply is fail-closed too). Loading the owner's own secrets is best-effort: if it fails (e.g. no `SECRETS_ENCRYPTION_KEY`), every tracked name is unset and the terminal starts without them. Existing processes retain their inherited environment until restarted. ### Encryption - Secrets encrypted at rest using configured encryption key @@ -265,6 +268,9 @@ Sandbox MCP Server - `src/integration-policies/api-clients/` — Gmail, GitHub, Drive, Calendar, Twitter API wrappers - `src/auth/pty-token.ts` — HMAC-SHA256 JWT token creation/verification +### OAuth connect/callback +- `state` is bound to the browser that started the flow: connect sets an HttpOnly `orcabot_oauth_` cookie (`SameSite=Lax`, `Path=/integrations`) whose SHA-256 is stored with the state; the callback requires it, or a session for the user that started the flow (`getState` in `src/integrations/handler.ts`). New connect handlers must return `oauthRedirect(...)`, and the callback host must be the connect host (`OAUTH_REDIRECT_BASE`) + ### Database Tables - `user_integrations` — OAuth tokens (encrypted), provider, account email - `terminal_integrations` — Which integrations are attached to which terminals @@ -277,6 +283,9 @@ Sandbox MCP Server 4. **Fail-closed** — Missing policy, expired token, unknown action → deny 5. **Audit before response** — Every request logged via `logAuditEntry()` before returning 6. **Context derived server-side** — Recipient, domain, etc. extracted from args in control plane +7. **Check what is sent** — `validateGatewayArgs` (`src/integration-policies/arg-safety.ts`) runs before `enforcePolicy`: IDs that clients put in URL paths must be single safe segments (no `/`, `..`, `?`, `#`, `%`), email headers can't contain CR/LF, and each recipient entry is exactly one address. Messaging targets are derived only from the field each client sends to (`MESSAGING_TARGET_FIELD`); channel names are resolved server-side, never taken from args. Routing values (WhatsApp `phone_number_id`, Matrix `homeserver`) come from the stored integration. Empty path IDs (`folderId: ""`) are dropped as omitted (`dropEmptyPathArgs`) +8. **Policy belongs to the account holder** — only the member who attached an integration can change its policy; they or the dashboard owner can detach it +9. **Discord is pinned to the stored guild** — every integration uses the shared platform bot, which is in every customer's guild. The gateway forces `guild_id` to the integration's stored guild (a different one → E79625, none stored → E79624), looks up every channel-taking action's channel (`GET /channels/{id}`) and denies DMs / other guilds (E79628) or failed lookups (E79627); actions outside `DISCORD_*_ACTIONS` are denied (E79629). The connect callback binds a guild only with the `bot` scope, from the token response, or from the `guild_id` param only if the user owns it or has MANAGE_GUILD / ADMINISTRATOR there ### Providers - **Gmail**: search, get, send, archive, trash, mark_read/unread, add/remove_label diff --git a/controlplane/src/asr/ASRStreamProxy.ts b/controlplane/src/asr/ASRStreamProxy.ts index 231fbc86..8c4f2e7a 100644 --- a/controlplane/src/asr/ASRStreamProxy.ts +++ b/controlplane/src/asr/ASRStreamProxy.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: asr-stream-proxy-v2-auth-fix +// REVISION: asr-stream-proxy-v3-error-codes -const proxyRevision = "asr-stream-proxy-v2-auth-fix"; +const proxyRevision = "asr-stream-proxy-v3-error-codes"; console.log(`[ASRStreamProxy] REVISION: ${proxyRevision} loaded at ${new Date().toISOString()}`); /** @@ -20,12 +20,12 @@ export class ASRStreamProxy implements DurableObject { async fetch(request: Request): Promise { if (request.headers.get('Upgrade')?.toLowerCase() !== 'websocket') { - return new Response('Expected WebSocket', { status: 426 }); + return new Response('E79176: Expected WebSocket', { status: 426 }); } const apiKey = request.headers.get('X-ASR-Api-Key'); if (!apiKey) { - return new Response('Missing API key', { status: 401 }); + return new Response('E79177: Missing API key', { status: 401 }); } // Create the client-facing WebSocket pair diff --git a/controlplane/src/asr/handler.ts b/controlplane/src/asr/handler.ts index 2b0ab9a6..92f70d42 100644 --- a/controlplane/src/asr/handler.ts +++ b/controlplane/src/asr/handler.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: asr-handler-v6-deepgram-rest-fallback +// REVISION: asr-handler-v7-error-codes -const handlerRevision = "asr-handler-v6-deepgram-rest-fallback"; +const handlerRevision = "asr-handler-v7-error-codes"; console.log(`[asr] REVISION: ${handlerRevision} loaded at ${new Date().toISOString()}`); import type { Env } from '../types'; @@ -65,13 +65,13 @@ export async function saveASRKey( ): Promise { const provider = data.provider as ASRProvider; if (!['assemblyai', 'openai', 'deepgram'].includes(provider)) { - return jsonResponse({ error: 'Invalid provider' }, 400); + return jsonResponse({ error: 'E79240: Invalid provider' }, 400); } if (!data.apiKey || typeof data.apiKey !== 'string' || data.apiKey.trim().length === 0) { - return jsonResponse({ error: 'API key is required' }, 400); + return jsonResponse({ error: 'E79241: API key is required' }, 400); } if (!hasEncryptionKey(env)) { - return jsonResponse({ error: 'Encryption not configured' }, 500); + return jsonResponse({ error: 'E79242: Encryption not configured' }, 500); } const key = await getEncryptionKey(env); @@ -109,7 +109,7 @@ export async function deleteASRKey( provider: string ): Promise { if (!['assemblyai', 'openai', 'deepgram'].includes(provider)) { - return jsonResponse({ error: 'Invalid provider' }, 400); + return jsonResponse({ error: 'E79243: Invalid provider' }, 400); } const p = provider as ASRProvider; @@ -153,7 +153,7 @@ async function getASRKey(env: Env, userId: string, provider: ASRProvider): Promi export async function getAssemblyAIToken(env: Env, userId: string): Promise { const apiKey = await getASRKey(env, userId, 'assemblyai'); if (!apiKey) { - return jsonResponse({ error: 'AssemblyAI API key not configured' }, 404); + return jsonResponse({ error: 'E79244: AssemblyAI API key not configured' }, 404); } const tokenResponse = await fetch('https://api.assemblyai.com/v2/realtime/token', { @@ -167,7 +167,7 @@ export async function getAssemblyAIToken(env: Env, userId: string): Promise { const apiKey = await getASRKey(env, userId, 'openai'); if (!apiKey) { - return jsonResponse({ error: 'OpenAI API key not configured' }, 404); + return jsonResponse({ error: 'E79246: OpenAI API key not configured' }, 404); } // Reject oversized payloads before parsing const contentLength = parseInt(request.headers.get('Content-Length') || '0', 10); if (contentLength > MAX_AUDIO_SIZE) { - return jsonResponse({ error: `File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); + return jsonResponse({ error: `E79247: File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); } const formData = await request.formData(); @@ -205,20 +205,20 @@ export async function transcribeOpenAI(env: Env, userId: string, request: Reques const upstreamForm = new FormData(); const audioFile = formData.get('file'); if (!audioFile || typeof audioFile === 'string') { - return jsonResponse({ error: 'No audio file provided' }, 400); + return jsonResponse({ error: 'E79248: No audio file provided' }, 400); } if ((audioFile as File).size > MAX_AUDIO_SIZE) { - return jsonResponse({ error: `File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); + return jsonResponse({ error: `E79249: File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); } // Allowlist model and response format — don't forward arbitrary client values const requestedModel = (formData.get('model') as string) || 'whisper-1'; const requestedFormat = (formData.get('response_format') as string) || 'json'; if (!ALLOWED_MODELS.includes(requestedModel)) { - return jsonResponse({ error: `Unsupported model. Allowed: ${ALLOWED_MODELS.join(', ')}` }, 400); + return jsonResponse({ error: `E79250: Unsupported model. Allowed: ${ALLOWED_MODELS.join(', ')}` }, 400); } if (!ALLOWED_RESPONSE_FORMATS.includes(requestedFormat)) { - return jsonResponse({ error: `Unsupported response format. Allowed: ${ALLOWED_RESPONSE_FORMATS.join(', ')}` }, 400); + return jsonResponse({ error: `E79251: Unsupported response format. Allowed: ${ALLOWED_RESPONSE_FORMATS.join(', ')}` }, 400); } // Preserve MIME type — Workers FormData can lose content-type when re-appending File objects. @@ -239,7 +239,7 @@ export async function transcribeOpenAI(env: Env, userId: string, request: Reques if (!whisperResponse.ok) { const text = await whisperResponse.text(); - return jsonResponse({ error: `OpenAI transcription failed: ${text}` }, 502); + return jsonResponse({ error: `E79252: OpenAI transcription failed: ${text}` }, 502); } const result = await whisperResponse.json() as { text: string }; @@ -259,7 +259,7 @@ export async function transcribeOpenAI(env: Env, userId: string, request: Reques export async function getDeepgramToken(env: Env, userId: string): Promise { const apiKey = await getASRKey(env, userId, 'deepgram'); if (!apiKey) { - return jsonResponse({ error: 'Deepgram API key not configured' }, 404); + return jsonResponse({ error: 'E79253: Deepgram API key not configured' }, 404); } const tokenResponse = await fetch('https://api.deepgram.com/v1/auth/grant', { @@ -274,7 +274,7 @@ export async function getDeepgramToken(env: Env, userId: string): Promise { const apiKey = await getASRKey(env, userId, 'deepgram'); if (!apiKey) { - return jsonResponse({ error: 'Deepgram API key not configured' }, 404); + return jsonResponse({ error: 'E79255: Deepgram API key not configured' }, 404); } const contentLength = parseInt(request.headers.get('Content-Length') || '0', 10); if (contentLength > MAX_AUDIO_SIZE) { - return jsonResponse({ error: `File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); + return jsonResponse({ error: `E79256: File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); } const audioData = await request.arrayBuffer(); if (audioData.byteLength === 0) { - return jsonResponse({ error: 'No audio data provided' }, 400); + return jsonResponse({ error: 'E79257: No audio data provided' }, 400); } if (audioData.byteLength > MAX_AUDIO_SIZE) { - return jsonResponse({ error: `File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); + return jsonResponse({ error: `E79258: File too large (max ${MAX_AUDIO_SIZE / 1024 / 1024}MB)` }, 413); } const contentType = request.headers.get('Content-Type') || 'audio/webm'; @@ -325,7 +325,7 @@ export async function transcribeDeepgram(env: Env, userId: string, request: Requ if (!dgResponse.ok) { const text = await dgResponse.text(); - return jsonResponse({ error: `Deepgram transcription failed: ${text}` }, 502); + return jsonResponse({ error: `E79259: Deepgram transcription failed: ${text}` }, 502); } const result = await dgResponse.json() as { diff --git a/controlplane/src/auth/csrf.test.ts b/controlplane/src/auth/csrf.test.ts new file mode 100644 index 00000000..69c7bf5e --- /dev/null +++ b/controlplane/src/auth/csrf.test.ts @@ -0,0 +1,44 @@ +// REVISION: csrf-tests-v2-same-origin +import { describe, it, expect } from 'vitest'; +import { checkCsrf } from '../index'; + +const allowed = new Set(['https://orcabot.com']); +const req = (method: string, headers: Record = {}) => + new Request('https://api.orcabot.com/secrets', { method, headers }); + +describe('checkCsrf()', () => { + it('allows same-origin POSTs from pages the control plane serves (Drive picker)', () => { + expect(checkCsrf(req('POST', { Origin: 'https://api.orcabot.com' }), 'session', allowed)).toBeNull(); + expect(checkCsrf(req('POST', { Origin: 'https://api.orcabot.com.evil.com' }), 'session', allowed)?.status).toBe(403); + }); + + it('allows cookie POST from an allowed origin', () => { + expect(checkCsrf(req('POST', { Origin: 'https://orcabot.com' }), 'session', allowed)).toBeNull(); + }); + + it('rejects cookie POST from a foreign origin', () => { + expect(checkCsrf(req('POST', { Origin: 'https://evil.com' }), 'session', allowed)?.status).toBe(403); + }); + + it('rejects the opaque "null" origin', () => { + expect(checkCsrf(req('DELETE', { Origin: 'null' }), 'session', allowed)?.status).toBe(403); + }); + + it('rejects origin-less requests the browser marks cross-site', () => { + expect(checkCsrf(req('POST', { 'Sec-Fetch-Site': 'cross-site' }), 'cf-access', allowed)?.status).toBe(403); + }); + + it('allows origin-less non-browser clients', () => { + expect(checkCsrf(req('POST'), 'session', allowed)).toBeNull(); + }); + + it('ignores safe methods and bearer-style auth', () => { + expect(checkCsrf(req('GET', { Origin: 'https://evil.com' }), 'session', allowed)).toBeNull(); + expect(checkCsrf(req('POST', { Origin: 'https://evil.com' }), 'pat', allowed)).toBeNull(); + expect(checkCsrf(req('POST', { Origin: 'https://evil.com' }), undefined, allowed)).toBeNull(); + }); + + it('is disabled when all origins are allowed (dev mode)', () => { + expect(checkCsrf(req('POST', { Origin: 'https://evil.com' }), 'session', null)).toBeNull(); + }); +}); diff --git a/controlplane/src/auth/dashboard-token.ts b/controlplane/src/auth/dashboard-token.ts index fc38167f..51ccc12f 100644 --- a/controlplane/src/auth/dashboard-token.ts +++ b/controlplane/src/auth/dashboard-token.ts @@ -1,6 +1,10 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: dashboard-token-v4-identifier-renames +const MODULE_REVISION = 'dashboard-token-v4-identifier-renames'; +console.log(`[dashboard-token] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + /** * Dashboard-scoped tokens for MCP proxy * @@ -10,7 +14,7 @@ * * Format: base64url(header).base64url(payload).base64url(signature) * - header: { alg: "HS256", typ: "JWT" } - * - payload: { dashboard_id, session_id, aud: "mcp-ui", exp, iat } + * - payload: { dashboard_id, session_id, aud: "mcp-ui", exp, iat, sbx_gen } * - signature: HMAC-SHA256(header.payload, secret) */ @@ -20,6 +24,10 @@ export interface DashboardTokenClaims { aud: string; // "mcp-ui" exp: number; // Unix timestamp iat: number; // Unix timestamp + // The sandbox the token was minted for: its dashboard_sandboxes.created_at + // (ms). Carried through refreshes so a token outlives its VM only until the + // VM is retired or replaced (see refreshSandbоxDashboardToken). + sbx_gen?: number; } const ALGORITHM = 'HS256'; @@ -77,16 +85,26 @@ async function hmacVerify(data: string, signature: string, secret: string): Prom return result === 0; } +/** A dashboard_sandboxes.created_at value as a token generation (ms). */ +export function sandboxGeneration(createdAt: string): number { + // Rows written by SQLite's datetime('now') have no zone marker; they're UTC. + const iso = /[zZ]|[+-]\d\d:?\d\d$/.test(createdAt) ? createdAt : `${createdAt.replace(' ', 'T')}Z`; + return Date.parse(iso); +} + /** * Create a dashboard-scoped token for MCP proxy * @param dashboardId - Required dashboard to scope the token to - * @param secret - Signing secret (INTERNAL_API_TOKEN) + * @param secret - Signing secret (tokenSigningSecret(env) — PTY_TOKEN_SECRET) * @param sessionId - Optional session ID for auditing (not enforced) + * @param generation - sandboxGeneration(createdAt) of the dashboard_sandboxes + * row the sandbox will be claimed with (it must be written with that same createdAt) */ export async function createDashboardToken( dashboardId: string, secret: string, - sessionId?: string + sessionId?: string, + generation?: number ): Promise { const now = Math.floor(Date.now() / 1000); @@ -97,6 +115,7 @@ export async function createDashboardToken( exp: now + TOKEN_EXPIRY_SECONDS, iat: now, ...(sessionId && { session_id: sessionId }), + ...(generation !== undefined && { sbx_gen: generation }), }; const headerB64 = base64UrlEncode(JSON.stringify(header)); @@ -110,10 +129,15 @@ export async function createDashboardToken( /** * Verify and decode a dashboard-scoped token * Returns the claims if valid, null if invalid + * + * expiredGraceSeconds: accept a correctly signed token up to this long past its + * expiry. Only the token refresh endpoint uses it (a suspended VM can wake after + * its token expired); every other caller gets strict expiry. */ export async function verifyDashboardToken( token: string, - secret: string + secret: string, + expiredGraceSeconds = 0 ): Promise { const parts = token.split('.'); if (parts.length !== 3) { @@ -155,7 +179,7 @@ export async function verifyDashboardToken( // Check expiry const now = Math.floor(Date.now() / 1000); - if (payload.exp < now) { + if (payload.exp + expiredGraceSeconds < now) { return null; } diff --git a/controlplane/src/auth/middleware.ts b/controlplane/src/auth/middleware.ts index 9e22eaa4..bdd3d684 100644 --- a/controlplane/src/auth/middleware.ts +++ b/controlplane/src/auth/middleware.ts @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: auth-middleware-v2-pat-method-gate +// REVISION: auth-middleware-v3-dedicated-token-secret /** * Auth Middleware @@ -297,6 +297,7 @@ function tokenFingerprint(token: string | null): string { } import { verifyDashboardToken, type DashboardTokenClaims } from './dashboard-token'; +import { tokenSigningSecret } from './pty-token'; export interface McpAuthResult { isValid: boolean; @@ -348,7 +349,7 @@ export async function validateMcpAuth( ), }; } - const claims = await verifyDashboardToken(dashboardToken, env.INTERNAL_API_TOKEN); + const claims = await verifyDashboardToken(dashboardToken, tokenSigningSecret(env)); if (claims) { return { isValid: true, diff --git a/controlplane/src/auth/pty-token.ts b/controlplane/src/auth/pty-token.ts index 01d518ef..df607682 100644 --- a/controlplane/src/auth/pty-token.ts +++ b/controlplane/src/auth/pty-token.ts @@ -20,6 +20,29 @@ * - signature: HMAC-SHA256(header.payload, secret) */ +// REVISION: pty-token-v2-dedicated-signing-secret +console.log(`[pty-token] REVISION: pty-token-v2-dedicated-signing-secret loaded at ${new Date().toISOString()}`); + +let warnedSigningFallback = false; + +/** + * Secret used to sign + verify PTY and dashboard tokens. + * + * SECURITY: this must be a control-plane-only secret. INTERNAL_API_TOKEN is + * shipped into every sandbox VM (where root can read it from /proc), so using it + * as the HMAC key lets any tenant mint tokens for any other tenant's terminals. + * Falls back to INTERNAL_API_TOKEN only so a deploy without PTY_TOKEN_SECRET set + * keeps working — set the secret to actually close the hole. + */ +export function tokenSigningSecret(env: { PTY_TOKEN_SECRET?: string; INTERNAL_API_TOKEN: string }): string { + if (env.PTY_TOKEN_SECRET) return env.PTY_TOKEN_SECRET; + if (!warnedSigningFallback) { + warnedSigningFallback = true; + console.error('[pty-token] SECURITY WARNING: PTY_TOKEN_SECRET is not set — signing PTY/dashboard tokens with INTERNAL_API_TOKEN, which sandbox VMs can read. Set PTY_TOKEN_SECRET.'); + } + return env.INTERNAL_API_TOKEN; +} + export interface PtyTokenClaims { terminal_id: string; // PTY ID - the primary binding for integration policies sandbox_id: string; // Sandbox session ID @@ -103,7 +126,7 @@ export async function createPtyToken( // An empty HMAC key still produces valid signatures, so anyone could forge // tokens if the signing secret is missing from the environment. if (!secret) { - throw new Error('E79850: Cannot create PTY token: signing secret (INTERNAL_API_TOKEN) is empty'); + throw new Error('E79850: Cannot create PTY token: signing secret (PTY_TOKEN_SECRET / INTERNAL_API_TOKEN) is empty'); } const now = Math.floor(Date.now() / 1000); @@ -142,7 +165,7 @@ export async function verifyPtyToken( // SECURITY: Reject empty secrets. HMAC with empty key still produces valid // signatures, so an attacker could forge tokens if the secret is missing. if (!secret) { - console.error('[pty-token] CRITICAL: verifyPtyToken called with empty secret — rejecting all tokens. Check INTERNAL_API_TOKEN env var.'); + console.error('[pty-token] CRITICAL: verifyPtyToken called with empty secret — rejecting all tokens. Check PTY_TOKEN_SECRET / INTERNAL_API_TOKEN env vars.'); return null; } diff --git a/controlplane/src/auth/sandbox-auth.test.ts b/controlplane/src/auth/sandbox-auth.test.ts new file mode 100644 index 00000000..bf7761e4 --- /dev/null +++ b/controlplane/src/auth/sandbox-auth.test.ts @@ -0,0 +1,192 @@ +// REVISION: sandbox-auth-tests-v3-identifier-renames +import { describe, it, expect, beforeEach } from 'vitest'; +import { + authorizeSandboxForDashboard, + dashboardIdForSandboxSession, + refreshSandbоxDashboardToken, + sandboxInternalTokenEnv, +} from './sandbox-auth'; +import { createDashboardToken, sandboxGeneration, verifyDashboardToken } from './dashboard-token'; +import { tokenSigningSecret } from './pty-token'; +import { createTestContext, seedDashboard, seedSession } from '../../tests/helpers'; +import type { TestContext } from '../../tests/helpers'; + +function req(headers: Record, method = 'GET'): Request { + return new Request('https://api.example.com/internal/drive/file?dashboard_id=x', { method, headers }); +} + +// Signed with the same secret, but with an arbitrary expiry (createDashboardToken always uses now + 7d). +async function tokenWithExp(dashboardId: string, secret: string, expOffsetSeconds: number): Promise { + const b64 = (s: string) => btoa(s).replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, ''); + const now = Math.floor(Date.now() / 1000); + const header = b64(JSON.stringify({ alg: 'HS256', typ: 'JWT' })); + const payload = b64(JSON.stringify({ dashboard_id: dashboardId, aud: 'mcp-ui', iat: now, exp: now + expOffsetSeconds })); + const key = await crypto.subtle.importKey('raw', new TextEncoder().encode(secret), { name: 'HMAC', hash: 'SHA-256' }, false, ['sign']); + const sig = new Uint8Array(await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(`${header}.${payload}`))); + let bin = ''; + for (const b of sig) bin += String.fromCharCode(b); + return `${header}.${payload}.${b64(bin)}`; +} + +describe('sandbox auth', () => { + let ctx: TestContext; + + beforeEach(async () => { + ctx = await createTestContext(); + ctx.env.PTY_TOKEN_SECRET = 'test-pty-secret'; + }); + + describe('authorizeSandboxForDashboard()', () => { + it('lets the internal token through for any dashboard', async () => { + expect(await authorizeSandboxForDashboard(req({ 'X-Internal-Token': 'test-internal-token' }), ctx.env, 'dash-a')).toBeNull(); + }); + + it('rejects a wrong internal token', async () => { + const res = await authorizeSandboxForDashboard(req({ 'X-Internal-Token': 'nope' }), ctx.env, 'dash-a'); + expect(res?.status).toBe(401); + }); + + it('lets a dashboard token through for its own dashboard', async () => { + const token = await createDashboardToken('dash-a', tokenSigningSecret(ctx.env)); + expect(await authorizeSandboxForDashboard(req({ 'X-Dashboard-Token': token }), ctx.env, 'dash-a')).toBeNull(); + }); + + it("rejects a dashboard token used against another dashboard", async () => { + const token = await createDashboardToken('dash-a', tokenSigningSecret(ctx.env)); + const res = await authorizeSandboxForDashboard(req({ 'X-Dashboard-Token': token }), ctx.env, 'dash-b'); + expect(res?.status).toBe(403); + }); + + it('rejects a dashboard token when the resource dashboard is unknown', async () => { + const token = await createDashboardToken('dash-a', tokenSigningSecret(ctx.env)); + const res = await authorizeSandboxForDashboard(req({ 'X-Dashboard-Token': token }), ctx.env, null); + expect(res?.status).toBe(403); + }); + + it('rejects a dashboard token signed with INTERNAL_API_TOKEN (the VM-readable secret)', async () => { + const forged = await createDashboardToken('dash-b', ctx.env.INTERNAL_API_TOKEN); + const res = await authorizeSandboxForDashboard(req({ 'X-Dashboard-Token': forged }), ctx.env, 'dash-b'); + expect(res?.status).toBe(401); + }); + + it('rejects an expired dashboard token', async () => { + const expired = await tokenWithExp('dash-a', tokenSigningSecret(ctx.env), -60); + const res = await authorizeSandboxForDashboard(req({ 'X-Dashboard-Token': expired }), ctx.env, 'dash-a'); + expect(res?.status).toBe(401); + }); + + it('rejects a request with no credential', async () => { + const res = await authorizeSandboxForDashboard(req({}), ctx.env, 'dash-a'); + expect(res?.status).toBe(401); + }); + }); + + describe('dashboardIdForSandboxSession()', () => { + it('resolves the owning dashboard of a sandbox session', async () => { + await seedSession(ctx.db, 'dash-a', 'item-1', { id: 'sess-1', sandboxSessionId: 'sbx-1' }); + expect(await dashboardIdForSandboxSession(ctx.env, 'sbx-1')).toBe('dash-a'); + }); + + it('returns null for an unknown session', async () => { + expect(await dashboardIdForSandboxSession(ctx.env, 'sbx-missing')).toBeNull(); + }); + }); + + describe('refreshSandboxDashboardToken()', () => { + const refreshReq = (headers: Record) => + new Request('https://api.example.com/internal/sandbox/dashboard-token/refresh', { method: 'POST', headers }); + + const CREATED_AT = '2026-10-01T12:00:00.000Z'; + const GEN = sandboxGeneration(CREATED_AT); + const setSandbox = (sessionId: string, createdAt = CREATED_AT) => ctx.db.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, created_at) + VALUES ('dash-a', ?, '', ?) + ON CONFLICT(dashboard_id) DO UPDATE SET sandbox_session_id = excluded.sandbox_session_id, created_at = excluded.created_at + `).bind(sessionId, createdAt).run(); + const refresh = async (token: string) => { + const res = await refreshSandbоxDashboardToken(refreshReq({ 'X-Dashboard-Token': token }), ctx.env); + return { res, body: await res.json() as { token?: string; error?: string } }; + }; + + beforeEach(async () => { + await seedDashboard(ctx.db, 'owner-1', { id: 'dash-a' }); + await setSandbox('sbx-a'); + }); + + it('keeps the token bound to the sandbox it was minted for', async () => { + const token = await createDashboardToken('dash-a', tokenSigningSecret(ctx.env), undefined, GEN); + const { res, body } = await refresh(token); + expect(res.status).toBe(200); + expect((await verifyDashboardToken(body.token!, tokenSigningSecret(ctx.env)))?.sbx_gen).toBe(GEN); + }); + + it('binds a token minted before sandbox generations to the current sandbox', async () => { + const { res, body } = await refresh(await createDashboardToken('dash-a', tokenSigningSecret(ctx.env))); + expect(res.status).toBe(200); + expect((await verifyDashboardToken(body.token!, tokenSigningSecret(ctx.env)))?.sbx_gen).toBe(GEN); + }); + + it('a token copied out of a VM stops refreshing once that sandbox is replaced', async () => { + let token = await createDashboardToken('dash-a', tokenSigningSecret(ctx.env), undefined, GEN); + token = (await refresh(token)).body.token!; + await setSandbox('sbx-b', '2026-10-09T08:00:00.000Z'); + const { res, body } = await refresh(token); + expect(res.status).toBe(401); + expect(body.error).toMatch(/^E79398/); + }); + + it('refuses while the dashboard has no live sandbox (retired)', async () => { + await setSandbox(''); + const { res, body } = await refresh(await createDashboardToken('dash-a', tokenSigningSecret(ctx.env), undefined, GEN)); + expect(res.status).toBe(401); + expect(body.error).toMatch(/^E79399/); + }); + + it('reads SQLite datetime() timestamps as UTC', () => { + expect(sandboxGeneration('2026-10-01 12:00:00')).toBe(Date.parse('2026-10-01T12:00:00Z')); + }); + + it('issues a fresh token for the same dashboard', async () => { + const token = await createDashboardToken('dash-a', tokenSigningSecret(ctx.env)); + const res = await refreshSandbоxDashboardToken(refreshReq({ 'X-Dashboard-Token': token }), ctx.env); + expect(res.status).toBe(200); + const body = await res.json() as { token: string }; + const claims = await verifyDashboardToken(body.token, tokenSigningSecret(ctx.env)); + expect(claims?.dashboard_id).toBe('dash-a'); + }); + + it('accepts a token that expired recently (VM was suspended)', async () => { + const expired = await tokenWithExp('dash-a', tokenSigningSecret(ctx.env), -3 * 24 * 60 * 60); + const res = await refreshSandbоxDashboardToken(refreshReq({ 'X-Dashboard-Token': expired }), ctx.env); + expect(res.status).toBe(200); + }); + + it('rejects a token that expired beyond the grace period', async () => { + const expired = await tokenWithExp('dash-a', tokenSigningSecret(ctx.env), -31 * 24 * 60 * 60); + const res = await refreshSandbоxDashboardToken(refreshReq({ 'X-Dashboard-Token': expired }), ctx.env); + expect(res.status).toBe(401); + }); + + it('refuses to mint from the internal token', async () => { + const res = await refreshSandbоxDashboardToken(refreshReq({ 'X-Internal-Token': 'test-internal-token' }), ctx.env); + expect(res.status).toBe(401); + }); + + it('refuses once the dashboard is gone', async () => { + const token = await createDashboardToken('dash-deleted', tokenSigningSecret(ctx.env)); + const res = await refreshSandbоxDashboardToken(refreshReq({ 'X-Dashboard-Token': token }), ctx.env); + expect(res.status).toBe(404); + }); + }); + + describe('sandboxInternalTokenEnv()', () => { + it('passes INTERNAL_API_TOKEN to new VMs by default', () => { + expect(sandboxInternalTokenEnv(ctx.env)).toEqual({ INTERNAL_API_TOKEN: 'test-internal-token' }); + }); + + it('withholds it when WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX=true', () => { + ctx.env.WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX = 'true'; + expect(sandboxInternalTokenEnv(ctx.env)).toEqual({}); + }); + }); +}); diff --git a/controlplane/src/auth/sandbox-auth.ts b/controlplane/src/auth/sandbox-auth.ts new file mode 100644 index 00000000..644f772d --- /dev/null +++ b/controlplane/src/auth/sandbox-auth.ts @@ -0,0 +1,166 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: sandbox-auth-v3-identifier-renames +const MODULE_REVISION = 'sandbox-auth-v3-identifier-renames'; +console.log(`[sandbox-auth] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + +/** + * Authentication for sandbox → control plane `/internal/*` calls. + * + * Every sandbox VM is root-readable by its own user, so any credential it holds + * must only open that VM's own dashboard. A VM authenticates with the + * dashboard-scoped token (X-Dashboard-Token) it receives at session creation, + * and each route checks that the resource it touches belongs to that dashboard. + * + * X-Internal-Token (INTERNAL_API_TOKEN) remains full trust for control-plane-side + * callers. VMs are weaned off it; once WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX is set + * new VMs no longer receive it at all. + */ + +import type { Env } from '../types'; +import { requireInternalAuth } from './middleware'; +import { createDashboardToken, sandboxGeneration, verifyDashboardToken } from './dashboard-token'; +import { tokenSigningSecret } from './pty-token'; + +export type SandboxAuth = + | { kind: 'full' } + | { kind: 'dashboard'; dashboardId: string }; + +/** + * Accepts X-Internal-Token (full trust) or X-Dashboard-Token (scoped to one dashboard). + */ +export async function authenticatеSandbox(request: Request, env: Env): Promise { + if (request.headers.get('X-Internal-Token')) { + const err = requireInternalAuth(request, env); + return err ?? { kind: 'full' }; + } + + const dashboardToken = request.headers.get('X-Dashboard-Token'); + if (dashboardToken) { + const claims = await verifyDashboardToken(dashboardToken, tokenSigningSecret(env)); + if (claims) { + return { kind: 'dashboard', dashboardId: claims.dashboard_id }; + } + } + + const url = new URL(request.url); + console.warn(`[sandbox-auth] REJECT ${request.method} ${url.pathname} - missing or invalid sandbox credential`); + return Response.json({ error: 'E79393: Invalid or missing sandbox credential' }, { status: 401 }); +} + +/** + * Full-trust callers pass. Dashboard-scoped callers pass only when the resource's + * dashboard is known and matches the credential. + */ +export function requireSandboxDashboard( + auth: SandboxAuth, + dashboardId: string | null | undefined, + request: Request +): Response | null { + if (auth.kind === 'full') return null; + if (dashboardId && dashboardId === auth.dashboardId) return null; + const url = new URL(request.url); + console.warn(`[sandbox-auth] REJECT ${request.method} ${url.pathname} - credential for dashboard ${auth.dashboardId} used against ${dashboardId || '(unknown)'}`); + return Response.json({ error: 'E79394: Sandbox credential is not valid for this dashboard' }, { status: 403 }); +} + +/** + * Authenticate and scope-check in one step. + */ +export async function authorizeSandboxForDashboard( + request: Request, + env: Env, + dashboardId: string | null | undefined +): Promise { + const auth = await authenticatеSandbox(request, env); + if (auth instanceof Response) return auth; + return requireSandboxDashboard(auth, dashboardId, request); +} + +/** + * Resolve the dashboard that owns a sandbox session id. + */ +export async function dashboardIdForSandboxSession(env: Env, sandboxSessionId: string): Promise { + if (!sandboxSessionId) return null; + const session = await env.DB.prepare( + 'SELECT dashboard_id FROM sessions WHERE sandbox_session_id = ? ORDER BY created_at DESC LIMIT 1' + ).bind(sandboxSessionId).first<{ dashboard_id: string }>(); + if (session?.dashboard_id) return session.dashboard_id; + const sandbox = await env.DB.prepare( + 'SELECT dashboard_id FROM dashboard_sandboxes WHERE sandbox_session_id = ?' + ).bind(sandboxSessionId).first<{ dashboard_id: string }>(); + return sandbox?.dashboard_id ?? null; +} + +/** + * Resolve the dashboard that owns a schedule execution (null for recipe schedules + * that have no dashboard — those are only reachable with full trust). + */ +export async function dashbоardIdForScheduleExecution(env: Env, executionId: string): Promise { + const row = await env.DB.prepare( + 'SELECT s.dashboard_id AS dashboard_id FROM schedule_executions e JOIN schedules s ON s.id = e.schedule_id WHERE e.id = ?' + ).bind(executionId).first<{ dashboard_id: string | null }>(); + return row?.dashboard_id ?? null; +} + +/** + * Env vars for a new sandbox VM carrying INTERNAL_API_TOKEN — omitted once + * WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX=true, after which the VM authenticates + * only with its dashboard token. + */ +export function sandboxInternalTokenEnv(env: Env): Record { + if (env.WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX === 'true') return {}; + return { INTERNAL_API_TOKEN: env.INTERNAL_API_TOKEN || '' }; +} + +/** + * POST /internal/sandbox/dashboard-token/refresh + * + * Sandbox sessions are created once per dashboard and live as long as the VM, + * which can outlast the 7-day token. The sandbox exchanges its token for a fresh + * one before it expires. A token that expired while the VM was suspended is still + * accepted here for REFRESH_EXPIRED_GRACE_SECONDS. + * + * Only the dashboard's current sandbox can refresh: the token's sbx_gen must + * match its dashboard_sandboxes row, which retiring (blank session) or + * replacing (new created_at) the sandbox invalidates. Otherwise a token copied + * out of a VM would renew itself forever. Tokens minted before sbx_gen existed + * are bound to the current sandbox on their first refresh. + */ +const REFRESH_EXPIRED_GRACE_SECONDS = 30 * 24 * 60 * 60; + +export async function refreshSandbоxDashboardToken(request: Request, env: Env): Promise { + const presented = request.headers.get('X-Dashboard-Token'); + const claims = presented + ? await verifyDashboardToken(presented, tokenSigningSecret(env), REFRESH_EXPIRED_GRACE_SECONDS) + : null; + if (!claims) { + console.warn('[sandbox-auth] refresh denied: missing or invalid dashboard token'); + return Response.json({ error: 'E79395: A valid dashboard token is required to refresh' }, { status: 401 }); + } + + const dashboard = await env.DB.prepare('SELECT id FROM dashboards WHERE id = ?') + .bind(claims.dashboard_id).first<{ id: string }>(); + if (!dashboard) { + console.warn(`[sandbox-auth] refresh denied: dashboard ${claims.dashboard_id} no longer exists`); + return Response.json({ error: 'E79396: Dashboard not found' }, { status: 404 }); + } + + const sandbox = await env.DB.prepare('SELECT sandbox_session_id, created_at FROM dashboard_sandboxes WHERE dashboard_id = ?') + .bind(claims.dashboard_id).first<{ sandbox_session_id: string; created_at: string }>(); + const generation = sandbox?.sandbox_session_id ? sandboxGeneration(sandbox.created_at) : NaN; + if (Number.isNaN(generation)) { + console.warn(`[sandbox-auth] refresh denied: dashboard ${claims.dashboard_id} has no live sandbox`); + return Response.json({ error: 'E79399: Dashboard has no live sandbox' }, { status: 401 }); + } + if (claims.sbx_gen === undefined) { + console.log(`[sandbox-auth] binding pre-generation token for dashboard ${claims.dashboard_id} to its current sandbox`); + } else if (claims.sbx_gen !== generation) { + console.warn(`[sandbox-auth] refresh denied: token for dashboard ${claims.dashboard_id} belongs to a replaced sandbox`); + return Response.json({ error: 'E79398: Token belongs to a sandbox that was replaced' }, { status: 401 }); + } + + const token = await createDashboardToken(claims.dashboard_id, tokenSigningSecret(env), undefined, generation); + return Response.json({ token, revision: MODULE_REVISION }); +} diff --git a/controlplane/src/auth/sessions.ts b/controlplane/src/auth/sessions.ts index b0c69033..f7f532ee 100644 --- a/controlplane/src/auth/sessions.ts +++ b/controlplane/src/auth/sessions.ts @@ -1,12 +1,16 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: sessions-v2-export-parse-cookies +const MODULE_REVISION = 'sessions-v2-export-parse-cookies'; +console.log(`[sessions] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + import type { Env, User } from '../types'; export const SESSION_COOKIE_NAME = 'orcabot_session'; const SESSION_MAX_AGE_DAYS = 30; -function parseCookies(header: string | null): Map { +export function parseCookies(header: string | null): Map { const cookies = new Map(); if (!header) { return cookies; diff --git a/controlplane/src/chat/handler.ts b/controlplane/src/chat/handler.ts index 7b9f6ca6..77eb018b 100644 --- a/controlplane/src/chat/handler.ts +++ b/controlplane/src/chat/handler.ts @@ -1,6 +1,6 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: chat-v23-list-secret-names +// REVISION: chat-v24-signed-sandbox-requests /** * Orcabot Chat Handler @@ -14,7 +14,7 @@ * - DELETE /chat/history - Clear conversation history */ -console.log(`[chat] REVISION: chat-v23-list-secret-names loaded at ${new Date().toISOString()}`); +console.log(`[chat] REVISION: chat-v24-signed-sandbox-requests loaded at ${new Date().toISOString()}`); import type { Env, ChatMessage, ChatToolCall, ChatToolResult, ChatStreamEvent, AnyUIGuidanceCommand } from '../types'; import { type GeminiTool } from '../gemini/client'; @@ -946,7 +946,7 @@ async function executeTool( // last (so the marker never appears before the output is flushed). const wrapped = `echo ${b64} | base64 -d | bash > ${outRel} 2>&1; echo $? > ${exitRel}`; - const client = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const client = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); // rel = path relative to the workspace root (/workspace). const readMarker = async (rel: string) => { const res = await sandboxFetch(env, `/sessions/${sessionId}/file?path=${encodeURIComponent(rel)}`, { machineId }); @@ -985,7 +985,7 @@ async function executeTool( for (const rel of [outRel, exitRel]) { try { await sandboxFetch(env, `/sessions/${sessionId}/file?path=${encodeURIComponent(rel)}`, { method: 'DELETE', machineId }); } catch { /* ignore */ } } - if (pty?.id) { try { await client.deletePty(sessionId, pty.id); } catch { /* ignore */ } } + if (pty?.id) { try { await client.deletePty(sessionId, pty.id, machineId); } catch { /* ignore */ } } if (exitRaw === null) { // The cleanup above deleted the PTY, which kills the process group — so @@ -1009,7 +1009,7 @@ async function executeTool( }; } catch (error) { console.error(`[run_command] THREW: ${error instanceof Error ? error.stack || error.message : String(error)}`); - if (pty?.id) { try { await client.deletePty(sessionId, pty.id); } catch { /* ignore */ } } + if (pty?.id) { try { await client.deletePty(sessionId, pty.id, machineId); } catch { /* ignore */ } } return { result: { error: `run_command failed: ${error instanceof Error ? error.message : String(error)}` }, isError: true }; } } @@ -1057,7 +1057,7 @@ async function executeTool( } // Send input to the PTY - const client = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const client = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); const textToSend = pressEnter ? `${input}\n` : input; await client.writePty( @@ -1126,7 +1126,7 @@ async function executeTool( } // Send the agent command - const client = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const client = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); await client.writePty( sessionRow.dashboard_sandbox_session_id, sessionRow.pty_id, diff --git a/controlplane/src/dashboards/DurableObject.ts b/controlplane/src/dashboards/DurableObject.ts index c48f46be..5685cb9b 100644 --- a/controlplane/src/dashboards/DurableObject.ts +++ b/controlplane/src/dashboards/DurableObject.ts @@ -1,5 +1,6 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: dashboards-do-v1-error-codes /** * Dashboard Durable Object @@ -194,14 +195,14 @@ export class DashboardDO implements DurableObject { // WebSocket upgrade for real-time collaboration if (path === '/ws') { if (request.headers.get('Upgrade') !== 'websocket') { - return new Response('Expected WebSocket', { status: 426 }); + return new Response('E79173: Expected WebSocket', { status: 426 }); } const userId = url.searchParams.get('user_id'); const userName = url.searchParams.get('user_name') || 'Anonymous'; if (!userId) { - return new Response('user_id required', { status: 400 }); + return new Response('E79174: user_id required', { status: 400 }); } const pair = new WebSocketPair(); @@ -418,7 +419,7 @@ export class DashboardDO implements DurableObject { return Response.json({ success: true }); } - return new Response('Not found', { status: 404 }); + return new Response('E79175: Not found', { status: 404 }); } private handleWebSocket(ws: WebSocket, userId: string, userName: string): void { diff --git a/controlplane/src/dashboards/handler.test.ts b/controlplane/src/dashboards/handler.test.ts index da0ce8a3..b6513397 100644 --- a/controlplane/src/dashboards/handler.test.ts +++ b/controlplane/src/dashboards/handler.test.ts @@ -10,11 +10,13 @@ import { listDashbоards, createDashbоard, upsertItem, + createEdge, } from './handler'; import { createTestContext, seedUser, seedDashboard, + seedDashboardItem, createTestUser, } from '../../tests/helpers'; import type { TestContext } from '../../tests/helpers'; @@ -154,4 +156,37 @@ describe('Dashboard Handlers', () => { expect(updated.item.content).toBe('Keep this'); }); }); + + describe('createEdge()', () => { + // Seed directly: the D1 mock's multi-line re-select in upsertItem returns the first row. + async function makeNote(dashboardId: string) { + return (await seedDashboardItem(ctx.db, dashboardId, { type: 'terminal' })).id; + } + + it('creates an edge between two items on the same dashboard', async () => { + const dashboard = await seedDashboard(ctx.db, testUser.id, { id: 'dash-edge-same' }); + const a = await makeNote(dashboard.id); + const b = await makeNote(dashboard.id); + const res = await createEdge(ctx.env, dashboard.id, testUser.id, { sourceItemId: a, targetItemId: b }); + expect(res.status).toBe(201); + }); + + it('rejects an edge whose target lives on another dashboard', async () => { + const mine = await seedDashboard(ctx.db, testUser.id, { id: 'dash-mine-1' }); + const other = await seedDashboard(ctx.db, testUser.id, { id: 'dash-other-1' }); + const source = await makeNote(mine.id); + const foreign = await makeNote(other.id); + const res = await createEdge(ctx.env, mine.id, testUser.id, { sourceItemId: source, targetItemId: foreign }); + expect(res.status).toBe(400); + }); + + it('rejects an edge whose source lives on another dashboard', async () => { + const mine = await seedDashboard(ctx.db, testUser.id, { id: 'dash-mine-2' }); + const other = await seedDashboard(ctx.db, testUser.id, { id: 'dash-other-2' }); + const target = await makeNote(mine.id); + const foreign = await makeNote(other.id); + const res = await createEdge(ctx.env, mine.id, testUser.id, { sourceItemId: foreign, targetItemId: target }); + expect(res.status).toBe(400); + }); + }); }); diff --git a/controlplane/src/dashboards/handler.ts b/controlplane/src/dashboards/handler.ts index 2e2c3af5..0ad597b9 100644 --- a/controlplane/src/dashboards/handler.ts +++ b/controlplane/src/dashboards/handler.ts @@ -5,7 +5,8 @@ * Dashboard API Handlers */ -// REVISION: dashboards-v8-teardown-without-snapshot +// REVISION: dashboards-v10-signed-sandbox-requests +console.log(`[dashboards] REVISION: dashboards-v10-signed-sandbox-requests loaded at ${new Date().toISOString()}`); import type { Env, Dashboard, DashboardItem, DashboardEdge } from '../types'; import { syncItemToLinked, syncEdgeToLinked } from '../links/handler'; @@ -381,7 +382,7 @@ export async function deleteDashbоard( // wedging it. if (sandboxRow?.sandbox_session_id && env.SANDBOX_URL) { try { - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); await sandbox.deleteSession( sandboxRow.sandbox_session_id, sandboxRow.sandbox_machine_id || undefined @@ -633,6 +634,23 @@ export async function createEdge( return Response.json({ error: 'E79303: Not found or no edit access' }, { status: 404 }); } + // SECURITY: both endpoints must be items on THIS dashboard. Edge consumers + // (schedule executor, messaging delivery) act on the target item, so an edge + // into another dashboard's terminal would let a caller drive someone else's PTY. + if (!edge.sourceItemId || !edge.targetItemId) { + return Response.json({ error: 'E79390: sourceItemId and targetItemId are required' }, { status: 400 }); + } + const itemOnDashboard = (itemId: string) => env.DB.prepare( + 'SELECT id FROM dashboard_items WHERE id = ? AND dashboard_id = ?' + ).bind(itemId, dashboardId).first(); + const [sourceItem, targetItem] = await Promise.all([ + itemOnDashboard(edge.sourceItemId), + itemOnDashboard(edge.targetItemId), + ]); + if (!sourceItem || !targetItem) { + return Response.json({ error: 'E79391: Edge endpoints must belong to this dashboard' }, { status: 400 }); + } + const existingEdge = await env.DB.prepare(` SELECT * FROM dashboard_edges WHERE dashboard_id = ? diff --git a/controlplane/src/egress/handler.ts b/controlplane/src/egress/handler.ts index 88bf7b06..e50e5bef 100644 --- a/controlplane/src/egress/handler.ts +++ b/controlplane/src/egress/handler.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: egress-handler-v9-deny-always -console.log(`[egress] REVISION: egress-handler-v9-deny-always loaded at ${new Date().toISOString()}`); +// REVISION: egress-handler-v10-signed-sandbox-requests +console.log(`[egress] REVISION: egress-handler-v10-signed-sandbox-requests loaded at ${new Date().toISOString()}`); /** * Egress Proxy Management @@ -89,7 +89,7 @@ export async function handleApproveEgress( try { const approveUrl = sandboxUrl(env, '/egress/approve'); - const headers = sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined); + const headers = await sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined, 'POST', approveUrl); headers.set('Content-Type', 'application/json'); const resp = await fetch(approveUrl.toString(), { @@ -232,7 +232,7 @@ export async function handleListPendingEgress( try { const pendingUrl = sandboxUrl(env, '/egress/pending'); - const headers = sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined); + const headers = await sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined, 'GET', pendingUrl); const resp = await fetch(pendingUrl.toString(), { method: 'GET', headers, @@ -304,7 +304,7 @@ export async function handleRevokeEgressDomain( if (sandbox) { try { const revokeUrl = sandboxUrl(env, '/egress/revoke'); - const headers = sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined); + const headers = await sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined, 'POST', revokeUrl); headers.set('Content-Type', 'application/json'); const resp = await fetch(revokeUrl.toString(), { @@ -389,7 +389,7 @@ export async function handleRevokeEgressDenied( if (sandbox) { try { const revokeUrl = sandboxUrl(env, '/egress/revoke'); - const headers = sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined); + const headers = await sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined, 'POST', revokeUrl); headers.set('Content-Type', 'application/json'); const resp = await fetch(revokeUrl.toString(), { @@ -484,7 +484,7 @@ export async function handleBlockDefault( if (sandbox) { try { const blockUrl = sandboxUrl(env, '/egress/block-default'); - const headers = sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined); + const headers = await sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined, 'POST', blockUrl); headers.set('Content-Type', 'application/json'); const resp = await fetch(blockUrl.toString(), { method: 'POST', @@ -549,7 +549,7 @@ export async function handleUnblockDefault( if (sandbox) { try { const unblockUrl = sandboxUrl(env, `/egress/block-default/${encodeURIComponent(normalizedPattern)}`); - const headers = sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined); + const headers = await sandboxHeaders(env, undefined, sandbox.sandbox_machine_id || undefined, 'DELETE', unblockUrl); const resp = await fetch(unblockUrl.toString(), { method: 'DELETE', headers, diff --git a/controlplane/src/health/checker.ts b/controlplane/src/health/checker.ts index 6e6a5288..8542f10d 100644 --- a/controlplane/src/health/checker.ts +++ b/controlplane/src/health/checker.ts @@ -1,6 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: health-checker-v2-signed-sandbox-requests + /** * Background Health Checker * @@ -11,6 +13,9 @@ import type { Env } from '../types'; import { SandboxClient } from '../sandbox/client'; +const MODULE_REVISION = 'health-checker-v2-signed-sandbox-requests'; +console.log(`[health-checker] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + export interface HealthStatus { service: string; isHealthy: boolean; @@ -23,7 +28,7 @@ export interface HealthStatus { * Check sandbox health and store result in D1 */ export async function checkAndCacheSandbоxHealth(env: Env): Promise { - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); const now = new Date().toISOString(); try { diff --git a/controlplane/src/index.ts b/controlplane/src/index.ts index 7c8c1cef..ff196d1e 100644 --- a/controlplane/src/index.ts +++ b/controlplane/src/index.ts @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: controlplane-v18-idor-input-validation -console.log(`[controlplane] REVISION: controlplane-v18-idor-input-validation loaded at ${new Date().toISOString()}`); +// REVISION: controlplane-v27-error-codes +console.log(`[controlplane] REVISION: controlplane-v27-error-codes loaded at ${new Date().toISOString()}`); /** * OrcaBot Control Plane - Cloudflare Worker Entry Point @@ -12,6 +12,7 @@ console.log(`[controlplane] REVISION: controlplane-v18-idor-input-validation loa import type { Env, DashboardItem, RecipeStep, Session } from './types'; import { authenticate, requireAuth, rejectPatAuth, requireInternalAuth, validateMcpAuth, devAuthSurfaceTrusted, type AuthContext } from './auth/middleware'; +import { authenticatеSandbox, authorizeSandboxForDashboard, requireSandboxDashboard, dashboardIdForSandboxSession, dashbоardIdForScheduleExecution, refreshSandbоxDashboardToken, sandboxInternalTokenEnv } from './auth/sandbox-auth'; import { checkRateLimitIp, checkRateLimitUser } from './ratelimit/middleware'; import { initializeDatabase } from './db/schema'; import { ensureDb, type EnvWithDb } from './db/remote'; @@ -55,6 +56,7 @@ import * as blog from './blog/handler'; import * as tts from './tts/handler'; import * as releases from './releases/handler'; import { sandboxHeaders, sandboxUrl } from './sandbox/fetch'; +import { sandboxAuthEnv } from './sandbox/request-signing'; // Export Durable Objects export { DashboardDO } from './dashboards/DurableObject'; @@ -94,6 +96,59 @@ function isOriginAllоwed(origin: string | null, allowedOrigins: Set | n return allowedOrigins.has(origin); } +// Error bodies always carry an `E79xxx:` code. Messages bubbled up from thrown +// errors may already have one; otherwise prefix the caller's site code. +function withErrorCode(message: string, code: string): string { + return /^E\d{5}:/.test(message) ? message : `${code}: ${message}`; +} + +const CSRF_CHECKED_METHODS = new Set(['POST', 'PUT', 'PATCH', 'DELETE']); +const CSRF_COOKIE_AUTH_METHODS = new Set(['session', 'cf-access']); + +/** + * Starting, stopping or steering the dashboard browser drives a shared VM + * browser, so it takes the same role as editing the dashboard (owner/editor). + */ +async function requireBrоwserControl(env: Env, dashboardId: string, userId: string): Promise { + const access = await env.DB.prepare( + `SELECT role FROM dashboard_members WHERE dashboard_id = ? AND user_id = ?` + ).bind(dashboardId, userId).first<{ role: string }>(); + if (!access) return Response.json({ error: 'E79301: Not found or no access' }, { status: 404 }); + if (access.role !== 'owner' && access.role !== 'editor') { + return Response.json({ error: 'E79618: Viewers cannot control the browser' }, { status: 403 }); + } + return null; +} + +/** + * Reject cross-site state-changing requests authenticated by an ambient cookie. + * Bearer/PAT/dev-header auth can't be attached cross-site, so they're exempt. + * Requests with no Origin are allowed unless Sec-Fetch-Site marks them cross-site + * (non-browser clients send neither header and aren't a CSRF vector). + * Pages the control plane serves itself (e.g. the Drive folder picker) post + * same-origin, so the request's own origin is always allowed. + */ +export function checkCsrf( + request: Request, + authMethod: string | undefined, + allowedOrigins: Set | null +): Response | null { + if (!CSRF_CHECKED_METHODS.has(request.method)) return null; + if (!authMethod || !CSRF_COOKIE_AUTH_METHODS.has(authMethod)) return null; + if (allowedOrigins === null) return null; // Dev mode: all origins allowed + + const origin = request.headers.get('Origin'); + if (origin !== null) { + if (allowedOrigins.has(origin) || origin === new URL(request.url).origin) return null; + } else { + const fetchSite = request.headers.get('Sec-Fetch-Site'); + if (fetchSite !== 'cross-site') return null; + } + + console.warn(`[csrf] Rejected ${request.method} ${new URL(request.url).pathname} origin=${origin ?? '(none)'}`); + return Response.json({ error: 'E79392: Cross-site request rejected' }, { status: 403 }); +} + const EMBED_ALLOWED_PROTOCOLS = new Set(['http:', 'https:']); const EMBED_FETCH_TIMEOUT_MS = 5_000; const EMBED_MAX_REDIRECTS = 5; @@ -359,7 +414,7 @@ async function prоxySandbоxWebSоcket( const requestUrl = new URL(request.url); const origin = request.headers.get('Origin') || ''; - const headers = sandboxHeaders(env, request.headers, machineId); + const headers = await sandboxHeaders(env, request.headers, machineId, request.method, sandboxUrlValue); headers.delete('Host'); const body = ['POST', 'PUT', 'PATCH'].includes(request.method) @@ -414,7 +469,7 @@ async function prоxySandbоxControlWebSоcket( ): Promise { const sandboxUrlValue = sandboxUrl(env, `/sessions/${sandboxSessionId}/control`); - const headers = sandboxHeaders(env, request.headers, machineId); + const headers = await sandboxHeaders(env, request.headers, machineId, request.method, sandboxUrlValue); headers.delete('Host'); const proxyRequest = new Request(sandboxUrlValue.toString(), { @@ -434,8 +489,10 @@ async function prоxySandbоxRequest( ): Promise { const sandboxUrlValue = sandboxUrl(env, path); sandboxUrlValue.search = new URL(request.url).search; + // The desktop surface token authenticates dev-auth; it must not reach the VM. + if (sandboxUrlValue.searchParams.has('surface')) sandboxUrlValue.searchParams.delete('surface'); - const headers = sandboxHeaders(env, request.headers, machineId); + const headers = await sandboxHeaders(env, request.headers, machineId, request.method, sandboxUrlValue); headers.delete('Host'); const body = request.method === 'GET' || request.method === 'HEAD' ? undefined : request.body; @@ -457,8 +514,10 @@ async function prоxySandbоxWebSоcketPath( ): Promise { const sandboxUrlValue = sandboxUrl(env, path); sandboxUrlValue.search = new URL(request.url).search; + // The desktop surface token authenticates dev-auth; it must not reach the VM. + if (sandboxUrlValue.searchParams.has('surface')) sandboxUrlValue.searchParams.delete('surface'); - const headers = sandboxHeaders(env, request.headers, machineId); + const headers = await sandboxHeaders(env, request.headers, machineId, request.method, sandboxUrlValue); headers.delete('Host'); const proxyRequest = new Request(sandboxUrlValue.toString(), { @@ -528,7 +587,7 @@ export default { } catch (error) { if (isDesktopFeatureDisabledError(error)) { return cоrsRespоnse(Response.json( - { error: 'Desktop feature disabled', message: (error as Error).message }, + { error: 'E79150: Desktop feature disabled', message: (error as Error).message }, { status: 501 } ), origin, allowedOrigins); } @@ -537,13 +596,13 @@ export default { if (error instanceof SyntaxError) { console.warn('Request body parse error (400):', error.message); return cоrsRespоnse(Response.json( - { error: 'E40001: Invalid JSON body' }, + { error: 'E79281: Invalid JSON body' }, { status: 400 } ), origin, allowedOrigins); } console.error('Request error:', error); return cоrsRespоnse(Response.json( - { error: error instanceof Error ? error.message : 'Internal server error' }, + { error: withErrorCode(error instanceof Error ? error.message : 'Internal server error', 'E79151') }, { status: 500 } ), origin, allowedOrigins); } @@ -717,9 +776,9 @@ async function replenishWarmPool(env: EnvWithBindings): Promise { // (WARM_POOL_TARGET) so the always-on cost is bounded. autostop: 'off', env: { - SANDBOX_INTERNAL_TOKEN: env.SANDBOX_INTERNAL_TOKEN || '', + ...(await sandboxAuthEnv(env)), CONTROLPLANE_URL: env.FLY_SANDBOX_CONTROLPLANE_URL || 'https://api.orcabot.com', - INTERNAL_API_TOKEN: env.INTERNAL_API_TOKEN || '', + ...sandboxInternalTokenEnv(env), ALLOWED_ORIGINS: env.ALLOWED_ORIGINS || 'https://orcabot.com', ...(env.EGRESS_PROXY_ENABLED ? { EGRESS_PROXY_ENABLED: env.EGRESS_PROXY_ENABLED } : {}), }, @@ -921,7 +980,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick POST would otherwise ride the cookie. + const csrfError = checkCsrf(request, auth.method, parseAllоwedOrigins(env)); + if (csrfError) return csrfError; + // Unauthenticated IP rate limit (after auth to avoid double-limiting) if (!auth.user) { const skipIpRateLimit = @@ -1000,7 +1065,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick[4]); @@ -1914,7 +1979,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick(); if (!access) { return Response.json({ error: 'E79301: Not found or no access' }, { status: 404 }); } + // The VNC websocket is full mouse/keyboard control (x11vnc has no per-client + // view-only mode here), so viewers get the UI assets but not the connection. + const wantsControl = segments.slice(3).join('/').startsWith('websockify') || + request.headers.get('Upgrade')?.toLowerCase() === 'websocket'; + if (wantsControl && access.role !== 'owner' && access.role !== 'editor') { + return Response.json({ error: 'E79618: Viewers cannot control the browser' }, { status: 403 }); + } } if (allowDevBypass) { @@ -3000,7 +3083,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick ({})) as { dashboardId?: unknown }; + const authError = await authorizeSandboxForDashboard(request, env, typeof progressBody.dashboardId === 'string' ? progressBody.dashboardId : null); if (authError) return authError; return integrations.updateDriveSyncPrоgressInternal(request, env); } // GET /internal/mirror/manifest if (segments[0] === 'internal' && segments[1] === 'mirror' && segments[2] === 'manifest' && method === 'GET') { - const authError = requireInternalAuth(request, env); + const authError = await authorizeSandboxForDashboard(request, env, url.searchParams.get('dashboard_id')); if (authError) return authError; return integrations.getMirrоrManifestInternal(request, env); } // GET /internal/mirror/file if (segments[0] === 'internal' && segments[1] === 'mirror' && segments[2] === 'file' && method === 'GET') { - const authError = requireInternalAuth(request, env); + const authError = await authorizeSandboxForDashboard(request, env, url.searchParams.get('dashboard_id')); if (authError) return authError; return integrations.getMirrоrFileInternal(request, env); } // POST /internal/mirror/sync/progress if (segments[0] === 'internal' && segments[1] === 'mirror' && segments[2] === 'sync' && segments[3] === 'progress' && method === 'POST') { - const authError = requireInternalAuth(request, env); + const progressBody = await request.clone().json().catch(() => ({})) as { dashboardId?: unknown }; + const authError = await authorizeSandboxForDashboard(request, env, typeof progressBody.dashboardId === 'string' ? progressBody.dashboardId : null); if (authError) return authError; return integrations.updateMirrоrSyncPrоgressInternal(request, env); } // POST /internal/browser/open - Notify browser open from sandbox session if (segments[0] === 'internal' && segments[1] === 'browser' && segments[2] === 'open' && method === 'POST') { - const authError = requireInternalAuth(request, env); - if (authError) return authError; + const sbAuth = await authenticatеSandbox(request, env); + if (sbAuth instanceof Response) return sbAuth; const data = await request.json() as { sandbox_session_id?: string; url?: string; pty_id?: string; }; const sandboxSessionId = typeof data.sandbox_session_id === 'string' ? data.sandbox_session_id : ''; + if (sbAuth.kind !== 'full') { + const scopeError = requireSandboxDashboard(sbAuth, await dashboardIdForSandboxSession(env, sandboxSessionId), request); + if (scopeError) return scopeError; + } const url = typeof data.url === 'string' ? data.url : ''; const ptyId = typeof data.pty_id === 'string' ? data.pty_id : undefined; return sessions.openBrowserFromSandbоxSessionInternal(env, sandboxSessionId, url, ptyId); @@ -3281,7 +3370,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick[2], - data.dashboardId, - data.userId - ); - } - - // POST /internal/gateway/:provider/validate-token - Validate using PTY token (SECURE) - // Security: terminal_id extracted from cryptographically verified token, not from body - // Body: { action?: string, args?: Record } - // NOTE: context is derived server-side from args (same as execute endpoint) to prevent spoofing - if (segments[0] === 'internal' && segments[1] === 'gateway' && segments.length === 4 && segments[3] === 'validate-token' && method === 'POST') { - const ptyToken = request.headers.get('X-PTY-Token'); - if (!ptyToken) { - return Response.json({ error: 'E79410: AUTH_DENIED', reason: 'Missing X-PTY-Token header' }, { status: 401 }); - } - let action: string | undefined; - let args: Record | undefined; - try { - const body = await request.json() as { - action?: string; - args?: Record; - }; - action = body.action; - args = body.args; - } catch { - // No body or invalid JSON - action and args remain undefined - } - // Derive context server-side from args (never trust body.context) - const { deriveEnforcementContext } = await import('./integration-policies/gateway'); - const context = args && action ? deriveEnforcementContext(action, args) : undefined; - return integrationPolicies.validateGatewayWithToken( - env, - ptyToken, - segments[2] as Parameters[2], - action, - context - ); - } - // ============================================ // Internal Agent State Gateway (Tasks & Memory) // NOTE: These MUST come before the generic /internal/gateway/:provider/execute route @@ -3374,7 +3408,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick(); if (!membership) { return Response.json({ error: 'E79420: Not found' }, { status: 404 }); } + // Subscribing wires a channel into the dashboard (same bar as editing the block). + if (membership.role !== 'owner' && membership.role !== 'editor') { + return Response.json({ error: 'E79619: Viewers cannot add channel subscriptions' }, { status: 403 }); + } // Verify itemId belongs to the dashboard AND is a messaging block matching the provider. // Without this, subscriptions could be created for any item (terminals, notes, etc.), // causing confusing delivery behavior. @@ -3641,7 +3684,7 @@ async function handleRequest(request: Request, env: EnvWithBindings, ctx: Pick}` act on primary. +function calendarIdArg(args: Record): string { + const value = args.calendarId ?? args.calendar_id; + return typeof value === 'string' && value ? value : 'primary'; +} + +function sendUpdatesArg(args: Record): string { + return args.sendUpdates === 'all' || args.sendUpdates === 'externalOnly' ? args.sendUpdates : 'none'; +} /** * Google Calendar API Client @@ -109,7 +120,7 @@ async function listEvents( args: Record, accessToken: string ): Promise { - const calendarId = args.calendarId as string || 'primary'; + const calendarId = calendarIdArg(args); const timeMin = args.timeMin as string || new Date().toISOString(); const timeMax = args.timeMax as string || undefined; const maxResults = Math.min(args.maxResults as number || 100, 2500); @@ -141,14 +152,14 @@ async function getEvent( args: Record, accessToken: string ): Promise { - const calendarId = args.calendarId as string || 'primary'; + const calendarId = calendarIdArg(args); const eventId = args.eventId as string; if (!eventId) { throw new Error('eventId is required'); } const response = await calendarFetch( - `/calendars/${encodeURIComponent(calendarId)}/events/${eventId}`, + `/calendars/${encodeURIComponent(calendarId)}/events/${encodeURIComponent(eventId)}`, accessToken ); @@ -159,7 +170,7 @@ async function createEvent( args: Record, accessToken: string ): Promise { - const calendarId = args.calendarId as string || 'primary'; + const calendarId = calendarIdArg(args); const summary = args.summary as string; if (!summary) { throw new Error('summary is required'); @@ -180,7 +191,7 @@ async function createEvent( attendees: args.attendees as CalendarEvent['attendees'] || undefined, }; - const sendUpdates = args.sendUpdates as string || 'none'; + const sendUpdates = sendUpdatesArg(args); const response = await calendarFetch( `/calendars/${encodeURIComponent(calendarId)}/events?sendUpdates=${sendUpdates}`, @@ -198,7 +209,7 @@ async function updateEvent( args: Record, accessToken: string ): Promise { - const calendarId = args.calendarId as string || 'primary'; + const calendarId = calendarIdArg(args); const eventId = args.eventId as string; if (!eventId) { throw new Error('eventId is required'); @@ -218,10 +229,10 @@ async function updateEvent( attendees: args.attendees as CalendarEvent['attendees'] || existing.attendees, }; - const sendUpdates = args.sendUpdates as string || 'none'; + const sendUpdates = sendUpdatesArg(args); const response = await calendarFetch( - `/calendars/${encodeURIComponent(calendarId)}/events/${eventId}?sendUpdates=${sendUpdates}`, + `/calendars/${encodeURIComponent(calendarId)}/events/${encodeURIComponent(eventId)}?sendUpdates=${sendUpdates}`, accessToken, { method: 'PUT', @@ -236,16 +247,16 @@ async function deleteEvent( args: Record, accessToken: string ): Promise<{ success: boolean }> { - const calendarId = args.calendarId as string || 'primary'; + const calendarId = calendarIdArg(args); const eventId = args.eventId as string; if (!eventId) { throw new Error('eventId is required'); } - const sendUpdates = args.sendUpdates as string || 'none'; + const sendUpdates = sendUpdatesArg(args); await calendarFetch( - `/calendars/${encodeURIComponent(calendarId)}/events/${eventId}?sendUpdates=${sendUpdates}`, + `/calendars/${encodeURIComponent(calendarId)}/events/${encodeURIComponent(eventId)}?sendUpdates=${sendUpdates}`, accessToken, { method: 'DELETE', @@ -259,7 +270,7 @@ async function searchEvents( args: Record, accessToken: string ): Promise { - const calendarId = args.calendarId as string || 'primary'; + const calendarId = calendarIdArg(args); const query = args.query as string; const timeMin = args.timeMin as string || new Date().toISOString(); const timeMax = args.timeMax as string || undefined; diff --git a/controlplane/src/integration-policies/api-clients/discord.ts b/controlplane/src/integration-policies/api-clients/discord.ts index 4e1e156c..106eb5ab 100644 --- a/controlplane/src/integration-policies/api-clients/discord.ts +++ b/controlplane/src/integration-policies/api-clients/discord.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: discord-client-v1-initial -const MODULE_REVISION = 'discord-client-v1-initial'; +// REVISION: discord-client-v3-identifier-renames +const MODULE_REVISION = 'discord-client-v3-identifier-renames'; console.log(`[discord-client] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); /** @@ -14,6 +14,38 @@ console.log(`[discord-client] REVISION: ${MODULE_REVISION} loaded at ${new Date( const DISCORD_API_BASE = 'https://discord.com/api/v10'; +/** + * Every Discord integration shares the platform bot token, which is in every + * customer's guild. The gateway must therefore pin each action to the guild + * stored on the caller's integration (gateway.ts step 9c): + * - guild actions get guild_id forced to the stored guild; + * - channel actions have their channel looked up and must belong to it. + * An action in none of these sets is denied, so a new action can't skip the check. + */ +export const DISCORD_GUILD_ACTIONS = new Set(['discord.list_channels']); +export const DISCORD_CHANNEL_ACTIONS = new Set([ + 'discord.read_messages', + 'discord.send_message', + 'discord.reply_thread', + 'discord.react', + 'discord.edit_message', + 'discord.delete_message', +]); +// Public user profile lookup; not tied to any guild's data. +export const DISCORD_UNSCOPED_ACTIONS = new Set(['discord.get_user_info']); + +/** + * The guild a channel (or thread) belongs to, or null for a DM / group DM. + * Throws on any lookup failure so callers fail closed. + */ +export async function getDiscоrdChannelGuildId(channelId: string, accessToken: string): Promise { + const data = await discordFetch(`/channels/${encodeURIComponent(channelId)}`, accessToken) as DiscordChannel; + if (!data || typeof data !== 'object' || data.id !== channelId) { + throw new Error('Discord channel lookup returned an unexpected response'); + } + return typeof data.guild_id === 'string' && data.guild_id ? data.guild_id : null; +} + interface DiscordMessage { id: string; content: string; diff --git a/controlplane/src/integration-policies/api-clients/drive.ts b/controlplane/src/integration-policies/api-clients/drive.ts index 7c35098a..6beedd4e 100644 --- a/controlplane/src/integration-policies/api-clients/drive.ts +++ b/controlplane/src/integration-policies/api-clients/drive.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: drive-client-v5-recursive-sync-list -console.log(`[drive-client] REVISION: drive-client-v5-recursive-sync-list loaded at ${new Date().toISOString()}`); +// REVISION: drive-client-v6-encoded-ids +console.log(`[drive-client] REVISION: drive-client-v6-encoded-ids loaded at ${new Date().toISOString()}`); /** * Google Drive API Client @@ -13,6 +13,11 @@ console.log(`[drive-client] REVISION: drive-client-v5-recursive-sync-list loaded const DRIVE_API_BASE = 'https://www.googleapis.com/drive/v3'; +/** Escapes a value for a single-quoted Drive search-query string. */ +function driveQueryString(value: string): string { + return value.replace(/\\/g, '\\\\').replace(/'/g, "\\'"); +} + interface DriveFile { id: string; name: string; @@ -107,7 +112,7 @@ async function listFiles( queryParts.push(query); } if (folderId) { - queryParts.push(`'${folderId}' in parents`); + queryParts.push(`'${driveQueryString(folderId)}' in parents`); } queryParts.push('trashed = false'); @@ -136,7 +141,7 @@ async function getFile( fields: 'id,name,mimeType,parents,webViewLink,webContentLink,size,modifiedTime,createdTime,owners', }); - const response = await driveFetch(`/files/${fileId}?${params}`, accessToken); + const response = await driveFetch(`/files/${encodeURIComponent(fileId)}?${params}`, accessToken); return response.json() as Promise; } @@ -179,13 +184,13 @@ async function downloadFile( // For Google Docs, Sheets, etc., use export (always text-safe) if (file.mimeType.startsWith('application/vnd.google-apps')) { const exportMimeType = getExportMimeType(file.mimeType); - const response = await driveFetch(`/files/${fileId}/export?mimeType=${encodeURIComponent(exportMimeType)}`, accessToken); + const response = await driveFetch(`/files/${encodeURIComponent(fileId)}/export?mimeType=${encodeURIComponent(exportMimeType)}`, accessToken); const content = await response.text(); return { content, mimeType: exportMimeType }; } // For regular files, download directly - const response = await driveFetch(`/files/${fileId}?alt=media`, accessToken); + const response = await driveFetch(`/files/${encodeURIComponent(fileId)}?alt=media`, accessToken); if (isTextMimeType(file.mimeType)) { // Text files: read as UTF-8 text @@ -311,7 +316,7 @@ async function updateFile( const body = buildMultipartBody(boundary, metadata, mimeType, content, encoding); - const response = await fetch(`https://www.googleapis.com/upload/drive/v3/files/${fileId}?uploadType=multipart&fields=id,name,mimeType,webViewLink`, { + const response = await fetch(`https://www.googleapis.com/upload/drive/v3/files/${encodeURIComponent(fileId)}?uploadType=multipart&fields=id,name,mimeType,webViewLink`, { method: 'PATCH', headers: { Authorization: `Bearer ${accessToken}`, @@ -337,7 +342,7 @@ async function deleteFile( throw new Error('fileId is required'); } - const response = await driveFetch(`/files/${fileId}`, accessToken, { + const response = await driveFetch(`/files/${encodeURIComponent(fileId)}`, accessToken, { method: 'DELETE', }); @@ -355,7 +360,7 @@ async function shareFile( throw new Error('fileId and email are required'); } - const response = await driveFetch(`/files/${fileId}/permissions`, accessToken, { + const response = await driveFetch(`/files/${encodeURIComponent(fileId)}/permissions`, accessToken, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ @@ -440,7 +445,7 @@ async function syncListFilesRecursive( const params = new URLSearchParams({ pageSize: '1000', fields: 'files(id,name,mimeType,parents,size,modifiedTime,md5Checksum,createdTime),nextPageToken', - q: `'${currentFolderId}' in parents and trashed = false`, + q: `'${driveQueryString(currentFolderId)}' in parents and trashed = false`, }); if (pageToken) { diff --git a/controlplane/src/integration-policies/api-clients/github.ts b/controlplane/src/integration-policies/api-clients/github.ts index bd2557d0..f2d2d615 100644 --- a/controlplane/src/integration-policies/api-clients/github.ts +++ b/controlplane/src/integration-policies/api-clients/github.ts @@ -1,8 +1,17 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: github-client-v1 -console.log(`[github-client] REVISION: github-client-v1 loaded at ${new Date().toISOString()}`); +// REVISION: github-client-v2-encoded-paths +console.log(`[github-client] REVISION: github-client-v2-encoded-paths loaded at ${new Date().toISOString()}`); + +/** + * Encode a repo file path one segment at a time, so names with spaces, `#`, `?` + * or `%` stay part of the filename. `.`/`..` segments are rejected earlier + * (arg-safety.ts) since encoding leaves dots unchanged. + */ +function encodeRepoPath(path: string): string { + return path.replace(/^\/+/, '').split('/').map(encodeURIComponent).join('/'); +} /** * GitHub API Client @@ -131,7 +140,7 @@ async function getRepo( throw new Error('owner and repo are required'); } - const response = await githubFetch(`/repos/${owner}/${repo}`, accessToken); + const response = await githubFetch(`/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}`, accessToken); return response.json() as Promise; } @@ -153,7 +162,7 @@ async function listIssues( per_page: perPage.toString(), }); - const response = await githubFetch(`/repos/${owner}/${repo}/issues?${params}`, accessToken); + const response = await githubFetch(`/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/issues?${params}`, accessToken); return response.json() as Promise; } @@ -171,7 +180,7 @@ async function createIssue( const body = args.body as string || ''; const labels = args.labels as string[] || []; - const response = await githubFetch(`/repos/${owner}/${repo}/issues`, accessToken, { + const response = await githubFetch(`/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/issues`, accessToken, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ title, body, labels }), @@ -198,7 +207,7 @@ async function listPRs( per_page: perPage.toString(), }); - const response = await githubFetch(`/repos/${owner}/${repo}/pulls?${params}`, accessToken); + const response = await githubFetch(`/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/pulls?${params}`, accessToken); return response.json() as Promise; } @@ -218,7 +227,7 @@ async function createPR( const body = args.body as string || ''; const draft = args.draft as boolean || false; - const response = await githubFetch(`/repos/${owner}/${repo}/pulls`, accessToken, { + const response = await githubFetch(`/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/pulls`, accessToken, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ title, body, head, base, draft }), @@ -242,7 +251,7 @@ async function getFileContent( const params = ref ? `?ref=${encodeURIComponent(ref)}` : ''; const response = await githubFetch( - `/repos/${owner}/${repo}/contents/${path}${params}`, + `/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/contents/${encodeRepoPath(path)}${params}`, accessToken ); @@ -264,7 +273,7 @@ async function listFiles( const params = ref ? `?ref=${encodeURIComponent(ref)}` : ''; const response = await githubFetch( - `/repos/${owner}/${repo}/contents/${path}${params}`, + `/repos/${encodeURIComponent(owner)}/${encodeURIComponent(repo)}/contents/${encodeRepoPath(path)}${params}`, accessToken ); diff --git a/controlplane/src/integration-policies/api-clients/gmail.ts b/controlplane/src/integration-policies/api-clients/gmail.ts index 047f8017..623b354a 100644 --- a/controlplane/src/integration-policies/api-clients/gmail.ts +++ b/controlplane/src/integration-policies/api-clients/gmail.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: gmail-client-v1 -console.log(`[gmail-client] REVISION: gmail-client-v1 loaded at ${new Date().toISOString()}`); +// REVISION: gmail-client-v2-header-safety +console.log(`[gmail-client] REVISION: gmail-client-v2-header-safety loaded at ${new Date().toISOString()}`); /** * Gmail API Client @@ -130,14 +130,25 @@ async function getMessage( return response.json() as Promise; } +/** to/cc/bcc as the same string list the policy check saw (a bare string is one entry). */ +function headerList(value: unknown): string[] { + if (typeof value === 'string') return [value]; + return Array.isArray(value) ? value.filter((v): v is string => typeof v === 'string') : []; +} + async function sendMessage( args: Record, accessToken: string ): Promise { - const to = args.to as string[] || []; - const cc = args.cc as string[] || []; - const bcc = args.bcc as string[] || []; - const subject = args.subject as string || ''; + const to = headerList(args.to); + const cc = headerList(args.cc); + const bcc = headerList(args.bcc); + const subject = typeof args.subject === 'string' ? args.subject : ''; + // The gateway rejects these (arg-safety.ts); refuse here too so a line break + // can never add headers such as Bcc to the raw message. + if ([...to, ...cc, ...bcc, subject].some((v) => /[\r\n]/.test(v))) { + throw new Error('Line breaks are not allowed in email headers'); + } const body = args.body as string || ''; const threadId = args.threadId as string || undefined; diff --git a/controlplane/src/integration-policies/api-clients/outlook-calendar.ts b/controlplane/src/integration-policies/api-clients/outlook-calendar.ts index 866c9e4b..3fc0e95b 100644 --- a/controlplane/src/integration-policies/api-clients/outlook-calendar.ts +++ b/controlplane/src/integration-policies/api-clients/outlook-calendar.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: outlook-calendar-v2-scope-by-calendar-id -console.log(`[outlook-calendar-client] REVISION: outlook-calendar-v2-scope-by-calendar-id loaded at ${new Date().toISOString()}`); +// REVISION: outlook-calendar-v3-event-ops-default-calendar +console.log(`[outlook-calendar-client] REVISION: outlook-calendar-v3-event-ops-default-calendar loaded at ${new Date().toISOString()}`); /** * Outlook Calendar API Client (Microsoft Graph) @@ -174,7 +174,7 @@ async function getEvent( const basePath = calendarId ? `/me/calendars/${encodeURIComponent(calendarId)}/events/${encodeURIComponent(eventId)}` - : `/me/events/${encodeURIComponent(eventId)}`; + : `/me/calendar/events/${encodeURIComponent(eventId)}`; // default calendar, as the policy check assumes const response = await fetch(`${GRAPH_API_BASE}${basePath}`, { headers: { @@ -363,7 +363,7 @@ async function updateEvent( const calendarId = (args.calendarId ?? args.calendar_id) as string | undefined; const basePath = calendarId ? `/me/calendars/${encodeURIComponent(calendarId)}/events/${encodeURIComponent(eventId)}` - : `/me/events/${encodeURIComponent(eventId)}`; + : `/me/calendar/events/${encodeURIComponent(eventId)}`; // default calendar, as the policy check assumes const response = await fetch(`${GRAPH_API_BASE}${basePath}`, { method: 'PATCH', @@ -395,7 +395,7 @@ async function deleteEvent( const basePath = calendarId ? `/me/calendars/${encodeURIComponent(calendarId)}/events/${encodeURIComponent(eventId)}` - : `/me/events/${encodeURIComponent(eventId)}`; + : `/me/calendar/events/${encodeURIComponent(eventId)}`; // default calendar, as the policy check assumes const response = await fetch(`${GRAPH_API_BASE}${basePath}`, { method: 'DELETE', diff --git a/controlplane/src/integration-policies/arg-safety.test.ts b/controlplane/src/integration-policies/arg-safety.test.ts new file mode 100644 index 00000000..19402a5b --- /dev/null +++ b/controlplane/src/integration-policies/arg-safety.test.ts @@ -0,0 +1,119 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +import { describe, it, expect } from 'vitest'; +import { validateGatewayArgs, dropEmptyPathArgs } from './arg-safety'; +import { deriveEnforcementContext } from './gateway'; + +describe('validateGatewayArgs: IDs in URL paths', () => { + it.each([ + ['github', { owner: 'myorg', repo: '../otherorg/secret' }], + ['github', { owner: 'myorg/../evil', repo: 'x' }], + ['github', { owner: 'myorg', repo: 'ok', path: '../../../otherorg/secret/contents/f' }], + ['github', { owner: 'myorg', repo: 'ok', path: 'docs/../../x' }], + ['github', { owner: 'myorg', repo: 'ok', path: './x' }], + ['github', { owner: 'myorg', repo: 'ok', path: 'docs/a\nb.md' }], + ['discord', { channel: 'C1', message_id: '../../C2/messages/9' }], + ['google_drive', { fileId: 'OTHER/permissions?x=' }], + ['gmail', { messageId: 'abc/trash' }], + ['teams', { team_id: 'T/channels/OTHER/messages#', channel_id: '19:ok@thread.tacv2' }], + ['google_calendar', { calendarId: 'mine', eventId: '../../other@group.calendar.google.com/events/1' }], + ['google_drive', { fileId: 'ID?alt=media#' }], + ['google_drive', { folderId: "x' in parents or 'y" }], + ['google_calendar', { calendarId: 'mine', event_id: 'a/../../b' }], + ['twitter', { tweet_id: '1?x=y' }], + ['google_chat', { space: 'spaces/A', message_id: 'spaces/B/messages/1' }], + ['google_chat', { space: 'spaces/../B' }], + ])('%s rejects %j', (provider, args) => { + expect(validateGatewayArgs(provider, args)).toMatch(/^E796/); + }); + + it.each([ + ['github', { owner: 'my-org', repo: 'repo.js', path: 'src/index.ts' }], + ['github', { owner: 'o', repo: 'r', path: '' }], + ['github', { owner: 'o', repo: 'r', path: 'docs/Getting Started.md' }], + ['github', { owner: 'o', repo: 'r', path: 'notes/#1 why?.md' }], + ['github', { owner: 'o', repo: 'r', path: 'data/100%/a..b.txt' }], + ['discord', { channel: '123456789', message_id: '987654321' }], + ['teams', { team_id: '8c1e-guid', channel_id: '19:abc@thread.tacv2', message_id: '1700000000000' }], + ['google_drive', { fileId: '1AbC_d-EfG' }], + ['google_chat', { space: 'spaces/AAAA', message_id: 'spaces/AAAA/messages/BBB.CCC' }], + ['google_chat', { space: 'AAAA', message_id: 'BBB' }], + ['google_calendar', { calendarId: 'en.usa#holiday@group.v.calendar.google.com', eventId: 'abc123' }], + ])('%s accepts %j', (provider, args) => { + expect(validateGatewayArgs(provider, args)).toBeNull(); + }); +}); + +describe('validateGatewayArgs: email headers and recipients', () => { + it('rejects header injection through the subject or a recipient', () => { + expect(validateGatewayArgs('gmail', { to: ['a@ok.com'], subject: 'hi\r\nBcc: evil@x.com' })).toMatch(/^E79613/); + expect(validateGatewayArgs('gmail', { to: ['a@ok.com\r\nBcc: evil@x.com'] })).toMatch(/^E79613/); + expect(validateGatewayArgs('outlook', { to: ['a@ok.com\nCc: evil@x.com'] })).toMatch(/^E79613/); + }); + + it.each([ + ['a@ok.com, evil@x.com'], + ['x , evil@x.com'], + ['a@ok.com ,evil@x.com'], + ['a@ok.com;evil@x.com'], + ['"Doe, John" '], + ])('rejects a recipient entry hiding several addresses: %s', (entry) => { + expect(validateGatewayArgs('gmail', { to: [entry] })).toMatch(/^E79614/); + }); + + it('accepts one address per entry, with or without a display name', () => { + expect(validateGatewayArgs('gmail', { to: ['a@ok.com', 'Jane Doe '], cc: 'b@ok.com', subject: 'Hello' })).toBeNull(); + }); + + it('rejects non-string recipients', () => { + expect(validateGatewayArgs('gmail', { to: ['a@ok.com', 42] })).toMatch(/^E79614/); + }); +}); + +describe('deriveEnforcementContext: checks what the client sends', () => { + it.each([ + ['telegram.send_message', { channel: 'ALLOWED', chat_id: 'TARGET' }], + ['whatsapp.send_message', { channel: 'ALLOWED', to: 'TARGET' }], + ['teams.send_message', { channel: 'ALLOWED', channel_id: 'TARGET' }], + ['matrix.send_message', { channel: 'ALLOWED', room_id: 'TARGET' }], + ['google_chat.send_message', { channel: 'ALLOWED', space: 'TARGET' }], + ['slack.send_message', { chat_id: 'ALLOWED', channel: 'TARGET' }], + ['discord.send_message', { chat_id: 'ALLOWED', channel: 'TARGET' }], + ])('%s checks the field the client sends to', (action, args) => { + expect(deriveEnforcementContext(action, args).channelId).toBe('TARGET'); + }); + + it('ignores an LLM-supplied channel_name', () => { + expect(deriveEnforcementContext('slack.send_message', { channel: 'C_SECRET', channel_name: 'general' }).channelName).toBeUndefined(); + }); + + it('counts Discord content and Slack blocks toward the message length', () => { + expect(deriveEnforcementContext('discord.send_message', { channel: 'C', content: 'x'.repeat(500) }).messageText).toHaveLength(500); + const blocks = [{ type: 'section', text: { type: 'mrkdwn', text: 'y'.repeat(800) } }]; + expect(deriveEnforcementContext('slack.send_message', { channel: 'C', text: 'hi', blocks }).messageText!.length).toBeGreaterThan(800); + }); +}); + +describe('empty path IDs are treated as omitted', () => { + it.each([ + ['google_drive', { folderId: '' }], + ['google_drive', { fileId: '', folderId: '' }], + ['gmail', { labelId: '' }], + ['discord', { guild_id: '' }], + ['github', { owner: '', repo: '' }], + ])('%s %j passes validation', (provider, args) => { + expect(validateGatewayArgs(provider, args)).toBeNull(); + }); + + it('dropEmptyPathArgs removes them so no client builds a URL from an empty segment', () => { + const args: Record = { folderId: '', fileId: 'abc', name: '' }; + dropEmptyPathArgs('google_drive', args); + expect(args).toEqual({ fileId: 'abc', name: '' }); + }); + + it('still rejects whitespace and traversal', () => { + expect(validateGatewayArgs('discord', { channel: ' ' })).toMatch(/^E79610/); + expect(validateGatewayArgs('google_drive', { folderId: '..' })).toMatch(/^E796/); + }); +}); diff --git a/controlplane/src/integration-policies/arg-safety.ts b/controlplane/src/integration-policies/arg-safety.ts new file mode 100644 index 00000000..bfcdaa8f --- /dev/null +++ b/controlplane/src/integration-policies/arg-safety.ts @@ -0,0 +1,199 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: arg-safety-v6-identifier-renames +import type { IntegrationProvider } from '../types'; + +const MODULE_REVISION = 'arg-safety-v6-identifier-renames'; +console.log(`[arg-safety] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + +/** + * Gateway argument validation, run before policy enforcement. + * + * enforcePolicy checks values derived from the LLM's arguments, and the API + * clients then build requests from those same arguments. Anything that lets one + * argument mean one thing to the check and another to the provider is a policy + * bypass, so it is rejected here: + * - IDs interpolated into URL paths must be one plain path segment. A `/`, + * `..`, `?`, `#` or `%` would let an allowed ID reach another repo, channel, + * calendar or file once the URL is normalised. + * - Email header values must not contain CR/LF (header injection, e.g. Bcc). + * - Each recipient entry must be exactly one address; "a@ok.com, b@evil.com" + * would otherwise be checked as a@ok.com and sent to both. + */ + +/** Messaging: the one arg each provider's client sends to (see api-clients/). */ +export const MESSAGING_TARGET_FIELD: Record = { + slack: 'channel', + discord: 'channel', + telegram: 'chat_id', + whatsapp: 'to', + teams: 'channel_id', + matrix: 'room_id', + google_chat: 'space', +}; + +/** + * Args each client interpolates into a URL path. Keyed by the gateway's + * provider identifiers (IntegrationProvider) — a key that matches no provider + * silently disables the check, so the type only admits real providers. + */ +export const PATH_SEGMENT_ARGS: Partial> = { + discord: ['channel', 'message_id', 'guild_id', 'user_id', 'user'], + github: ['owner', 'repo'], + google_drive: ['fileId', 'folderId'], + gmail: ['messageId', 'threadId', 'labelId'], + google_chat: ['member_id'], + teams: ['team_id', 'channel_id', 'message_id', 'conversation_id', 'user_id'], + whatsapp: ['phone_number_id', 'message_id'], + twitter: ['tweet_id', 'user_id', 'username', 'target_user_id', 'account_id'], + google_calendar: ['eventId', 'event_id'], + outlook_calendar: ['event_id', 'eventId', 'id'], +}; + +/** + * Args the gateway reads into the policy context (gateway.ts buildContext), + * each only when it is a string. Any other type was skipped by the check but + * still used by the client — `calendarId: ["FORBIDDEN"]` was checked as the + * default calendar and sent as `/me/calendars/FORBIDDEN` — so a non-string + * here is rejected for every provider. + */ +const POLICY_SCALAR_ARGS = [ + 'fileId', 'folderId', 'name', 'mimeType', + 'messageId', 'message_id', 'eventId', 'event_id', 'ts', 'timestamp', + 'owner', 'repo', 'calendarId', 'calendar_id', + 'thread_ts', 'reply_to_message_id', 'thread_key', 'user', 'user_id', +]; + +const EMAIL_PROVIDERS = new Set(['gmail', 'outlook']); +const EMAIL_HEADER_ARGS = ['subject', 'to', 'cc', 'bcc']; +const RECIPIENT_ARGS = ['to', 'cc', 'bcc']; + +// Drive file/folder IDs are URL-safe base64-ish; folderId is also quoted into a +// Drive search query (`'' in parents`), so a quote must never get through. +const DRIVE_ID = /^[A-Za-z0-9_-]+$/; + +const CONTROL_OR_SPACE = /[\s\x00-\x1f\x7f]/; +const UNSAFE_PATH_CHARS = /[\/\\?#%]/; +const ADDR = '[^\\s<>@,;"]+@[^\\s<>@,;"]+'; +// `addr` or `Display Name `; the display name can't hold separators or quotes. +const SINGLE_RECIPIENT = new RegExp(`^\\s*(?:${ADDR}|[^<>,;"\\r\\n]*<\\s*${ADDR}\\s*>)\\s*$`); + +export function isSafеPathSegment(value: string): boolean { + return value.length > 0 && value !== '.' && value !== '..' && + !UNSAFE_PATH_CHARS.test(value) && !CONTROL_OR_SPACE.test(value); +} + +/** + * A relative file path: no `.`/`..` segments (traversal) and no control + * characters. Anything else is a legitimate filename character; the client + * URL-encodes each segment (github.ts encodeRepoPath). + */ +function isSafeRelativePath(value: string): boolean { + if (/[\x00-\x1f\x7f]/.test(value)) return false; + return value.split('/').every((segment) => segment !== '.' && segment !== '..'); +} + +/** Google Chat space: `spaces/` or a bare ``. */ +function chatSpacеName(space: string): string | null { + const id = space.startsWith('spaces/') ? space.slice('spaces/'.length) : space; + return isSafеPathSegment(id) ? `spaces/${id}` : null; +} + +function stringValues(value: unknown): string[] { + if (typeof value === 'string') return [value]; + if (Array.isArray(value)) return value.filter((v): v is string => typeof v === 'string'); + return []; +} + +/** + * An empty path ID means "not given" (LLMs often send `folderId: ""`). Remove it + * so the policy check and the client both see it as omitted; left in place, a + * client could still interpolate '' and reach a different endpoint (e.g. + * `/files/` instead of `/files/`). Call before validateGatewayArgs. + */ +export function dropEmptyPathArgs(provider: string, args: Record): void { + for (const field of PATH_SEGMENT_ARGS[provider as IntegrationProvider] ?? []) { + if (args[field] === '') delete args[field]; + } +} + +/** Returns a denial reason, or null if the args are safe to enforce and execute. */ +export function validateGatewayArgs(provider: string, args: Record): string | null { + // Plus the messaging target (Gmail's `to` is a recipient list, WhatsApp's is a number). + const targetField = MESSAGING_TARGET_FIELD[provider]; + for (const field of targetField ? [...POLICY_SCALAR_ARGS, targetField] : POLICY_SCALAR_ARGS) { + const value = args[field]; + if (value !== undefined && value !== null && typeof value !== 'string') { + return `E79622: Invalid ${field}: must be a string`; + } + } + + for (const field of PATH_SEGMENT_ARGS[provider as IntegrationProvider] ?? []) { + const value = args[field]; + // '' is treated as omitted; the gateway removes it first (dropEmptyPathArgs). + if (value === undefined || value === null || value === '') continue; + if (typeof value !== 'string' || !isSafеPathSegment(value)) { + return `E79610: Invalid ${field}: must be a single ID without '/', '..', '?', '#', '%' or spaces`; + } + } + + if (provider === 'google_drive') { + for (const field of ['fileId', 'folderId']) { + const value = args[field]; + if (value === undefined || value === null || value === '') continue; + if (typeof value !== 'string' || !DRIVE_ID.test(value)) { + return `E79621: Invalid ${field}: must be a Drive ID (letters, digits, '-' and '_')`; + } + } + } + + if (provider === 'github' && args.path !== undefined) { + if (typeof args.path !== 'string' || !isSafeRelativePath(args.path.replace(/^\/+/, ''))) { + return "E79611: Invalid path: '.' and '..' segments are not allowed"; + } + } + + if (provider === 'google_chat') { + let spaceName: string | null = null; + if (args.space !== undefined) { + spaceName = typeof args.space === 'string' ? chatSpacеName(args.space) : null; + if (!spaceName) return 'E79612: Invalid space'; + } + if (args.message_id !== undefined) { + const id = args.message_id; + if (typeof id !== 'string') return 'E79612: Invalid message_id'; + if (id.startsWith('spaces/')) { + // A full resource name replaces the space in the client, so it must be in the checked space. + const parts = id.split('/'); + if (!spaceName || parts.length !== 4 || `${parts[0]}/${parts[1]}` !== spaceName || + parts[2] !== 'messages' || !isSafеPathSegment(parts[3])) { + return 'E79612: message_id must belong to the given space'; + } + } else if (!isSafеPathSegment(id)) { + return 'E79612: Invalid message_id'; + } + } + } + + if (EMAIL_PROVIDERS.has(provider)) { + for (const field of EMAIL_HEADER_ARGS) { + if (stringValues(args[field]).some((v) => /[\r\n]/.test(v))) { + return `E79613: Invalid ${field}: line breaks are not allowed`; + } + } + for (const field of RECIPIENT_ARGS) { + const value = args[field]; + if (value === undefined || value === null) continue; + const entries = stringValues(value); + if (entries.length !== (Array.isArray(value) ? value.length : 1)) { + return `E79614: Invalid ${field}: recipients must be strings`; + } + if (entries.some((e) => !SINGLE_RECIPIENT.test(e))) { + return `E79614: Invalid ${field}: give one address per entry`; + } + } + } + + return null; +} diff --git a/controlplane/src/integration-policies/discord-guild.test.ts b/controlplane/src/integration-policies/discord-guild.test.ts new file mode 100644 index 00000000..927b5f5b --- /dev/null +++ b/controlplane/src/integration-policies/discord-guild.test.ts @@ -0,0 +1,175 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: discord-guild-tests-v1-guild-pinning +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { handleGatewayExecute } from './gateway'; +import { createPtyToken } from '../auth/pty-token'; +import type { Env } from '../types'; + +// Every Discord integration uses the shared platform bot, which sits in every +// customer's guild: the gateway must pin each action to the stored guild. +const SECRET = 'pty-secret'; +const OWN_GUILD = '111111111111111111'; +const OTHER_GUILD = '999999999999999999'; +const OWN_CHANNEL = '222222222222222222'; +const OTHER_CHANNEL = '888888888888888888'; +const DM_CHANNEL = '777777777777777777'; + +const POLICY = { + canReceive: true, canSend: true, canReadHistory: true, canReact: true, + canEditMessages: true, canDeleteMessages: true, +}; + +function fakeEnv(metadata: Record | null) { + const audits: unknown[][] = []; + const stmt = (sql: string, args: unknown[] = []) => ({ + bind: (...a: unknown[]) => stmt(sql, a), + async first() { + if (sql.includes('FROM terminal_integrations ti')) { + return { + id: 'ti-1', terminal_id: 'term-1', item_id: null, dashboard_id: 'dash-1', user_id: 'alice', + provider: 'discord', user_integration_id: 'ui-1', active_policy_id: 'pol-1', + policy: JSON.stringify(POLICY), security_level: 'restricted', + policy_id: 'pol-1', policy_version: 1, + }; + } + if (sql.includes('high_risk_confirmations')) return { id: 'confirmed' }; + if (sql.includes('FROM user_integrations WHERE id')) { + return { access_token: 'platform-bot-token', refresh_token: null, expires_at: null, metadata: metadata ? JSON.stringify(metadata) : null }; + } + return null; + }, + async all() { return { results: [] }; }, + async run() { + if (sql.includes('integration_audit_log')) audits.push(args); + return { success: true }; + }, + }); + const env = { + DB: { prepare: (sql: string) => stmt(sql), batch: async () => [] }, + PTY_TOKEN_SECRET: SECRET, + INTERNAL_API_TOKEN: 'internal', + } as unknown as Env; + return { env, audits }; +} + +/** Fake Discord API: channel lookups resolve to guilds; everything else is recorded. */ +function stubDiscord(opts: { lookupFails?: boolean } = {}) { + const calls: Array<{ method: string; path: string }> = []; + const upstream = vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + const url = new URL(typeof input === 'string' ? input : input instanceof URL ? input.href : input.url); + const path = url.pathname.replace('/api/v10', ''); + const method = init?.method || 'GET'; + calls.push({ method, path }); + const lookup = path.match(/^\/channels\/(\d+)$/); + if (lookup && method === 'GET') { + if (opts.lookupFails) return new Response('{"message":"Unknown Channel"}', { status: 404 }); + const id = lookup[1]; + const guild = id === OWN_CHANNEL ? OWN_GUILD : id === OTHER_CHANNEL ? OTHER_GUILD : undefined; + return Response.json({ id, type: guild ? 0 : 1, ...(guild ? { guild_id: guild } : {}) }); + } + if (path.startsWith('/guilds/')) return Response.json([]); + return Response.json({ id: 'm1', channel_id: OWN_CHANNEL }); + }); + vi.stubGlobal('fetch', upstream); + return calls; +} + +async function execute(action: string, args: Record, metadata: Record | null = { guild_id: OWN_GUILD }) { + const { env, audits } = fakeEnv(metadata); + const token = await createPtyToken('term-1', 'sbx-1', 'dash-1', 'alice', SECRET); + const req = new Request('https://cp.test/internal/gateway/discord/execute', { + method: 'POST', + headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, + body: JSON.stringify({ action, args }), + }); + const res = await handleGatewayExecute(req, env, 'discord'); + return { res, audits }; +} + +describe('gateway pins Discord to the stored guild', () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('list_channels always uses the stored guild', async () => { + const calls = stubDiscord(); + const { res } = await execute('discord.list_channels', {}); + expect(res.status).toBe(200); + expect(calls.map((c) => c.path)).toEqual([`/guilds/${OWN_GUILD}/channels`]); + }); + + it('denies an LLM-supplied foreign guild_id before any Discord call', async () => { + const calls = stubDiscord(); + const { res, audits } = await execute('discord.list_channels', { guild_id: OTHER_GUILD }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ decision: 'denied', reason: expect.stringMatching(/^E79625/) }); + expect(calls).toEqual([]); + expect(audits.length).toBe(1); + }); + + it('denies when the integration has no stored guild', async () => { + const calls = stubDiscord(); + const { res } = await execute('discord.list_channels', {}, { guild_id: null }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ reason: expect.stringMatching(/^E79624/) }); + expect(calls).toEqual([]); + }); + + const channelActions: Array<[string, Record]> = [ + ['discord.read_messages', {}], + ['discord.send_message', { text: 'hi' }], + ['discord.reply_thread', { message_id: '5', text: 'hi' }], + ['discord.react', { message_id: '5', emoji: 'x' }], + ['discord.edit_message', { message_id: '5', text: 'hi' }], + ['discord.delete_message', { message_id: '5' }], + ]; + + it.each(channelActions)('%s in another guild is denied after only the lookup', async (action, extra) => { + const calls = stubDiscord(); + const { res } = await execute(action, { channel: OTHER_CHANNEL, ...extra }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ reason: expect.stringMatching(/^E79628/) }); + expect(calls).toEqual([{ method: 'GET', path: `/channels/${OTHER_CHANNEL}` }]); + }); + + it.each(channelActions)('%s in a DM channel is denied', async (action, extra) => { + const calls = stubDiscord(); + const { res } = await execute(action, { channel: DM_CHANNEL, ...extra }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ reason: expect.stringMatching(/^E79628/) }); + expect(calls.length).toBe(1); + }); + + it.each(channelActions)('%s fails closed when the channel lookup fails', async (action, extra) => { + const calls = stubDiscord({ lookupFails: true }); + const { res } = await execute(action, { channel: OWN_CHANNEL, ...extra }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ reason: expect.stringMatching(/^E79627/) }); + expect(calls.length).toBe(1); + }); + + it.each(channelActions)('%s in the stored guild goes through', async (action, extra) => { + const calls = stubDiscord(); + const { res } = await execute(action, { channel: OWN_CHANNEL, ...extra }); + expect(res.status).toBe(200); + expect(calls[0]).toEqual({ method: 'GET', path: `/channels/${OWN_CHANNEL}` }); + expect(calls.length).toBe(2); + expect(calls[1].path.startsWith(`/channels/${OWN_CHANNEL}/messages`)).toBe(true); + }); + + it('a channel action without a channel is denied', async () => { + const calls = stubDiscord(); + const { res } = await execute('discord.send_message', { text: 'hi' }); + expect(res.status).toBe(403); + expect(calls).toEqual([]); + }); + + it('an empty channel is treated as omitted and denied', async () => { + const calls = stubDiscord(); + const { res } = await execute('discord.send_message', { channel: '', text: 'hi' }); + expect(res.status).toBe(403); + expect(calls).toEqual([]); + }); +}); diff --git a/controlplane/src/integration-policies/gateway-arg-safety.test.ts b/controlplane/src/integration-policies/gateway-arg-safety.test.ts new file mode 100644 index 00000000..10c8abe3 --- /dev/null +++ b/controlplane/src/integration-policies/gateway-arg-safety.test.ts @@ -0,0 +1,124 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: gateway-arg-safety-tests-v3-empty-ids +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { handleGatewayExecute } from './gateway'; +import { createPtyToken } from '../auth/pty-token'; +import type { Env, IntegrationProvider } from '../types'; + +// Arg validation is keyed by provider identifier, so it has to be exercised +// with the identifiers the gateway really receives (google_drive, not drive). +const SECRET = 'pty-secret'; + +function fakeEnv(provider: IntegrationProvider, policy: Record = { canRead: true, canWrite: true }) { + const audits: unknown[][] = []; + const stmt = (sql: string, args: unknown[] = []) => ({ + bind: (...a: unknown[]) => stmt(sql, a), + async first() { + if (sql.includes('FROM terminal_integrations ti')) { + return { + id: 'ti-1', terminal_id: 'term-1', item_id: null, dashboard_id: 'dash-1', user_id: 'alice', + provider, user_integration_id: 'ui-1', active_policy_id: 'pol-1', + policy: JSON.stringify(policy), security_level: 'restricted', + policy_id: 'pol-1', policy_version: 1, + }; + } + if (sql.includes('FROM user_integrations WHERE id')) { + return { access_token: 'tok', refresh_token: null, expires_at: null, metadata: null }; + } + return null; + }, + async all() { return { results: [] }; }, + async run() { + if (sql.includes('integration_audit_log')) audits.push(args); + return { success: true }; + }, + }); + const env = { + DB: { prepare: (sql: string) => stmt(sql), batch: async () => [] }, + PTY_TOKEN_SECRET: SECRET, + INTERNAL_API_TOKEN: 'internal', + } as unknown as Env; + return { env, audits }; +} + +async function execute(provider: IntegrationProvider, action: string, args: Record, policy?: Record) { + const { env } = fakeEnv(provider, policy); + const token = await createPtyToken('term-1', 'sbx-1', 'dash-1', 'alice', SECRET); + const req = new Request(`https://cp.test/internal/gateway/${provider}/execute`, { + method: 'POST', + headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' }, + body: JSON.stringify({ action, args }), + }); + return handleGatewayExecute(req, env, provider); +} + +describe('gateway rejects unsafe IDs for Google providers', () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it.each([ + ['google_drive', 'drive.get', { fileId: 'ID?alt=media#' }], + ['google_drive', 'drive.download', { fileId: 'OTHER/permissions' }], + ['google_drive', 'drive.list', { folderId: "x' in parents or 'y" }], + ['google_calendar', 'calendar.get_event', { calendarId: 'primary', eventId: '../../other/events/1' }], + ] as const)('%s %s %j is denied before any provider call', async (provider, action, args) => { + const upstream = vi.fn(async () => new Response('{}')); + vi.stubGlobal('fetch', upstream); + const res = await execute(provider, action, args); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ decision: 'denied', reason: expect.stringMatching(/^E796/) }); + expect(upstream).not.toHaveBeenCalled(); + }); +}); + +describe('gateway rejects args the policy check would skip', () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + const onlyAllowed = { canRead: true, calendarFilter: { mode: 'allowlist', calendarIds: ['ALLOWED'] } }; + + it.each([ + // An array was skipped by the calendar check (so it counted as the default + // calendar) but sent as /me/calendars/FORBIDDEN. + ['outlook_calendar.list_events', { calendarId: ['FORBIDDEN'] }], + ['outlook_calendar.get_event', { calendar_id: { id: 'FORBIDDEN' }, event_id: 'e1' }], + ['outlook_calendar.get_event', { calendarId: 'ALLOWED', event_id: '../../calendars/FORBIDDEN/events/e1' }], + ] as const)('%s %j is denied before any provider call', async (action, args) => { + const upstream = vi.fn(async () => new Response('{}')); + vi.stubGlobal('fetch', upstream); + const res = await execute('outlook_calendar', action, args, onlyAllowed); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ decision: 'denied', reason: expect.stringMatching(/^E796/) }); + expect(upstream).not.toHaveBeenCalled(); + }); + + it('a non-allowlisted string calendarId is still denied by policy', async () => { + const upstream = vi.fn(async () => new Response('{}')); + vi.stubGlobal('fetch', upstream); + const res = await execute('outlook_calendar', 'outlook_calendar.list_events', { calendarId: 'FORBIDDEN' }, onlyAllowed); + expect(res.status).toBe(403); + expect(upstream).not.toHaveBeenCalled(); + }); +}); + +describe('gateway treats empty path IDs as omitted', () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('drive.list with folderId "" is not denied and sends no empty folder', async () => { + const urls: string[] = []; + vi.stubGlobal('fetch', vi.fn(async (input: RequestInfo | URL) => { + urls.push(typeof input === 'string' ? input : input instanceof URL ? input.href : input.url); + return Response.json({ files: [] }); + })); + const res = await execute('google_drive', 'drive.list', { folderId: '' }); + expect(res.status).toBe(200); + expect(urls.length).toBe(1); + expect(decodeURIComponent(urls[0])).not.toContain("'' in parents"); + }); +}); diff --git a/controlplane/src/integration-policies/gateway.ts b/controlplane/src/integration-policies/gateway.ts index 8a53c2be..26fc7e01 100644 --- a/controlplane/src/integration-policies/gateway.ts +++ b/controlplane/src/integration-policies/gateway.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: gateway-v30-forward-as-send-category -const MODULE_REVISION = 'gateway-v30-forward-as-send-category'; +// REVISION: gateway-v35-error-codes +const MODULE_REVISION = 'gateway-v35-error-codes'; console.log(`[integration-gateway] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); /** @@ -38,8 +38,9 @@ import type { OutlookPolicy, OutlookCalendarPolicy, } from '../types'; -import { verifyPtyToken, type PtyTokenClaims } from '../auth/pty-token'; +import { verifyPtyToken, tokenSigningSecret, type PtyTokenClaims } from '../auth/pty-token'; import { enforcePolicy, type EnforcementResult } from './handler'; +import { validateGatewayArgs, dropEmptyPathArgs, MESSAGING_TARGET_FIELD } from './arg-safety'; import { getAccessToken } from './token-refresh'; import { filterResponse, type FilterResult } from './response-filter'; import { executeGmailAction } from './api-clients/gmail'; @@ -47,7 +48,13 @@ import { executeGitHubAction } from './api-clients/github'; import { executeDriveAction } from './api-clients/drive'; import { executeCalendarAction } from './api-clients/calendar'; import { executeSlackAction } from './api-clients/slack'; -import { executeDiscordAction } from './api-clients/discord'; +import { + executeDiscordAction, + getDiscоrdChannelGuildId, + DISCORD_CHANNEL_ACTIONS, + DISCORD_GUILD_ACTIONS, + DISCORD_UNSCOPED_ACTIONS, +} from './api-clients/discord'; import { executeTelegramAction, type TelegramD1Context } from './api-clients/telegram'; import { executeWhatsAppAction } from './api-clients/whatsapp'; import { executeTeamsAction } from './api-clients/teams'; @@ -185,29 +192,21 @@ export function deriveEnforcementContext(action: string, args: Record typeof v === 'string'); + if (args.blocks !== undefined) messageParts.push(JSON.stringify(args.blocks)); + if (messageParts.length > 0) { + ctx.messageText = messageParts.join('\n'); } // Extract thread ID for requireThreadReply enforcement. @@ -410,7 +409,7 @@ async function checkRateLimit( if (limit === 0) { return { allowed: false, - reason: `Rate limit is 0 for ${category} (all ${category} blocked)`, + reason: `E79260: Rate limit is 0 for ${category} (all ${category} blocked)`, }; } @@ -428,20 +427,20 @@ async function checkRateLimit( if (!res.ok) { // Fail closed on error - deny requests when rate limiter is unavailable console.error(`[gateway] Rate limit check failed (failing closed): ${res.status}`); - return { allowed: false, reason: 'Rate limiter unavailable - request denied for safety' }; + return { allowed: false, reason: 'E79261: Rate limiter unavailable - request denied for safety' }; } const result = await res.json() as { allowed: boolean; remaining?: number }; if (!result.allowed) { return { allowed: false, - reason: `Rate limit exceeded for ${category} (${limit}/${window})`, + reason: `E79262: Rate limit exceeded for ${category} (${limit}/${window})`, }; } } catch (err) { // Fail closed on error - deny requests when rate limiter is unavailable console.error(`[gateway] Rate limit check error (failing closed):`, err); - return { allowed: false, reason: 'Rate limiter unavailable - request denied for safety' }; + return { allowed: false, reason: 'E79263: Rate limiter unavailable - request denied for safety' }; } return { allowed: true }; @@ -577,17 +576,17 @@ export async function handleGatewayExecute( const authHeader = request.headers.get('Authorization'); if (!authHeader?.startsWith('Bearer ')) { return Response.json( - { error: 'AUTH_DENIED', reason: 'Missing Authorization header' }, + { error: 'AUTH_DENIED', reason: 'E79264: Missing Authorization header' }, { status: 401 } ); } const ptyToken = authHeader.slice(7); - const claims = await verifyPtyToken(ptyToken, env.INTERNAL_API_TOKEN); + const claims = await verifyPtyToken(ptyToken, tokenSigningSecret(env)); if (!claims) { return Response.json( - { error: 'AUTH_DENIED', reason: 'Invalid or expired PTY token' }, + { error: 'AUTH_DENIED', reason: 'E79265: Invalid or expired PTY token' }, { status: 401 } ); } @@ -600,17 +599,20 @@ export async function handleGatewayExecute( body = await request.json() as GatewayExecuteRequest; } catch { return Response.json( - { error: 'INVALID_REQUEST', reason: 'Invalid JSON body' }, + { error: 'INVALID_REQUEST', reason: 'E79266: Invalid JSON body' }, { status: 400 } ); } if (!body.action) { return Response.json( - { error: 'INVALID_REQUEST', reason: 'Missing action' }, + { error: 'INVALID_REQUEST', reason: 'E79267: Missing action' }, { status: 400 } ); } + if (!body.args || typeof body.args !== 'object' || Array.isArray(body.args)) { + body.args = {}; + } // 3. Load terminal_integration and active policy // Defense-in-depth: JOIN also verifies the policy belongs to this terminal_integration @@ -636,7 +638,7 @@ export async function handleGatewayExecute( if (!ti) { return Response.json( - { error: 'NOT_ATTACHED', reason: `${provider} not attached to this terminal` }, + { error: 'NOT_ATTACHED', reason: `E79268: ${provider} not attached to this terminal` }, { status: 403 } ); } @@ -644,7 +646,7 @@ export async function handleGatewayExecute( // 4. Defense-in-depth: Verify terminal_integration matches token context if (ti.dashboard_id !== dashboardId || ti.user_id !== userId) { return Response.json( - { error: 'AUTH_DENIED', reason: 'Dashboard mismatch' }, + { error: 'AUTH_DENIED', reason: 'E79269: Dashboard mismatch' }, { status: 403 } ); } @@ -652,7 +654,7 @@ export async function handleGatewayExecute( // 5. Verify policy exists if (!ti.active_policy_id || !ti.policy) { return Response.json( - { error: 'POLICY_DENIED', reason: 'No policy configured' }, + { error: 'POLICY_DENIED', reason: 'E79270: No policy configured' }, { status: 403 } ); } @@ -689,6 +691,29 @@ export async function handleGatewayExecute( ); } + // 6b. Reject args that would mean one thing to enforcePolicy and another to the + // provider (path traversal in IDs, header injection, multi-address recipients). + dropEmptyPathArgs(provider, body.args); + const argError = validateGatewayArgs(provider, body.args); + if (argError) { + await logAuditEntry(env, { + terminalIntegrationId: ti.id, + terminalId, + dashboardId, + userId, + provider, + action: body.action, + policyId, + policyVersion, + decision: 'denied', + denialReason: argError, + }); + return Response.json( + { allowed: false, decision: 'denied' as const, reason: argError, policyId, policyVersion }, + { status: 403 } + ); + } + // 7. Derive enforcement context server-side from args (NEVER trust body.context) const derivedContext = deriveEnforcementContext(body.action, body.args); @@ -705,6 +730,12 @@ export async function handleGatewayExecute( `https://graph.microsoft.com/v1.0/me/messages/${encodeURIComponent(replyMsgId)}?$select=from`, { headers: { Authorization: `Bearer ${token}` } }, ); + // The reply goes to the original sender, whatever `to` the LLM passed. + // Fail closed: with no sender to check, enforce against no recipient. + derivedContext.recipient = undefined; + derivedContext.recipients = undefined; + derivedContext.recipientDomain = undefined; + derivedContext.recipientDomains = undefined; if (msgResp.ok) { const msg = await msgResp.json() as { from?: { emailAddress?: { address?: string } } }; const senderAddr = msg.from?.emailAddress?.address; @@ -858,14 +889,14 @@ export async function handleGatewayExecute( if (!bridgeSub) { return Response.json( - { error: 'AUTH_DENIED', reason: 'No active bridge WhatsApp connection found' }, + { error: 'AUTH_DENIED', reason: 'E79271: No active bridge WhatsApp connection found' }, { status: 403 } ); } if (!env.BRIDGE_URL || !env.BRIDGE_INTERNAL_TOKEN) { return Response.json( - { error: 'API_ERROR', reason: 'Bridge service not configured' }, + { error: 'API_ERROR', reason: 'E79272: Bridge service not configured' }, { status: 503 } ); } @@ -886,7 +917,7 @@ export async function handleGatewayExecute( } default: return Response.json( - { error: 'API_ERROR', reason: `Action ${body.action} is not supported for bridge WhatsApp connections` }, + { error: 'API_ERROR', reason: `E79273: Action ${body.action} is not supported for bridge WhatsApp connections` }, { status: 400 } ); } @@ -908,7 +939,7 @@ export async function handleGatewayExecute( }); return Response.json( - { error: 'API_ERROR', reason: errorMessage }, + { error: 'API_ERROR', reason: `E79284: ${errorMessage}` }, { status: 502 } ); } @@ -939,9 +970,8 @@ export async function handleGatewayExecute( // WhatsApp can use platform-level credentials when no per-user integration exists if (provider === 'whatsapp' && env.WHATSAPP_ACCESS_TOKEN && env.WHATSAPP_PHONE_NUMBER_ID) { prefetchedAccessToken = env.WHATSAPP_ACCESS_TOKEN; - if (!body.args.phone_number_id) { - body.args.phone_number_id = env.WHATSAPP_PHONE_NUMBER_ID; - } + // Always the platform number: the platform token can send from others. + body.args.phone_number_id = env.WHATSAPP_PHONE_NUMBER_ID; // Auto-fill recipient from hybrid subscription's linked phone when agent doesn't specify one if (!body.args.to) { const hybridSub = await env.DB.prepare(` @@ -957,7 +987,7 @@ export async function handleGatewayExecute( } } else { return Response.json( - { error: 'AUTH_DENIED', reason: 'OAuth connection not found' }, + { error: 'AUTH_DENIED', reason: 'E79274: OAuth connection not found' }, { status: 403 } ); } @@ -966,7 +996,7 @@ export async function handleGatewayExecute( const accessToken = prefetchedAccessToken ?? await getAccessToken(env, ti.user_integration_id!, provider); if (!accessToken) { return Response.json( - { error: 'AUTH_DENIED', reason: 'OAuth token expired. Reconnect required.' }, + { error: 'AUTH_DENIED', reason: 'E79275: OAuth token expired. Reconnect required.' }, { status: 403 } ); } @@ -1009,14 +1039,18 @@ export async function handleGatewayExecute( return Response.json({ allowed: false, decision: 'denied' as const, - reason: `Action denied: unable to verify file against policy`, + reason: `E79276: Action denied: unable to verify file against policy`, policyId, policyVersion, }, { status: 403 }); } + // Fail closed unless this is the file's own metadata: the single-file filter + // only applies to objects with an `id`, so anything else would pass unchecked. + const isFileMetadata = !!metadata && typeof metadata === 'object' && + (metadata as { id?: unknown }).id === fileId; const metadataFilter = filterResponse('google_drive', 'drive.get', metadata, policy); - if (metadataFilter.filtered && metadataFilter.data === null) { + if (!isFileMetadata || (metadataFilter.filtered && metadataFilter.data === null)) { await logAuditEntry(env, { terminalIntegrationId: ti.id, terminalId, @@ -1034,7 +1068,7 @@ export async function handleGatewayExecute( return Response.json({ allowed: false, decision: 'denied' as const, - reason: 'File filtered by policy (folder/filetype restriction)', + reason: 'E79277: File filtered by policy (folder/filetype restriction)', policyId, policyVersion, }, { status: 403 }); @@ -1042,31 +1076,84 @@ export async function handleGatewayExecute( } } - // 9c. Discord: inject guild_id from stored metadata so LLM doesn't need to know it + // 9c. Discord: every integration uses the shared platform bot, which is in every + // customer's guild, so the guild comes from the stored integration and every + // channel must belong to it. Anything unverifiable is denied (fail closed). if (provider === 'discord') { + const denyDiscord = async (reason: string, resourceId?: string) => { + await logAuditEntry(env, { + terminalIntegrationId: ti.id, + terminalId, + dashboardId, + userId, + provider, + action: body.action, + resourceId, + policyId, + policyVersion, + decision: 'denied', + denialReason: reason, + }); + return Response.json( + { allowed: false, decision: 'denied' as const, reason, policyId, policyVersion }, + { status: 403 } + ); + }; + + let storedGuildId: string | null = null; const discordMeta = await env.DB.prepare(` SELECT metadata FROM user_integrations WHERE id = ? `).bind(ti.user_integration_id).first<{ metadata: string | null }>(); if (discordMeta?.metadata) { try { - const meta = JSON.parse(discordMeta.metadata) as { guild_id?: string }; - if (meta.guild_id && !body.args.guild_id) { - body.args.guild_id = meta.guild_id; - } - } catch { /* ignore parse errors */ } + const meta = JSON.parse(discordMeta.metadata) as { guild_id?: unknown }; + if (typeof meta.guild_id === 'string' && meta.guild_id) storedGuildId = meta.guild_id; + } catch { /* treated as no guild */ } + } + if (!storedGuildId) { + return denyDiscord('E79624: Discord integration has no verified server. Reconnect Discord.'); + } + const llmGuildId = body.args.guild_id; + if (llmGuildId !== undefined && llmGuildId !== null && llmGuildId !== '' && llmGuildId !== storedGuildId) { + return denyDiscord('E79625: guild_id must be the connected Discord server'); + } + body.args.guild_id = storedGuildId; + + if (DISCORD_CHANNEL_ACTIONS.has(body.action)) { + const channel = body.args.channel; + if (typeof channel !== 'string' || !channel) { + return denyDiscord('E79626: channel is required'); + } + let channelGuildId: string | null; + try { + channelGuildId = await getDiscоrdChannelGuildId(channel, accessToken); + } catch (err) { + const msg = err instanceof Error ? err.message : 'Unknown error'; + console.warn(`[gateway] Discord channel lookup failed for ${channel}: ${msg}`); + return denyDiscord('E79627: Unable to verify the Discord channel belongs to the connected server', channel); + } + if (channelGuildId !== storedGuildId) { + return denyDiscord('E79628: Discord channel is not in the connected server', channel); + } + } else if (!DISCORD_GUILD_ACTIONS.has(body.action) && !DISCORD_UNSCOPED_ACTIONS.has(body.action)) { + return denyDiscord(`E79629: Unsupported Discord action: ${body.action}`); } } - // 9d. WhatsApp Cloud API: inject phone_number_id from stored metadata so LLM doesn't need to know it - if (provider === 'whatsapp' && ti.user_integration_id && !body.args.phone_number_id) { - const waMeta = await env.DB.prepare(` + // 9d. WhatsApp Cloud API / Matrix: routing comes from the stored integration, + // never from the LLM. An LLM-supplied phone_number_id could send from another + // number on the account; an LLM-supplied homeserver would receive the token. + if ((provider === 'whatsapp' || provider === 'matrix') && ti.user_integration_id) { + const field = provider === 'whatsapp' ? 'phone_number_id' : 'homeserver'; + delete body.args[field]; + const storedMeta = await env.DB.prepare(` SELECT metadata FROM user_integrations WHERE id = ? `).bind(ti.user_integration_id).first<{ metadata: string | null }>(); - if (waMeta?.metadata) { + if (storedMeta?.metadata) { try { - const meta = JSON.parse(waMeta.metadata) as { phone_number_id?: string }; - if (meta.phone_number_id) { - body.args.phone_number_id = meta.phone_number_id; + const meta = JSON.parse(storedMeta.metadata) as Record; + if (typeof meta[field] === 'string' && meta[field]) { + body.args[field] = meta[field]; } } catch { /* ignore parse errors */ } } @@ -1148,7 +1235,7 @@ export async function handleGatewayExecute( // Pass through provider API error messages (e.g. "Slack API error: not_in_channel") // These are public error codes that help the LLM and user diagnose the issue. return Response.json( - { error: 'API_ERROR', reason: errorMessage }, + { error: 'API_ERROR', reason: `E79285: ${errorMessage}` }, { status: 502 } ); } @@ -1213,18 +1300,18 @@ export async function handleListTerminalIntegrations( if (!authHeader?.startsWith('Bearer ')) { console.log(`[gateway] ListTerminalIntegrations: missing Authorization header`); return Response.json( - { error: 'AUTH_DENIED', reason: 'Missing Authorization header' }, + { error: 'AUTH_DENIED', reason: 'E79278: Missing Authorization header' }, { status: 401 } ); } const ptyToken = authHeader.slice(7); - const claims = await verifyPtyToken(ptyToken, env.INTERNAL_API_TOKEN); + const claims = await verifyPtyToken(ptyToken, tokenSigningSecret(env)); if (!claims) { console.log(`[gateway] ListTerminalIntegrations: invalid/expired PTY token`); return Response.json( - { error: 'AUTH_DENIED', reason: 'Invalid or expired PTY token' }, + { error: 'AUTH_DENIED', reason: 'E79279: Invalid or expired PTY token' }, { status: 401 } ); } @@ -1233,7 +1320,7 @@ export async function handleListTerminalIntegrations( if (claims.terminal_id !== ptyId) { console.log(`[gateway] ListTerminalIntegrations: PTY ID mismatch token=${claims.terminal_id} url=${ptyId}`); return Response.json( - { error: 'AUTH_DENIED', reason: 'PTY ID mismatch' }, + { error: 'AUTH_DENIED', reason: 'E79280: PTY ID mismatch' }, { status: 403 } ); } @@ -1264,7 +1351,7 @@ export async function handleListTerminalIntegrations( * * GET /internal/dashboards/:dashboardId/terminal-integrations * - * Auth: X-Internal-Token (sandbox internal auth) + * Auth: X-Internal-Token (full trust) or X-Dashboard-Token scoped to :dashboardId * * Response: * { @@ -1280,14 +1367,10 @@ export async function handleBatchListTerminalIntegrations( env: Env, dashboardId: string ): Promise { - // Auth: require internal token - const internalToken = request.headers.get('X-Internal-Token'); - if (!internalToken || internalToken !== env.INTERNAL_API_TOKEN) { - return Response.json( - { error: 'AUTH_DENIED', reason: 'Invalid or missing internal token' }, - { status: 401 } - ); - } + // Auth: internal token (full trust) or a dashboard token scoped to this dashboard + const { authorizeSandboxForDashboard } = await import('../auth/sandbox-auth'); + const authError = await authorizeSandboxForDashboard(request, env, dashboardId); + if (authError) return authError; // Single query for all terminals in this dashboard const integrations = await env.DB.prepare(` diff --git a/controlplane/src/integration-policies/github-path.test.ts b/controlplane/src/integration-policies/github-path.test.ts new file mode 100644 index 00000000..46ca6709 --- /dev/null +++ b/controlplane/src/integration-policies/github-path.test.ts @@ -0,0 +1,20 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { executeGitHubAction } from './api-clients/github'; + +describe('GitHub client encodes each path segment', () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('keeps spaces, # and ? inside the filename', async () => { + const fetchMock = vi.fn<[RequestInfo | URL], Promise>(async () => new Response('{}')); + vi.stubGlobal('fetch', fetchMock); + await executeGitHubAction('github.get_file', { owner: 'o', repo: 'r', path: 'docs/Getting Started #1?.md', ref: 'main' }, 'tok'); + const url = new URL(String(fetchMock.mock.calls[0][0])); + expect(url.pathname).toBe('/repos/o/r/contents/docs/Getting%20Started%20%231%3F.md'); + expect(url.searchParams.get('ref')).toBe('main'); + }); +}); diff --git a/controlplane/src/integration-policies/handler.ts b/controlplane/src/integration-policies/handler.ts index e7744824..a972a874 100644 --- a/controlplane/src/integration-policies/handler.ts +++ b/controlplane/src/integration-policies/handler.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: handler-v16-attach-orphan-cleanup -console.log(`[integration-handler] REVISION: handler-v16-attach-orphan-cleanup loaded at ${new Date().toISOString()}`); +// REVISION: handler-v19-error-codes +console.log(`[integration-handler] REVISION: handler-v19-error-codes loaded at ${new Date().toISOString()}`); import type { Env, @@ -29,7 +29,6 @@ import type { } from '../types'; import { HIGH_RISK_CAPABILITIES } from '../types'; import { verifyPtyToken, type PtyTokenClaims } from '../auth/pty-token'; -import { getAccessToken } from './token-refresh'; // ============================================ // Helper Functions @@ -1404,7 +1403,10 @@ export async function enforcePolicy( 'canSend', 'canReadHistory', 'canReact', 'canEditMessages', 'canDeleteMessages', ]); - if (msgPolicy.channelFilter && CHANNEL_TARGETED_CAPABILITIES.has(capability)) { + // Read actions that target one specific chat despite a non-targeted capability. + const CHANNEL_TARGETED_ACTIONS = new Set(['telegram.get_chat_info']); + + if (msgPolicy.channelFilter && (CHANNEL_TARGETED_CAPABILITIES.has(capability) || CHANNEL_TARGETED_ACTIONS.has(action))) { // mode: 'all' = no channel restriction — skip filter entirely if (msgPolicy.channelFilter.mode === 'allowlist') { const { channelIds, channelNames } = msgPolicy.channelFilter; @@ -1805,7 +1807,7 @@ export async function attachIntegration( const browserPolicy = providedPolicy as BrowserPolicy | undefined; if (!browserPolicy?.urlFilter?.patterns?.length) { return Response.json( - { error: 'Browser integration requires at least one URL pattern' }, + { error: 'E79178: Browser integration requires at least one URL pattern' }, { status: 400 } ); } @@ -1825,7 +1827,7 @@ export async function attachIntegration( if (!bridgeSub) { return Response.json( - { error: 'No WhatsApp connection found. Configure platform credentials or connect via QR code first.' }, + { error: 'E79179: No WhatsApp connection found. Configure platform credentials or connect via QR code first.' }, { status: 400 } ); } @@ -1834,7 +1836,7 @@ export async function attachIntegration( // Non-browser providers require userIntegrationId if (!userIntegrationId) { return Response.json( - { error: 'userIntegrationId is required for non-browser providers' }, + { error: 'E79180: userIntegrationId is required for non-browser providers' }, { status: 400 } ); } @@ -1848,7 +1850,7 @@ export async function attachIntegration( if (!userInt || userInt.user_id !== userId) { return Response.json( - { error: 'OAuth connection does not belong to user' }, + { error: 'E79181: OAuth connection does not belong to user' }, { status: 403 } ); } @@ -1877,7 +1879,7 @@ export async function attachIntegration( await env.DB.prepare(`DELETE FROM terminal_integrations WHERE id = ?`).bind(existing.id).run(); } else { return Response.json( - { error: `${provider} is already attached to this terminal` }, + { error: `E79182: ${provider} is already attached to this terminal` }, { status: 409 } ); } @@ -1998,26 +2000,36 @@ export async function updateIntegrationPolicy( // Find existing terminal integration const existing = await env.DB.prepare(` - SELECT id, active_policy_id + SELECT id, active_policy_id, user_id FROM terminal_integrations WHERE terminal_id = ? AND dashboard_id = ? AND provider = ? AND deleted_at IS NULL `) .bind(terminalId, dashboardId, provider) - .first<{ id: string; active_policy_id: string | null }>(); + .first<{ id: string; active_policy_id: string | null; user_id: string }>(); if (!existing) { return Response.json( - { error: `${provider} is not attached to this terminal` }, + { error: `E79183: ${provider} is not attached to this terminal` }, { status: 404 } ); } + // The policy guards the account of whoever attached the integration (the + // gateway uses their OAuth token), so only they may change it — not another + // editor, and not the dashboard owner. + if (existing.user_id !== userId) { + return Response.json( + { error: 'E79615: Only the member who attached this integration can change its policy' }, + { status: 403 } + ); + } + // Browser requires URL patterns if (provider === 'browser') { const browserPolicy = policy as BrowserPolicy; if (!browserPolicy?.urlFilter?.patterns?.length) { return Response.json( - { error: 'Browser policy requires at least one URL pattern' }, + { error: 'E79184: Browser policy requires at least one URL pattern' }, { status: 400 } ); } @@ -2100,19 +2112,27 @@ export async function detachIntegration( // Find existing terminal integration const existing = await env.DB.prepare(` - SELECT id FROM terminal_integrations + SELECT id, user_id FROM terminal_integrations WHERE terminal_id = ? AND dashboard_id = ? AND provider = ? AND deleted_at IS NULL `) .bind(terminalId, dashboardId, provider) - .first<{ id: string }>(); + .first<{ id: string; user_id: string }>(); if (!existing) { return Response.json( - { error: `${provider} is not attached to this terminal` }, + { error: `E79185: ${provider} is not attached to this terminal` }, { status: 404 } ); } + // Detaching only removes access, so the dashboard owner may also do it. + if (existing.user_id !== userId && access.role !== 'owner') { + return Response.json( + { error: 'E79616: Only the member who attached this integration, or the dashboard owner, can detach it' }, + { status: 403 } + ); + } + // Soft delete (preserve audit history) await env.DB.prepare(` UPDATE terminal_integrations @@ -2161,7 +2181,7 @@ export async function getPolicyHistory( if (!ti) { return Response.json( - { error: `${provider} has never been attached to this terminal` }, + { error: `E79186: ${provider} has never been attached to this terminal` }, { status: 404 } ); } @@ -2279,278 +2299,6 @@ export async function getDashboardAuditLog( // Internal Gateway Handlers (Sandbox → Control Plane) // ============================================ -/** - * Validate a gateway request and return the active policy - * Called by sandbox MCP server before executing integration tools - */ -export async function validateGatewayRequest( - env: Env, - terminalId: string, - provider: IntegrationProvider, - dashboardId: string, - userId: string -): Promise { - // Look up terminal_integration by (terminal_id, provider) - const ti = await env.DB.prepare(` - SELECT id, active_policy_id, user_integration_id, dashboard_id, user_id - FROM terminal_integrations - WHERE terminal_id = ? AND provider = ? AND deleted_at IS NULL - `) - .bind(terminalId, provider) - .first<{ - id: string; - active_policy_id: string | null; - user_integration_id: string | null; - dashboard_id: string; - user_id: string; - }>(); - - if (!ti) { - return Response.json( - { error: 'NOT_ATTACHED', reason: `${provider} not attached to terminal` }, - { status: 404 } - ); - } - - // Defense-in-depth: Verify terminal_integration belongs to same dashboard/user - if (ti.dashboard_id !== dashboardId || ti.user_id !== userId) { - return Response.json( - { error: 'AUTH_DENIED', reason: 'Terminal integration does not match token context' }, - { status: 403 } - ); - } - - // Load active policy - if (!ti.active_policy_id) { - return Response.json( - { error: 'POLICY_DENIED', reason: 'No policy configured for this integration' }, - { status: 403 } - ); - } - - const policy = await env.DB.prepare( - `SELECT * FROM integration_policies WHERE id = ?` - ) - .bind(ti.active_policy_id) - .first>(); - - if (!policy) { - return Response.json( - { error: 'POLICY_DENIED', reason: 'Policy configuration error' }, - { status: 500 } - ); - } - - // Defense-in-depth: Verify policy belongs to this terminal_integration - if (policy.terminal_integration_id !== ti.id) { - return Response.json( - { error: 'POLICY_DENIED', reason: 'Policy configuration error' }, - { status: 500 } - ); - } - - // Get OAuth token if needed (non-browser providers), with automatic refresh - let accessToken: string | null = null; - if (provider !== 'browser' && ti.user_integration_id) { - accessToken = await getAccessToken(env, ti.user_integration_id, provider); - if (!accessToken) { - return Response.json( - { error: 'AUTH_DENIED', reason: 'OAuth token expired. Reconnect required.' }, - { status: 403 } - ); - } - } - - return Response.json({ - terminalIntegrationId: ti.id, - policyId: policy.id, - policyVersion: policy.version, - policy: JSON.parse(policy.policy as string), - securityLevel: policy.security_level, - accessToken, - }); -} - -/** - * Validate a gateway request using a PTY token - * This is the secure entry point - terminal_id is extracted from the verified token - * - * Security: The terminal_id comes FROM the cryptographically verified token, - * not from an untrusted header. This prevents a compromised sandbox from - * impersonating other terminals. - * - * @param action - The action being performed (e.g., "gmail_search", "calendar_create_event") - * Used for rate limiting based on action category - */ -export async function validateGatewayWithToken( - env: Env, - ptyToken: string, - provider: IntegrationProvider, - action?: string, - context?: { - url?: string; - recipient?: string; - recipientDomain?: string; - sender?: string; - senderDomain?: string; - resourceId?: string; - } -): Promise { - // 1. Verify PTY token signature and extract claims - const claims = await verifyPtyToken(ptyToken, env.INTERNAL_API_TOKEN); - - if (!claims) { - return Response.json( - { error: 'AUTH_DENIED', reason: 'Invalid or expired PTY token' }, - { status: 401 } - ); - } - - // 2. Extract terminal_id from verified token (not from untrusted source) - const terminalId = claims.terminal_id; - const dashboardId = claims.dashboard_id; - const userId = claims.user_id; - - // 3. Look up terminal_integration and policy - const ti = await env.DB.prepare(` - SELECT id, active_policy_id, user_integration_id, dashboard_id, user_id - FROM terminal_integrations - WHERE terminal_id = ? AND provider = ? AND deleted_at IS NULL - `) - .bind(terminalId, provider) - .first<{ - id: string; - active_policy_id: string | null; - user_integration_id: string | null; - dashboard_id: string; - user_id: string; - }>(); - - if (!ti) { - return Response.json( - { error: 'NOT_ATTACHED', reason: `${provider} not attached to terminal` }, - { status: 404 } - ); - } - - // Defense-in-depth: Verify terminal_integration belongs to same dashboard/user - if (ti.dashboard_id !== dashboardId || ti.user_id !== userId) { - return Response.json( - { error: 'AUTH_DENIED', reason: 'Terminal integration does not match token context' }, - { status: 403 } - ); - } - - // Load active policy - if (!ti.active_policy_id) { - return Response.json( - { error: 'POLICY_DENIED', reason: 'No policy configured for this integration' }, - { status: 403 } - ); - } - - const policyRow = await env.DB.prepare( - `SELECT * FROM integration_policies WHERE id = ?` - ) - .bind(ti.active_policy_id) - .first>(); - - if (!policyRow) { - return Response.json( - { error: 'POLICY_DENIED', reason: 'Policy configuration error' }, - { status: 500 } - ); - } - - // Defense-in-depth: Verify policy belongs to this terminal_integration - if (policyRow.terminal_integration_id !== ti.id) { - return Response.json( - { error: 'POLICY_DENIED', reason: 'Policy configuration error' }, - { status: 500 } - ); - } - - const policy = JSON.parse(policyRow.policy as string) as AnyPolicy; - - // 4. Check rate limits if action is provided - if (action) { - const rateLimitResult = await checkRateLimit(env, ti.id, provider, action, policy); - if (!rateLimitResult.allowed) { - // Log rate limit denial - await logAuditEntryInternal(env, { - terminalIntegrationId: ti.id, - terminalId, - dashboardId, - userId, - provider, - action, - resourceId: context?.resourceId, - policyId: policyRow.id as string, - policyVersion: policyRow.version as number, - decision: 'denied', - denialReason: rateLimitResult.reason, - }); - - return Response.json( - { error: 'RATE_LIMITED', reason: rateLimitResult.reason }, - { status: 429 } - ); - } - } - - // 5. Enforce policy for the specific action - if (action) { - const enforcement = await enforcePolicy(env, provider, action, policy, ti.id, context); - - // Log the enforcement decision - await logAuditEntryInternal(env, { - terminalIntegrationId: ti.id, - terminalId, - dashboardId, - userId, - provider, - action, - resourceId: context?.resourceId, - policyId: policyRow.id as string, - policyVersion: policyRow.version as number, - decision: enforcement.decision, - denialReason: enforcement.reason, - }); - - if (!enforcement.allowed) { - return Response.json( - { - error: enforcement.decision === 'filtered' ? 'FILTERED' : 'POLICY_DENIED', - reason: enforcement.reason, - decision: enforcement.decision, - }, - { status: 403 } - ); - } - } - - // 6. Get OAuth token if needed (non-browser providers), with automatic refresh - let accessToken: string | null = null; - if (provider !== 'browser' && ti.user_integration_id) { - accessToken = await getAccessToken(env, ti.user_integration_id, provider); - if (!accessToken) { - return Response.json( - { error: 'AUTH_DENIED', reason: 'OAuth token expired. Reconnect required.' }, - { status: 403 } - ); - } - } - - return Response.json({ - terminalIntegrationId: ti.id, - policyId: policyRow.id, - policyVersion: policyRow.version, - policy, - securityLevel: policyRow.security_level, - accessToken, - }); -} - /** * Internal helper to log an audit entry (used by gateway functions) */ diff --git a/controlplane/src/integration-policies/policy-ownership.test.ts b/controlplane/src/integration-policies/policy-ownership.test.ts new file mode 100644 index 00000000..28acc653 --- /dev/null +++ b/controlplane/src/integration-policies/policy-ownership.test.ts @@ -0,0 +1,67 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +import { describe, it, expect } from 'vitest'; +import { updateIntegrationPolicy, detachIntegration } from './handler'; +import type { Env } from '../types'; + +// The terminal-access helper uses a JOIN the shared D1 mock can't run, so this +// fake answers just the queries these two handlers make before their writes. +function fakeEnv(roles: Record, attachedBy: string) { + const writes: string[] = []; + const stmt = (sql: string, args: unknown[] = []) => ({ + bind: (...a: unknown[]) => stmt(sql, a), + async first() { + if (sql.includes('FROM dashboard_members WHERE dashboard_id')) { + const role = roles[args[1] as string]; + return role ? { role } : null; + } + if (sql.includes('FROM sessions s')) return roles[args[1] as string] ? { dashboard_id: 'dash-1' } : null; + if (sql.includes('FROM terminal_integrations')) return { id: 'ti-1', active_policy_id: 'pol-1', user_id: attachedBy }; + return null; + }, + async run() { + writes.push(sql); + return { success: true }; + }, + async all() { + return { results: [] }; + }, + }); + const env = { DB: { prepare: (sql: string) => stmt(sql), batch: async () => [] } } as unknown as Env; + return { env, writes }; +} + +const roles = { alice: 'editor', bob: 'editor', olivia: 'owner' }; + +describe('integration policy changes are limited to the account holder', () => { + it('another editor cannot rewrite the policy on an integration attached with alice\'s account', async () => { + const { env, writes } = fakeEnv(roles, 'alice'); + const res = await updateIntegrationPolicy(env, 'dash-1', 'pty-1', 'gmail', 'bob', { policy: {} as never }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ error: expect.stringContaining('E79615') }); + expect(writes).toHaveLength(0); + }); + + it('the dashboard owner cannot rewrite it either', async () => { + const { env } = fakeEnv(roles, 'alice'); + const res = await updateIntegrationPolicy(env, 'dash-1', 'pty-1', 'gmail', 'olivia', { policy: {} as never }); + expect(res.status).toBe(403); + }); + + it('another editor cannot detach it', async () => { + const { env, writes } = fakeEnv(roles, 'alice'); + const res = await detachIntegration(env, 'dash-1', 'pty-1', 'gmail', 'bob'); + expect(res.status).toBe(403); + expect(writes).toHaveLength(0); + }); + + it('the attacher and the dashboard owner can detach it', async () => { + for (const user of ['alice', 'olivia']) { + const { env, writes } = fakeEnv(roles, 'alice'); + const res = await detachIntegration(env, 'dash-1', 'pty-1', 'gmail', user); + expect(res.status).toBe(200); + expect(writes.some((w) => w.includes('UPDATE terminal_integrations'))).toBe(true); + } + }); +}); diff --git a/controlplane/src/integrations/discord-guild-binding.test.ts b/controlplane/src/integrations/discord-guild-binding.test.ts new file mode 100644 index 00000000..a79c81d4 --- /dev/null +++ b/controlplane/src/integrations/discord-guild-binding.test.ts @@ -0,0 +1,67 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: discord-guild-binding-tests-v2-identifier-renames +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { resоlveDiscordGuildId, canManagеDiscоrdGuild } from './handler'; + +// The callback's guild_id param is user-editable: binding the shared bot to a +// guild must need the bot scope and the right to manage that guild. +function stubGuilds(guilds: unknown, ok = true) { + const upstream = vi.fn(async () => ok ? Response.json(guilds) : new Response('', { status: 500 })); + vi.stubGlobal('fetch', upstream); + return upstream; +} + +describe('resolveDiscordGuildId', () => { + afterEach(() => { vi.unstubAllGlobals(); }); + + it('uses the token response guild when the bot scope was granted', async () => { + const upstream = stubGuilds([]); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'identify bot guilds', guild: { id: 'G1' } }, 'G2')).toBe('G1'); + expect(upstream).not.toHaveBeenCalled(); + }); + + it('binds nothing without the bot scope, even with a token guild', async () => { + stubGuilds([{ id: 'G1', owner: true }]); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'identify guilds', guild: { id: 'G1' } }, 'G1')).toBeNull(); + expect(await resоlveDiscordGuildId({ access_token: 't', guild: { id: 'G1' } }, 'G1')).toBeNull(); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'robot identify' }, 'G1')).toBeNull(); + }); + + it('rejects a query guild the user is merely a member of', async () => { + stubGuilds([{ id: 'G2', owner: false, permissions: String(1024 + 2048) }]); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'bot identify guilds' }, 'G2')).toBeNull(); + }); + + it.each([ + [{ id: 'G2', owner: true, permissions: '0' }], + [{ id: 'G2', owner: false, permissions: String(0x20) }], + [{ id: 'G2', owner: false, permissions: String(0x8) }], + [{ id: 'G2', owner: false, permissions: '1099511627775' }], + ])('accepts a query guild the user can manage: %j', async (entry) => { + stubGuilds([entry]); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'bot identify guilds' }, 'G2')).toBe('G2'); + }); + + it('fails closed when the guild list cannot be read or lacks the guild', async () => { + stubGuilds([], false); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'bot' }, 'G2')).toBeNull(); + stubGuilds([{ id: 'G3', owner: true }]); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'bot' }, 'G2')).toBeNull(); + stubGuilds({ not: 'an array' }); + expect(await resоlveDiscordGuildId({ access_token: 't', scope: 'bot' }, 'G2')).toBeNull(); + }); +}); + +describe('canManageDiscordGuild', () => { + it.each([ + [{ owner: 'true' }, false], + [{ permissions: 'not-a-number' }, false], + [{ permissions: { x: 1 } }, false], + [{}, false], + [{ permissions: 0x20 }, true], + ])('%j -> %s', (entry, expected) => { + expect(canManagеDiscоrdGuild(entry as never)).toBe(expected); + }); +}); diff --git a/controlplane/src/integrations/handler.ts b/controlplane/src/integrations/handler.ts index c07e67f9..be92585b 100644 --- a/controlplane/src/integrations/handler.ts +++ b/controlplane/src/integrations/handler.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: integrations-v36-outlook-mirror-v1-email-sync -const integrationsRevision = "integrations-v36-outlook-mirror-v1-email-sync"; +// REVISION: integrations-v40-error-codes +const integrationsRevision = "integrations-v40-error-codes"; console.log(`[integrations] REVISION: ${integrationsRevision} loaded at ${new Date().toISOString()}`); @@ -10,6 +10,7 @@ import type { EnvWithDriveCache } from '../storage/drive-cache'; import type { AuthContext } from '../auth/middleware'; import type { IntegrationProvider as GlobalIntegrationProvider } from '../types'; import { requireAuth } from '../auth/middleware'; +import { getUserForSession, parseCookies } from '../auth/sessions'; import { sandboxFetch } from '../sandbox/fetch'; import { getAccessToken } from '../integration-policies/token-refresh'; @@ -379,21 +380,102 @@ function buildState(): string { return crypto.randomUUID(); } +// OAuth state is bound to the browser that started the flow. Without this, an +// attacker could start a flow (state tied to their account) and get a victim to +// finish it, storing the victim's provider token under the attacker's account. +// Connect sets a per-state HttpOnly cookie holding a random binding value whose +// hash is stored with the state; the callback must present it, or else carry a +// session for the same user that started the flow. +const OAUTH_BINDING_COOKIE_PREFIX = 'orcabot_oauth_'; +const OAUTH_BINDING_MAX_AGE_SECONDS = 3600; // matches the state TTL in getState + +async function sha256Hex(value: string): Promise { + const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(value)); + return Array.from(new Uint8Array(digest), (b) => b.toString(16).padStart(2, '0')).join(''); +} + +/** Creates the state row and returns the browser binding value for oauthRedirect. */ async function createState( env: EnvWithDriveCache, userId: string, provider: string, state: string, metadata: Record = {} -) { - console.log(`[oauth-state] create provider=${provider} user=${userId} state=${state}`); +): Promise { + console.log(`[oauth-state] create provider=${provider} user=${userId}`); + const binding = crypto.randomUUID(); + const stored = { ...metadata, binding_hash: await sha256Hex(binding) }; await env.DB.prepare(` INSERT INTO oauth_states (state, user_id, provider, metadata) VALUES (?, ?, ?, ?) - `).bind(state, userId, provider, JSON.stringify(metadata)).run(); + `).bind(state, userId, provider, JSON.stringify(stored)).run(); + return binding; +} + +const LOOPBACK_HOSTNAMES = new Set(['localhost', '127.0.0.1', '[::1]']); + +/** + * The binding cookie is host-only, so connect has to run on the host the + * provider redirects back to. Desktop opens connect on 127.0.0.1 (the baked API + * base) while its callback is OAUTH_REDIRECT_BASE=localhost, so bounce once to + * the callback host with the same path and query (desktop passes its dev-auth + * identity and surface token in the query, so it authenticates there too). + * Only between loopback aliases of the same port: a remote host's session + * cookie wouldn't follow, and an open redirect isn't wanted. + */ +export function cоnnectOnCallbackHost(request: Request, env: EnvWithDriveCache): Response | null { + if (!env.OAUTH_REDIRECT_BASE) return null; // callback uses the request's own origin + let base: URL; + try { + base = new URL(env.OAUTH_REDIRECT_BASE); + } catch { + return null; + } + const current = new URL(request.url); + if (current.host === base.host && current.protocol === base.protocol) return null; + if (!LOOPBACK_HOSTNAMES.has(current.hostname) || !LOOPBACK_HOSTNAMES.has(base.hostname)) return null; + if (current.port !== base.port || current.protocol !== base.protocol) return null; + const target = new URL(current.pathname + current.search, base.origin); + return new Response(null, { status: 302, headers: { Location: target.toString() } }); +} + +/** 302 to the provider, setting the cookie that binds `state` to this browser. */ +function oauthRedirect(request: Request, location: string, state: string, binding: string): Response { + const parts = [ + `${OAUTH_BINDING_COOKIE_PREFIX}${state}=${binding}`, + 'Path=/integrations', + `Max-Age=${OAUTH_BINDING_MAX_AGE_SECONDS}`, + 'HttpOnly', + // Lax is sent on the provider's top-level redirect back to the callback. + 'SameSite=Lax', + ]; + if (new URL(request.url).protocol === 'https:') parts.push('Secure'); + return new Response(null, { status: 302, headers: { Location: location, 'Set-Cookie': parts.join('; ') } }); } -async function getState(env: EnvWithDriveCache, state: string, provider: string) { +/** + * Who is finishing the flow: a browser callback (`request`), or an + * authenticated poll with no redirect, such as the GitHub device flow (`userId`). + */ +type StateCaller = { request: Request } | { userId: string }; + +async function statеBoundToCaller( + env: EnvWithDriveCache, + state: string, + bindingHash: unknown, + stateUserId: string, + caller: StateCaller +): Promise { + if ('userId' in caller) return caller.userId === stateUserId; + const cookie = parseCookies(caller.request.headers.get('Cookie')).get(`${OAUTH_BINDING_COOKIE_PREFIX}${state}`); + if (cookie && typeof bindingHash === 'string' && (await sha256Hex(cookie)) === bindingHash) return true; + // No binding cookie (e.g. the callback host differs from the connect host): + // accept a session for the same user that started the flow. + const sessionUser = await getUserForSession(caller.request, env); + return sessionUser?.id === stateUserId; +} + +async function getState(env: EnvWithDriveCache, state: string, provider: string, caller: StateCaller) { // D1 read replicas can lag behind writes. Force this lookup to the primary by // batching it with a harmless cleanup write. const results = await env.DB.batch([ @@ -407,7 +489,7 @@ async function getState(env: EnvWithDriveCache, state: string, provider: string) const rows = results[1].results as Array<{ userId: string; metadata: string }> | undefined; const record = rows?.[0] ?? null; - console.log(`[oauth-state] lookup provider=${provider} state=${state} found=${record ? 'yes' : 'no'}`); + console.log(`[oauth-state] lookup provider=${provider} found=${record ? 'yes' : 'no'}`); if (!record) { return null; @@ -420,18 +502,24 @@ async function getState(env: EnvWithDriveCache, state: string, provider: string) metadata = {}; } - return { userId: record.userId, metadata }; + const { binding_hash: bindingHash, ...rest } = metadata; + if (!(await statеBoundToCaller(env, state, bindingHash, record.userId, caller))) { + console.warn(`[oauth-state] REJECT provider=${provider}: callback not from the browser or user that started the flow`); + return null; + } + + return { userId: record.userId, metadata: rest }; } async function deleteState(env: EnvWithDriveCache, state: string) { - console.log(`[oauth-state] delete state=${state}`); + console.log('[oauth-state] delete'); await env.DB.prepare(` DELETE FROM oauth_states WHERE state = ? `).bind(state).run(); } -async function consumeState(env: EnvWithDriveCache, state: string, provider: string) { - const record = await getState(env, state, provider); +async function consumеState(env: EnvWithDriveCache, state: string, provider: string, caller: StateCaller) { + const record = await getState(env, state, provider, caller); if (!record) { return null; } @@ -455,7 +543,7 @@ function usePublicClient(env: EnvWithDriveCache): boolean { /** * On the public-client path, generate a PKCE verifier, stash it in `metadata` - * (so the callback can read it back via consumeState), and return the S256 + * (so the callback can read it back via consumеState), and return the S256 * challenge to place on the auth URL. Returns null on the confidential path. * Mutates `metadata` — call before createState so the verifier is persisted. */ @@ -1216,7 +1304,7 @@ export async function cоnnectGооgleDrive( popup: mode === 'popup', }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'google_drive', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'google_drive', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/google/drive/callback`; @@ -1235,7 +1323,7 @@ export async function cоnnectGооgleDrive( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackGооgleDrive( @@ -1253,7 +1341,7 @@ export async function callbackGооgleDrive( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'google_drive'); + const stateData = await consumеState(env, state, 'google_drive', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -1617,7 +1705,7 @@ export async function setGithubRepо( const result = await runGithubSync(env, auth.user!.id, data.dashboardId); return Response.json({ ok: true, hasMore: result.hasMore }); } catch (error) { - return Response.json({ error: error instanceof Error ? error.message : 'E79843: GitHub sync failed' }, { status: 500 }); + return Response.json({ error: 'E79843: ' + (error instanceof Error ? error.message : 'GitHub sync failed') }, { status: 500 }); } } @@ -2361,7 +2449,7 @@ export async function setBоxFоlder( try { await runBoxSync(env, auth.user!.id, data.dashboardId); } catch (error) { - return Response.json({ error: error instanceof Error ? error.message : 'E79862: Box sync failed' }, { status: 500 }); + return Response.json({ error: 'E79862: ' + (error instanceof Error ? error.message : 'Box sync failed') }, { status: 500 }); } return Response.json({ ok: true }); @@ -2964,7 +3052,7 @@ export async function setОnedriveFоlder( try { await runOnedriveSync(env, auth.user!.id, data.dashboardId); } catch (error) { - return Response.json({ error: error instanceof Error ? error.message : 'E79882: OneDrive sync failed' }, { status: 500 }); + return Response.json({ error: 'E79882: ' + (error instanceof Error ? error.message : 'OneDrive sync failed') }, { status: 500 }); } return Response.json({ ok: true }); @@ -4618,7 +4706,7 @@ export async function cоnnectGithub( const mode = requestUrl.searchParams.get('mode'); const dashboardId = requestUrl.searchParams.get('dashboard_id'); const state = buildState(); - await createState(env, auth.user!.id, 'github', state, { + const oauthBinding = await createState(env, auth.user!.id, 'github', state, { mode, dashboardId, }); @@ -4632,7 +4720,7 @@ export async function cоnnectGithub( authUrl.searchParams.set('scope', GITHUB_SCOPE.join(' ')); authUrl.searchParams.set('state', state); - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackGithub( @@ -4650,7 +4738,7 @@ export async function callbackGithub( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'github'); + const stateData = await consumеState(env, state, 'github', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -4744,10 +4832,10 @@ export async function deviceStartGithub( const authError = requireAuth(auth); if (authError) return authError; if (!usePublicClient(env)) { - return deviceJson({ error: 'Device flow is only available on the desktop build.' }, 400); + return deviceJson({ error: 'E79191: Device flow is only available on the desktop build.' }, 400); } if (!env.GITHUB_CLIENT_ID) { - return deviceJson({ error: 'GitHub OAuth is not configured.' }, 400); + return deviceJson({ error: 'E79192: GitHub OAuth is not configured.' }, 400); } let dashboardId: string | null = null; @@ -4765,14 +4853,14 @@ export async function deviceStartGithub( body: reqBody, }); if (!resp.ok) { - return deviceJson({ error: 'Failed to start GitHub device flow.' }, 502); + return deviceJson({ error: 'E79193: Failed to start GitHub device flow.' }, 502); } const data = await resp.json() as { device_code?: string; user_code?: string; verification_uri?: string; expires_in?: number; interval?: number; error?: string; }; if (!data.device_code || !data.user_code || !data.verification_uri) { - return deviceJson({ error: data.error || 'GitHub device flow response was incomplete.' }, 502); + return deviceJson({ error: `E79194: ${data.error || 'GitHub device flow response was incomplete.'}` }, 502); } // Stash the device_code under a random state handle; the frontend polls with it. @@ -4812,7 +4900,7 @@ export async function devicePollGithub( } // Look up (do NOT consume) the device_code so the frontend can poll repeatedly. - const stateData = await getState(env, state, 'github_device'); + const stateData = await getState(env, state, 'github_device', { userId: auth.user!.id }); if (!stateData) { return deviceJson({ status: 'error', error: 'expired' }); } @@ -4889,7 +4977,7 @@ export async function cоnnectBоx( const mode = url.searchParams.get('mode'); const dashboardId = url.searchParams.get('dashboard_id'); const state = buildState(); - await createState(env, auth.user!.id, 'box', state, { + const oauthBinding = await createState(env, auth.user!.id, 'box', state, { mode, dashboardId, }); @@ -4904,7 +4992,7 @@ export async function cоnnectBоx( authUrl.searchParams.set('state', state); authUrl.searchParams.set('scope', BOX_SCOPE.join(' ')); - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackBоx( @@ -4922,7 +5010,7 @@ export async function callbackBоx( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'box'); + const stateData = await consumеState(env, state, 'box', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -5017,7 +5105,7 @@ export async function cоnnectОnedrive( dashboardId, }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'onedrive', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'onedrive', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/onedrive/callback`; @@ -5035,7 +5123,7 @@ export async function cоnnectОnedrive( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackОnedrive( @@ -5053,7 +5141,7 @@ export async function callbackОnedrive( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'onedrive'); + const stateData = await consumеState(env, state, 'onedrive', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -5438,7 +5526,7 @@ export async function connectGmail( popup: mode === 'popup', }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'gmail', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'gmail', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/google/gmail/callback`; @@ -5457,7 +5545,7 @@ export async function connectGmail( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackGmail( @@ -5475,7 +5563,7 @@ export async function callbackGmail( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'gmail'); + const stateData = await consumеState(env, state, 'gmail', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -6658,7 +6746,7 @@ export async function connectCalendar( popup: mode === 'popup', }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'google_calendar', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'google_calendar', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/google/calendar/callback`; @@ -6677,7 +6765,7 @@ export async function connectCalendar( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackCalendar( @@ -6695,7 +6783,7 @@ export async function callbackCalendar( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'google_calendar'); + const stateData = await consumеState(env, state, 'google_calendar', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -7608,7 +7696,7 @@ export async function connectContacts( popup: mode === 'popup', }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'google_contacts', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'google_contacts', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/google/contacts/callback`; @@ -7627,7 +7715,7 @@ export async function connectContacts( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackContacts( @@ -7645,7 +7733,7 @@ export async function callbackContacts( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'google_contacts'); + const stateData = await consumеState(env, state, 'google_contacts', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -8578,7 +8666,7 @@ export async function connectSheets( popup: mode === 'popup', }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'google_sheets', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'google_sheets', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/google/sheets/callback`; @@ -8597,7 +8685,7 @@ export async function connectSheets( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackSheets( @@ -8615,7 +8703,7 @@ export async function callbackSheets( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'google_sheets'); + const stateData = await consumеState(env, state, 'google_sheets', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -9287,7 +9375,7 @@ export async function connectForms( popup: mode === 'popup', }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'google_forms', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'google_forms', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/google/forms/callback`; @@ -9306,7 +9394,7 @@ export async function connectForms( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackForms( @@ -9324,7 +9412,7 @@ export async function callbackForms( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'google_forms'); + const stateData = await consumеState(env, state, 'google_forms', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -9814,7 +9902,7 @@ export async function connectSlack( const dashboardId = requestUrl.searchParams.get('dashboard_id'); const mode = requestUrl.searchParams.get('mode'); const state = buildState(); - await createState(env, auth.user!.id, 'slack', state, { + const oauthBinding = await createState(env, auth.user!.id, 'slack', state, { dashboard_id: dashboardId, popup: mode === 'popup', }); @@ -9828,7 +9916,7 @@ export async function connectSlack( authUrl.searchParams.set('scope', SLACK_SCOPE.join(',')); authUrl.searchParams.set('state', state); - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackSlack( @@ -9846,7 +9934,7 @@ export async function callbackSlack( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'slack'); + const stateData = await consumеState(env, state, 'slack', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -10164,7 +10252,7 @@ export async function connectDiscord( const dashboardId = requestUrl.searchParams.get('dashboard_id'); const mode = requestUrl.searchParams.get('mode'); const state = buildState(); - await createState(env, auth.user!.id, 'discord', state, { + const oauthBinding = await createState(env, auth.user!.id, 'discord', state, { dashboard_id: dashboardId, popup: mode === 'popup', }); @@ -10180,7 +10268,55 @@ export async function connectDiscord( authUrl.searchParams.set('permissions', DISCORD_BOT_PERMISSIONS); authUrl.searchParams.set('state', state); - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); +} + +const DISCORD_PERM_ADMINISTRATOR = BigInt(0x8); +const DISCORD_PERM_MANAGE_GUILD = BigInt(0x20); + +function discordScоpeIncludesBot(scope: string | undefined): boolean { + return typeof scope === 'string' && scope.split(/\s+/).includes('bot'); +} + +/** Whether a /users/@me/guilds entry lets the user manage that guild. */ +export function canManagеDiscоrdGuild(entry: { owner?: unknown; permissions?: unknown }): boolean { + if (entry.owner === true) return true; + const raw = entry.permissions; + if (typeof raw !== 'string' && typeof raw !== 'number') return false; + let perms: bigint; + try { + perms = BigInt(raw); + } catch { + return false; + } + return (perms & DISCORD_PERM_ADMINISTRATOR) !== BigInt(0) || (perms & DISCORD_PERM_MANAGE_GUILD) !== BigInt(0); +} + +/** + * The guild a Discord authorization is bound to, or null when it can't be + * verified (fail closed). Requires the `bot` scope; prefers the token + * response's guild; falls back to the callback's guild_id only if the user + * owns it or holds MANAGE_GUILD / ADMINISTRATOR there. + */ +export async function resоlveDiscordGuildId( + tokenData: { access_token?: string; scope?: string; guild?: { id?: string } }, + queryGuildId: string | null, +): Promise { + if (!discordScоpeIncludesBot(tokenData.scope)) return null; + if (typeof tokenData.guild?.id === 'string' && tokenData.guild.id) return tokenData.guild.id; + if (!queryGuildId || !tokenData.access_token) return null; + try { + const guildsResp = await fetch('https://discord.com/api/v10/users/@me/guilds', { + headers: { Authorization: `Bearer ${tokenData.access_token}` }, + }); + if (!guildsResp.ok) return null; + const guilds = await guildsResp.json() as Array<{ id?: unknown; owner?: unknown; permissions?: unknown }>; + if (!Array.isArray(guilds)) return null; + const entry = guilds.find((g) => g && g.id === queryGuildId); + return entry && canManagеDiscоrdGuild(entry) ? queryGuildId : null; + } catch { + return null; + } } export async function callbackDiscord( @@ -10202,7 +10338,7 @@ export async function callbackDiscord( return renderErrorPage('Missing authorization code.'); } - const stateData = await consumeState(env, state, 'discord'); + const stateData = await consumеState(env, state, 'discord', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -10247,9 +10383,18 @@ export async function callbackDiscord( return renderErrorPage('Discord authorization failed: no access token.'); } - // Use the guild from token response, or from query param - const resolvedGuildId = tokenData.guild?.id || guildId; - const resolvedGuildName = tokenData.guild?.name || null; + // The guild from the token response is Discord's word for where the bot was + // installed by this authorization; it is only trusted when the bot scope was + // granted. The callback's guild_id query param is user-editable, so it is only + // used if the user can manage that guild (owner, MANAGE_GUILD or ADMINISTRATOR + // in their own guild list) — mere membership would let any member bind the + // shared bot to someone else's server. Anything unverified records no guild, + // and the gateway and subscriptions then fail closed. + // Subscriptions trust this guild_id (messaging/webhook-handler.ts). + const resolvedGuildId = await resоlveDiscordGuildId(tokenData, guildId); + const resolvedGuildName = resolvedGuildId && tokenData.guild?.id === resolvedGuildId + ? tokenData.guild.name || null + : null; // Fetch user info using the user OAuth token let discordUser: { id: string; username: string; discriminator?: string; avatar?: string | null } | null = null; @@ -11187,7 +11332,7 @@ export async function connectWhatsAppPersonal( if (!env.BRIDGE_URL || !env.BRIDGE_INTERNAL_TOKEN) { return Response.json( - { error: 'Bridge service not configured (BRIDGE_URL / BRIDGE_INTERNAL_TOKEN missing)' }, + { error: 'E79187: Bridge service not configured (BRIDGE_URL / BRIDGE_INTERNAL_TOKEN missing)' }, { status: 503 }, ); } @@ -11346,7 +11491,7 @@ export async function connectWhatsAppPersonal( await env.DB.prepare('DELETE FROM messaging_subscriptions WHERE id = ?').bind(subscriptionId).run(); console.error(`[integrations] Bridge startSession failed: dashboardId=${body.dashboardId}`, err); return Response.json( - { error: 'Failed to start WhatsApp connection — bridge service unavailable' }, + { error: 'E79188: Failed to start WhatsApp connection — bridge service unavailable' }, { status: 502 }, ); } @@ -11481,12 +11626,12 @@ export async function saveTwitterCredentials( try { body = await request.json() as { bearer_token?: string }; } catch { - return Response.json({ error: 'Invalid request body' }, { status: 400 }); + return Response.json({ error: 'E79189: Invalid request body' }, { status: 400 }); } const bearerToken = (body.bearer_token || '').trim(); if (!bearerToken) { - return Response.json({ error: 'bearer_token is required' }, { status: 400 }); + return Response.json({ error: 'E79190: bearer_token is required' }, { status: 400 }); } // App-only Bearer tokens have no user identity — store directly without verification @@ -11606,7 +11751,7 @@ export async function connectTeams( dashboardId, }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'teams', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'teams', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/teams/callback`; @@ -11624,7 +11769,7 @@ export async function connectTeams( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackTeams( @@ -11649,7 +11794,7 @@ export async function callbackTeams( // Teams popup callbacks can be re-requested by intermediate challenge pages. // Do not delete the state until after the token exchange succeeds. - const stateData = await getState(env, state, 'teams'); + const stateData = await getState(env, state, 'teams', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -11798,7 +11943,7 @@ export async function connectOutlook( dashboardId, }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'outlook', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'outlook', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/outlook/callback`; @@ -11816,7 +11961,7 @@ export async function connectOutlook( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackOutlook( @@ -11837,7 +11982,7 @@ export async function callbackOutlook( } // Do not delete the state until after the token exchange succeeds. - const stateData = await getState(env, state, 'outlook'); + const stateData = await getState(env, state, 'outlook', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } @@ -12550,7 +12695,7 @@ export async function connectOutlookCalendar( dashboardId, }; const codeChallenge = await pkceChallengeForState(env, metadata); - await createState(env, auth.user!.id, 'outlook_calendar', state, metadata); + const oauthBinding = await createState(env, auth.user!.id, 'outlook_calendar', state, metadata); const redirectBase = getRedirectBase(request, env); const redirectUri = `${redirectBase}/integrations/outlook/calendar/callback`; @@ -12568,7 +12713,7 @@ export async function connectOutlookCalendar( authUrl.searchParams.set('code_challenge_method', 'S256'); } - return Response.redirect(authUrl.toString(), 302); + return oauthRedirect(request, authUrl.toString(), state, oauthBinding); } export async function callbackOutlookCalendar( @@ -12593,7 +12738,7 @@ export async function callbackOutlookCalendar( // Do not delete the state until after the token exchange succeeds. // Transient Microsoft failures would otherwise burn the state and block retries. - const stateData = await getState(env, state, 'outlook_calendar'); + const stateData = await getState(env, state, 'outlook_calendar', { request }); if (!stateData) { return renderErrorPage('Invalid or expired state.'); } diff --git a/controlplane/src/integrations/oauth-state.test.ts b/controlplane/src/integrations/oauth-state.test.ts new file mode 100644 index 00000000..5e5e06f8 --- /dev/null +++ b/controlplane/src/integrations/oauth-state.test.ts @@ -0,0 +1,180 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: integrations-oauth-state-test-v1-identifier-renames + +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import { cоnnectGithub, callbackGithub, cоnnectOnCallbackHost } from './handler'; +import type { AuthContext } from '../auth/middleware'; +import type { EnvWithDriveCache } from '../storage/drive-cache'; + +// Minimal D1 fake covering the oauth_states and user_sessions queries the OAuth +// state helpers make; anything else returns no rows. +function fakeDb(sessions: Record) { + const states = new Map(); + const stmt = (sql: string, args: unknown[] = []) => ({ + bind: (...a: unknown[]) => stmt(sql, a), + async run() { + if (sql.includes('INSERT INTO oauth_states')) { + const [state, user_id, provider, metadata] = args as string[]; + states.set(state, { user_id, provider, metadata }); + } else if (sql.includes('DELETE FROM oauth_states WHERE state')) { + states.delete(args[0] as string); + } + return { success: true }; + }, + async all() { + if (sql.includes('FROM oauth_states WHERE state')) { + const row = states.get(args[0] as string); + return { results: row && row.provider === args[1] ? [{ userId: row.user_id, metadata: row.metadata }] : [] }; + } + return { results: [] }; + }, + async first() { + if (sql.includes('FROM user_sessions')) { + const userId = sessions[args[0] as string]; + return userId ? { id: userId, email: `${userId}@x.test`, name: userId, created_at: '', trial_started_at: null } : null; + } + return null; + }, + }); + return { + states, + prepare: (sql: string) => stmt(sql), + batch: async (stmts: Array>) => Promise.all(stmts.map((s) => s.all())), + }; +} + +const auth = (id: string) => ({ isAuthenticated: true, user: { id, email: `${id}@x.test`, name: id, createdAt: '' } }) as AuthContext; + +describe('OAuth state is bound to the browser that started the flow', () => { + let env: EnvWithDriveCache; + let fetchMock: ReturnType>>; + + beforeEach(() => { + env = { + DB: fakeDb({ 'victim-session': 'victim', 'attacker-session': 'attacker' }), + GITHUB_CLIENT_ID: 'cid', + GITHUB_CLIENT_SECRET: 'csecret', + } as unknown as EnvWithDriveCache; + // The token exchange is the first network call after the state check. + fetchMock = vi.fn<[], Promise>(async () => new Response(JSON.stringify({ error: 'bad_verification_code' }), { status: 200 })); + vi.stubGlobal('fetch', fetchMock); + }); + afterEach(() => { + vi.unstubAllGlobals(); + }); + + async function startFlow(userId: string) { + const res = await cоnnectGithub(new Request('https://api.test/integrations/github/connect'), env, auth(userId)); + expect(res.status).toBe(302); + const state = new URL(res.headers.get('Location')!).searchParams.get('state')!; + const setCookie = res.headers.get('Set-Cookie')!; + return { state, cookie: setCookie.split(';')[0], setCookie }; + } + + const callback = (state: string, cookie?: string) => + callbackGithub( + new Request(`https://api.test/integrations/github/callback?code=c&state=${state}`, { + headers: cookie ? { Cookie: cookie } : {}, + }), + env + ); + + it('sets an HttpOnly, Lax, Secure cookie scoped to the state', async () => { + const { state, setCookie } = await startFlow('attacker'); + expect(setCookie).toMatch(new RegExp(`^orcabot_oauth_${state}=[0-9a-f-]{36}; `)); + expect(setCookie).toContain('Path=/integrations'); + expect(setCookie).toContain('HttpOnly'); + expect(setCookie).toContain('SameSite=Lax'); + expect(setCookie).toContain('Secure'); + // Only the hash is stored server-side. + const stored = (env.DB as unknown as ReturnType).states.get(state)!; + expect(stored.metadata).not.toContain(setCookie.split(';')[0].split('=')[1]); + }); + + it('rejects a callback finished in another browser (attacker state, victim browser)', async () => { + const { state } = await startFlow('attacker'); + const res = await callback(state, 'orcabot_session=victim-session'); + expect(await res.text()).toContain('Invalid or expired state'); + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it('rejects a callback with no cookies at all', async () => { + const { state } = await startFlow('attacker'); + expect(await (await callback(state)).text()).toContain('Invalid or expired state'); + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it('rejects a forged binding cookie', async () => { + const { state } = await startFlow('attacker'); + const res = await callback(state, `orcabot_oauth_${state}=00000000-0000-0000-0000-000000000000`); + expect(await res.text()).toContain('Invalid or expired state'); + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it('rejects one flow\'s cookie presented for another flow\'s state', async () => { + const mine = await startFlow('victim'); + const theirs = await startFlow('attacker'); + const value = mine.cookie.split('=')[1]; + const res = await callback(theirs.state, `orcabot_oauth_${theirs.state}=${value}`); + expect(await res.text()).toContain('Invalid or expired state'); + }); + + it('accepts the browser that started the flow (binding cookie)', async () => { + const { state, cookie } = await startFlow('victim'); + await callback(state, cookie); + expect(fetchMock).toHaveBeenCalledTimes(1); // got past the state check to the token exchange + }); + + it('accepts a session for the same user when the binding cookie is missing', async () => { + const { state } = await startFlow('victim'); + await callback(state, 'orcabot_session=victim-session'); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it('is single use', async () => { + const { state, cookie } = await startFlow('victim'); + await callback(state, cookie); + const again = await callback(state, cookie); + expect(await again.text()).toContain('Invalid or expired state'); + }); +}); + +describe('connect runs on the callback host', () => { + const env = (base?: string) => ({ OAUTH_REDIRECT_BASE: base }) as unknown as EnvWithDriveCache; + const hop = (url: string, base?: string) => cоnnectOnCallbackHost(new Request(url), env(base)); + + it('bounces desktop 127.0.0.1 connect to the localhost callback host, keeping the query', () => { + const res = hop('http://127.0.0.1:8787/integrations/github/connect?dashboard_id=d&user_id=u&surface=s', 'http://localhost:8787'); + expect(res?.status).toBe(302); + expect(res?.headers.get('Location')).toBe('http://localhost:8787/integrations/github/connect?dashboard_id=d&user_id=u&surface=s'); + }); + + it.each([ + ['same host', 'http://localhost:8787/integrations/github/connect', 'http://localhost:8787'], + ['no OAUTH_REDIRECT_BASE', 'http://127.0.0.1:8787/integrations/github/connect', undefined], + ['different port', 'http://127.0.0.1:9999/integrations/github/connect', 'http://localhost:8787'], + ['remote hosts', 'https://api.orcabot.com/integrations/github/connect', 'https://other.orcabot.com'], + ['remote callback host', 'http://127.0.0.1:8787/integrations/github/connect', 'https://evil.example'], + ])('does not bounce: %s', (_label, url, base) => { + expect(hop(url, base)).toBeNull(); + }); + + it('desktop flow: the cookie set after the bounce is sent to the callback', async () => { + const fetchMock = vi.fn<[], Promise>(async () => new Response(JSON.stringify({ error: 'x' }))); + vi.stubGlobal('fetch', fetchMock); + const e = { + DB: fakeDb({}), GITHUB_CLIENT_ID: 'cid', GITHUB_CLIENT_SECRET: 's', OAUTH_REDIRECT_BASE: 'http://localhost:8787', + } as unknown as EnvWithDriveCache; + const first = cоnnectOnCallbackHost(new Request('http://127.0.0.1:8787/integrations/github/connect?user_id=victim'), e)!; + const res = await cоnnectGithub(new Request(first.headers.get('Location')!), e, auth('victim')); + const location = new URL(res.headers.get('Location')!); + expect(location.searchParams.get('redirect_uri')).toBe('http://localhost:8787/integrations/github/callback'); + // The cookie was set on localhost, where the callback lands; no session cookie needed. + const cookie = res.headers.get('Set-Cookie')!.split(';')[0]; + await callbackGithub(new Request(`http://localhost:8787/integrations/github/callback?code=c&state=${location.searchParams.get('state')}`, { headers: { Cookie: cookie } }), e); + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(res.headers.get('Set-Cookie')).not.toContain('Secure'); + vi.unstubAllGlobals(); + }); +}); diff --git a/controlplane/src/mcp-tools/handler.ts b/controlplane/src/mcp-tools/handler.ts index 004b3450..605512f5 100644 --- a/controlplane/src/mcp-tools/handler.ts +++ b/controlplane/src/mcp-tools/handler.ts @@ -1,5 +1,6 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: mcp-tools-handler-v1-error-codes import type { Env, UserMcpTool } from '../types'; @@ -52,7 +53,7 @@ export async function createMcpTооl( const transport = data.transport || 'stdio'; if (!validTransports.includes(transport)) { return Response.json( - { error: `transport must be one of: ${validTransports.join(', ')}` }, + { error: `E79195: transport must be one of: ${validTransports.join(', ')}` }, { status: 400 } ); } diff --git a/controlplane/src/mcp-ui/handler.ts b/controlplane/src/mcp-ui/handler.ts index 5ecc827e..6cd67489 100644 --- a/controlplane/src/mcp-ui/handler.ts +++ b/controlplane/src/mcp-ui/handler.ts @@ -1,5 +1,6 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: mcp-ui-handler-v1-error-codes /** * MCP UI Server Handler @@ -464,7 +465,7 @@ export async function callTool( const dashboardId = args.dashboard_id as string; if (!dashboardId) { return Response.json( - { error: 'dashboard_id is required' }, + { error: 'E79162: dashboard_id is required' }, { status: 400 } ); } @@ -639,7 +640,7 @@ export async function callTool( default: return Response.json( - { error: `Unknown tool: ${toolName}` }, + { error: `E79163: Unknown tool: ${toolName}` }, { status: 400 } ); } @@ -657,7 +658,7 @@ export async function callTool( if (!response.ok) { const error = await response.text(); return Response.json( - { error: `Failed to send UI command: ${error}` }, + { error: `E79164: Failed to send UI command: ${error}` }, { status: 500 } ); } @@ -700,7 +701,7 @@ export async function getItems(env: Env, dashboardId: string, userId?: string): if (!response.ok) { return Response.json( - { error: 'Failed to get dashboard items' }, + { error: 'E79165: Failed to get dashboard items' }, { status: 500 } ); } diff --git a/controlplane/src/messaging/delivery.ts b/controlplane/src/messaging/delivery.ts index 6b79e474..7a10033b 100644 --- a/controlplane/src/messaging/delivery.ts +++ b/controlplane/src/messaging/delivery.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: messaging-delivery-v27-fix-retry-keepbuffered-condition -const MODULE_REVISION = 'messaging-delivery-v27-fix-retry-keepbuffered-condition'; +// REVISION: messaging-delivery-v29-signed-sandbox-requests +const MODULE_REVISION = 'messaging-delivery-v29-signed-sandbox-requests'; console.log(`[messaging-delivery] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); /** @@ -25,7 +25,7 @@ import type { Env, MessagingPolicy } from '../types'; import type { EnvWithDriveCache } from '../storage/drive-cache'; import { ensureDashbоardSandbоx } from '../sessions/handler'; import { SandboxClient } from '../sandbox/client'; -import { createPtyToken } from '../auth/pty-token'; +import { createPtyToken, tokenSigningSecret } from '../auth/pty-token'; import { channelMatchesFilter } from './webhook-handler'; import { formatItem } from '../dashboards/handler'; @@ -85,21 +85,21 @@ export async function deliverOrWakeAndDrain( ? env.DB.prepare(` SELECT DISTINCT de.target_item_id FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' - WHERE de.source_item_id = ? + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(messagingItemId).all<{ target_item_id: string }>() : env.DB.prepare(` SELECT DISTINCT de.target_item_id FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' JOIN terminal_integrations ti ON ti.item_id = de.target_item_id AND ti.provider = ? AND ti.deleted_at IS NULL AND ti.active_policy_id IS NOT NULL - WHERE de.source_item_id = ? + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(provider, messagingItemId).all<{ target_item_id: string }>(), env.DB.prepare(` SELECT DISTINCT de.target_item_id, di.type FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type IN ('note', 'prompt') - WHERE de.source_item_id = ? + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type IN ('note', 'prompt') + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(messagingItemId).all<{ target_item_id: string; type: string }>(), ]); @@ -162,7 +162,7 @@ export async function deliverOrWakeAndDrain( // Don't return — we can still deliver to note/prompt items } else { const { sandboxSessionId, sandboxMachineId } = sandboxResult; - sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); for (const terminal of connectedTerminals.results) { try { @@ -249,9 +249,9 @@ async function resolveTerminalPty( const activeSession = await env.DB.prepare(` SELECT id, pty_id, sandbox_session_id, sandbox_machine_id FROM sessions - WHERE item_id = ? AND status = 'active' + WHERE item_id = ? AND dashboard_id = ? AND status = 'active' ORDER BY created_at DESC LIMIT 1 - `).bind(terminalItemId).first(); + `).bind(terminalItemId, dashboardId).first(); if (activeSession?.pty_id) { return { @@ -269,7 +269,7 @@ async function resolveTerminalPty( sandboxSessionId, dashboardId, effectiveUserId, - env.INTERNAL_API_TOKEN, + tokenSigningSecret(env), ); await sandbox.createPty( @@ -365,10 +365,10 @@ async function checkPerTerminalPolicy( const terminalPolicies = await env.DB.prepare(` SELECT de.target_item_id AS terminal_item_id, ip.policy FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' JOIN terminal_integrations ti ON ti.item_id = de.target_item_id AND ti.provider = ? AND ti.deleted_at IS NULL JOIN integration_policies ip ON ip.id = ti.active_policy_id - WHERE de.source_item_id = ? + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(msg.provider, messagingItemId).all<{ terminal_item_id: string; policy: string }>(); if (!terminalPolicies.results?.length) return 'no_policy'; @@ -624,13 +624,13 @@ async function deliverToItem( const newContent = currentContent + separator + formattedText; await env.DB.prepare( - 'UPDATE dashboard_items SET content = ?, updated_at = ? WHERE id = ?' - ).bind(newContent, now, target.itemId).run(); + 'UPDATE dashboard_items SET content = ?, updated_at = ? WHERE id = ? AND dashboard_id = ?' + ).bind(newContent, now, target.itemId, dashboardId).run(); } else { // Prompt: replace content with message text (raw, no formatting wrapper) await env.DB.prepare( - 'UPDATE dashboard_items SET content = ?, updated_at = ? WHERE id = ?' - ).bind(msg.message_text || '', now, target.itemId).run(); + 'UPDATE dashboard_items SET content = ?, updated_at = ? WHERE id = ? AND dashboard_id = ?' + ).bind(msg.message_text || '', now, target.itemId, dashboardId).run(); } // Notify Durable Object for real-time WebSocket broadcast @@ -900,7 +900,7 @@ export async function retryBufferedMessages(env: Env): Promise { return; } - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); for (const row of targets.results) { try { @@ -912,21 +912,21 @@ export async function retryBufferedMessages(env: Env): Promise { ? env.DB.prepare(` SELECT DISTINCT de.target_item_id FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' - WHERE de.source_item_id = ? + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(row.messaging_item_id).all<{ target_item_id: string }>() : env.DB.prepare(` SELECT DISTINCT de.target_item_id FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' JOIN terminal_integrations ti ON ti.item_id = de.target_item_id AND ti.provider = ? AND ti.deleted_at IS NULL AND ti.active_policy_id IS NOT NULL - WHERE de.source_item_id = ? + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(row.provider, row.messaging_item_id).all<{ target_item_id: string }>(), env.DB.prepare(` SELECT DISTINCT de.target_item_id, di.type FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type IN ('note', 'prompt') - WHERE de.source_item_id = ? + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type IN ('note', 'prompt') + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(row.messaging_item_id).all<{ target_item_id: string; type: string }>(), ]); diff --git a/controlplane/src/messaging/send.ts b/controlplane/src/messaging/send.ts index d95cd660..633ea388 100644 --- a/controlplane/src/messaging/send.ts +++ b/controlplane/src/messaging/send.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: messaging-send-v6-teams-require-channel-context -const MODULE_REVISION = 'messaging-send-v6-teams-require-channel-context'; +// REVISION: messaging-send-v9-error-codes +const MODULE_REVISION = 'messaging-send-v9-error-codes'; console.log(`[messaging-send] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); /** @@ -19,7 +19,7 @@ console.log(`[messaging-send] REVISION: ${MODULE_REVISION} loaded at ${new Date( import type { Env, MessagingProvider } from '../types'; import { executeWhatsAppAction } from '../integration-policies/api-clients/whatsapp'; import { executeSlackAction } from '../integration-policies/api-clients/slack'; -import { executeDiscordAction } from '../integration-policies/api-clients/discord'; +import { executeDiscordAction, getDiscоrdChannelGuildId } from '../integration-policies/api-clients/discord'; import { executeTeamsAction } from '../integration-policies/api-clients/teams'; import { getAccessToken } from '../integration-policies/token-refresh'; import { BridgeClient } from '../bridge/client'; @@ -74,7 +74,7 @@ export async function handleMessagingSend( if (!dashboardId || !itemId || !text?.trim()) { return Response.json( - { error: 'dashboardId, itemId, and text are required' }, + { error: 'E79166: dashboardId, itemId, and text are required' }, { status: 400 }, ); } @@ -84,7 +84,12 @@ export async function handleMessagingSend( `SELECT role FROM dashboard_members WHERE dashboard_id = ? AND user_id = ?` ).bind(dashboardId, userId).first<{ role: string }>(); if (!membership) { - return Response.json({ error: 'Not a member of this dashboard' }, { status: 403 }); + return Response.json({ error: 'E79167: Not a member of this dashboard' }, { status: 403 }); + } + // Sending posts with the subscription creator's account (their Slack/WhatsApp + // etc.), so viewers can't — same bar as editing the messaging block. + if (membership.role !== 'owner' && membership.role !== 'editor') { + return Response.json({ error: 'E79617: Viewers cannot send messages' }, { status: 403 }); } // Find active subscriptions for this messaging block item @@ -95,7 +100,7 @@ export async function handleMessagingSend( if (!subs.results?.length) { return Response.json( - { error: 'No active messaging subscriptions for this block' }, + { error: 'E79168: No active messaging subscriptions for this block' }, { status: 404 }, ); } @@ -283,6 +288,24 @@ async function sendDiscord( throw new Error('No channel ID — no inbound messages received yet and no channel configured'); } + // The bot token is shared by every tenant, so the bot being able to post in + // a channel proves nothing: the channel must be in the guild this + // integration was connected to (same rule as gateway step 9c). + const meta = await env.DB.prepare( + `SELECT metadata FROM user_integrations WHERE id = ?` + ).bind(integrationId).first<{ metadata: string | null }>(); + let guildId: string | undefined; + try { + guildId = meta?.metadata ? (JSON.parse(meta.metadata) as { guild_id?: string }).guild_id : undefined; + } catch { /* treated as no guild below */ } + if (!guildId) { + throw new Error('E79630: Discord integration has no connected server; reconnect Discord'); + } + const channelGuildId = await getDiscоrdChannelGuildId(channelId, token); + if (channelGuildId !== guildId) { + throw new Error('E79631: Discord channel is not in the connected server'); + } + // Send message to channel const result = await executeDiscordAction('discord.send_message', { channel: channelId, diff --git a/controlplane/src/messaging/subscription-auth.test.ts b/controlplane/src/messaging/subscription-auth.test.ts new file mode 100644 index 00000000..3cc94015 --- /dev/null +++ b/controlplane/src/messaging/subscription-auth.test.ts @@ -0,0 +1,111 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import { createTestContext, seedUser, seedDashboard, type TestContext } from '../../tests/helpers'; +import { createSubscription } from './webhook-handler'; +import { handleMessagingSend } from './send'; + +const owner = { id: 'owner-1', email: 'o@x.test', name: 'Owner' }; +const viewer = { id: 'viewer-1', email: 'v@x.test', name: 'Viewer' }; + +async function seedItem(ctx: TestContext, dashboardId: string, type: string) { + const id = `item-${type}`; + const now = new Date().toISOString(); + await ctx.db.prepare( + `INSERT INTO dashboard_items (id, dashboard_id, type, content, position_x, position_y, width, height, created_at, updated_at) VALUES (?, ?, ?, '', 0, 0, 200, 150, ?, ?)` + ).bind(id, dashboardId, type, now, now).run(); + return id; +} + +async function seedIntegration(ctx: TestContext, userId: string, provider: string, metadata: Record) { + await ctx.db.prepare( + `INSERT INTO user_integrations (id, user_id, provider, access_token, metadata) VALUES (?, ?, ?, ?, ?)` + ).bind(`int-${provider}-${userId}`, userId, provider, `tok-${provider}`, JSON.stringify(metadata)).run(); +} + +describe('messaging subscriptions are scoped to the caller\'s own workspace', () => { + let ctx: TestContext; + let dashboardId: string; + let fetchMock: ReturnType>>; + + beforeEach(async () => { + ctx = await createTestContext(); + await seedUser(ctx.db, owner); + dashboardId = (await seedDashboard(ctx.db, owner.id, { id: 'dash-1' })).id; + fetchMock = vi.fn<[RequestInfo | URL], Promise>(); + vi.stubGlobal('fetch', fetchMock); + }); + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('rejects a Slack teamId that is not one of the caller\'s workspaces', async () => { + const itemId = await seedItem(ctx, dashboardId, 'slack'); + await seedIntegration(ctx, owner.id, 'slack', { team_id: 'T_MINE' }); + await expect( + createSubscription(ctx.env, dashboardId, itemId, owner.id, 'slack', { channelId: 'C_THEIRS', teamId: 'T_OTHER' }, 'https://api.test') + ).rejects.toMatchObject({ code: 'SLACK_WORKSPACE_MISMATCH' }); + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it('rejects a Slack channel that does not exist in the caller\'s workspace', async () => { + const itemId = await seedItem(ctx, dashboardId, 'slack'); + await seedIntegration(ctx, owner.id, 'slack', { team_id: 'T_MINE' }); + fetchMock.mockResolvedValue(new Response(JSON.stringify({ ok: false, error: 'channel_not_found' }))); + await expect( + createSubscription(ctx.env, dashboardId, itemId, owner.id, 'slack', { channelId: 'C_THEIRS' }, 'https://api.test') + ).rejects.toMatchObject({ code: 'SLACK_CHANNEL_NOT_FOUND' }); + expect(String(fetchMock.mock.calls[0][0])).toContain('conversations.info?channel=C_THEIRS'); + }); + + it('accepts a Slack channel in the caller\'s workspace', async () => { + const itemId = await seedItem(ctx, dashboardId, 'slack'); + await seedIntegration(ctx, owner.id, 'slack', { team_id: 'T_MINE' }); + fetchMock.mockResolvedValue(new Response(JSON.stringify({ ok: true, channel: { id: 'C_MINE' } }))); + const sub = await createSubscription(ctx.env, dashboardId, itemId, owner.id, 'slack', { channelId: 'C_MINE', teamId: 'T_MINE' }, 'https://api.test'); + expect(sub.id).toBeTruthy(); + }); + + it('rejects a Discord channel in another guild (shared bot can see it)', async () => { + const itemId = await seedItem(ctx, dashboardId, 'discord'); + await seedIntegration(ctx, owner.id, 'discord', { guild_id: 'G_MINE' }); + fetchMock.mockResolvedValue(new Response(JSON.stringify({ id: '555', guild_id: 'G_OTHER' }))); + await expect( + createSubscription(ctx.env, dashboardId, itemId, owner.id, 'discord', { channelId: '555' }, 'https://api.test') + ).rejects.toMatchObject({ code: 'DISCORD_CHANNEL_NOT_FOUND' }); + }); + + it('rejects Discord when the integration has no recorded guild', async () => { + const itemId = await seedItem(ctx, dashboardId, 'discord'); + await seedIntegration(ctx, owner.id, 'discord', {}); + fetchMock.mockResolvedValue(new Response(JSON.stringify({ id: '555', guild_id: 'G_ANY' }))); + await expect( + createSubscription(ctx.env, dashboardId, itemId, owner.id, 'discord', { channelId: '555' }, 'https://api.test') + ).rejects.toMatchObject({ code: 'DISCORD_CHANNEL_NOT_FOUND' }); + }); + + it('accepts a Discord channel in the caller\'s guild', async () => { + const itemId = await seedItem(ctx, dashboardId, 'discord'); + await seedIntegration(ctx, owner.id, 'discord', { guild_id: 'G_MINE' }); + fetchMock.mockResolvedValue(new Response(JSON.stringify({ id: '555', guild_id: 'G_MINE' }))); + const sub = await createSubscription(ctx.env, dashboardId, itemId, owner.id, 'discord', { channelId: '555' }, 'https://api.test'); + expect(sub.id).toBeTruthy(); + }); +}); + +describe('messaging send requires editor', () => { + it('rejects a viewer', async () => { + const ctx = await createTestContext(); + await seedUser(ctx.db, owner); + await seedUser(ctx.db, viewer); + const { id: dashboardId } = await seedDashboard(ctx.db, owner.id, { id: 'dash-2' }); + await ctx.db.prepare( + `INSERT INTO dashboard_members (dashboard_id, user_id, role, added_at) VALUES (?, ?, 'viewer', ?)` + ).bind(dashboardId, viewer.id, new Date().toISOString()).run(); + const itemId = await seedItem(ctx, dashboardId, 'slack'); + const res = await handleMessagingSend(ctx.env, viewer.id, { dashboardId, itemId, text: 'hi' }); + expect(res.status).toBe(403); + expect(await res.json()).toMatchObject({ error: expect.stringContaining('E79617') }); + }); +}); diff --git a/controlplane/src/messaging/webhook-handler.ts b/controlplane/src/messaging/webhook-handler.ts index a9b2eda9..8571af7b 100644 --- a/controlplane/src/messaging/webhook-handler.ts +++ b/controlplane/src/messaging/webhook-handler.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: messaging-webhook-v56-teams-require-team-id -const MODULE_REVISION = 'messaging-webhook-v56-teams-require-team-id'; +// REVISION: messaging-webhook-v59-identifier-renames +const MODULE_REVISION = 'messaging-webhook-v59-identifier-renames'; console.log(`[messaging-webhook] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); /** @@ -1344,10 +1344,10 @@ async function processSubscriptionMessage( const msgPolicies = await env.DB.prepare(` SELECT ip.policy FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' JOIN terminal_integrations ti ON ti.item_id = de.target_item_id AND ti.provider = ? AND ti.deleted_at IS NULL JOIN integration_policies ip ON ip.id = ti.active_policy_id - WHERE de.source_item_id = ? + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(provider, subscription.item_id).all<{ policy: string }>(); const parsedMsgPolicies = (msgPolicies.results || []).map(row => { @@ -1374,10 +1374,10 @@ async function processSubscriptionMessage( const terminalPolicies = await env.DB.prepare(` SELECT ip.policy FROM dashboard_edges de - JOIN dashboard_items di ON di.id = de.target_item_id AND di.type = 'terminal' + JOIN dashboard_items di ON di.id = de.target_item_id AND di.dashboard_id = de.dashboard_id AND di.type = 'terminal' JOIN terminal_integrations ti ON ti.item_id = de.target_item_id AND ti.provider = ? AND ti.deleted_at IS NULL JOIN integration_policies ip ON ip.id = ti.active_policy_id - WHERE de.source_item_id = ? + WHERE de.source_item_id = ? AND de.dashboard_id = (SELECT si.dashboard_id FROM dashboard_items si WHERE si.id = de.source_item_id) `).bind(provider, subscription.item_id).all<{ policy: string }>(); const policies = (terminalPolicies.results || []).map(row => { @@ -1544,6 +1544,49 @@ async function matchIntegrationByMetadata( /** * Synchronous helper: find integration whose metadata[key] matches the given value. */ +/** True if `channelId` is a channel in the workspace this Slack token belongs to. */ +async function slackChannеlInWorkspace(token: string, channelId: string): Promise { + try { + const resp = await fetch(`https://slack.com/api/conversations.info?channel=${encodeURIComponent(channelId)}`, { + headers: { Authorization: `Bearer ${token}` }, + }); + if (!resp.ok) return false; + const result = await resp.json() as { ok: boolean; channel?: { id?: string } }; + return result.ok && result.channel?.id === channelId; + } catch { + return false; + } +} + +/** + * The caller's Discord integration whose recorded guild owns `channelId`. + * Every integration stores the platform's shared bot token, so the bot seeing + * the channel proves nothing — the guild match is the check. An integration + * with no recorded guild (legacy) can't subscribe until reconnected. + */ +async function discordIntegratiоnForChannel( + intRows: Array<{ id: string; access_token: string; metadata: string | null }>, + channelId: string, +): Promise { + for (const row of intRows) { + if (!row.access_token) continue; + try { + const resp = await fetch(`https://discord.com/api/v10/channels/${encodeURIComponent(channelId)}`, { + headers: { Authorization: `Bot ${row.access_token}` }, + }); + if (!resp.ok) continue; + const ch = await resp.json() as { id?: string; guild_id?: string }; + if (ch.id !== channelId) continue; + let guildId: string | undefined; + try { + guildId = row.metadata ? (JSON.parse(row.metadata) as { guild_id?: string }).guild_id : undefined; + } catch { /* no recorded guild */ } + if (guildId && ch.guild_id === guildId) return row.id; + } catch { /* try next integration */ } + } + return null; +} + function matchIntegrationByMetadataSync( rows: Array<{ id: string; metadata: string | null }>, metaKey: string, @@ -1699,9 +1742,17 @@ export async function createSubscription( } if (resolvedTeamId) { - // team_id already provided — match integration by team_id + // team_id already provided — it must be one of the caller's own workspaces. + // Inbound Slack events route by team_id + channel_id, so accepting any + // team_id would let a user subscribe to another tenant's channel. const match = intTeamMap.find((r) => r.teamId === resolvedTeamId); - resolvedIntegrationId = match?.id || intTeamMap[0].id; + if (!match) { + throw new SubscriptionError( + 'SLACK_WORKSPACE_MISMATCH', + 'That Slack workspace is not connected to your account.', + ); + } + resolvedIntegrationId = match.id; } else if (intRows.length === 1) { // Single integration — no ambiguity resolvedIntegrationId = intTeamMap[0].id; @@ -1711,7 +1762,7 @@ export async function createSubscription( for (const row of intTeamMap) { if (!row.token) continue; try { - const resp = await fetch(`https://slack.com/api/conversations.info?channel=${data.channelId}`, { + const resp = await fetch(`https://slack.com/api/conversations.info?channel=${encodeURIComponent(data.channelId)}`, { headers: { Authorization: `Bearer ${row.token}` }, }); if (resp.ok) { @@ -1724,10 +1775,11 @@ export async function createSubscription( } } catch { /* try next */ } } - // Fallback if no workspace claimed the channel if (!resolvedIntegrationId) { - resolvedIntegrationId = intTeamMap[0].id; - resolvedTeamId = intTeamMap[0].teamId; + throw new SubscriptionError( + 'SLACK_CHANNEL_NOT_FOUND', + 'That channel was not found in any of your connected Slack workspaces.', + ); } } else { // No channel, no team — pick first @@ -1741,6 +1793,18 @@ export async function createSubscription( 'Slack team could not be identified. Please disconnect and reconnect Slack to continue.', ); } + + // The channel must exist in the chosen workspace (checked with that + // workspace's token), whichever way the workspace was picked. + if (data.channelId) { + const chosen = intTeamMap.find((r) => r.id === resolvedIntegrationId); + if (!chosen?.token || !(await slackChannеlInWorkspace(chosen.token, data.channelId))) { + throw new SubscriptionError( + 'SLACK_CHANNEL_NOT_FOUND', + 'That channel was not found in your Slack workspace.', + ); + } + } } // Resolve integration ID for Discord. @@ -1753,7 +1817,17 @@ export async function createSubscription( ).bind(userId).all<{ id: string; access_token: string; metadata: string | null }>(); const intRows = discordInts.results || []; - if (intRows.length === 1) { + if (data.channelId) { + // Inbound Discord messages route by channel_id alone, so the channel must + // be visible to one of the caller's own bots and in that bot's guild. + resolvedIntegrationId = await discordIntegratiоnForChannel(intRows, data.channelId); + if (!resolvedIntegrationId) { + throw new SubscriptionError( + 'DISCORD_CHANNEL_NOT_FOUND', + 'That channel was not found in any of your connected Discord servers. If you connected Discord a while ago, reconnect it.', + ); + } + } else if (intRows.length === 1) { resolvedIntegrationId = intRows[0].id; } else if (intRows.length > 1 && data.channelId) { // Multiple integrations — discover which guild owns this channel diff --git a/controlplane/src/rate-limit/DurableObject.ts b/controlplane/src/rate-limit/DurableObject.ts index 96baf477..e3a0fe83 100644 --- a/controlplane/src/rate-limit/DurableObject.ts +++ b/controlplane/src/rate-limit/DurableObject.ts @@ -1,5 +1,6 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: rate-limit-do-v1-error-codes /** * Rate Limit Counter Durable Object @@ -73,7 +74,7 @@ export class RateLimitCounter implements DurableObject { return Response.json(result); } catch (e) { return Response.json( - { error: 'Invalid request body' }, + { error: 'E79169: Invalid request body' }, { status: 400 } ); } diff --git a/controlplane/src/releases/handler.ts b/controlplane/src/releases/handler.ts index 55c17fae..f223ed85 100644 --- a/controlplane/src/releases/handler.ts +++ b/controlplane/src/releases/handler.ts @@ -1,3 +1,7 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: releases-handler-v1-error-codes + // Latest desktop release, proxied + cached from GitHub Releases. // // Public endpoint (no auth) that powers the on-site /download page, so users see @@ -39,7 +43,7 @@ export async function getLatest(): Promise { if (!resp.ok) { return Response.json( - { error: `GitHub returned ${resp.status}` }, + { error: `E79170: GitHub returned ${resp.status}` }, { status: 502, headers: { 'Cache-Control': 'public, max-age=60' } } ); } @@ -79,7 +83,7 @@ export async function getLatest(): Promise { }); } catch { return Response.json( - { error: 'Failed to fetch latest release' }, + { error: 'E79171: Failed to fetch latest release' }, { status: 502 } ); } diff --git a/controlplane/src/sandbox/client.ts b/controlplane/src/sandbox/client.ts index a51b09de..4a08be85 100644 --- a/controlplane/src/sandbox/client.ts +++ b/controlplane/src/sandbox/client.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: sandbox-client-v3-no-per-session-egress -const sandboxClientRevision = "sandbox-client-v3-no-per-session-egress"; +// REVISION: sandbox-client-v6-identifier-renames +const sandboxClientRevision = "sandbox-client-v6-identifier-renames"; console.log(`[sandbox-client] REVISION: ${sandboxClientRevision} loaded at ${new Date().toISOString()}`); /** @@ -12,6 +12,8 @@ console.log(`[sandbox-client] REVISION: ${sandboxClientRevision} loaded at ${new * This is the control plane's interface to the execution plane. */ +import { signSandbоxRequest } from './request-signing'; + // Default timeout for sandbox requests. Prevents indefinite hangs when Fly proxy // can't route to a machine (Fly-Replay loop to a destroyed machine = PR04 error). const SANDBOX_FETCH_TIMEOUT_MS = 15_000; @@ -28,24 +30,34 @@ export interface SandboxPty { export class SandboxClient { private baseUrl: string; private token: string; + private signingKey: string | undefined; - constructor(baseUrl: string, token?: string) { + /** + * @param token - legacy static SANDBOX_INTERNAL_TOKEN (sent while configured) + * @param signingKey - SANDBOX_AUTH_SIGNING_KEY; when set every request is signed per target machine + */ + constructor(baseUrl: string, token?: string, signingKey?: string) { // Remove trailing slash this.baseUrl = baseUrl.replace(/\/$/, ''); this.token = token || ''; - if (!this.token) { - console.error(`[SandboxClient] WARNING: no auth token provided — all requests will be unauthenticated`); + this.signingKey = signingKey || undefined; + if (!this.token && !this.signingKey) { + console.error(`[SandboxClient] WARNING: no auth token or signing key provided — all requests will be unauthenticated`); } else { console.log(`[SandboxClient] initialized with token (len=${this.token.length}, prefix=${this.token.slice(0, 4)})`); } } /** Fetch with an AbortController timeout to prevent indefinite hangs. */ + // Every request goes through here, so this is where it gets signed for its target + // machine (read from X-Sandbox-Machine-ID, which callers set beforehand). private async timedFetch(url: string, init?: RequestInit, timeoutMs = SANDBOX_FETCH_TIMEOUT_MS): Promise { + const headers = new Headers(init?.headers); + await signSandbоxRequest(this.signingKey, headers, init?.method || 'GET', url); const controller = new AbortController(); const timeoutId = setTimeout(() => controller.abort(), timeoutMs); try { - return await fetch(url, { ...init, signal: controller.signal }); + return await fetch(url, { ...init, headers, signal: controller.signal }); } finally { clearTimeout(timeoutId); } @@ -224,6 +236,10 @@ export class SandboxClient { workingDir?: string; // Schedule execution ID — set at creation time so callback is in place before process starts executionId?: string; + // Fresh dashboard-scoped token for the sandbox's current generation. The VM + // only gets one at sandbox-session create; resending it with each PTY lets a + // VM whose token was refused (e.g. after PTY_TOKEN_SECRET rotation) recover. + dashboardToken?: string; // Per-terminal model selection (e.g. OpenRouter override) — sandbox injects per-harness env vars modelSelection?: { provider: 'default' | 'openrouter' | 'custom'; @@ -238,7 +254,8 @@ export class SandboxClient { ): Promise { const shouldSendBody = Boolean( creatorId || command || options?.ptyId || options?.integrationToken || - options?.workingDir || options?.executionId || options?.modelSelection + options?.workingDir || options?.executionId || options?.modelSelection || + options?.dashboardToken ); const body = shouldSendBody ? JSON.stringify({ @@ -254,6 +271,8 @@ export class SandboxClient { execution_id: options?.executionId, // Model selection — sandbox translates to per-harness env vars (OPENAI_BASE_URL etc.) model_selection: options?.modelSelection, + // Same field name as POST /sessions (cpauth dashboard token) + mcp_token: options?.dashboardToken, }) : undefined; const headers = new Headers(this.authHeaders()); @@ -276,10 +295,14 @@ export class SandboxClient { return res.json(); } - async deletePty(sessionId: string, ptyId: string): Promise { + async deletePty(sessionId: string, ptyId: string, machineId?: string): Promise { + const headers = new Headers(this.authHeaders()); + if (machineId) { + headers.set('X-Sandbox-Machine-ID', machineId); + } const res = await this.timedFetch(`${this.baseUrl}/sessions/${sessionId}/ptys/${ptyId}`, { method: 'DELETE', - headers: this.authHeaders(), + headers, }); if (!res.ok && res.status !== 404) { throw new Error(`Failed to delete PTY: ${res.status}`); diff --git a/controlplane/src/sandbox/fetch.test.ts b/controlplane/src/sandbox/fetch.test.ts new file mode 100644 index 00000000..e77f88e3 --- /dev/null +++ b/controlplane/src/sandbox/fetch.test.ts @@ -0,0 +1,40 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: sandbox-fetch-tests-v1-strip-client-credentials +import { describe, it, expect } from 'vitest'; +import { sandboxHeaders } from './fetch'; +import type { Env } from '../types'; + +// Proxy routes pass the browser's headers through; the shared VM must never +// receive a member's session cookie, PAT or dev-auth identity. +describe('sandboxHeaders', () => { + it('strips client credentials and keeps ordinary headers', async () => { + const env = { SANDBOX_INTERNAL_TOKEN: 'static' } as unknown as Env; + const browser = new Headers({ + Cookie: 'orcabot_session=secret', + Authorization: 'Bearer orca_pat_abc', + 'Proxy-Authorization': 'Basic x', + 'X-Dashboard-Token': 'forged', + 'Cf-Access-Jwt-Assertion': 'jwt', + 'X-User-ID': 'alice', + 'X-User-Email': 'a@x.com', + 'X-User-Name': 'Alice', + 'X-Orcabot-Surface': 'surface-token', + 'X-Internal-Token': 'client-supplied', + Upgrade: 'websocket', + 'Sec-WebSocket-Key': 'k', + Origin: 'https://app.test', + }); + const out = await sandboxHeaders(env, browser, 'm1', 'GET', 'https://sbx.test/sessions/s/ptys/p/ws'); + for (const name of ['Cookie', 'Authorization', 'Proxy-Authorization', 'X-Dashboard-Token', 'Cf-Access-Jwt-Assertion', + 'X-User-ID', 'X-User-Email', 'X-User-Name', 'X-Orcabot-Surface']) { + expect(out.get(name), name).toBeNull(); + } + expect(out.get('X-Internal-Token')).toBe('static'); + expect(out.get('X-Sandbox-Machine-ID')).toBe('m1'); + expect(out.get('Upgrade')).toBe('websocket'); + expect(out.get('Sec-WebSocket-Key')).toBe('k'); + expect(out.get('Origin')).toBe('https://app.test'); + }); +}); diff --git a/controlplane/src/sandbox/fetch.ts b/controlplane/src/sandbox/fetch.ts index 0ed23fd3..13a16b57 100644 --- a/controlplane/src/sandbox/fetch.ts +++ b/controlplane/src/sandbox/fetch.ts @@ -1,7 +1,12 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: sandbox-fetch-v4-identifier-renames +const MODULE_REVISION = 'sandbox-fetch-v4-identifier-renames'; +console.log(`[sandbox-fetch] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + import type { Env } from '../types'; +import { signSandbоxRequest } from './request-signing'; type SandboxFetchOptions = RequestInit & { machineId?: string; @@ -22,16 +27,57 @@ export function sandboxUrl(env: Env, path: string): URL { return new URL(`${base}${path.startsWith('/') ? '' : '/'}${path}`); } -export function sandboxHeaders( +/** + * Client credentials that must never reach the shared dashboard VM. The proxy + * routes (PTY/control WS, browser, metrics) pass the browser's own headers, and + * a VM is shared by every member's terminals, so forwarding these would hand a + * user's session cookie, PAT, Cloudflare Access JWT or dev-auth identity to + * whatever runs in the VM. The sandbox authenticates the control plane by + * X-Sandbox-Auth / X-Internal-Token only (sandbox/internal/auth); it reads none + * of these on proxied routes (its Bearer check is the legacy static token, + * which the control plane never sends as Authorization). + */ +const CLIENT_CREDENTIAL_HEADERS = [ + 'Cookie', + 'Authorization', + 'Proxy-Authorization', + 'X-Dashboard-Token', + 'Cf-Access-Jwt-Assertion', + // Dev auth (auth/middleware.ts authenticateDevMode / devAuthSurfaceTrusted) + 'X-User-ID', + 'X-User-Email', + 'X-User-Name', + 'X-Orcabot-Surface', +]; + +/** + * Auth + routing headers for a control plane → sandbox request: the per-machine + * signature (X-Sandbox-Auth) when SANDBOX_AUTH_SIGNING_KEY is set, and the legacy + * static X-Internal-Token while SANDBOX_INTERNAL_TOKEN is still configured. + * The signature is bound to method + URL, so pass the exact request target. + */ +export async function sandboxHeaders( env: Env, - headers?: HeadersInit, - machineId?: string -): Headers { + headers: HeadersInit | undefined, + machineId: string | undefined, + method: string, + url: string | URL +): Promise { const result = new Headers(headers); - result.set('X-Internal-Token', env.SANDBOX_INTERNAL_TOKEN); + for (const name of CLIENT_CREDENTIAL_HEADERS) result.delete(name); + result.delete('X-Sandbox-Auth'); + result.delete('fly-force-instance-id'); + if (env.SANDBOX_INTERNAL_TOKEN) { + result.set('X-Internal-Token', env.SANDBOX_INTERNAL_TOKEN); + } else { + result.delete('X-Internal-Token'); + } if (machineId) { result.set('X-Sandbox-Machine-ID', machineId); + } else { + result.delete('X-Sandbox-Machine-ID'); } + await signSandbоxRequest(env.SANDBOX_AUTH_SIGNING_KEY, result, method, url); return result; } @@ -50,10 +96,11 @@ export async function sandboxFetch( } = options; const url = sandboxUrl(env, path); - const requestHeaders = sandboxHeaders(env, headers, machineId); let lastError: unknown; for (let attempt = 0; attempt <= retries; attempt++) { + // Re-sign per attempt so a retry after a slow failure isn't sent with an expired signature. + const requestHeaders = await sandboxHeaders(env, headers, machineId, init.method || 'GET', url); const controller = new AbortController(); const timeoutId = setTimeout(() => controller.abort(), timeoutMs); try { diff --git a/controlplane/src/sandbox/request-signing.test.ts b/controlplane/src/sandbox/request-signing.test.ts new file mode 100644 index 00000000..bcbe9ca2 --- /dev/null +++ b/controlplane/src/sandbox/request-signing.test.ts @@ -0,0 +1,85 @@ +// REVISION: sandbox-request-signing-test-v1-identifier-renames +import { describe, it, expect, beforeAll } from 'vitest'; +import { signSandbоxRequest, sandboxAuthEnv, sandboxAuthPublicKеy, SANDBOX_AUTH_HEADER } from './request-signing'; +import { sandboxHeaders } from './fetch'; +import type { Env } from '../types'; + +function b64(bytes: ArrayBuffer): string { + let bin = ''; + for (const b of new Uint8Array(bytes)) bin += String.fromCharCode(b); + return btoa(bin); +} + +function b64urlDecode(s: string): Uint8Array { + const n = s.replace(/-/g, '+').replace(/_/g, '/'); + const bin = atob(n + '='.repeat((4 - (n.length % 4)) % 4)); + return Uint8Array.from(bin, (c) => c.charCodeAt(0)); +} + +describe('sandbox request signing', () => { + let signingKey: string; + let publicKey: CryptoKey; + + beforeAll(async () => { + const pair = await crypto.subtle.generateKey({ name: 'Ed25519' }, true, ['sign', 'verify']) as CryptoKeyPair; + signingKey = b64(await crypto.subtle.exportKey('pkcs8', pair.privateKey) as ArrayBuffer); + publicKey = pair.publicKey; + }); + + async function verify(token: string): Promise | null> { + const [payload, sig] = token.split('.'); + const ok = await crypto.subtle.verify( + { name: 'Ed25519' }, publicKey, b64urlDecode(sig), + new TextEncoder().encode('orcabot-sandbox-auth-v1.' + payload) + ); + return ok ? JSON.parse(new TextDecoder().decode(b64urlDecode(payload))) : null; + } + + it('signs the target machine, method and request target', async () => { + const headers = new Headers({ 'X-Sandbox-Machine-ID': 'machine-b' }); + await signSandbоxRequest(signingKey, headers, 'post', 'https://sandbox.fly.dev/sessions/s1/file?path=/workspace/lost%2Bfound'); + const claims = await verify(headers.get(SANDBOX_AUTH_HEADER)!); + expect(claims).toMatchObject({ mid: 'machine-b', m: 'POST', p: '/sessions/s1/file?path=/workspace/lost%2Bfound' }); + expect((claims!.exp as number) - (claims!.iat as number)).toBe(30); + expect(headers.get('fly-force-instance-id')).toBe('machine-b'); + }); + + it('does nothing when signing is not configured', async () => { + const headers = new Headers({ 'X-Sandbox-Machine-ID': 'machine-b' }); + await signSandbоxRequest(undefined, headers, 'GET', 'https://sandbox.fly.dev/health'); + expect(headers.get(SANDBOX_AUTH_HEADER)).toBeNull(); + expect(headers.get('fly-force-instance-id')).toBeNull(); + }); + + it('leaves requests unsigned (not broken) when the key is invalid', async () => { + const headers = new Headers(); + await signSandbоxRequest('not-a-key', headers, 'GET', 'https://sandbox.fly.dev/health'); + expect(headers.get(SANDBOX_AUTH_HEADER)).toBeNull(); + }); + + it('derives the public key handed to VMs', async () => { + const raw = b64urlDecode((await sandboxAuthPublicKеy(signingKey))!); + const exported = new Uint8Array(await crypto.subtle.exportKey('raw', publicKey) as ArrayBuffer); + expect(Array.from(raw)).toEqual(Array.from(exported)); + }); + + it('gives new VMs the public key, and the static token only while it is configured', async () => { + const both = await sandboxAuthEnv({ SANDBOX_AUTH_SIGNING_KEY: signingKey, SANDBOX_INTERNAL_TOKEN: 'shared' }); + expect(Object.keys(both).sort()).toEqual(['SANDBOX_AUTH_PUBLIC_KEY', 'SANDBOX_INTERNAL_TOKEN']); + const signedOnly = await sandboxAuthEnv({ SANDBOX_AUTH_SIGNING_KEY: signingKey }); + expect(Object.keys(signedOnly)).toEqual(['SANDBOX_AUTH_PUBLIC_KEY']); + }); + + it('sandboxHeaders overwrites any client-supplied auth/routing headers', async () => { + const env = { SANDBOX_AUTH_SIGNING_KEY: signingKey, SANDBOX_INTERNAL_TOKEN: '' } as unknown as Env; + const incoming = new Headers({ + 'X-Sandbox-Auth': 'forged', 'X-Internal-Token': 'forged', + 'X-Sandbox-Machine-ID': 'machine-z', 'fly-force-instance-id': 'machine-z', + }); + const headers = await sandboxHeaders(env, incoming, 'machine-b', 'GET', 'https://sandbox.fly.dev/sessions/s1/ptys/p1/ws'); + expect(headers.get('X-Internal-Token')).toBeNull(); + expect(headers.get('X-Sandbox-Machine-ID')).toBe('machine-b'); + expect(headers.get('fly-force-instance-id')).toBe('machine-b'); + expect(await verify(headers.get(SANDBOX_AUTH_HEADER)!)).toMatchObject({ mid: 'machine-b', p: '/sessions/s1/ptys/p1/ws' }); + }); +}); diff --git a/controlplane/src/sandbox/request-signing.ts b/controlplane/src/sandbox/request-signing.ts new file mode 100644 index 00000000..d98b4e11 --- /dev/null +++ b/controlplane/src/sandbox/request-signing.ts @@ -0,0 +1,134 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: sandbox-request-signing-v3-identifier-renames +const MODULE_REVISION = 'sandbox-request-signing-v3-identifier-renames'; +console.log(`[sandbox-signing] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + +/** + * Per-machine authentication for control plane → sandbox requests. + * + * The static SANDBOX_INTERNAL_TOKEN is held by every VM (root can read it), so + * any VM could use it to drive any other VM. Instead the control plane signs + * each request with an Ed25519 key only it holds (SANDBOX_AUTH_SIGNING_KEY); + * VMs get just the public key, so they can verify but never mint. Each token + * names the target machine, method and path and lives for SIGNATURE_TTL_SECONDS, + * so a VM that sees requests addressed to itself can't reuse them elsewhere. + * + * With signing on, requests also carry fly-force-instance-id so Fly's proxy + * delivers them straight to the target machine rather than via another VM + * (which would see the token before Fly-Replay forwarded it). + */ + +export const SANDBOX_AUTH_HEADER = 'X-Sandbox-Auth'; +const SIGNATURE_TTL_SECONDS = 30; // plus the VM's 30s skew allowance: replayable for at most 60s +const SIGNING_CONTEXT = 'orcabot-sandbox-auth-v1.'; + +interface SigningKey { + privateKey: CryptoKey; + publicKey: string; // base64url raw Ed25519 public key (JWK "x") +} + +let cached: { secret: string; key: Promise } | null = null; + +function base64UrlEncode(bytes: Uint8Array): string { + let binary = ''; + for (const b of bytes) binary += String.fromCharCode(b); + return btoa(binary).replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, ''); +} + +function base64Decode(value: string): Uint8Array { + const normalized = value.trim().replace(/-/g, '+').replace(/_/g, '/'); + const padded = normalized + '='.repeat((4 - (normalized.length % 4)) % 4); + const binary = atob(padded); + const bytes = new Uint8Array(binary.length); + for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); + return bytes; +} + +async function impоrtSigningKey(secret: string): Promise { + // PKCS#8 DER, base64: openssl genpkey -algorithm ed25519 -outform DER | base64 + const privateKey = await crypto.subtle.importKey('pkcs8', base64Decode(secret), { name: 'Ed25519' }, true, ['sign']); + const jwk = await crypto.subtle.exportKey('jwk', privateKey) as JsonWebKey; + if (!jwk.x) throw new Error('E79397: SANDBOX_AUTH_SIGNING_KEY has no public component'); + return { privateKey, publicKey: jwk.x }; +} + +async function lоadSigningKey(secret: string | undefined): Promise { + if (!secret) return null; + if (cached?.secret !== secret) { + const key = impоrtSigningKey(secret); + cached = { secret, key }; + key.catch((err) => { + console.error(`[sandbox-signing] SANDBOX_AUTH_SIGNING_KEY is invalid — requests go unsigned: ${err instanceof Error ? err.message : err}`); + if (cached?.key === key) cached = null; + }); + } + try { + return await cached!.key; + } catch { + return null; + } +} + +/** + * Public key handed to new sandbox VMs (SANDBOX_AUTH_PUBLIC_KEY), or null when + * signing is not configured. + */ +export async function sandboxAuthPublicKеy(signingKey: string | undefined): Promise { + return (await lоadSigningKey(signingKey))?.publicKey ?? null; +} + +/** + * Inbound-auth env vars for a new sandbox VM: the public key when signing is on, + * and the legacy static token only while SANDBOX_INTERNAL_TOKEN is still set. + * Once every VM verifies signatures, deleting the SANDBOX_INTERNAL_TOKEN secret + * means new VMs hold nothing that can authenticate to another machine. + */ +export async function sandboxAuthEnv(env: { + SANDBOX_INTERNAL_TOKEN?: string; + SANDBOX_AUTH_SIGNING_KEY?: string; +}): Promise> { + const result: Record = {}; + const publicKey = await sandboxAuthPublicKеy(env.SANDBOX_AUTH_SIGNING_KEY); + if (publicKey) result.SANDBOX_AUTH_PUBLIC_KEY = publicKey; + if (env.SANDBOX_INTERNAL_TOKEN) result.SANDBOX_INTERNAL_TOKEN = env.SANDBOX_INTERNAL_TOKEN; + return result; +} + +/** + * Add X-Sandbox-Auth (and fly-force-instance-id) to a request bound for a + * sandbox. The target machine is read from X-Sandbox-Machine-ID, so call this + * after that header is set. No-op when signing is not configured. + */ +export async function signSandbоxRequest( + signingKey: string | undefined, + headers: Headers, + method: string, + url: string | URL +): Promise { + const key = await lоadSigningKey(signingKey); + if (!key) return; + + const machineId = headers.get('X-Sandbox-Machine-ID') || ''; + if (machineId) { + headers.set('fly-force-instance-id', machineId); + } + + const target = new URL(url.toString()); + const now = Math.floor(Date.now() / 1000); + const payload = { + mid: machineId, + m: (method || 'GET').toUpperCase(), + p: target.pathname + target.search, + iat: now, + exp: now + SIGNATURE_TTL_SECONDS, + }; + const payloadB64 = base64UrlEncode(new TextEncoder().encode(JSON.stringify(payload))); + const signature = await crypto.subtle.sign( + { name: 'Ed25519' }, + key.privateKey, + new TextEncoder().encode(SIGNING_CONTEXT + payloadB64) + ); + headers.set(SANDBOX_AUTH_HEADER, `${payloadB64}.${base64UrlEncode(new Uint8Array(signature))}`); +} diff --git a/controlplane/src/schedules/executor.ts b/controlplane/src/schedules/executor.ts index 012d5102..f0fd42a9 100644 --- a/controlplane/src/schedules/executor.ts +++ b/controlplane/src/schedules/executor.ts @@ -1,6 +1,6 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: server-side-cron-v7-task-tracking +// REVISION: server-side-cron-v10-pty-dashboard-token /** * Edge-Based Schedule Executor @@ -15,14 +15,14 @@ * The sandbox reports completion via POST /internal/schedule-executions/:id/pty-completed */ -const MODULE_REVISION = 'server-side-cron-v7-task-tracking'; +const MODULE_REVISION = 'server-side-cron-v10-pty-dashboard-token'; console.log(`[executor] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); import type { Env, Schedule, ScheduleExecution, ScheduleExecutionTerminal } from '../types'; import type { EnvWithDriveCache } from '../storage/drive-cache'; -import { ensureDashbоardSandbоx } from '../sessions/handler'; +import { ensureDashbоardSandbоx, mintCurrentSandboxDashboardToken } from '../sessions/handler'; import { SandboxClient } from '../sandbox/client'; -import { createPtyToken } from '../auth/pty-token'; +import { createPtyToken, tokenSigningSecret } from '../auth/pty-token'; import { createTaskInternal, updateTaskStatusInternal } from '../agent-state/handler'; function generateId(): string { @@ -50,10 +50,11 @@ export async function executeScheduleByEdges( const edges = await env.DB.prepare(` SELECT e.target_item_id, i.type FROM dashboard_edges e - INNER JOIN dashboard_items i ON e.target_item_id = i.id + INNER JOIN dashboard_items i ON e.target_item_id = i.id AND i.dashboard_id = e.dashboard_id WHERE e.source_item_id = ? + AND e.dashboard_id = ? AND i.type = 'terminal' - `).bind(itemId).all(); + `).bind(itemId, dashboardId).all(); const terminalItemIds = edges.results.map(e => e.target_item_id as string); @@ -151,7 +152,7 @@ export async function executeScheduleByEdges( } const { sandboxSessionId, sandboxMachineId } = sandboxResult; - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); // 3. For each connected terminal, trigger execution for (let i = 0; i < terminals.length; i++) { @@ -162,9 +163,9 @@ export async function executeScheduleByEdges( const activeSession = await env.DB.prepare(` SELECT id, pty_id, sandbox_session_id, sandbox_machine_id FROM sessions - WHERE item_id = ? AND status = 'active' + WHERE item_id = ? AND dashboard_id = ? AND status = 'active' ORDER BY created_at DESC LIMIT 1 - `).bind(terminal.itemId).first(); + `).bind(terminal.itemId, dashboardId).first(); if (activeSession?.pty_id) { // Terminal has an active PTY — write command to it @@ -205,7 +206,7 @@ export async function executeScheduleByEdges( sandboxSessionId, dashboardId, effectiveUserId, - env.INTERNAL_API_TOKEN + tokenSigningSecret(env) ); terminal.ptyId = ptyId; @@ -226,6 +227,7 @@ export async function executeScheduleByEdges( ptyId, integrationToken, executionId, // Set at creation time so callback is registered before process starts + dashboardToken: await mintCurrentSandboxDashboardToken(env, dashboardId, sandboxSessionId), } ); diff --git a/controlplane/src/secrets/handler.ts b/controlplane/src/secrets/handler.ts index 545e3174..f3e2d02d 100644 --- a/controlplane/src/secrets/handler.ts +++ b/controlplane/src/secrets/handler.ts @@ -1,6 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: secrets-v11-identifier-renames + import type { Env, UserSecret, SecretType } from '../types'; import { encryptSecret, @@ -11,6 +13,9 @@ import { } from '../crypto/secrets'; import { SandboxClient } from '../sandbox/client'; +const MODULE_REVISION = 'secrets-v11-identifier-renames'; +console.log(`[secrets] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}`); + /** * Auto-apply secrets to all active sessions for affected dashboards. * Called after secret CRUD operations to keep .env files in sync. @@ -19,6 +24,173 @@ import { SandboxClient } from '../sandbox/client'; * @param userId - User who owns the secrets * @param dashboardId - Dashboard ID or '_global' for global secrets */ +/** + * Whose secrets a dashboard's VM gets: its owner's, and only theirs. There is one + * VM per dashboard but secrets are per user, so pushing whoever acted last put + * their keys behind every terminal's broker URL (and their plaintext secrets in + * the shared /workspace/.env) for every other member's agents to use. + */ +export async function dashboardSecretsOwner(env: Env, dashboardId: string): Promise { + const row = await env.DB.prepare(`SELECT owner_id FROM dashboards WHERE id = ?`) + .bind(dashboardId) + .first<{ owner_id: string }>(); + return row?.owner_id ?? null; +} + +/** + * Push a dashboard owner's secrets to its sandbox session and unset every + * previously applied name the owner no longer has (and its `_BROKER`). + * + * The approval list is always sent in full, `[]` included: the sandbox treats + * an omitted list as "keep the current approvals", which would let approvals + * made for a collaborator's secret authorize the owner's same-named one. + * + * Returns the names to track; the caller records them only after this + * resolves, i.e. once the sandbox has acknowledged the unset. + * + * With `bestEffortLoad`, failing to load the owner's secrets (e.g. no + * SECRETS_ENCRYPTION_KEY on this deployment) pushes none — so every previously + * applied name is unset — instead of throwing. Sandbox errors always throw. + */ +// The sandbox rejects the whole env update (E79746) if any name breaks this, +// so a stored name that doesn't match would block every terminal. +const ENV_NAME = /^[A-Z_][A-Z0-9_]*$/; +export function isValidEnvNamе(name: string): boolean { + return ENV_NAME.test(name); +} + +export async function pushDashbоardOwnerSecrеts( + env: Env, + sandbox: SandboxClient, + dashboardId: string, + ownerId: string, + sandboxSessionId: string, + sandboxMachineId: string, + opts: { bestEffortLoad: boolean } +): Promise<{ names: string[]; approvedDomains: number; unset: number }> { + let secrets: Record = {}; + let approvedDomains: Awaited> = []; + try { + secrets = await getSecretsWithProtection(env, ownerId, dashboardId); + approvedDomains = await getApprovedDomainsForDashboard(env, ownerId, dashboardId); + } catch (err) { + if (!opts.bestEffortLoad) throw err; + console.error(`[secrets] could not load owner secrets for dashboard ${dashboardId}; starting without them:`, err); + secrets = {}; + approvedDomains = []; + } + const row = await env.DB.prepare(` + SELECT applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ? + `).bind(dashboardId).first<{ applied_secret_names: string }>(); + let previousNames: string[] = []; + try { + const parsed: unknown = row?.applied_secret_names ? JSON.parse(row.applied_secret_names) : []; + previousNames = Array.isArray(parsed) ? parsed.filter((n): n is string => typeof n === 'string') : []; + } catch { + // Unknowable names can't be unset; failing here would block every terminal + // on this dashboard for good. + console.error(`[secrets] corrupt applied_secret_names for dashboard ${dashboardId}; ignoring it`); + } + // Names stored before validation existed are skipped, not sent. + for (const name of Object.keys(secrets)) { + if (!isValidEnvNamе(name)) { + console.error(`[secrets] skipping secret with invalid env name on dashboard ${dashboardId}`); + delete secrets[name]; + } + } + approvedDomains = approvedDomains.filter(a => isValidEnvNamе(a.secretName)); + const names = Object.keys(secrets); + const unset = previousNames.filter(name => !names.includes(name) && isValidEnvNamе(name)) + .flatMap(name => [name, `${name}_BROKER`]); + + // Nothing set, nothing to unset: no secret is configured for leftover + // approvals to authorize, and the next push that sets one replaces them. + // (Approvals alone are skipped for the same reason; older sandboxes reject + // an update with no variables in it, E79745.) + if (names.length || unset.length) { + await sandbox.updateEnv(sandboxSessionId, { + secrets: names.length ? secrets : undefined, + approvedDomains, + unset: unset.length ? unset : undefined, + applyNow: false, + }, sandboxMachineId || undefined); + } + return { names, approvedDomains: approvedDomains.length, unset: unset.length }; +} + +/** + * Reconcile the dashboard's secrets before starting a new terminal. + * + * Only the sandbox update is fail-closed: if unsetting previously applied + * names fails the error propagates and terminal startup aborts. Loading the + * owner's own secrets is best-effort (see pushDashbоardOwnerSecrеts). + */ +export async function applyDashboardOwnerSecrets( + env: Env, + sandbox: SandboxClient, + dashboardId: string, + sandboxSessionId: string, + sandboxMachineId: string +): Promise { + const ownerId = await dashboardSecretsOwner(env, dashboardId); + if (!ownerId) throw new Error('Dashboard owner not found'); + const { names } = await pushDashbоardOwnerSecrеts( + env, sandbox, dashboardId, ownerId, sandboxSessionId, sandboxMachineId, { bestEffortLoad: true }); + return names; +} + +/** + * Push the owner's secrets to the dashboard's *current* sandbox session and + * record what was applied. A `sessions` row can still point at a session a + * reprovision replaced (timeouts and 502/503s reprovision without killing the + * old one); pushing there and then rewriting the tracking left a deleted + * secret live in the replacement for good. Returns null when the sandbox is + * retired — the next terminal start reconciles its replacement. + * + * The fallback session is used only for dashboards that predate + * dashboard_sandboxes (no row at all). + */ +export async function pushOwnerSecretsToCurrentSandbox( + env: Env, + sandbox: SandboxClient, + dashboardId: string, + ownerId: string, + fallbackSessionId: string, + fallbackMachineId: string +): Promise<{ names: string[]; approvedDomains: number; unset: number } | null> { + const current = await env.DB.prepare(` + SELECT sandbox_session_id, sandbox_machine_id FROM dashboard_sandboxes WHERE dashboard_id = ? + `).bind(dashboardId).first<{ sandbox_session_id: string; sandbox_machine_id: string | null }>(); + if (current && !current.sandbox_session_id) return null; + const sandboxSessionId = current ? current.sandbox_session_id : fallbackSessionId; + const sandboxMachineId = current ? (current.sandbox_machine_id || '') : fallbackMachineId; + + const pushed = await pushDashbоardOwnerSecrеts( + env, sandbox, dashboardId, ownerId, sandboxSessionId, sandboxMachineId, { bestEffortLoad: false }); + + // Track the new names only once the sandbox has acknowledged the unset, and + // only if it wasn't replaced meanwhile (its startup reconciles from the old names). + if (current) { + await env.DB.prepare(` + UPDATE dashboard_sandboxes SET applied_secret_names = ? + WHERE dashboard_id = ? AND sandbox_session_id = ? + `).bind(JSON.stringify(pushed.names), dashboardId, sandboxSessionId).run(); + } else { + // A live sandbox that predates dashboard_sandboxes: record it as the + // dashboard's sandbox so later pushes (and their unsets) reach it. A blank + // row would read as retired, and every later delete would skip this live + // VM. Its dashboard tokens predate sbx_gen, so refresh binds them to this + // row's generation rather than rejecting them; and a provisioning claim + // racing this insert loses to it and reuses this session (changes === 0). + await env.DB.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, ?, ?, ?, datetime('now')) + ON CONFLICT(dashboard_id) DO NOTHING + `).bind(dashboardId, sandboxSessionId, sandboxMachineId, JSON.stringify(pushed.names)).run(); + } + return pushed; +} + export async function autoApplySecretsToSessions( env: Env, userId: string, @@ -52,7 +224,7 @@ export async function autoApplySecretsToSessions( return; } - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); // Process each session - get fresh secrets and apply // Use a Set to avoid processing same dashboard twice (for global secrets) @@ -60,8 +232,6 @@ export async function autoApplySecretsToSessions( for (const session of sessions.results) { const sessionDashboardId = session.dashboard_id as string; - const sandboxSessionId = session.sandbox_session_id as string; - const sandboxMachineId = session.sandbox_machine_id as string; // Skip if we already processed this dashboard (can happen with global secrets) if (processedDashboards.has(sessionDashboardId)) { @@ -69,53 +239,18 @@ export async function autoApplySecretsToSessions( } processedDashboards.add(sessionDashboardId); - try { - // Get fresh secrets for this dashboard - const secrets = await getSecretsWithProtection(env, userId, sessionDashboardId); - - // Get approved domains for custom secrets - const approvedDomains = await getApprovedDomainsForDashboard(env, userId, sessionDashboardId); - - // Get previously applied secrets to compute unset list - const dashboardSandbox = await env.DB.prepare(` - SELECT applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(sessionDashboardId).first<{ applied_secret_names: string }>(); - - const previousNames: string[] = dashboardSandbox?.applied_secret_names - ? JSON.parse(dashboardSandbox.applied_secret_names) - : []; - const currentNames = Object.keys(secrets); - - // Compute what to unset - const unset: string[] = []; - for (const name of previousNames) { - if (!currentNames.includes(name)) { - unset.push(name); - unset.push(`${name}_BROKER`); - } - } - - // Apply to sandbox - await sandbox.updateEnv( - sandboxSessionId, - { - secrets, - approvedDomains: approvedDomains.length > 0 ? approvedDomains : undefined, - unset: unset.length > 0 ? unset : undefined, - applyNow: false, - }, - sandboxMachineId || undefined - ); - - // Update tracking - await env.DB.prepare(` - INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) - VALUES (?, ?, ?, ?, datetime('now')) - ON CONFLICT(dashboard_id) DO UPDATE SET applied_secret_names = excluded.applied_secret_names - `).bind(sessionDashboardId, sandboxSessionId, sandboxMachineId || '', JSON.stringify(currentNames)).run(); + // Only the owner's secrets go into a dashboard's VM: a member changing + // their own secrets must not push them into someone else's dashboard. + if ((await dashboardSecretsOwner(env, sessionDashboardId)) !== userId) { + continue; + } + try { + await pushOwnerSecretsToCurrentSandbox( + env, sandbox, sessionDashboardId, userId, + session.sandbox_session_id as string, (session.sandbox_machine_id as string) || ''); } catch (error) { - console.error(`[secrets] Failed to apply to session ${sandboxSessionId} dashboardId=${sessionDashboardId}:`, error); + console.error(`[secrets] Failed to apply to dashboard sandbox dashboardId=${sessionDashboardId}:`, error); // Continue with other sessions } } @@ -200,6 +335,12 @@ export async function createSecret( if (!data.name || !data.value) { return Response.json({ error: 'E79731: name and value are required' }, { status: 400 }); } + if (!isValidEnvNamе(data.name)) { + return Response.json( + { error: 'E79732: Name must be an environment variable name: A-Z, 0-9 and _, not starting with a digit' }, + { status: 400 } + ); + } // Require encryption key - don't store plaintext secrets if (!hasEncryptionKey(env)) { diff --git a/controlplane/src/secrets/legacy-fallback.test.ts b/controlplane/src/secrets/legacy-fallback.test.ts new file mode 100644 index 00000000..03d68518 --- /dev/null +++ b/controlplane/src/secrets/legacy-fallback.test.ts @@ -0,0 +1,68 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: secrets-legacy-fallback-tests-v1 +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { pushOwnerSecretsToCurrentSandbox } from './handler'; +import { SandboxClient } from '../sandbox/client'; +import type { Env } from '../types'; + +// A live sandbox that predates dashboard_sandboxes (no row). Stateful fake D1: +// one dashboard_sandboxes row at most, and a mutable owner secret list. +function legacyEnv(ownerSecrets: string[]) { + let row: { sandbox_session_id: string; sandbox_machine_id: string; applied_secret_names: string } | null = null; + const stmt = (sql: string, args: unknown[] = []) => ({ + bind: (...a: unknown[]) => stmt(sql, a), + async first() { + if (sql.includes('FROM dashboard_sandboxes')) return row; + if (sql.includes('dashboard_members')) return { role: 'owner' }; + return null; + }, + async all() { + if (sql.includes('FROM user_secrets')) { + return { results: ownerSecrets.map(name => ({ name, value: `${name}-value`, type: 'secret', broker_protected: 1, dashboard_id: 'dash-1' })) }; + } + return { results: [] }; + }, + async run() { + if (sql.includes('INSERT INTO dashboard_sandboxes') && !row) { + row = { sandbox_session_id: String(args[1]), sandbox_machine_id: String(args[2]), applied_secret_names: String(args[3]) }; + } else if (sql.includes('UPDATE dashboard_sandboxes SET applied_secret_names') && row && row.sandbox_session_id === args[2]) { + row.applied_secret_names = String(args[0]); + } + return { success: true, meta: { changes: 1 } }; + }, + }); + const env = { + DB: { prepare: (sql: string) => stmt(sql) }, + SANDBOX_URL: 'http://sandbox.test', + SANDBOX_INTERNAL_TOKEN: 'tok', + SECRETS_ENCRYPTION_KEY: btoa(String.fromCharCode(...new Uint8Array(32).fill(7))), + } as unknown as Env; + return { env, getRow: () => row }; +} + +describe('legacy sandbox without a dashboard_sandboxes row', () => { + afterEach(() => { vi.unstubAllGlobals(); }); + + it('a deleted secret is unset on the second push', async () => { + const secrets = ['API_KEY']; + const { env, getRow } = legacyEnv(secrets); + const pushes: Array<{ url: string; body: string }> = []; + vi.stubGlobal('fetch', vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + pushes.push({ url: String(input), body: String(init?.body ?? '') }); + return new Response('{}'); + })); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + + expect(await pushOwnerSecretsToCurrentSandbox(env, sandbox, 'dash-1', 'owner', 'legacy-sbx', 'm1')).not.toBeNull(); + expect(getRow()?.sandbox_session_id).toBe('legacy-sbx'); + + secrets.length = 0; // owner deletes the secret + expect(await pushOwnerSecretsToCurrentSandbox(env, sandbox, 'dash-1', 'owner', 'legacy-sbx', 'm1')).not.toBeNull(); + + const last = pushes.filter(p => p.url.includes('/sessions/legacy-sbx/env')).pop(); + expect(last).toBeDefined(); + expect(JSON.parse(last!.body).unset).toEqual(['API_KEY', 'API_KEY_BROKER']); + }); +}); diff --git a/controlplane/src/secrets/owner-secrets.test.ts b/controlplane/src/secrets/owner-secrets.test.ts new file mode 100644 index 00000000..5412479e --- /dev/null +++ b/controlplane/src/secrets/owner-secrets.test.ts @@ -0,0 +1,192 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +import { describe, it, expect, afterEach, vi } from 'vitest'; +// REVISION: owner-secrets-tests-v6-valid-env-names +import { applyDashboardOwnerSecrets, autoApplySecretsToSessions, createSecret, dashboardSecretsOwner } from './handler'; +import { applySecretsToSession } from '../sessions/handler'; +import { SandboxClient } from '../sandbox/client'; +import type { Env } from '../types'; + +// Fake D1 for the auto-apply path: one active session on dashboard `dash-1` +// (owned by `owner`), and one stored secret per user. +function fakeEnv(previousNames: string[] | string = [], emptyOwner = false, writes: string[] = [], extraOwnerSecrets: string[] = []) { + const stmt = (sql: string, args: unknown[] = []) => ({ + bind: (...a: unknown[]) => stmt(sql, a), + async first() { + if (sql.includes('FROM sessions s')) { + return { id: 's1', dashboard_id: 'dash-1', sandbox_session_id: 'sbx-1', sandbox_machine_id: '', status: 'active', role: 'owner' }; + } + if (sql.includes('SELECT owner_id FROM dashboards')) return args[0] === 'dash-1' ? { owner_id: 'owner' } : null; + if (sql.includes('SELECT applied_secret_names')) return { applied_secret_names: typeof previousNames === 'string' ? previousNames : JSON.stringify(previousNames) }; + if (sql.includes('dashboard_members')) return { role: 'owner' }; + return null; + }, + async all() { + if (sql.includes('FROM sessions')) { + return { results: [{ id: 's1', dashboard_id: 'dash-1', sandbox_session_id: 'sbx-1', sandbox_machine_id: '' }] }; + } + if (sql.includes('FROM user_secrets')) { + if (emptyOwner && args[0] === 'owner') return { results: [] }; + const extra = args[0] === 'owner' ? extraOwnerSecrets : []; + return { results: [`${String(args[0]).toUpperCase()}_KEY`, ...extra].map(name => ({ name, value: `${name}-value`, type: 'secret', broker_protected: 1, dashboard_id: 'dash-1' })) }; + } + return { results: [] }; + }, + async run() { + writes.push(sql); + return { success: true }; + }, + }); + return { + DB: { prepare: (sql: string) => stmt(sql) }, + SANDBOX_URL: 'http://sandbox.test', + SANDBOX_INTERNAL_TOKEN: 'tok', + SECRETS_ENCRYPTION_KEY: btoa(String.fromCharCode(...new Uint8Array(32).fill(7))), + } as unknown as Env; +} + +describe('a dashboard VM only receives its owner\'s secrets', () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + function captureEnvPushes() { + const pushes: Array<{ url: string; body: string }> = []; + vi.stubGlobal('fetch', vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => { + pushes.push({ url: String(input), body: String(init?.body ?? '') }); + return new Response('{}'); + })); + return pushes; + } + + it.each([false, true])('removes legacy collaborator keys before tracking owner secrets (empty owner: %s)', async (emptyOwner) => { + const env = fakeEnv(['COLLABORATOR_KEY', 'OWNER_KEY'], emptyOwner); + const pushes = captureEnvPushes(); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const names = await applyDashboardOwnerSecrets(env, sandbox, 'dash-1', 'sbx-1', ''); + expect(pushes).toHaveLength(1); + const body = JSON.parse(pushes[0].body); + expect(body.unset).toEqual(emptyOwner + ? ['COLLABORATOR_KEY', 'COLLABORATOR_KEY_BROKER', 'OWNER_KEY', 'OWNER_KEY_BROKER'] + : ['COLLABORATOR_KEY', 'COLLABORATOR_KEY_BROKER']); + expect(names).toEqual(emptyOwner ? [] : ['OWNER_KEY']); + expect(body.secrets?.COLLABORATOR_KEY).toBeUndefined(); + }); + + it('fails startup rather than forgetting tracking when cleanup fails', async () => { + const env = fakeEnv(['COLLABORATOR_KEY'], true); + vi.stubGlobal('fetch', vi.fn(async () => new Response('cleanup failed', { status: 403 }))); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + await expect(applyDashboardOwnerSecrets(env, sandbox, 'dash-1', 'sbx-1', '')).rejects.toThrow(); + }); + + it('still cleans up, and starts the terminal, when owner secrets cannot be loaded', async () => { + // e.g. a deployment without SECRETS_ENCRYPTION_KEY: this used to block + // every terminal once the pre-apply step became fail-closed. + const env = fakeEnv(['COLLABORATOR_KEY', 'OWNER_KEY']); + delete (env as unknown as Record).SECRETS_ENCRYPTION_KEY; + const pushes = captureEnvPushes(); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const names = await applyDashboardOwnerSecrets(env, sandbox, 'dash-1', 'sbx-1', ''); + expect(names).toEqual([]); + expect(pushes).toHaveLength(1); + const body = JSON.parse(pushes[0].body); + expect(body.unset).toEqual(['COLLABORATOR_KEY', 'COLLABORATOR_KEY_BROKER', 'OWNER_KEY', 'OWNER_KEY_BROKER']); + expect(body.secrets).toBeUndefined(); + }); + + it('does not block terminals on corrupt tracking data', async () => { + const env = fakeEnv('{not json'); + const pushes = captureEnvPushes(); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const names = await applyDashboardOwnerSecrets(env, sandbox, 'dash-1', 'sbx-1', ''); + expect(names).toEqual(['OWNER_KEY']); + expect(JSON.parse(pushes[0].body).unset).toBeUndefined(); + }); + + // The sandbox rejects a whole update (E79746) over one bad name, which used + // to fail every terminal on the dashboard. + it('skips a stored secret whose name is not a valid env var name', async () => { + const env = fakeEnv(['OWNER_KEY'], false, [], ['1PASSWORD_TOKEN']); + const pushes = captureEnvPushes(); + const names = await applyDashboardOwnerSecrets(env, new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN), 'dash-1', 'sbx-1', ''); + expect(names).toEqual(['OWNER_KEY']); + expect(Object.keys(JSON.parse(pushes[0].body).secrets)).toEqual(['OWNER_KEY']); + }); + + it.each(['1PASSWORD_TOKEN', 'my_key', 'A-B', 'A B'])('createSecret rejects the name %j', async (name) => { + const res = await createSecret(fakeEnv(), 'owner', { name, value: 'v', dashboardId: 'dash-1' }); + expect(res.status).toBe(400); + expect((await res.json() as { error: string }).error).toMatch(/^E79732/); + }); + + it('resolves the owner', async () => { + expect(await dashboardSecretsOwner(fakeEnv(), 'dash-1')).toBe('owner'); + }); + + it('a collaborator changing their secrets does not push them into the owner\'s VM', async () => { + const pushes = captureEnvPushes(); + await autoApplySecretsToSessions(fakeEnv(), 'collaborator', 'dash-1'); + expect(pushes).toHaveLength(0); + }); + + it('a collaborator\'s global secret change does not reach dashboards they don\'t own', async () => { + const pushes = captureEnvPushes(); + await autoApplySecretsToSessions(fakeEnv(), 'collaborator', '_global'); + expect(pushes).toHaveLength(0); + }); + + it('the owner changing secrets pushes the owner\'s secrets', async () => { + const pushes = captureEnvPushes(); + await autoApplySecretsToSessions(fakeEnv(), 'owner', 'dash-1'); + const envPush = pushes.find((p) => p.url.includes('/sessions/sbx-1/env')); + expect(envPush?.body).toContain('OWNER_KEY'); + expect(envPush?.body).not.toContain('COLLABORATOR'); + }); + + // The sandbox reads an omitted list as "keep the current approvals", so a + // collaborator's approvals for a same-named secret would authorize the + // owner's value. Every push must carry the full list, [] included. + it.each([ + ['terminal start', async (env: Env) => { + await applyDashboardOwnerSecrets(env, new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN), 'dash-1', 'sbx-1', ''); + }], + ['owner secret change', (env: Env) => autoApplySecretsToSessions(env, 'owner', 'dash-1')], + ['explicit apply', async (env: Env) => { + expect((await applySecretsToSession(env as never, 's1', 'owner')).status).toBe(200); + }], + ] as const)('%s replaces approvals with the owner\'s (empty) list', async (_path, run) => { + const env = fakeEnv(['OWNER_KEY']); + const pushes = captureEnvPushes(); + await run(env); + const body = JSON.parse(pushes.find((p) => p.url.includes('/sessions/sbx-1/env'))!.body); + expect(Object.keys(body.secrets)).toEqual(['OWNER_KEY']); + expect(body.approved_domains).toEqual([]); + }); + + describe('explicit apply endpoint', () => { + const tracked = (writes: string[]) => writes.filter((sql) => sql.includes('applied_secret_names')); + + it('keeps the old tracking when the sandbox rejects the cleanup', async () => { + const writes: string[] = []; + const env = fakeEnv(['COLLABORATOR_KEY'], false, writes); + vi.stubGlobal('fetch', vi.fn(async () => new Response('cleanup failed', { status: 403 }))); + const res = await applySecretsToSession(env as never, 's1', 'owner'); + expect(res.status).toBe(500); + expect(tracked(writes)).toHaveLength(0); + }); + + it('unsets collaborator names (and _BROKER), then updates tracking', async () => { + const writes: string[] = []; + const env = fakeEnv(['COLLABORATOR_KEY'], false, writes); + const pushes = captureEnvPushes(); + const res = await applySecretsToSession(env as never, 's1', 'owner'); + expect(res.status).toBe(200); + const body = JSON.parse(pushes.find((p) => p.url.includes('/sessions/sbx-1/env'))!.body); + expect(body.unset).toEqual(['COLLABORATOR_KEY', 'COLLABORATOR_KEY_BROKER']); + expect(Object.keys(body.secrets)).toEqual(['OWNER_KEY']); + expect(tracked(writes)).toHaveLength(1); + }); + }); +}); diff --git a/controlplane/src/sessions/handler.test.ts b/controlplane/src/sessions/handler.test.ts index f80de8ba..e2f3baf7 100644 --- a/controlplane/src/sessions/handler.test.ts +++ b/controlplane/src/sessions/handler.test.ts @@ -1,3 +1,4 @@ +// REVISION: session-tests-v6-push-current-sandbox /** * Session Handler Tests * @@ -5,7 +6,9 @@ * Full integration tests should use a real D1 database. */ -import { describe, it, expect, beforeEach, beforeAll, afterAll, afterEach } from 'vitest'; +import { describe, it, expect, beforeEach, beforeAll, afterAll, afterEach, vi } from 'vitest'; +import { http, HttpResponse } from 'msw'; +import * as secretsHandler from '../secrets/handler'; import { createTestContext, seedUser, @@ -16,7 +19,7 @@ import { } from '../../tests/helpers'; import { createMockSandboxServer } from '../../tests/mocks/sandbox'; import { SandboxClient } from '../sandbox/client'; -import { createSessiоn } from './handler'; +import { createSessiоn, applySecretsToSession } from './handler'; import { ensureDriveCache } from '../storage/drive-cache'; import type { TestContext } from '../../tests/helpers'; @@ -29,8 +32,13 @@ describe('Session Handlers', () => { mockSandbox.server.listen({ onUnhandledRequest: 'error' }); }); + // The terminal must not start: no PTY in any sandbox session. + const ptysStarted = () => [...mockSandbox.sessions.values()].reduce((n, s) => n + s.ptys.size, 0); + afterEach(() => { mockSandbox.reset(); + mockSandbox.server.resetHandlers(); + vi.restoreAllMocks(); }); afterAll(() => { @@ -39,6 +47,7 @@ describe('Session Handlers', () => { beforeEach(async () => { ctx = await createTestContext(); + ctx.env.SECRETS_ENCRYPTION_KEY = btoa(String.fromCharCode(...new Uint8Array(32).fill(7))); testUser = createTestUser({ id: 'user-1' }); await seedUser(ctx.db, testUser); }); @@ -86,6 +95,160 @@ describe('Session Handlers', () => { ).toBe(true); }); + it('reprovisions instead of failing when the sandbox session is gone at secret cleanup', async () => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + // The first sandbox session vanishes (redeploy) before secret cleanup: + // updateEnv 404s. That must take the reprovision path, not fail creation. + const calls: string[] = []; + vi.spyOn(secretsHandler, 'applyDashboardOwnerSecrets').mockImplementation(async (_e, _s, _d, sessionId) => { + calls.push(sessionId); + if (calls.length === 1) throw new Error('Failed to update env: 404'); + return ['OWNER_KEY']; + }); + + const response = await createSessiоn(ensureDriveCache(ctx.env), dashboard.id, item.id, testUser.id, testUser.name); + expect(response.status).toBe(201); + expect(calls).toHaveLength(2); + expect(calls[1]).not.toBe(calls[0]); + const data = await response.json() as Record; + expect(data.session.status).toBe('active'); + expect(data.session.sandboxSessionId).toBe(calls[1]); + }); + + it('keeps secret tracking across a reprovision and unsets it in the replacement session', async () => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + // A live sandbox session with a collaborator's secret still tracked. + const old = await new SandboxClient(ctx.env.SANDBOX_URL).createSessiоn(); + await ctx.db.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, ?, '', ?, ?) + `).bind(dashboard.id, old.id, JSON.stringify(['COLLAB_KEY']), new Date().toISOString()).run(); + + // It vanishes between validation and cleanup. On desktop the replacement + // shares its workspace .env, so the replacement must get the unset. + const pushes: Array<{ session: string; body: Record }> = []; + mockSandbox.server.use(http.post(`${ctx.env.SANDBOX_URL}/sessions/:id/env`, async ({ params, request }) => { + if (params.id === old.id) return HttpResponse.json({ error: 'session not found' }, { status: 404 }); + pushes.push({ session: params.id as string, body: await request.json() as Record }); + return HttpResponse.json({ status: 'ok' }); + })); + + const response = await createSessiоn(ensureDriveCache(ctx.env), dashboard.id, item.id, testUser.id, testUser.name); + expect(response.status).toBe(201); + const data = await response.json() as Record; + expect(data.session.sandboxSessionId).not.toBe(old.id); + expect(pushes).toHaveLength(1); + expect(pushes[0].session).toBe(data.session.sandboxSessionId); + expect(pushes[0].body.unset).toEqual(['COLLAB_KEY', 'COLLAB_KEY_BROKER']); + const row = await ctx.db.prepare(`SELECT sandbox_session_id, applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ?`) + .bind(dashboard.id).first<{ sandbox_session_id: string; applied_secret_names: string }>(); + expect(row?.sandbox_session_id).toBe(data.session.sandboxSessionId); + expect(JSON.parse(row!.applied_secret_names)).toEqual([]); + }); + + it('keeps secret tracking when validation finds the sandbox session gone', async () => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + // Not known to the mock sandbox: the /ptys validation 404s up front. + await ctx.db.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, 'gone-session', '', ?, ?) + `).bind(dashboard.id, JSON.stringify(['COLLAB_KEY']), new Date().toISOString()).run(); + const pushes: Array> = []; + mockSandbox.server.use(http.post(`${ctx.env.SANDBOX_URL}/sessions/:id/env`, async ({ request }) => { + pushes.push(await request.json() as Record); + return HttpResponse.json({ status: 'ok' }); + })); + + const response = await createSessiоn(ensureDriveCache(ctx.env), dashboard.id, item.id, testUser.id, testUser.name); + expect(response.status).toBe(201); + expect(pushes.map((b) => b.unset)).toEqual([['COLLAB_KEY', 'COLLAB_KEY_BROKER']]); + }); + + // A reprovision (timeout, 502/503) leaves older `sessions` rows pointing at + // the replaced sandbox session, which may still be alive. + it.each([ + ['secret change', (dashboardId: string, sessionId: string) => + secretsHandler.autoApplySecretsToSessions(ctx.env, testUser.id, dashboardId).then(() => sessionId)], + ['explicit apply', async (_dashboardId: string, sessionId: string) => { + expect((await applySecretsToSession(ensureDriveCache(ctx.env), sessionId, testUser.id)).status).toBe(200); + }], + ] as const)('%s pushes to the current sandbox session, not a replaced one', async (_name, run) => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + const client = new SandboxClient(ctx.env.SANDBOX_URL); + const old = await client.createSessiоn(); + const current = await client.createSessiоn(); + const stale = await seedSession(ctx.db, dashboard.id, item.id, { sandboxSessionId: old.id, ownerUserId: testUser.id }); + await ctx.db.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, ?, '', ?, ?) + `).bind(dashboard.id, current.id, JSON.stringify(['DELETED_KEY']), new Date().toISOString()).run(); + const pushes: Array<{ session: string; body: Record }> = []; + mockSandbox.server.use(http.post(`${ctx.env.SANDBOX_URL}/sessions/:id/env`, async ({ params, request }) => { + pushes.push({ session: params.id as string, body: await request.json() as Record }); + return HttpResponse.json({ status: 'ok' }); + })); + + await run(dashboard.id, stale.id); + + expect(pushes.map((p) => p.session)).toEqual([current.id]); + expect(pushes[0].body.unset).toEqual(['DELETED_KEY', 'DELETED_KEY_BROKER']); + const row = await ctx.db.prepare(`SELECT sandbox_session_id, applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ?`) + .bind(dashboard.id).first<{ sandbox_session_id: string; applied_secret_names: string }>(); + expect(row?.sandbox_session_id).toBe(current.id); + expect(JSON.parse(row!.applied_secret_names)).toEqual([]); + }); + + it('does not push to a retired sandbox', async () => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + const old = await new SandboxClient(ctx.env.SANDBOX_URL).createSessiоn(); + await seedSession(ctx.db, dashboard.id, item.id, { sandboxSessionId: old.id, ownerUserId: testUser.id }); + await ctx.db.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, '', '', ?, ?) + `).bind(dashboard.id, JSON.stringify(['DELETED_KEY']), new Date().toISOString()).run(); + const pushes: string[] = []; + mockSandbox.server.use(http.post(`${ctx.env.SANDBOX_URL}/sessions/:id/env`, ({ params }) => { + pushes.push(params.id as string); + return HttpResponse.json({ status: 'ok' }); + })); + await secretsHandler.autoApplySecretsToSessions(ctx.env, testUser.id, dashboard.id); + expect(pushes).toEqual([]); + const row = await ctx.db.prepare(`SELECT applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ?`) + .bind(dashboard.id).first<{ applied_secret_names: string }>(); + expect(JSON.parse(row!.applied_secret_names)).toEqual(['DELETED_KEY']); + }); + + it.each([ + ['cleanup rejected by the sandbox', 'Failed to update env: 403'], + ['tracking could not be written', 'D1 error'], + ])('still fails closed when %s', async (_name, message) => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + vi.spyOn(secretsHandler, 'applyDashboardOwnerSecrets').mockRejectedValue(new Error(message)); + + const response = await createSessiоn(ensureDriveCache(ctx.env), dashboard.id, item.id, testUser.id, testUser.name); + expect(response.status).toBeGreaterThanOrEqual(500); + expect(ptysStarted()).toBe(0); + }); + + it('fails closed when cleanup fails on the reprovisioned sandbox too', async () => { + const dashboard = await seedDashboard(ctx.db, testUser.id); + const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); + const spy = vi.spyOn(secretsHandler, 'applyDashboardOwnerSecrets') + .mockRejectedValueOnce(new Error('Failed to update env: 404')) + .mockRejectedValueOnce(new Error('Failed to update env: 403')); + + const response = await createSessiоn(ensureDriveCache(ctx.env), dashboard.id, item.id, testUser.id, testUser.name); + expect(response.status).toBeGreaterThanOrEqual(500); + expect(spy).toHaveBeenCalledTimes(2); + expect(ptysStarted()).toBe(0); + }); + it('should seed and query sessions', async () => { const dashboard = await seedDashboard(ctx.db, testUser.id); const item = await seedDashboardItem(ctx.db, dashboard.id, { type: 'terminal' }); diff --git a/controlplane/src/sessions/handler.ts b/controlplane/src/sessions/handler.ts index 992f3fe1..0c1ae9fd 100644 --- a/controlplane/src/sessions/handler.ts +++ b/controlplane/src/sessions/handler.ts @@ -1,8 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: sessions-v20-no-per-session-egress -const sessionsRevision = "sessions-v20-no-per-session-egress"; +// REVISION: sessions-v31-error-codes +const sessionsRevision = "sessions-v31-error-codes"; console.log(`[sessions] REVISION: ${sessionsRevision} loaded at ${new Date().toISOString()}`); /** @@ -12,11 +12,13 @@ console.log(`[sessions] REVISION: ${sessionsRevision} loaded at ${new Date().toI * This is the bridge between the control plane and execution plane. */ -import type { Session, DashboardItem } from '../types'; +import type { Env, Session, DashboardItem } from '../types'; import { isDesktopFeatureDisabledError, type EnvWithDriveCache } from '../storage/drive-cache'; import { SandboxClient } from '../sandbox/client'; -import { createDashboardToken } from '../auth/dashboard-token'; -import { createPtyToken } from '../auth/pty-token'; +import { createDashboardToken, sandboxGeneration } from '../auth/dashboard-token'; +import { sandboxInternalTokenEnv } from '../auth/sandbox-auth'; +import { sandboxAuthEnv } from '../sandbox/request-signing'; +import { createPtyToken, tokenSigningSecret } from '../auth/pty-token'; import { sandboxFetch } from '../sandbox/fetch'; import { requireAuth, type AuthContext } from '../auth/middleware'; import { FlyMachinesClient, FlyMachineNotFoundError, FlyApiError } from '../sandbox/fly-machines'; @@ -255,6 +257,118 @@ async function getDashbоardSandbоx(env: EnvWithDriveCache, dashboardId: string }>(); } +async function readAppliеdSecretNames(env: Env, dashboardId: string): Promise { + const row = await env.DB.prepare(` + SELECT applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ? + `).bind(dashboardId).first<{ applied_secret_names: string | null }>(); + return row?.applied_secret_names ?? null; +} + +function parsеSecretNames(raw: string | null): string[] { + try { + const parsed: unknown = raw ? JSON.parse(raw) : []; + return Array.isArray(parsed) ? parsed.filter((n): n is string => typeof n === 'string') : []; + } catch { + return []; + } +} + +/** + * Record the names a terminal start's pre-apply left set in the sandbox. + * + * If the tracking is unchanged since that pre-apply read it, it is replaced + * exactly. If it changed, an owner secret push ran concurrently and recorded + * names this start never saw: overwriting would drop them, and a later delete + * of that secret would never unset it. Then the union is written instead — an + * extra name only costs a redundant unset, and the next push records the exact + * set. The write is compare-and-swap so a push landing between read and write + * is merged too. + */ +export async function recоrdAppliedSecretNames( + env: Env, + dashboardId: string, + sandboxSessionId: string, + sandboxMachineId: string, + names: string[], + trackedBeforePreApply: string | null, + createdAt: string, +): Promise { + const insert = await env.DB.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, ?, ?, ?, ?) + ON CONFLICT(dashboard_id) DO NOTHING + `).bind(dashboardId, sandboxSessionId, sandboxMachineId, JSON.stringify(names), createdAt).run(); + if (insert.meta.changes) return; + + for (let attempt = 0; attempt < 5; attempt++) { + const current = await readAppliеdSecretNames(env, dashboardId); + if (current === null) return; + const next = current === trackedBeforePreApply + ? names + : Array.from(new Set([...parsеSecretNames(current), ...names])); + const nextRaw = JSON.stringify(next); + if (nextRaw === current) return; + const update = await env.DB.prepare(` + UPDATE dashboard_sandboxes SET applied_secret_names = ? + WHERE dashboard_id = ? AND applied_secret_names = ? + `).bind(nextRaw, dashboardId, current).run(); + if (update.meta.changes) return; + } + console.error(`[createSession] could not record applied secret names for dashboard ${dashboardId} (concurrent updates)`); +} + +/** + * A fresh dashboard token for the dashboard's current sandbox, bound to its + * generation (dashboard_sandboxes.created_at) so the VM can keep refreshing it. + * Sent with each new PTY: the VM otherwise only receives a token when its + * sandbox session is created, so one whose token was refused (signing secret + * rotated, E79393/E79395) would never get another. Undefined when the row is not + * this sandbox session (it was replaced meanwhile) — never mint for a stale one. + */ +export async function mintCurrentSandboxDashboardToken( + env: Env, + dashboardId: string, + sandboxSessionId: string, +): Promise { + const row = await env.DB.prepare(` + SELECT sandbox_session_id, created_at FROM dashboard_sandboxes WHERE dashboard_id = ? + `).bind(dashboardId).first<{ sandbox_session_id: string | null; created_at: string | null }>(); + if (!row?.sandbox_session_id || row.sandbox_session_id !== sandboxSessionId || !row.created_at) { + return undefined; + } + const generation = sandboxGeneration(row.created_at); + if (Number.isNaN(generation)) return undefined; + return createDashboardToken(dashboardId, tokenSigningSecret(env), undefined, generation); +} + +/** + * Forget a dashboard's dead sandbox so the next ensure provisions a new one. + * + * The row is blanked rather than deleted so applied_secret_names survives: the + * replacement can reuse the workspace (on desktop it always does, .env + * included), and its pre-apply must still unset every name tracked here. The + * inserts in ensureDashboardSandboxCreate / provisioning claim a blank row. + */ +async function retirеDashboardSandbоx(env: EnvWithDriveCache, dashboardId: string): Promise { + await env.DB.prepare(` + UPDATE dashboard_sandboxes + SET sandbox_session_id = '', sandbox_machine_id = '', fly_volume_id = '', machine_state = 'unknown' + WHERE dashboard_id = ? + `).bind(dashboardId).run(); +} + +// A blank row (retirеDashboardSandbоx, or the idle-machine cleanup) is free to +// claim; a live one means a concurrent request already provisioned this +// dashboard, so the insert changes nothing and the caller defers to it. +const CLAIM_DASHBOARD_SANDBOX = ` + ON CONFLICT(dashboard_id) DO UPDATE SET + sandbox_session_id = excluded.sandbox_session_id, + sandbox_machine_id = excluded.sandbox_machine_id, + fly_volume_id = excluded.fly_volume_id, + machine_state = excluded.machine_state, + created_at = excluded.created_at + WHERE dashboard_sandboxes.sandbox_session_id = ''`; + // ── Fly machine-state cache (anti-429) ───────────────────────────────── // REVISION: fly-getmachine-cache-v1-anti-429 // Every browser asset request (noVNC loads 20+ assets at once: svg/css/js) and @@ -397,9 +511,7 @@ export async function ensureDashbоardSandbоx( } catch (waitErr) { if (waitErr instanceof FlyMachineNotFoundError) { console.log(`[ensureDashboardSandbox] Machine ${existingSandbox.sandbox_machine_id} disappeared while waiting, clearing for reprovision`); - await env.DB.prepare(` - DELETE FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(dashboardId).run(); + await retirеDashboardSandbоx(env, dashboardId); return ensureDashbоardSandbоxCreate(env, dashboardId, flyProvisioningEnabled, preferredRegion); } // Otherwise fall through to session validation (may work if machine just came up) @@ -410,9 +522,7 @@ export async function ensureDashbоardSandbоx( if (err instanceof FlyMachineNotFoundError) { // Machine was destroyed externally — clear and reprovision below console.log(`[ensureDashboardSandbox] Machine ${existingSandbox.sandbox_machine_id} not found, clearing for reprovision`); - await env.DB.prepare(` - DELETE FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(dashboardId).run(); + await retirеDashboardSandbоx(env, dashboardId); // Fall through to provisioning below return ensureDashbоardSandbоxCreate(env, dashboardId, flyProvisioningEnabled, preferredRegion); } @@ -457,18 +567,14 @@ export async function ensureDashbоardSandbоx( if (checkRes.status === 404) { // Session definitively gone (sandbox process restarted, lost in-memory state) console.log(`[ensureDashboardSandbox] Session ${existingSandbox.sandbox_session_id} not found on sandbox (404), clearing for reprovision`); - await env.DB.prepare(` - DELETE FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(dashboardId).run(); + await retirеDashboardSandbоx(env, dashboardId); // Fall through to create new sandbox below } else if ( (checkRes.status === 502 || checkRes.status === 503) && existingSandbox.sandbox_machine_id ) { console.warn(`[ensureDashboardSandbox] Session validation returned ${checkRes.status} for machine ${existingSandbox.sandbox_machine_id}, clearing for reprovision`); - await env.DB.prepare(` - DELETE FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(dashboardId).run(); + await retirеDashboardSandbоx(env, dashboardId); // Fall through to create new sandbox below } else { // Other non-404 responses may be transient; keep existing mapping. @@ -486,9 +592,7 @@ export async function ensureDashbоardSandbоx( (fetchErr.name === 'AbortError' || fetchErr.message.includes('The operation was aborted')); if (isTimeout) { console.warn(`[ensureDashboardSandbox] Session validation timed out — sandbox unreachable, clearing for reprovision`); - await env.DB.prepare(` - DELETE FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(dashboardId).run(); + await retirеDashboardSandbоx(env, dashboardId); // Fall through to create new sandbox below } else { // Non-timeout error (e.g. connection reset) — could be transient. @@ -517,9 +621,10 @@ async function ensureDashbоardSandbоxCreate( flyProvisioningEnabled: boolean, preferredRegion?: string ): Promise<{ sandboxSessionId: string; sandboxMachineId: string }> { - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); const now = new Date().toISOString(); - const mcpToken = await createDashboardToken(dashboardId, env.INTERNAL_API_TOKEN); + // Bound to the row this claim writes (created_at = now): see refreshSandbоxDashboardToken. + const mcpToken = await createDashboardToken(dashboardId, tokenSigningSecret(env), undefined, sandboxGeneration(now)); if (flyProvisioningEnabled) { return provisionDedicatedMachine(env, dashboardId, sandbox, mcpToken, now, preferredRegion); @@ -557,8 +662,9 @@ async function ensureDashbоardSandbоxCreate( throw lastCreateErr instanceof Error ? lastCreateErr : new Error('Failed to create sandbox session'); } const insertResult = await env.DB.prepare(` - INSERT OR IGNORE INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, created_at) - VALUES (?, ?, ?, ?) + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, fly_volume_id, machine_state, created_at) + VALUES (?, ?, ?, '', 'unknown', ?) + ${CLAIM_DASHBOARD_SANDBOX} `).bind(dashboardId, sandboxSession.id, sandboxSession.machineId || '', now).run(); if (insertResult.meta.changes === 0) { @@ -606,9 +712,10 @@ export async function preProvisionDashboardSandbox( if (existing?.sandbox_session_id) return; const fly = new FlyMachinesClient(env.FLY_APP_NAME!, env.FLY_API_TOKEN!); - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); const now = new Date().toISOString(); - const mcpToken = await createDashboardToken(dashboardId, env.INTERNAL_API_TOKEN); + // Bound to the row this claim writes (created_at = now): see refreshSandbоxDashboardToken. + const mcpToken = await createDashboardToken(dashboardId, tokenSigningSecret(env), undefined, sandboxGeneration(now)); const result = await claimWarmMachine( env, fly, dashboardId, sandbox, mcpToken, now, preferredRegion, @@ -712,12 +819,10 @@ async function claimWarmMachine( // Clean up lost+found (ext4 creates this on new volumes) try { - const headers = new Headers({ 'Content-Type': 'application/json' }); - headers.set('Authorization', `Bearer ${env.SANDBOX_INTERNAL_TOKEN}`); - headers.set('X-Sandbox-Machine-ID', machineId); - await fetch(`${env.SANDBOX_URL}/sessions/${sandboxSession.id}/file?path=/workspace/lost%2Bfound`, { + await sandboxFetch(env, `/sessions/${sandboxSession.id}/file?path=/workspace/lost%2Bfound`, { method: 'DELETE', - headers, + machineId, + headers: { 'Content-Type': 'application/json' }, }); } catch { // Best-effort cleanup @@ -725,9 +830,10 @@ async function claimWarmMachine( // Store in DB const insertResult = await env.DB.prepare(` - INSERT OR IGNORE INTO dashboard_sandboxes + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, fly_volume_id, machine_state, created_at) VALUES (?, ?, ?, ?, 'started', ?) + ${CLAIM_DASHBOARD_SANDBOX} `).bind(dashboardId, sandboxSession.id, machineId, volumeId, now).run(); if (insertResult.meta.changes === 0) { @@ -802,9 +908,9 @@ async function coldProvisionMachine( // connecting"). The cron escalates long-idle VMs from suspended → stopped. autostop: 'suspend', env: { - SANDBOX_INTERNAL_TOKEN: env.SANDBOX_INTERNAL_TOKEN, + ...(await sandboxAuthEnv(env)), CONTROLPLANE_URL: env.FLY_SANDBOX_CONTROLPLANE_URL || 'https://api.orcabot.com', - INTERNAL_API_TOKEN: env.INTERNAL_API_TOKEN, + ...sandboxInternalTokenEnv(env), DASHBOARD_ID: dashboardId, ALLOWED_ORIGINS: env.ALLOWED_ORIGINS || 'https://orcabot.com', ...(env.EGRESS_PROXY_ENABLED ? { EGRESS_PROXY_ENABLED: env.EGRESS_PROXY_ENABLED } : {}), @@ -837,12 +943,10 @@ async function coldProvisionMachine( // Step 4b: Clean up lost+found (ext4 creates this on new volumes) try { - const headers = new Headers({ 'Content-Type': 'application/json' }); - headers.set('Authorization', `Bearer ${env.SANDBOX_INTERNAL_TOKEN}`); - headers.set('X-Sandbox-Machine-ID', machineId); - await fetch(`${env.SANDBOX_URL}/sessions/${sandboxSession.id}/file?path=/workspace/lost%2Bfound`, { + await sandboxFetch(env, `/sessions/${sandboxSession.id}/file?path=/workspace/lost%2Bfound`, { method: 'DELETE', - headers, + machineId, + headers: { 'Content-Type': 'application/json' }, }); } catch { // Best-effort cleanup @@ -852,9 +956,10 @@ async function coldProvisionMachine( let insertResult; try { insertResult = await env.DB.prepare(` - INSERT OR IGNORE INTO dashboard_sandboxes + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, fly_volume_id, machine_state, created_at) VALUES (?, ?, ?, ?, 'started', ?) + ${CLAIM_DASHBOARD_SANDBOX} `).bind(dashboardId, sandboxSession.id, machineId, volumeId, now).run(); } catch (dbErr) { // FK constraint = dashboard was deleted during provisioning; clean up @@ -1387,6 +1492,16 @@ export async function createSessiоn( } } +function isUnrеachableSandbox(err: unknown): boolean { + return err instanceof Error && + (err.name === 'AbortError' || err.message.includes('The operation was aborted')); +} + +// Definitively gone (404) or completely unreachable (timeout/abort): reprovision. +function isStalеOrUnreachableSandbox(err: unknown): boolean { + return (err instanceof Error && err.message.includes('404')) || isUnrеachableSandbox(err); +} + async function createSessionImpl( env: EnvWithDriveCache, dashboardId: string, @@ -1415,7 +1530,7 @@ async function createSessionImpl( return Response.json({ error: 'E79202: Terminal item not found' }, { status: 404 }); } - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); // Check if session already exists for this item const existingSession = await env.DB.prepare(` @@ -1493,30 +1608,17 @@ async function createSessionImpl( // uniformly to every dashboard-scoped execution path, not just this UI-terminal // one. `workingDir` here is only the terminal's own relative sub-path (if any). let appliedSecretNames: string[] = []; + // The tracked names as they were before the latest pre-apply read them; see + // recоrdAppliedSecretNames for why the final write compares against this. + let trackedBeforePreApply: string | null = null; const applyDashboardSecretsBeforePtyStart = async ( sandboxSessionId: string, sandboxMachineId: string ): Promise => { - const { getSecretsWithProtection, getApprovedDomainsForDashboard } = await import('../secrets/handler'); - const secrets = await getSecretsWithProtection(env, userId, dashboardId); - const approvedDomains = await getApprovedDomainsForDashboard(env, userId, dashboardId); - const secretNames = Object.keys(secrets); - - if (secretNames.length > 0 || approvedDomains.length > 0) { - await sandbox.updateEnv( - sandboxSessionId, - { - secrets: secretNames.length > 0 ? secrets : undefined, - approvedDomains: approvedDomains.length > 0 ? approvedDomains : undefined, - // PTYs started after this call load from .env at process start. - applyNow: false, - }, - sandboxMachineId || undefined - ); - } - - return secretNames; + trackedBeforePreApply = await readAppliеdSecretNames(env, dashboardId); + const { applyDashboardOwnerSecrets } = await import('../secrets/handler'); + return applyDashboardOwnerSecrets(env, sandbox, dashboardId, sandboxSessionId, sandboxMachineId); }; // Use ensureDashboardSandbox which checks machine health via Fly API, @@ -1535,11 +1637,20 @@ async function createSessionImpl( let { sandboxSessionId, sandboxMachineId } = sandboxResult; // Apply dashboard secrets before PTY start so agent apps inherit env at launch. + // A stale (404) or unreachable sandbox is not a cleanup failure: skip straight + // to the reprovision path below. Its tracking is kept (retirеDashboardSandbоx), + // so the replacement session's own pre-apply — which still fails closed — + // unsets those names in the workspace it may reuse. + let sandboxGoneErr: Error | null = null; try { appliedSecretNames = await applyDashboardSecretsBeforePtyStart(sandboxSessionId, sandboxMachineId); } catch (err) { - console.error('[createSession] Failed to pre-apply secrets before PTY start:', err); - // Non-fatal - continue with session creation + if (!isStalеOrUnreachableSandbox(err)) { + console.error('[createSession] Failed to pre-apply secrets before PTY start:', err); + // Fail closed: never launch with legacy secrets or discard their tracking. + throw err; + } + sandboxGoneErr = err as Error; } // Create PTY within the dashboard sandbox, assigning control to the creator @@ -1551,11 +1662,13 @@ async function createSessionImpl( sandboxSessionId, dashboardId, userId, - env.INTERNAL_API_TOKEN + tokenSigningSecret(env) ); + const dashboardToken = await mintCurrentSandboxDashboardToken(env, dashboardId, sandboxSessionId); let pty: { id: string }; try { + if (sandboxGoneErr) throw sandboxGoneErr; // Inner try: handle working directory not found (E79708) — fall back to workspace root. // This happens when reopening an old dashboard whose GitHub mirror / workspace is gone. try { @@ -1564,6 +1677,7 @@ async function createSessionImpl( integrationToken, workingDir, modelSelection, + dashboardToken, }); } catch (wdErr) { if (workingDir && wdErr instanceof Error && wdErr.message.includes('E79708')) { @@ -1572,6 +1686,7 @@ async function createSessionImpl( ptyId, integrationToken, modelSelection, + dashboardToken, }); } else { throw wdErr; @@ -1582,18 +1697,14 @@ async function createSessionImpl( // If the sandbox timed out, the Fly machine is effectively gone — retrying the same // machine won't help. Reprovision to get a working sandbox. // Other errors (502, 409 Fly-Replay) may be transient — let them propagate. - const isStaleSession = err instanceof Error && err.message.includes('404'); - const isUnreachable = err instanceof Error && - (err.name === 'AbortError' || err.message.includes('The operation was aborted')); - if (!isStaleSession && !isUnreachable) { + if (!isStalеOrUnreachableSandbox(err)) { throw err; } + const isUnreachable = isUnrеachableSandbox(err); // Stale/unreachable session — clear it and create a fresh sandbox console.log(`[createSession] Sandbox ${isUnreachable ? 'unreachable (timeout)' : 'stale (404)'} for session ${sandboxSessionId}, reprovisioning`); - await env.DB.prepare(` - DELETE FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(dashboardId).run(); + await retirеDashboardSandbоx(env, dashboardId); // Create fresh sandbox session (uses Fly provisioning if enabled) const flyProvisioningEnabled = env.FLY_PROVISIONING_ENABLED === 'true' @@ -1602,12 +1713,13 @@ async function createSessionImpl( sandboxSessionId = freshResult.sandboxSessionId; sandboxMachineId = freshResult.sandboxMachineId; - // Fresh sandbox starts empty; pre-apply secrets before launching PTY. + // Pre-apply before launching the PTY; this also unsets the names tracked for + // the old session, since the new one may share its workspace and .env. try { appliedSecretNames = await applyDashboardSecretsBeforePtyStart(sandboxSessionId, sandboxMachineId); } catch (secretErr) { console.error('[createSession] Failed to pre-apply secrets on reprovisioned sandbox:', secretErr); - // Non-fatal - continue with session creation + throw secretErr; } // Regenerate integration token with fresh sandbox session ID @@ -1616,7 +1728,7 @@ async function createSessionImpl( sandboxSessionId, dashboardId, userId, - env.INTERNAL_API_TOKEN + tokenSigningSecret(env) ); // Retry PTY creation on fresh session — omit workingDir (fresh sandbox has empty workspace) @@ -1624,6 +1736,7 @@ async function createSessionImpl( ptyId, integrationToken: freshIntegrationToken, modelSelection, + dashboardToken: await mintCurrentSandboxDashboardToken(env, dashboardId, sandboxSessionId), }); } @@ -1779,12 +1892,9 @@ async function createSessionImpl( console.error('[createSession] Failed to migrate integrations:', err); } - // Track applied secret names for deletion tracking (upsert in case row exists from sandbox creation) - await env.DB.prepare(` - INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) - VALUES (?, ?, ?, ?, ?) - ON CONFLICT(dashboard_id) DO UPDATE SET applied_secret_names = excluded.applied_secret_names - `).bind(dashboardId, sandboxSessionId, sandboxMachineId || '', JSON.stringify(appliedSecretNames), now).run(); + // Track applied secret names for deletion tracking. + await recоrdAppliedSecretNames( + env, dashboardId, sandboxSessionId, sandboxMachineId || '', appliedSecretNames, trackedBeforePreApply, now); const session: Session = { id, @@ -1892,7 +2002,7 @@ async function createSessionImpl( `).bind(id).run(); return Response.json({ - error: `Failed to create sandbox session: ${error instanceof Error ? error.message : 'Unknown error'}` + error: `E79172: Failed to create sandbox session: ${error instanceof Error ? error.message : 'Unknown error'}` }, { status: 500 }); } } @@ -2006,7 +2116,7 @@ export async function getDashbоardSandbоxStatus( // app restarts / sandbox crashes, but the actual sandbox session is in-memory — so // trusting the row alone shows green while the VM is still booting or has crashed. // Probe the local sandbox /health instead; not-yet-serving reads as "starting". - const localSandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const localSandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); const healthy = await localSandbox.health(undefined, 3000); return Response.json({ state: healthy ? 'ready' : 'starting', @@ -2388,7 +2498,7 @@ export async function updateSessiоnEnv( } } - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); if (payload.applyNow) { console.warn(`[updateSessionEnv] Ignoring applyNow=true for session ${sessionId}; env changes apply on next PTY start`); } @@ -2422,7 +2532,7 @@ export async function stоpSessiоn( return Response.json({ error: 'E79204: Session already stopped' }, { status: 400 }); } - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); // REVISION: preserve-sandbox-v1-stop-keeps-sandbox // Preserve the sandbox (and dashboard_sandboxes mapping) when stopping the last session. @@ -2448,7 +2558,7 @@ export async function stоpSessiоn( // Always just delete this session's PTY (not the sandbox session) if (session.pty_id) { - await sandbox.deletePty(session.sandbox_session_id as string, session.pty_id as string); + await sandbox.deletePty(session.sandbox_session_id as string, session.pty_id as string, (session.sandbox_machine_id as string) || undefined); } } catch { // Ignore errors - sandbox might already be gone @@ -2522,7 +2632,7 @@ export async function sandboxKeepalive( return Response.json({ error: 'E79751: No sandbox for this dashboard' }, { status: 404 }); } - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); const healthy = await sandbox.health(sandboxInfo.sandbox_machine_id || undefined); return Response.json({ alive: healthy }); @@ -2571,7 +2681,7 @@ export async function applySecretsToSession( userId: string ): Promise { // Import dynamically to avoid circular dependency - const { getSecretsWithProtection, getApprovedDomainsForDashboard } = await import('../secrets/handler'); + const { pushOwnerSecretsToCurrentSandbox, dashboardSecretsOwner } = await import('../secrets/handler'); const session = await env.DB.prepare(` SELECT s.*, dm.role FROM sessions s @@ -2588,71 +2698,26 @@ export async function applySecretsToSession( } try { - const secrets = await getSecretsWithProtection( - env, - userId, - session.dashboard_id as string - ); - - // Get approved domains for custom secrets - const approvedDomains = await getApprovedDomainsForDashboard( - env, - userId, - session.dashboard_id as string - ); - - // Get previously applied secret names to compute what to unset - const dashboardSandbox = await env.DB.prepare(` - SELECT applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ? - `).bind(session.dashboard_id).first<{ applied_secret_names: string }>(); - - const previousNames: string[] = dashboardSandbox?.applied_secret_names - ? JSON.parse(dashboardSandbox.applied_secret_names) - : []; - const currentNames = Object.keys(secrets); - - // Compute secrets to unset (were applied before but not in current set) - const unset: string[] = []; - for (const name of previousNames) { - if (!currentNames.includes(name)) { - unset.push(name); - // Also unset the _BROKER suffix for custom secrets - unset.push(`${name}_BROKER`); - } + // The dashboard's VM only ever gets its owner's secrets (see dashboardSecretsOwner). + const secretsOwnerId = await dashboardSecretsOwner(env, session.dashboard_id as string); + if (!secretsOwnerId) { + return Response.json({ error: 'E79620: Dashboard owner not found' }, { status: 404 }); } - - // Update the applied secret names for next time (upsert in case row doesn't exist) - await env.DB.prepare(` - INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, applied_secret_names) - VALUES (?, ?, ?) - ON CONFLICT(dashboard_id) DO UPDATE SET applied_secret_names = excluded.applied_secret_names - `).bind(session.dashboard_id, session.sandbox_session_id, JSON.stringify(currentNames)).run(); - - // Skip updateEnv call if there's nothing to do - const hasSecrets = Object.keys(secrets).length > 0; - const hasUnset = unset.length > 0; - const hasApprovedDomains = approvedDomains.length > 0; - - if (!hasSecrets && !hasUnset && !hasApprovedDomains) { - // Nothing to apply - return success without calling sandbox - return Response.json({ applied: 0, approvedDomains: 0, unset: 0 }); - } - - const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN); - // Don't use applyNow - new PTYs will load from .env automatically - // Existing PTYs can run `source ~/.env` if needed - await sandbox.updateEnv( + const sandbox = new SandboxClient(env.SANDBOX_URL, env.SANDBOX_INTERNAL_TOKEN, env.SANDBOX_AUTH_SIGNING_KEY); + const pushed = await pushOwnerSecretsToCurrentSandbox( + env, + sandbox, + session.dashboard_id as string, + secretsOwnerId, session.sandbox_session_id as string, - { - secrets: hasSecrets ? secrets : undefined, - approvedDomains: hasApprovedDomains ? approvedDomains : undefined, - unset: hasUnset ? unset : undefined, - applyNow: false, - }, - (session.sandbox_machine_id as string) || undefined + (session.sandbox_machine_id as string) || '' ); + if (!pushed) { + return Response.json({ error: 'E79623: The dashboard sandbox is being replaced; secrets apply when a terminal starts' }, { status: 409 }); + } + const currentNames = pushed.names; - return Response.json({ applied: Object.keys(secrets).length, approvedDomains: approvedDomains.length, unset: unset.length }); + return Response.json({ applied: currentNames.length, approvedDomains: pushed.approvedDomains, unset: pushed.unset }); } catch (error) { console.error('Failed to apply secrets:', error); return Response.json( diff --git a/controlplane/src/sessions/pty-dashboard-token.test.ts b/controlplane/src/sessions/pty-dashboard-token.test.ts new file mode 100644 index 00000000..c1edc756 --- /dev/null +++ b/controlplane/src/sessions/pty-dashboard-token.test.ts @@ -0,0 +1,79 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: pty-dashboard-token-tests-v2-identifier-renames +import { describe, it, expect, afterEach, vi } from 'vitest'; +import { mintCurrentSandboxDashboardToken, recоrdAppliedSecretNames } from './handler'; +import { createTestContext } from '../../tests/helpers'; +import { verifyDashboardToken, sandboxGeneration } from '../auth/dashboard-token'; +import { SandboxClient } from '../sandbox/client'; +import type { Env } from '../types'; + +// A VM only got a dashboard token at sandbox-session create; each new PTY now +// carries a fresh one for the current generation so a VM whose token was +// refused (PTY_TOKEN_SECRET rotated) recovers on the next terminal. +function envWithRow(row: { sandbox_session_id: string; created_at: string } | null) { + return { + DB: { prepare: () => ({ bind: () => ({ first: async () => row }) }) }, + PTY_TOKEN_SECRET: 'cp-only-secret', + INTERNAL_API_TOKEN: 'vm-readable', + } as unknown as Env; +} + +describe('mintCurrentSandboxDashboardToken', () => { + afterEach(() => { vi.unstubAllGlobals(); }); + + it('mints a token bound to the current row generation, signed with PTY_TOKEN_SECRET', async () => { + const createdAt = '2026-10-01 12:00:00'; + const token = await mintCurrentSandboxDashboardToken(envWithRow({ sandbox_session_id: 's1', created_at: createdAt }), 'd1', 's1'); + expect(token).toBeTruthy(); + const claims = await verifyDashboardToken(token!, 'cp-only-secret'); + expect(claims?.dashboard_id).toBe('d1'); + expect(claims?.sbx_gen).toBe(sandboxGeneration(createdAt)); + expect(await verifyDashboardToken(token!, 'vm-readable')).toBeNull(); + }); + + it('mints nothing for a replaced or retired sandbox', async () => { + expect(await mintCurrentSandboxDashboardToken(envWithRow({ sandbox_session_id: 's2', created_at: '2026-10-01 12:00:00' }), 'd1', 's1')).toBeUndefined(); + expect(await mintCurrentSandboxDashboardToken(envWithRow({ sandbox_session_id: '', created_at: '2026-10-01 12:00:00' }), 'd1', '')).toBeUndefined(); + expect(await mintCurrentSandboxDashboardToken(envWithRow(null), 'd1', 's1')).toBeUndefined(); + }); + + it('createPty sends it as mcp_token', async () => { + let body: Record = {}; + vi.stubGlobal('fetch', vi.fn(async (_input: RequestInfo | URL, init?: RequestInit) => { + body = JSON.parse(String(init?.body)); + return Response.json({ id: 'p1' }); + })); + await new SandboxClient('http://sbx.test').createPty('s1', 'alice', undefined, undefined, { dashboardToken: 'tok' }); + expect(body.mcp_token).toBe('tok'); + }); +}); + +// A concurrent owner secret push recorded names this terminal start never saw: +// the final tracking write must keep them, or deleting that secret later +// would never unset it in the VM. +describe('recordAppliedSecretNames', () => { + async function seeded(names: string[]) { + const ctx = await createTestContext(); + await ctx.db.prepare(` + INSERT INTO dashboard_sandboxes (dashboard_id, sandbox_session_id, sandbox_machine_id, applied_secret_names, created_at) + VALUES (?, ?, '', ?, ?) + `).bind('d1', 's1', JSON.stringify(names), '2026-10-01 12:00:00').run(); + const read = async () => JSON.parse((await ctx.db.prepare( + 'SELECT applied_secret_names FROM dashboard_sandboxes WHERE dashboard_id = ?').bind('d1').first<{ applied_secret_names: string }>())!.applied_secret_names); + return { env: ctx.env as unknown as Env, read }; + } + + it('replaces the tracking exactly when nothing changed since the pre-apply', async () => { + const { env, read } = await seeded(['OLD']); + await recоrdAppliedSecretNames(env, 'd1', 's1', '', ['A'], JSON.stringify(['OLD']), 'x'); + expect(await read()).toEqual(['A']); + }); + + it('merges names a concurrent push recorded', async () => { + const { env, read } = await seeded(['A', 'PUSHED']); + await recоrdAppliedSecretNames(env, 'd1', 's1', '', ['A'], JSON.stringify(['A']), 'x'); + expect((await read()).sort()).toEqual(['A', 'PUSHED']); + }); +}); diff --git a/controlplane/src/types.ts b/controlplane/src/types.ts index e292a81d..3e09ac80 100644 --- a/controlplane/src/types.ts +++ b/controlplane/src/types.ts @@ -29,6 +29,15 @@ export interface Env { /** Rate limiter for authenticated requests (200/min) */ RATE_LIMITER_AUTH: RateLimiter; INTERNAL_API_TOKEN: string; + /** Control-plane-only HMAC key for PTY + dashboard tokens. Never sent to sandboxes. */ + PTY_TOKEN_SECRET?: string; + /** 'true' = new sandbox VMs no longer receive INTERNAL_API_TOKEN; they authenticate + * to /internal/* only with their dashboard-scoped token (X-Dashboard-Token). */ + WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX?: string; + /** Ed25519 private key (PKCS#8 DER, base64) used to sign every control plane → + * sandbox request per target machine. VMs receive only the public key + * (SANDBOX_AUTH_PUBLIC_KEY). Set via: wrangler secret put SANDBOX_AUTH_SIGNING_KEY */ + SANDBOX_AUTH_SIGNING_KEY?: string; SANDBOX_INTERNAL_TOKEN: string; DEV_AUTH_ENABLED?: string; /** Desktop per-boot token: when set, dev-auth is only honored for requests diff --git a/controlplane/tests/mocks/d1.ts b/controlplane/tests/mocks/d1.ts index 75c4a1cf..f0551214 100644 --- a/controlplane/tests/mocks/d1.ts +++ b/controlplane/tests/mocks/d1.ts @@ -200,18 +200,39 @@ class MockD1PreparedStatement { const columns = colMatch[1].split(',').map(c => c.trim()); const row: Row = {}; + // VALUES may mix `?` with literals ('' / 'started' / datetime('now')). + const valuesStart = this.query.search(/VALUES\s*\(/i); + const values = splitTopLevel(this.query.slice(this.query.indexOf('(', valuesStart) + 1, matchingParen(this.query, this.query.indexOf('(', valuesStart)))); + let bindingIndex = 0; columns.forEach((col, i) => { - row[col] = this.bindings[i]; + const expr = values[i]?.trim() ?? '?'; + row[col] = expr === '?' ? this.bindings[bindingIndex++] : literalValue(expr); }); const isIgnore = /INSERT\s+OR\s+IGNORE/i.test(this.query); - if (isIgnore && columns.length > 0) { - const primaryKey = columns[0]; - const existing = this.tables.get(tableName)!.some(r => r[primaryKey] === row[primaryKey]); - if (existing) { - this.changes = 0; - return [] as T[]; + const upsert = this.query.match(/ON\s+CONFLICT\s*\((\w+)\)\s*DO\s+(NOTHING|UPDATE\s+SET\s+([\s\S]+?))(?:\s+WHERE\s+([\s\S]+?))?\s*$/i); + const conflictKey = upsert ? upsert[1] : columns[0]; + const existing = (isIgnore || upsert) && columns.length > 0 + ? this.tables.get(tableName)!.find(r => r[conflictKey] === row[conflictKey]) + : undefined; + if (existing) { + this.changes = 0; + if (!upsert || /^NOTHING$/i.test(upsert[2])) return [] as T[]; + // `WHERE table.col = 'literal'` guards the update (a no-op if it fails). + if (upsert[4]) { + const guard = upsert[4].match(/(?:\w+\.)?(\w+)\s*=\s*('(?:[^']|'')*')/); + if (guard && existing[guard[1]] !== literalValue(guard[2])) return [] as T[]; + } + for (const part of splitTopLevel(upsert[3])) { + const eq = part.indexOf('='); + if (eq === -1) continue; + const col = part.slice(0, eq).trim(); + const expr = part.slice(eq + 1).trim(); + const excluded = expr.match(/^excluded\.(\w+)$/i); + existing[col] = excluded ? row[excluded[1]] : expr === '?' ? this.bindings[bindingIndex++] : literalValue(expr); } + this.changes = 1; + return [] as T[]; } this.tables.get(tableName)!.push(row); @@ -247,6 +268,8 @@ class MockD1PreparedStatement { if (value !== null) { updates[col] = value; } + } else if (/^'(?:[^']|'')*'$/.test(part.substring(eqIndex + 1).trim())) { + updates[col] = literalValue(part.substring(eqIndex + 1).trim()); } else { updates[col] = this.bindings[bindingIndex++]; } @@ -327,3 +350,46 @@ class MockD1PreparedStatement { }); } } + +/** Splits a SQL list on commas outside parentheses and quotes. */ +function splitTopLevel(list: string): string[] { + const parts: string[] = []; + let depth = 0; + let quoted = false; + let current = ''; + for (const ch of list) { + if (ch === "'") quoted = !quoted; + if (!quoted && ch === '(') depth++; + if (!quoted && ch === ')') depth--; + if (!quoted && depth === 0 && ch === ',') { + parts.push(current); + current = ''; + } else { + current += ch; + } + } + if (current.trim()) parts.push(current); + return parts; +} + +function matchingParen(sql: string, open: number): number { + let depth = 0; + let quoted = false; + for (let i = open; i < sql.length; i++) { + if (sql[i] === "'") quoted = !quoted; + if (quoted) continue; + if (sql[i] === '(') depth++; + if (sql[i] === ')' && --depth === 0) return i; + } + return sql.length; +} + +/** A SQL literal as the mock stores it: strings unquoted, datetime('now') as an ISO time. */ +function literalValue(expr: string): unknown { + const str = expr.match(/^'((?:[^']|'')*)'$/); + if (str) return str[1].replace(/''/g, "'"); + if (/^datetime\('now'\)$/i.test(expr)) return new Date().toISOString(); + if (/^-?\d+(\.\d+)?$/.test(expr)) return Number(expr); + if (/^NULL$/i.test(expr)) return null; + return expr; +} diff --git a/desktop/CLAUDE.md b/desktop/CLAUDE.md index f5894579..3df1dbce 100644 --- a/desktop/CLAUDE.md +++ b/desktop/CLAUDE.md @@ -148,6 +148,24 @@ heredoc, not MININIT.** rc.local: console (root-only), readable from `/tmp/vz-console.log` — a non-root agent can't read it. This is the primitive the `orcabot` CLI uses to run guest shell commands. +### Guest clock sync +The VM has no NTP and its kernel reads the RTC only at boot, so while the host +sleeps the guest clock stops and then lags (signed control-plane requests then +fail with "issued in the future"). The host pushes its UTC time to a root-only +guest daemon, `orcabot-guest-timesync` (`sandbox/cmd/guest-timesync`, started by +`rc.local`), over **vsock port 1023**: right after boot, on host wake, and every +60s. The guest steps the clock only when drift > 2s and logs each step (old/new +delta) to the console. The port is reserved (<1024) and the daemon drops any peer +that isn't the host CID, so agents can't set the time. +- macOS: the loop lives in **vz-helper** (`GuestTimeSync`, `--guest-time-sync-port`; + NSWorkspace wake notification + wall-vs-uptime jump check). `macos.rs` passes + the flag only if the helper's `--help` lists it (a stale dev helper would + otherwise refuse to start the VM) — rebuild with `vz-helper/build.sh`. +- Linux QEMU: `vm/timesync.rs` thread over a `vhost-vsock-pci` device, enabled + only when `/dev/vhost-vsock` is accessible. +- Needs an image with the daemon (rc.local `rclocal-v3-guest-timesync`); on an + older image the host logs one "sync failed" line and carries on. + ### Image staging `vm/image.rs` stages `resources/vm/sandbox.img` → the **cache dir** (`~/Library/Caches/com.orcabot.desktop/vm/`, resolved via `app_cache_dir()` in @@ -237,10 +255,20 @@ matches** (`devAuthSurfaceTrusted` in `controlplane/src/auth/middleware.ts`). Th VM never gets the token (filtered from PTY env) and can't reach the frontend on `:8788` to scrape it (only `:8080` and `:8787` are bridged), so a VM process is confined to the token-authed paths it's supposed to use: the integration gateway -(PTY token) and `/internal/*` (`INTERNAL_API_TOKEN`). When `SURFACE_TOKEN` is unset +(PTY token) and `/internal/*` (its dashboard-scoped tokens). When `SURFACE_TOKEN` is unset (cloud, local `wrangler dev`, older builds) enforcement is off and behavior is unchanged. +**No baked VM tokens.** The image holds no shared secret. Each boot `main.rs` +generates a fresh Ed25519 key (`sandbox_auth_keys()`): the control-plane worker +gets the private key (`SANDBOX_AUTH_SIGNING_KEY`) and signs every request to the +VM; the guest gets only the public key on the kernel command line +(`orcabot.sandbox_auth_public_key=`, read by `rc.local`). `INTERNAL_API_TOKEN` is +also per-boot and host-only — the VM calls `/internal/*` with dashboard-scoped +tokens. Setting `SANDBOX_INTERNAL_TOKEN`/`INTERNAL_API_TOKEN` in the host env still +works (e.g. `scripts/dev.sh` against a hand-run sandbox). Changing `rc.local` means +a new VM image version (see `RELEASE.md`); a new app on an old image fails closed. + **Desktop-only**: production runs `DEV_AUTH_ENABLED=false` (real Cloudflare Access), where header spoofing doesn't work in the first place. @@ -298,7 +326,7 @@ production wrangler.production.toml — refer to that for descriptions. - OAuth client IDs + secrets: `GOOGLE_CLIENT_ID/SECRET`, `GITHUB_CLIENT_ID/SECRET`, `MICROSOFT_CLIENT_ID/SECRET`, `ONEDRIVE_CLIENT_ID/SECRET`, `BOX_CLIENT_ID/SECRET`, `TWITTER_CLIENT_ID/SECRET`, `DISCORD_CLIENT_ID/SECRET`, `SLACK_CLIENT_ID/SECRET` - `GOOGLE_API_KEY` — server-side Google APIs (Drive metadata etc.) - `RESEND_API_KEY` — transactional email -- `EGRESS_PROXY_ENABLED=true` — turn on the network egress proxy inside the sandbox VM (off by default; requires iptables setup) +- `EGRESS_PROXY_ENABLED=true` — turn on the network egress proxy inside the sandbox VM (off by default). It reaches the guest as `orcabot.egress_proxy_enabled=1` on the kernel command line (read by rc.local `rclocal-v4-egress-flag`; `VMConfig.env` is never delivered to the macOS/Linux guest). With it on, terminals run as the unprivileged pool UIDs with kernel default-deny egress; **without it every terminal and agent runs as root in the VM**, so the broker's keys, the browser user's isolation and other dashboards' workspaces are all within an agent's reach - `OAUTH_REDIRECT_BASE` — overrides the localhost OAuth callback base - `EMAIL_FROM` — overrides the default "OrcaBot Desktop " diff --git a/desktop/app/src-tauri/Cargo.lock b/desktop/app/src-tauri/Cargo.lock index c355f689..ce87edec 100644 --- a/desktop/app/src-tauri/Cargo.lock +++ b/desktop/app/src-tauri/Cargo.lock @@ -246,6 +246,12 @@ version = "0.22.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" +[[package]] +name = "base64ct" +version = "1.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" + [[package]] name = "bit-set" version = "0.8.0" @@ -540,6 +546,12 @@ dependencies = [ "crossbeam-utils", ] +[[package]] +name = "const-oid" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" + [[package]] name = "cookie" version = "0.18.1" @@ -708,6 +720,33 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "curve25519-dalek" +version = "4.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be" +dependencies = [ + "cfg-if", + "cpufeatures", + "curve25519-dalek-derive", + "digest", + "fiat-crypto", + "rustc_version", + "subtle", + "zeroize", +] + +[[package]] +name = "curve25519-dalek-derive" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "darling" version = "0.23.0" @@ -759,6 +798,16 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "der" +version = "0.7.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" +dependencies = [ + "const-oid", + "zeroize", +] + [[package]] name = "deranged" version = "0.5.8" @@ -943,6 +992,30 @@ version = "1.0.20" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555" +[[package]] +name = "ed25519" +version = "2.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53" +dependencies = [ + "pkcs8", + "signature", +] + +[[package]] +name = "ed25519-dalek" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9" +dependencies = [ + "curve25519-dalek", + "ed25519", + "serde", + "sha2", + "subtle", + "zeroize", +] + [[package]] name = "either" version = "1.16.0" @@ -1059,6 +1132,12 @@ dependencies = [ "simd-adler32", ] +[[package]] +name = "fiat-crypto" +version = "0.2.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" + [[package]] name = "field-offset" version = "0.3.6" @@ -2557,9 +2636,12 @@ checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" name = "orcabot-desktop" version = "0.9.3" dependencies = [ + "base64 0.22.1", "ctrlc", + "ed25519-dalek", "filetime", "flate2", + "getrandom 0.2.17", "libc", "ratatui", "serde", @@ -2739,6 +2821,16 @@ dependencies = [ "futures-io", ] +[[package]] +name = "pkcs8" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" +dependencies = [ + "der", + "spki", +] + [[package]] name = "pkg-config" version = "0.3.33" @@ -3594,6 +3686,15 @@ dependencies = [ "libc", ] +[[package]] +name = "signature" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" +dependencies = [ + "rand_core", +] + [[package]] name = "simd-adler32" version = "0.3.9" @@ -3692,6 +3793,16 @@ dependencies = [ "system-deps", ] +[[package]] +name = "spki" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +dependencies = [ + "base64ct", + "der", +] + [[package]] name = "stable_deref_trait" version = "1.2.1" diff --git a/desktop/app/src-tauri/Cargo.toml b/desktop/app/src-tauri/Cargo.toml index 0e1682c7..f1b74639 100644 --- a/desktop/app/src-tauri/Cargo.toml +++ b/desktop/app/src-tauri/Cargo.toml @@ -35,6 +35,13 @@ flate2 = "1" tar = "0.4" # Verify the on-demand VM image download against a hash baked into the binary. sha2 = "0.10" +# Per-boot Ed25519 key that signs control plane → sandbox VM requests; the VM +# gets only the public key (see sandbox_auth_keys in main.rs). +ed25519-dalek = "2" +# OS CSPRNG on every platform for per-boot secrets (already in the lockfile via +# ring/rand_core). random_bytes() in main.rs panics rather than fall back to zeros. +getrandom = "0.2" +base64 = "0.22" filetime = "0.2" ctrlc = "3" # SIGUSR1/SIGUSR2 surface toggle (show/hide the GUI window without restarting). diff --git a/desktop/app/src-tauri/build.rs b/desktop/app/src-tauri/build.rs index bc83ea8c..8d0034b3 100644 --- a/desktop/app/src-tauri/build.rs +++ b/desktop/app/src-tauri/build.rs @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + fn main() { // The packaged frontend is a REMOTE-origin webview (http://localhost:8788), and // Tauri v2's ACL rejects custom app commands from remote origins unless they're diff --git a/desktop/app/src-tauri/src/bin/orcabot.rs b/desktop/app/src-tauri/src/bin/orcabot.rs index 7044841b..b99780f9 100644 --- a/desktop/app/src-tauri/src/bin/orcabot.rs +++ b/desktop/app/src-tauri/src/bin/orcabot.rs @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + // REVISION: orcabot-cli-v24-import-stage-then-merge // // `orcabot` — command-line control for the Orcabot desktop stack. diff --git a/desktop/app/src-tauri/src/main.rs b/desktop/app/src-tauri/src/main.rs index 75a71d69..ea3f6ee1 100644 --- a/desktop/app/src-tauri/src/main.rs +++ b/desktop/app/src-tauri/src/main.rs @@ -1,7 +1,10 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + #![cfg_attr(not(debug_assertions), windows_subsystem = "windows")] -// REVISION: main-v13-vm-cache-dir -const MODULE_REVISION: &str = "main-v13-vm-cache-dir"; +// REVISION: main-v17-csprng-fail-closed +const MODULE_REVISION: &str = "main-v17-csprng-fail-closed"; mod commands; mod vm; @@ -162,6 +165,14 @@ fn passthrough_env(workerd_env: &mut Vec<(&'static str, String)>, key: &'static } } +/// Whether the user asked for the VM's egress proxy (EGRESS_PROXY_ENABLED=true), +/// matching the sandbox's own check. +fn egress_proxy_requested() -> bool { + std::env::var("EGRESS_PROXY_ENABLED") + .map(|v| v.trim().eq_ignore_ascii_case("true")) + .unwrap_or(false) +} + /// First free TCP port at/after `preferred` on loopback, skipping `used`. Falls /// back to `preferred` if nothing is free in range (the later bind then fails /// loudly). Used so the app boots even when a default port is occupied (e.g. a @@ -239,6 +250,23 @@ fn tee_child_stream( }); } +/// 32 bytes from the OS CSPRNG (getrandom: getentropy/getrandom/BCryptGenRandom), +/// on every platform. Panics if randomness is unavailable: every per-boot secret +/// (surface token, PTY HMAC key, INTERNAL_API_TOKEN, sandbox signing seed) must be +/// unpredictable, and an all-zero fallback would make tokens forgeable — failing +/// startup is the only safe outcome. +fn random_bytes_32() -> [u8; 32] { + let mut buf = [0u8; 32]; + if let Err(e) = getrandom::getrandom(&mut buf) { + panic!("[main] FATAL: OS random number generator unavailable ({e}); refusing to start with predictable secrets"); + } + buf +} + +fn random_hex_32() -> String { + random_bytes_32().iter().map(|b| format!("{:02x}", b)).collect() +} + /// Per-boot token that gates dev-auth to the trusted host frontend. Generated /// once at startup, passed to the control-plane worker (`SURFACE_TOKEN`) and /// handed to the GUI webview via the `get_surface_token` command. The sandbox VM @@ -248,19 +276,72 @@ fn tee_child_stream( static SURFACE_TOKEN: std::sync::OnceLock = std::sync::OnceLock::new(); pub fn surface_token() -> &'static str { - SURFACE_TOKEN.get_or_init(|| { - let mut buf = [0u8; 32]; - #[cfg(unix)] - { - use std::io::Read; - if let Ok(mut f) = std::fs::File::open("/dev/urandom") { - let _ = f.read_exact(&mut buf); + SURFACE_TOKEN.get_or_init(random_hex_32) +} + +/// Per-boot HMAC key for PTY + dashboard tokens (`PTY_TOKEN_SECRET`). Only the +/// host control-plane worker gets it — unlike INTERNAL_API_TOKEN, which is baked +/// into the sandbox VM image — so a VM process can't mint tokens for other +/// terminals. Rotating per boot just invalidates tokens of PTYs from the last run. +static PTY_TOKEN_SECRET: std::sync::OnceLock = std::sync::OnceLock::new(); + +fn pty_token_secret() -> &'static str { + PTY_TOKEN_SECRET.get_or_init(random_hex_32) +} + +/// Per-boot INTERNAL_API_TOKEN for host-side callers of the control plane's +/// /internal routes. The VM never gets it: it authenticates with its +/// dashboard-scoped tokens instead. Honors an explicit INTERNAL_API_TOKEN env. +static INTERNAL_API_TOKEN: std::sync::OnceLock = std::sync::OnceLock::new(); + +fn internal_api_token() -> &'static str { + INTERNAL_API_TOKEN.get_or_init(|| { + if let Ok(value) = std::env::var("INTERNAL_API_TOKEN") { + if !value.is_empty() { + return value; } } - buf.iter().map(|b| format!("{:02x}", b)).collect() + // Independent draw from pty_token_secret so the two keys stay unrelated. + random_hex_32() }) } +/// Per-boot Ed25519 key for control plane → sandbox auth (see +/// controlplane/src/sandbox/request-signing.ts). The control-plane worker gets +/// the private key (`SANDBOX_AUTH_SIGNING_KEY`, base64 PKCS#8); the VM gets only +/// the public key (`orcabot.sandbox_auth_public_key=` on the kernel command +/// line, base64url), which agents may read but can't sign with. Nothing the VM +/// holds authenticates a request to it, and there is no shared static token. +pub struct SandboxAuthKeys { + pub signing_key_pkcs8_b64: String, + pub public_key_b64url: String, +} + +static SANDBOX_AUTH_KEYS: std::sync::OnceLock> = std::sync::OnceLock::new(); + +/// Always Some today: random_bytes_32 panics (fails startup) when randomness is +/// unavailable rather than produce a predictable key. The Option is kept so the +/// sandbox side stays fail-closed if a None path is ever reintroduced. +pub fn sandbox_auth_keys() -> Option<&'static SandboxAuthKeys> { + SANDBOX_AUTH_KEYS + .get_or_init(|| { + use base64::Engine; + let seed = random_bytes_32(); + let signing = ed25519_dalek::SigningKey::from_bytes(&seed); + // PKCS#8 v1 wrapping of an Ed25519 seed (RFC 8410): fixed 16-byte prefix + seed. + let mut pkcs8 = vec![ + 0x30, 0x2e, 0x02, 0x01, 0x00, 0x30, 0x05, 0x06, 0x03, 0x2b, 0x65, 0x70, 0x04, 0x22, 0x04, 0x20, + ]; + pkcs8.extend_from_slice(&seed); + Some(SandboxAuthKeys { + signing_key_pkcs8_b64: base64::engine::general_purpose::STANDARD.encode(&pkcs8), + public_key_b64url: base64::engine::general_purpose::URL_SAFE_NO_PAD + .encode(signing.verifying_key().as_bytes()), + }) + }) + .as_ref() +} + /// Persist the surface token to a host-only file (0600) so trusted host clients /// that use dev-auth — the `orcabot` CLI and scripts — can read it and send the /// X-Orcabot-Surface header. The app-data dir is NOT shared into the sandbox VM @@ -659,10 +740,6 @@ impl DesktopServices { std::env::var("CONTROLPLANE_PORT").unwrap_or_else(|_| "8787".to_string()); let sandbox_url = std::env::var("SANDBOX_URL").unwrap_or_else(|_| "http://127.0.0.1:8080".to_string()); - let sandbox_internal_token = - std::env::var("SANDBOX_INTERNAL_TOKEN").unwrap_or_else(|_| "dev-sandbox-token".to_string()); - let internal_api_token = - std::env::var("INTERNAL_API_TOKEN").unwrap_or_else(|_| "dev-internal-token".to_string()); let dev_auth_enabled = std::env::var("DEV_AUTH_ENABLED").unwrap_or_else(|_| "true".to_string()); @@ -684,12 +761,12 @@ impl DesktopServices { let mut workerd_env = vec![ ("D1_HTTP_URL", "http://d1-shim".to_string()), ("SANDBOX_URL", sandbox_url), - ("SANDBOX_INTERNAL_TOKEN", sandbox_internal_token), - ("INTERNAL_API_TOKEN", internal_api_token.clone()), + ("INTERNAL_API_TOKEN", internal_api_token().to_string()), ("DEV_AUTH_ENABLED", dev_auth_enabled), // Gates dev-auth to the host frontend (which sends X-Orcabot-Surface with // this value); the sandbox VM never gets it, so it can't spoof user auth. ("SURFACE_TOKEN", surface_token().to_string()), + ("PTY_TOKEN_SECRET", pty_token_secret().to_string()), ("SECRETS_ENCRYPTION_KEY", secrets_key), ("OAUTH_REDIRECT_BASE", std::env::var("OAUTH_REDIRECT_BASE") .unwrap_or_else(|_| format!("http://localhost:{}", controlplane_port))), @@ -712,6 +789,17 @@ impl DesktopServices { if let Some(value) = d1_shim_debug { workerd_env.push(("D1_SHIM_DEBUG", value)); } + match sandbox_auth_keys() { + Some(keys) => workerd_env.push(("SANDBOX_AUTH_SIGNING_KEY", keys.signing_key_pkcs8_b64.clone())), + None => eprintln!("[sandbox-auth] ERROR: no randomness for the signing key — the sandbox will reject all requests"), + } + // Legacy static control plane → sandbox token: only when explicitly provided + // (e.g. a hand-run sandbox). Never baked; the VM doesn't get one. + if let Ok(value) = std::env::var("SANDBOX_INTERNAL_TOKEN") { + if !value.is_empty() { + workerd_env.push(("SANDBOX_INTERNAL_TOKEN", value)); + } + } // Optional pass-through for OAuth client IDs/secrets, Resend, etc. Users // who want these features set the env vars before launching the app; @@ -770,7 +858,7 @@ impl DesktopServices { // Apply the D1 schema on every launch (idempotent CREATE TABLE IF NOT EXISTS). // Without this, schema changes shipped in an app update never reach an existing // user's DB — the worker only runs init on a brand-new DB's first /health. - apply_schema(&controlplane_port, &internal_api_token); + apply_schema(&controlplane_port, internal_api_token()); // Write PID file so next launch can clean up orphans if we crash if let Ok(children) = self.children.lock() { @@ -858,9 +946,6 @@ impl DesktopServices { .and_then(|s| s.parse().ok()) .unwrap_or(8080); - let sandbox_internal_token = - std::env::var("SANDBOX_INTERNAL_TOKEN").unwrap_or_else(|_| "dev-sandbox-token".to_string()); - let allowed_origins = std::env::var("ALLOWED_ORIGINS").unwrap_or_else(|_| "http://localhost:8788".to_string()); @@ -875,9 +960,6 @@ impl DesktopServices { // CONTROLPLANE_URL explicitly when the controlplane lives elsewhere. vm::host_loopback_url(&port) }); - let internal_api_token = - std::env::var("INTERNAL_API_TOKEN").unwrap_or_else(|_| "dev-internal-token".to_string()); - // Host-side control-plane port for the guest→host reverse bridge. Matches the // port the control-plane workerd actually bound to (possibly dynamic). The // guest side of the bridge stays baked at 8787. @@ -894,21 +976,17 @@ impl DesktopServices { // Guest binds 8080 (image default); the host→guest forward maps the dynamic // host port to that. PORT here is the guest bind, not the host listen. .with_env("PORT", vm::SANDBOX_GUEST_PORT.to_string()) - .with_env("SANDBOX_INTERNAL_TOKEN", sandbox_internal_token) .with_env("ALLOWED_ORIGINS", allowed_origins) .with_env("WORKSPACE_BASE", "/workspace") - .with_env("CONTROLPLANE_URL", controlplane_url) - .with_env("INTERNAL_API_TOKEN", internal_api_token); - - // Opt-in: enable the network egress proxy inside the VM. Off by default - // because it requires iptables setup at boot; users who want it set the - // env var before launching. - if let Ok(value) = std::env::var("EGRESS_PROXY_ENABLED") { - if !value.is_empty() { - config = config.with_env("EGRESS_PROXY_ENABLED", value); - } + .with_env("CONTROLPLANE_URL", controlplane_url); + // No SANDBOX_INTERNAL_TOKEN / INTERNAL_API_TOKEN: inbound requests are signed + // per boot (public key on the kernel command line below), and outbound calls + // use the dashboard-scoped tokens the control plane hands each session. + if let Some(keys) = sandbox_auth_keys() { + config = config.with_env("SANDBOX_AUTH_PUBLIC_KEY", keys.public_key_b64url.clone()); } + // Add kernel/initrd/vz-helper for macOS direct boot if let Some(kernel) = staged_paths.kernel { config = config.with_kernel(kernel); @@ -931,6 +1009,24 @@ impl DesktopServices { } else { "console=ttyS0 root=/dev/vda rw net.ifnames=0 biosdevname=0 quiet" }; + // The kernel command line is the one host → guest channel on every backend. + // Agents can read /proc/cmdline, which is fine for a public key and is why + // no secret travels this way. + let mut cmdline = match sandbox_auth_keys() { + Some(keys) => format!("{} orcabot.sandbox_auth_public_key={}", cmdline, keys.public_key_b64url), + None => cmdline.to_string(), + }; + // Opt-in: enable the network egress proxy inside the VM (off by default; it + // prompts for unknown domains). With it on, terminals run as the unprivileged + // pool UIDs instead of root, which is what keeps the broker's keys, the + // browser user and other dashboards out of an agent's reach. The guest + // never sees config.env, so it travels on the command line like the key; + // rc.local exports it (rclocal-v4-egress-flag). Same rule as the sandbox: + // only "true" (any case) turns it on. + // REVISION: main-v16-egress-flag-cmdline + if egress_proxy_requested() { + cmdline.push_str(" orcabot.egress_proxy_enabled=1"); + } config = config.with_cmdline(cmdline); // Create and start VM — unless an update was accepted while we were staging. diff --git a/desktop/app/src-tauri/src/vm/config.rs b/desktop/app/src-tauri/src/vm/config.rs index 280c66e4..24f6934d 100644 --- a/desktop/app/src-tauri/src/vm/config.rs +++ b/desktop/app/src-tauri/src/vm/config.rs @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + use std::collections::HashMap; use std::path::PathBuf; @@ -82,7 +85,9 @@ impl VMConfig { self } - /// Add an environment variable. + /// Add an environment variable. Only the Windows stub delivers these: the + /// macOS (vz-helper) and Linux (QEMU) guests never see them, so anything the + /// guest must act on goes on the kernel command line (see main.rs). pub fn with_env(mut self, key: impl Into, value: impl Into) -> Self { self.env.insert(key.into(), value.into()); self diff --git a/desktop/app/src-tauri/src/vm/error.rs b/desktop/app/src-tauri/src/vm/error.rs index b9a411b2..a9567720 100644 --- a/desktop/app/src-tauri/src/vm/error.rs +++ b/desktop/app/src-tauri/src/vm/error.rs @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + // REVISION: vm-image-ondemand-v1 use std::path::PathBuf; use std::time::Duration; diff --git a/desktop/app/src-tauri/src/vm/image.rs b/desktop/app/src-tauri/src/vm/image.rs index f0a8f4b1..aed11ac2 100644 --- a/desktop/app/src-tauri/src/vm/image.rs +++ b/desktop/app/src-tauri/src/vm/image.rs @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + //! VM image staging, download-on-demand, and decompression utilities. //! //! The multi-GB disk image is NOT bundled in the app — it's fetched on demand diff --git a/desktop/app/src-tauri/src/vm/linux.rs b/desktop/app/src-tauri/src/vm/linux.rs index 5dc32d3c..a02b8237 100644 --- a/desktop/app/src-tauri/src/vm/linux.rs +++ b/desktop/app/src-tauri/src/vm/linux.rs @@ -1,3 +1,7 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v1-host-push //! Linux VM implementation using QEMU/KVM. //! //! This implementation spawns a QEMU process with KVM acceleration. @@ -8,6 +12,8 @@ use super::{VMConfig, VMError, VirtualMachine}; use std::io::{Read, Write}; use std::net::TcpStream; use std::process::{Child, Command, Stdio}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; use std::time::{Duration, Instant}; /// Linux VM using QEMU/KVM. @@ -24,6 +30,11 @@ pub struct QemuVM { sandbox_url: String, /// Path to virtiofsd socket virtiofs_socket: Option, + /// Guest CID of the vhost-vsock device (None when /dev/vhost-vsock is not + /// usable), used for the guest clock sync channel. + vsock_cid: Option, + /// Stops the guest clock sync thread. + timesync_stop: Option>, } impl QemuVM { @@ -35,9 +46,33 @@ impl QemuVM { running: false, sandbox_url: "http://127.0.0.1:8080".to_string(), virtiofs_socket: None, + vsock_cid: None, + timesync_stop: None, } } + /// A vhost-vsock device needs read/write access to /dev/vhost-vsock (often + /// root/kvm-group only). Without it the VM still boots, just without the + /// host→guest clock sync channel. + fn vhost_vsock_usable() -> bool { + std::fs::OpenOptions::new() + .read(true) + .write(true) + .open("/dev/vhost-vsock") + .is_ok() + } + + /// Guest CIDs are host-global; pick one unlikely to collide with another VM + /// (0-2 and u32::MAX are reserved). + fn pick_guest_cid() -> u32 { + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.subsec_nanos() as u64) + .unwrap_or(0); + let mix = (std::process::id() as u64).wrapping_mul(2_654_435_761) ^ nanos; + 3 + (mix % (u32::MAX as u64 - 16)) as u32 + } + /// Check if KVM is available. fn is_kvm_available() -> bool { std::path::Path::new("/dev/kvm").exists() @@ -149,6 +184,12 @@ impl QemuVM { ]); cmd.args(["-device", "virtio-net-pci,netdev=net0"]); + // vsock: host→guest clock sync channel (see vm/timesync.rs). Only the + // guest's root daemon listens; it accepts the host CID only. + if let Some(cid) = self.vsock_cid { + cmd.args(["-device", &format!("vhost-vsock-pci,guest-cid={}", cid)]); + } + // VirtioFS for shared workspace (if virtiofsd is running) if let Some(ref socket_path) = self.virtiofs_socket { cmd.args([ @@ -219,6 +260,16 @@ impl VirtualMachine for QemuVM { eprintln!("Warning: virtiofsd not found, using 9p for shared filesystem"); } + self.vsock_cid = if Self::vhost_vsock_usable() { + Some(Self::pick_guest_cid()) + } else { + eprintln!( + "[vm] /dev/vhost-vsock not accessible: guest clock sync disabled \ + (guest clock may drift after host suspend)" + ); + None + }; + // Build and start QEMU let mut cmd = self.build_qemu_command(config, use_kvm); cmd.stdout(Stdio::inherit()); @@ -233,10 +284,37 @@ impl VirtualMachine for QemuVM { self.running = true; self.sandbox_url = format!("http://127.0.0.1:{}", config.sandbox_port); + // Keep the guest wall clock in step with the host: after boot, after + // host suspend (wall-vs-monotonic jump) and every 60s. + if let Some(cid) = self.vsock_cid { + eprintln!( + "[vm] REVISION: {} guest clock sync -> vsock cid {} port {}", + super::timesync::TIMESYNC_REVISION, + cid, + super::GUEST_TIMESYNC_VSOCK_PORT + ); + let stop = Arc::new(AtomicBool::new(false)); + super::timesync::spawn_sync_thread( + move || { + super::timesync::connect_vsock( + cid, + super::GUEST_TIMESYNC_VSOCK_PORT, + Duration::from_secs(5), + ) + }, + stop.clone(), + ); + self.timesync_stop = Some(stop); + } + Ok(()) } fn stop(&mut self) -> Result<(), VMError> { + if let Some(stop) = self.timesync_stop.take() { + stop.store(true, Ordering::Relaxed); + } + // Stop QEMU if let Some(ref mut child) = self.qemu_process { let _ = child.kill(); diff --git a/desktop/app/src-tauri/src/vm/macos.rs b/desktop/app/src-tauri/src/vm/macos.rs index 03391c18..daf36d2f 100644 --- a/desktop/app/src-tauri/src/vm/macos.rs +++ b/desktop/app/src-tauri/src/vm/macos.rs @@ -1,3 +1,7 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v1-host-push //! macOS VM implementation using Apple Virtualization.framework. //! //! This implementation uses native macOS Virtualization.framework to boot @@ -89,6 +93,19 @@ impl MacOSVM { .unwrap_or(false) } + /// Whether this vz-helper build accepts `--guest-time-sync-port`. + fn helper_supports_timesync(helper_path: &Path) -> bool { + Command::new(helper_path) + .arg("--help") + .stdin(Stdio::null()) + .output() + .map(|o| { + String::from_utf8_lossy(&o.stdout).contains("--guest-time-sync-port") + || String::from_utf8_lossy(&o.stderr).contains("--guest-time-sync-port") + }) + .unwrap_or(false) + } + /// Start VM using native Virtualization.framework via Swift helper. /// /// This spawns a small Swift helper process that manages the VZ VM, @@ -165,6 +182,24 @@ impl MacOSVM { "--reverse-port-forward", &format!("{}:{}", CONTROLPLANE_PORT, config.controlplane_host_port), ]); + // Guest clock sync: vz-helper pushes host UTC to the guest's root-only + // daemon on this vsock port after boot, on host wake and every 60s (the + // guest has no NTP and its clock stops while the Mac sleeps, which breaks + // every signed control-plane request). See vm/timesync.rs. Only passed + // when the helper knows the flag: a stale dev vz-helper would otherwise + // reject the unknown option and the VM would not boot at all. + if Self::helper_supports_timesync(&helper_path) { + cmd.args([ + "--guest-time-sync-port", + &super::GUEST_TIMESYNC_VSOCK_PORT.to_string(), + ]); + } else { + eprintln!( + "[vm] WARNING: vz-helper at {} predates --guest-time-sync-port; guest \ + clock sync DISABLED (rebuild vz-helper: vz-helper/build.sh)", + helper_path.display() + ); + } cmd.stdout(Stdio::inherit()); cmd.stderr(Stdio::inherit()); @@ -191,6 +226,15 @@ impl MacOSVM { (guest→host control plane :{}) active.", config.sandbox_port, config.controlplane_host_port ); + eprintln!( + "[vm] REVISION: {} guest clock sync via vz-helper -> vsock:{} (at {})", + super::timesync::TIMESYNC_REVISION, + super::GUEST_TIMESYNC_VSOCK_PORT, + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| format!("{}s", d.as_secs())) + .unwrap_or_default() + ); Ok(()) } diff --git a/desktop/app/src-tauri/src/vm/mod.rs b/desktop/app/src-tauri/src/vm/mod.rs index b6351b22..ef82df61 100644 --- a/desktop/app/src-tauri/src/vm/mod.rs +++ b/desktop/app/src-tauri/src/vm/mod.rs @@ -1,3 +1,7 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v1-host-push //! Virtual machine abstraction for running the sandbox server. //! //! This module provides a platform-agnostic interface for managing VMs: @@ -18,6 +22,7 @@ pub mod windows; pub mod linux; pub mod image; +pub mod timesync; pub use config::VMConfig; pub use error::VMError; @@ -28,6 +33,11 @@ pub use error::VMError; /// (`VMConfig.sandbox_port`) may be dynamic when 8080 is busy on the host. pub const SANDBOX_GUEST_PORT: u16 = 8080; +/// Guest vsock port of the root-only clock-sync daemon (`orcabot-guest-timesync`, +/// started by the image's rc.local). Reserved range (<1024) so no unprivileged +/// guest process can bind it; baked into the image. See `timesync.rs`. +pub const GUEST_TIMESYNC_VSOCK_PORT: u32 = 1023; + use std::time::Duration; /// Trait for platform-specific VM implementations. diff --git a/desktop/app/src-tauri/src/vm/timesync.rs b/desktop/app/src-tauri/src/vm/timesync.rs new file mode 100644 index 00000000..37b2fae8 --- /dev/null +++ b/desktop/app/src-tauri/src/vm/timesync.rs @@ -0,0 +1,375 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v1-host-push +//! Host side of the guest clock sync. +//! +//! The sandbox VM has no NTP client and its kernel reads the RTC only at boot, +//! so after the host sleeps the guest wall clock lags by the sleep duration. +//! Control-plane -> sandbox requests are Ed25519-signed with iat/exp and the +//! guest rejects an iat more than 60s in its future, so a lagging guest fails +//! every signed request until its clock is fixed. +//! +//! The host pushes its UTC time to a root-only daemon in the guest +//! (`orcabot-guest-timesync`, sandbox/cmd/guest-timesync, started by rc.local) +//! over vsock port [`super::GUEST_TIMESYNC_VSOCK_PORT`]. The guest steps its +//! clock when drift > 2s and replies with the before/after delta. The guest only +//! accepts the host's CID on a reserved port, so agent processes in the VM can't +//! set the time. +//! +//! - macOS (Virtualization.framework): the push loop lives in vz-helper +//! (`GuestTimeSync` in vz-helper/Sources/main.swift), which owns the vsock +//! device and observes NSWorkspace wake notifications. Nothing here runs. +//! - Linux (QEMU/KVM): [`spawn_sync_thread`] below, over a vhost-vsock device. +//! +//! Wire protocol (one line each way): +//! host -> guest: `ORCABOT-TIMESYNC 1 \n` +//! guest -> host: `OK stepped=<0|1> before_ms= after_ms= rev=\n` +//! or `ERR \n` + +#![cfg_attr(not(target_os = "linux"), allow(dead_code))] + +use std::io::{self, Read, Write}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; + +pub const TIMESYNC_REVISION: &str = "guest-timesync-v1-host-push"; + +const REQUEST_PREFIX: &str = "ORCABOT-TIMESYNC 1 "; +/// Scheduler tick; also the wake-detection granularity. +pub const TICK: Duration = Duration::from_secs(5); +/// Backstop: push at least this often once in sync. +pub const PERIOD: Duration = Duration::from_secs(60); +/// Wall time advancing this much more than monotonic time between two ticks +/// means the host slept (monotonic clocks stop during suspend) or was reset. +const WAKE_JUMP: Duration = Duration::from_secs(3); +/// Fast (every tick) attempts before the first success: covers guest boot. +const BOOT_FAST_ATTEMPTS: u32 = 24; +const RETRY_AFTER_FAILURE: Duration = Duration::from_secs(10); +const FAILURE_LOG_INTERVAL: Duration = Duration::from_secs(300); + +fn now_epoch_secs() -> String { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| format!("{}s", d.as_secs())) + .unwrap_or_default() +} + +pub fn request_line(now: SystemTime) -> String { + let ns = now + .duration_since(UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + format!("{}{}\n", REQUEST_PREFIX, ns) +} + +#[derive(Debug, PartialEq, Eq)] +pub struct Reply { + pub stepped: bool, + pub before_ms: i64, + pub after_ms: i64, + pub rev: String, +} + +pub fn parse_reply(line: &str) -> Result { + let line = line.trim(); + if let Some(err) = line.strip_prefix("ERR ") { + return Err(format!("guest error: {}", err)); + } + let rest = line + .strip_prefix("OK ") + .ok_or_else(|| format!("unexpected reply: {:?}", line))?; + let mut reply = Reply { stepped: false, before_ms: 0, after_ms: 0, rev: String::new() }; + for kv in rest.split_whitespace() { + match kv.split_once('=') { + Some(("stepped", v)) => reply.stepped = v == "1", + Some(("before_ms", v)) => reply.before_ms = v.parse().map_err(|_| format!("bad before_ms {:?}", v))?, + Some(("after_ms", v)) => reply.after_ms = v.parse().map_err(|_| format!("bad after_ms {:?}", v))?, + Some(("rev", v)) => reply.rev = v.to_string(), + _ => {} + } + } + Ok(reply) +} + +/// One request/reply on an already-connected stream. The host time is taken +/// right before the write so connect latency doesn't count as drift. +pub fn exchange(stream: &mut S) -> Result { + stream + .write_all(request_line(SystemTime::now()).as_bytes()) + .map_err(|e| format!("write: {}", e))?; + let mut out = Vec::with_capacity(128); + let mut buf = [0u8; 256]; + while out.len() < 512 { + match stream.read(&mut buf) { + Ok(0) => break, + Ok(n) => { + out.extend_from_slice(&buf[..n]); + if out.contains(&b'\n') { + break; + } + } + Err(e) if e.kind() == io::ErrorKind::Interrupted => continue, + Err(e) => return Err(format!("read: {}", e)), + } + } + let line = String::from_utf8_lossy(&out); + if line.trim().is_empty() { + return Err("no reply (guest daemon missing on an older VM image?)".into()); + } + parse_reply(&line) +} + +/// Decides when to push. Pure, so it is unit-testable without a VM. +pub struct Scheduler { + next_due: Instant, + ever_synced: bool, + consecutive_failures: u32, + last_wall: SystemTime, + last_mono: Instant, + last_failure_log: Option, +} + +impl Scheduler { + pub fn new(now_wall: SystemTime, now_mono: Instant) -> Self { + Self { + next_due: now_mono, + ever_synced: false, + consecutive_failures: 0, + last_wall: now_wall, + last_mono: now_mono, + last_failure_log: None, + } + } + + /// Returns why a push is due now, if it is. + pub fn due(&mut self, now_wall: SystemTime, now_mono: Instant) -> Option<&'static str> { + let wall = match now_wall.duration_since(self.last_wall) { + Ok(d) => d.as_secs_f64(), + Err(e) => -e.duration().as_secs_f64(), + }; + let mono = now_mono.saturating_duration_since(self.last_mono).as_secs_f64(); + self.last_wall = now_wall; + self.last_mono = now_mono; + if (wall - mono).abs() > WAKE_JUMP.as_secs_f64() { + return Some("wake"); + } + if now_mono >= self.next_due { + return Some(if self.ever_synced { "periodic" } else { "boot" }); + } + None + } + + pub fn record_success(&mut self, now_mono: Instant) { + self.ever_synced = true; + self.consecutive_failures = 0; + self.next_due = now_mono + PERIOD; + } + + /// Records a failure; returns true when it should be logged (quiet while the + /// guest is still booting, then at most every FAILURE_LOG_INTERVAL). + pub fn record_failure(&mut self, now_mono: Instant) -> bool { + self.consecutive_failures += 1; + let booting = !self.ever_synced && self.consecutive_failures < BOOT_FAST_ATTEMPTS; + self.next_due = now_mono + + if self.ever_synced { + RETRY_AFTER_FAILURE + } else if booting { + // Due again on the very next tick, not the one after. + TICK - Duration::from_secs(1) + } else { + PERIOD + }; + if booting { + return false; + } + let log = self + .last_failure_log + .map_or(true, |t| now_mono.saturating_duration_since(t) >= FAILURE_LOG_INTERVAL); + if log { + self.last_failure_log = Some(now_mono); + } + log + } + + pub fn ever_synced(&self) -> bool { + self.ever_synced + } + + pub fn consecutive_failures(&self) -> u32 { + self.consecutive_failures + } +} + +/// Runs the push loop on a background thread until `stop` is set. `connect` +/// opens a fresh stream to the guest daemon for each push. +pub fn spawn_sync_thread(mut connect: F, stop: Arc) -> std::thread::JoinHandle<()> +where + S: Read + Write, + F: FnMut() -> io::Result + Send + 'static, +{ + std::thread::Builder::new() + .name("guest-timesync".into()) + .spawn(move || { + eprintln!( + "[timesync] REVISION: {} guest clock sync loop started at {} (tick {}s, period {}s)", + TIMESYNC_REVISION, + now_epoch_secs(), + TICK.as_secs(), + PERIOD.as_secs() + ); + let mut sched = Scheduler::new(SystemTime::now(), Instant::now()); + while !stop.load(Ordering::Relaxed) { + if let Some(reason) = sched.due(SystemTime::now(), Instant::now()) { + if reason == "wake" { + eprintln!("[timesync] host wake / wall-clock jump detected -> syncing guest clock"); + } + let result = connect() + .map_err(|e| format!("vsock connect: {}", e)) + .and_then(|mut s| exchange(&mut s)); + match result { + Ok(r) => { + if r.stepped { + eprintln!( + "[timesync] guest clock STEPPED ({}): delta before={}ms after={}ms (positive = guest was behind)", + reason, r.before_ms, r.after_ms + ); + } else if !sched.ever_synced() { + eprintln!( + "[timesync] guest clock in sync ({}): delta={}ms, guest {}", + reason, r.before_ms, r.rev + ); + } + sched.record_success(Instant::now()); + } + Err(e) => { + if sched.record_failure(Instant::now()) { + eprintln!( + "[timesync] guest clock sync failed ({}, {} in a row): {}", + reason, + sched.consecutive_failures(), + e + ); + } + } + } + } + std::thread::sleep(TICK); + } + }) + .expect("spawn guest-timesync thread") +} + +/// Connects to a guest vsock port (Linux host with a vhost-vsock device). +#[cfg(target_os = "linux")] +pub fn connect_vsock(cid: u32, port: u32, timeout: Duration) -> io::Result { + use std::os::fd::{FromRawFd, OwnedFd}; + unsafe { + let fd = libc::socket(libc::AF_VSOCK, libc::SOCK_STREAM | libc::SOCK_CLOEXEC, 0); + if fd < 0 { + return Err(io::Error::last_os_error()); + } + let owned = OwnedFd::from_raw_fd(fd); + let mut addr: libc::sockaddr_vm = std::mem::zeroed(); + addr.svm_family = libc::AF_VSOCK as libc::sa_family_t; + addr.svm_port = port; + addr.svm_cid = cid; + if libc::connect( + fd, + &addr as *const libc::sockaddr_vm as *const libc::sockaddr, + std::mem::size_of::() as libc::socklen_t, + ) < 0 + { + return Err(io::Error::last_os_error()); + } + let tv = libc::timeval { + tv_sec: timeout.as_secs() as libc::time_t, + tv_usec: 0, + }; + for opt in [libc::SO_RCVTIMEO, libc::SO_SNDTIMEO] { + libc::setsockopt( + fd, + libc::SOL_SOCKET, + opt, + &tv as *const libc::timeval as *const libc::c_void, + std::mem::size_of::() as libc::socklen_t, + ); + } + Ok(std::fs::File::from(owned)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::net::{TcpListener, TcpStream}; + + #[test] + fn request_and_reply_roundtrip() { + let t = UNIX_EPOCH + Duration::from_nanos(1_760_000_000_123_456_789); + assert_eq!(request_line(t), "ORCABOT-TIMESYNC 1 1760000000123456789\n"); + let r = parse_reply("OK stepped=1 before_ms=401000 after_ms=2 rev=guest-timesync-v1\n").unwrap(); + assert_eq!( + r, + Reply { stepped: true, before_ms: 401000, after_ms: 2, rev: "guest-timesync-v1".into() } + ); + assert!(parse_reply("ERR settimeofday: EPERM").is_err()); + assert!(parse_reply("HTTP/1.1 400").is_err()); + } + + #[test] + fn exchange_against_fake_guest() { + let l = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = l.local_addr().unwrap(); + let guest = std::thread::spawn(move || { + let (mut s, _) = l.accept().unwrap(); + let mut buf = [0u8; 128]; + let n = s.read(&mut buf).unwrap(); + let line = String::from_utf8_lossy(&buf[..n]).to_string(); + s.write_all(b"OK stepped=0 before_ms=12 after_ms=12 rev=x\n").unwrap(); + line + }); + let mut s = TcpStream::connect(addr).unwrap(); + let r = exchange(&mut s).unwrap(); + assert!(!r.stepped); + assert_eq!(r.before_ms, 12); + assert!(guest.join().unwrap().starts_with("ORCABOT-TIMESYNC 1 ")); + } + + #[test] + fn scheduler_boot_period_and_wake() { + let w0 = SystemTime::now(); + let m0 = Instant::now(); + let mut s = Scheduler::new(w0, m0); + // Due immediately at boot. + assert_eq!(s.due(w0, m0), Some("boot")); + // Boot failures are retried each tick and not logged. + assert!(!s.record_failure(m0)); + assert_eq!(s.due(w0 + TICK, m0 + TICK), Some("boot")); + s.record_success(m0 + TICK); + // Not due again until the period elapses. + assert_eq!(s.due(w0 + TICK * 2, m0 + TICK * 2), None); + let m = m0 + TICK + PERIOD; + let w = w0 + TICK + PERIOD; + assert_eq!(s.due(w, m), Some("periodic")); + s.record_success(m); + // Host slept 400s: wall advanced, monotonic didn't -> immediate push. + assert_eq!(s.due(w + Duration::from_secs(400) + TICK, m + TICK), Some("wake")); + } + + #[test] + fn scheduler_failure_logging_is_rate_limited() { + let w0 = SystemTime::now(); + let m0 = Instant::now(); + let mut s = Scheduler::new(w0, m0); + let mut logged = 0; + for i in 0..BOOT_FAST_ATTEMPTS + 5 { + if s.record_failure(m0 + TICK * i) { + logged += 1; + } + } + // One log once the boot window is exhausted, then quiet for 5 min. + assert_eq!(logged, 1); + } +} diff --git a/desktop/app/src-tauri/src/vm/windows.rs b/desktop/app/src-tauri/src/vm/windows.rs index d0ac8df1..1d73c12c 100644 --- a/desktop/app/src-tauri/src/vm/windows.rs +++ b/desktop/app/src-tauri/src/vm/windows.rs @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + //! Windows VM implementation using WSL2. //! //! This implementation manages a custom WSL2 distribution containing diff --git a/desktop/app/src-tauri/vz-helper/Package.swift b/desktop/app/src-tauri/vz-helper/Package.swift index 21039b3b..1803658a 100644 --- a/desktop/app/src-tauri/vz-helper/Package.swift +++ b/desktop/app/src-tauri/vz-helper/Package.swift @@ -1,4 +1,7 @@ // swift-tools-version:5.9 +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + import PackageDescription let package = Package( diff --git a/desktop/app/src-tauri/vz-helper/Sources/main.swift b/desktop/app/src-tauri/vz-helper/Sources/main.swift index de20d7ea..46929790 100644 --- a/desktop/app/src-tauri/vz-helper/Sources/main.swift +++ b/desktop/app/src-tauri/vz-helper/Sources/main.swift @@ -1,14 +1,22 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: vz-helper-v5-copyright-header +import AppKit import Foundation import Virtualization import ArgumentParser import Network +let vzHelperRevision = "vz-helper-v5-copyright-header" + // MARK: - Global references to prevent deallocation var globalVM: VZVirtualMachine? var globalForwarders: [TCPToVsockForwarder] = [] var globalVsockListeners: [VsockListener] = [] var globalConsolePipe: Pipe? var globalConsoleLogFile: FileHandle? +var globalTimeSync: GuestTimeSync? @main struct VZHelper: ParsableCommand { @@ -44,6 +52,9 @@ struct VZHelper: ParsableCommand { @Option(name: .long, help: "Reverse forward: guest vsock port to host TCP port (vsockPort:hostPort). Lets the guest reach host loopback services, e.g. the control plane.") var reversePortForward: [String] = [] + @Option(name: .long, help: "Guest vsock port of the root time-sync daemon (orcabot-guest-timesync). When set, the host pushes its UTC time to the guest after boot, on host wake and every 60s. 0 = off.") + var guestTimeSyncPort: UInt32 = 0 + @Flag(name: .long, help: "Disable directory sharing (for debugging)") var noShare: Bool = false @@ -54,6 +65,7 @@ struct VZHelper: ParsableCommand { var minimal: Bool = false mutating func run() throws { + print("[vz-helper] REVISION: \(vzHelperRevision) loaded at \(ISO8601DateFormatter().string(from: Date()))") print("Starting Orcabot sandbox VM...") print(" Kernel: \(kernel)") print(" Initrd: \(initrd)") @@ -362,6 +374,15 @@ struct VZHelper: ParsableCommand { fflush(stdout) } + // Keep the guest wall clock in step with the host. The guest has no NTP and + // its clock stops while the host sleeps, so without this every signed + // control-plane request fails after a sleep ("issued in the future"). + if guestTimeSyncPort > 0 { + let ts = GuestTimeSync(vm: vm, port: guestTimeSyncPort) + globalTimeSync = ts + ts.start() + } + // Handle signals for graceful shutdown signal(SIGINT) { _ in print("\nReceived SIGINT, shutting down...") @@ -580,6 +601,215 @@ class VsockListener: NSObject, VZVirtioSocketListenerDelegate { } } +// MARK: - Guest clock sync (host -> guest) + +/// Pushes the host's UTC time to the guest's root-only time-sync daemon +/// (`orcabot-guest-timesync`, sandbox/cmd/guest-timesync) over vsock. The guest +/// steps its clock when the drift exceeds 2s and replies with the delta. +/// +/// When: right after boot (retried every 5s until the guest daemon answers), on +/// host wake (NSWorkspace.didWakeNotification, plus a wall-clock-vs-uptime jump +/// check as a backstop since uptime stops while the Mac sleeps), and every 60s. +/// +/// Only the host can reach this: the guest listens on a reserved vsock port and +/// drops any peer that isn't CID 2 (the host), so agent processes can't set the +/// time. Nothing is exposed on a host TCP port either. +final class GuestTimeSync { + let port: UInt32 + weak var vm: VZVirtualMachine? + + static let tick: TimeInterval = 5 + static let period: TimeInterval = 60 + static let wakeJumpThreshold: TimeInterval = 3 + /// Before the first success, retry every tick for this many attempts (~2 min, + /// covers guest boot), then fall back to the period (e.g. an old VM image + /// without the daemon) so we don't spin forever. + static let bootFastAttempts = 24 + + private var timer: DispatchSourceTimer? + private var wakeObserver: NSObjectProtocol? + private var lastWall = Date() + private var lastUptime = ProcessInfo.processInfo.systemUptime + private var nextDue = Date() + private var everSynced = false + private var inFlight = false + private var inFlightSince = Date.distantPast + /// Bumped per attempt so a late completion of an abandoned attempt is ignored. + private var attempt: UInt64 = 0 + static let attemptTimeout: TimeInterval = 20 + private var consecutiveFailures = 0 + private var lastFailureLog: Date = .distantPast + private let ioQueue = DispatchQueue(label: "guest-timesync-io") + + init(vm: VZVirtualMachine, port: UInt32) { + self.vm = vm + self.port = port + } + + private func log(_ msg: String) { + print("[timesync] \(msg)") + fflush(stdout) + } + + /// Everything runs on the main queue (VZ socket APIs require it); only the + /// blocking socket I/O hops to ioQueue. + func start() { + log("REVISION: \(vzHelperRevision) guest clock sync -> vsock:\(port) (tick \(Int(Self.tick))s, period \(Int(Self.period))s) started at \(ISO8601DateFormatter().string(from: Date()))") + wakeObserver = NSWorkspace.shared.notificationCenter.addObserver( + forName: NSWorkspace.didWakeNotification, object: nil, queue: .main + ) { [weak self] _ in + guard let self = self else { return } + self.log("host wake notification -> syncing guest clock") + self.syncNоw(reason: "wake") + // The VM may need a moment to resume; push once more shortly after. + DispatchQueue.main.asyncAfter(deadline: .now() + 3) { [weak self] in + self?.syncNоw(reason: "wake+3s") + } + } + let t = DispatchSource.makeTimerSource(queue: .main) + t.schedule(deadline: .now() + 1, repeating: Self.tick, leeway: .milliseconds(500)) + t.setEventHandler { [weak self] in self?.оnTick() } + t.resume() + timer = t + } + + private func оnTick() { + let now = Date() + let up = ProcessInfo.processInfo.systemUptime + // systemUptime does not advance while the Mac sleeps, wall time does: a + // gap between the two means we just woke (or the host clock was changed). + let jump = now.timeIntervalSince(lastWall) - (up - lastUptime) + lastWall = now + lastUptime = up + if abs(jump) > Self.wakeJumpThreshold { + log(String(format: "host wall clock jumped %.1fs relative to uptime (sleep/wake or clock change) -> syncing", jump)) + syncNоw(reason: "wall-jump") + return + } + if now >= nextDue { + syncNоw(reason: everSynced ? "periodic" : "boot") + } + } + + func syncNоw(reason: String) { + if inFlight { + // Never let one hung attempt (e.g. a vsock connect that never + // completes) wedge the sync forever: abandon it after a timeout. + guard Date().timeIntervalSince(inFlightSince) > Self.attemptTimeout else { return } + finish(reason: reason, reply: nil, error: "previous attempt hung > \(Int(Self.attemptTimeout))s", id: attempt) + } + guard let vm = vm, vm.state == .running, + let device = vm.socketDevices.first as? VZVirtioSocketDevice else { + return + } + attempt &+= 1 + let id = attempt + inFlight = true + inFlightSince = Date() + let port = self.port + device.connect(toPort: port) { [weak self] result in + guard let self = self else { return } + switch result { + case .failure(let error): + self.finish(reason: reason, reply: nil, error: "vsock connect: \(error.localizedDescription)", id: id) + case .success(let conn): + self.ioQueue.async { + let (reply, err) = GuestTimeSync.еxchange(fd: conn.fileDescriptor) + conn.close() + DispatchQueue.main.async { + self.finish(reason: reason, reply: reply, error: err, id: id) + } + } + } + } + } + + /// One request/reply on a connected vsock fd. Host time is read immediately + /// before the write so connection setup latency doesn't count as drift. + static func еxchange(fd: Int32) -> (String?, String?) { + let flags = fcntl(fd, F_GETFL) + if flags >= 0 { _ = fcntl(fd, F_SETFL, flags & ~O_NONBLOCK) } + var tv = timeval(tv_sec: 5, tv_usec: 0) + _ = setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, socklen_t(MemoryLayout.size)) + _ = setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &tv, socklen_t(MemoryLayout.size)) + + var ts = timespec() + clock_gettime(CLOCK_REALTIME, &ts) + let ns = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec) + let req = Data("ORCABOT-TIMESYNC 1 \(ns)\n".utf8) + if !ConnectionBridge.writеAll(fd, req) { + return (nil, "write failed: errno=\(errno)") + } + var out = Data() + var buf = [UInt8](repeating: 0, count: 256) + while out.count < 512 { + let n = Darwin.read(fd, &buf, buf.count) + if n > 0 { + out.append(contentsOf: buf[0.. (String, String)? in + let p = kv.split(separator: "=", maxSplits: 1) + return p.count == 2 ? (String(p[0]), String(p[1])) : nil + }, + uniquingKeysWith: { a, _ in a }) + let before = fields["before_ms"] ?? "?" + let after = fields["after_ms"] ?? "?" + if fields["stepped"] == "1" { + log("guest clock STEPPED (\(reason)): delta before=\(before)ms after=\(after)ms (positive = guest was behind)") + } else if !everSynced { + log("guest clock in sync (\(reason)): delta=\(before)ms, guest \(fields["rev"] ?? "?")") + } + if consecutiveFailures > 0 && everSynced { + log("guest clock sync recovered after \(consecutiveFailures) failure(s)") + } + everSynced = true + consecutiveFailures = 0 + nextDue = now.addingTimeInterval(Self.period) + return + } + + consecutiveFailures += 1 + let detail = error ?? "unexpected reply: \(reply ?? "")" + if everSynced { + nextDue = now.addingTimeInterval(10) + } else if consecutiveFailures < Self.bootFastAttempts { + // tick - 1: due again on the very next timer tick, not the one after. + nextDue = now.addingTimeInterval(Self.tick - 1) + } else { + nextDue = now.addingTimeInterval(Self.period) + } + // Quiet while the guest is still booting; afterwards at most every 5 min. + let booting = !everSynced && consecutiveFailures < Self.bootFastAttempts + if !booting && now.timeIntervalSince(lastFailureLog) >= 300 { + lastFailureLog = now + log("guest clock sync failed (\(reason), \(consecutiveFailures) in a row): \(detail)") + } + } +} + // MARK: - Bidirectional Connection Bridge /// Bridges data between a TCP NWConnection and a VZVirtioSocketConnection @@ -593,7 +823,17 @@ class ConnectionBridge { private var vsockOpen = true private let queue = DispatchQueue(label: "connection-bridge") private var vsockReadSource: DispatchSourceRead? + // Armed only while the guest's vsock buffer is full; cancelled once drained. private var vsockWriteSource: DispatchSourceWrite? + // TCP->vsock bytes not yet accepted by the guest. Writes never block the + // queue (that also stopped vsock->TCP draining, deadlocking against the + // guest's one-direction-at-a-time socat); instead TCP reads pause while this + // is above the high-water mark and resume once it's flushed (backpressure). + private var pendingOut = Data() + private var pendingOffset = 0 + private var tcpReadPaused = false + private var tcpEOF = false + private static let pendingHighWater = 1 << 20 private static let verboseLogs = ProcessInfo.processInfo.environment["VZ_BRIDGE_VERBOSE"] == "1" init(id: UUID, tcpConnection: NWConnection, vsockConnection: VZVirtioSocketConnection, onClose: @escaping (UUID) -> Void) { @@ -659,6 +899,10 @@ class ConnectionBridge { self.vsockOpen = false self.vsockReadSource?.cancel() self.vsockReadSource = nil + self.vsockWriteSource?.cancel() + self.vsockWriteSource = nil + self.pendingOut = Data() + self.pendingOffset = 0 Darwin.close(self.vsockConnection.fileDescriptor) } self.onClose(self.id) @@ -681,23 +925,132 @@ class ConnectionBridge { if ConnectionBridge.verboseLogs { print("[BRIDGE] \(connId): TCP->Vsock \(data.count) bytes") } - // Write to vsock using file descriptor - data.withUnsafeBytes { buffer in - if let ptr = buffer.baseAddress { - let written = Darwin.write(self.vsockConnection.fileDescriptor, ptr, buffer.count) - if written < 0 { - print("[BRIDGE] \(connId): Vsock write error: \(errno)") - } - } - } + // The vsock fd is non-blocking (see startReadingFromVsock), so one + // write() can take only part of the data, or none (EAGAIN) once the + // guest's buffer is full (~8KB). Every byte is queued and flushed + // in order (dropping the rest truncated larger transfers, e.g. file + // uploads and big control-plane responses). + guard self.vsockOpen else { return } + self.pendingOut.append(data) + if !self.flushPendingTоVsоck() { return } } if isComplete { - print("[BRIDGE] \(connId): TCP complete, closing") - self.close() + // Close only after everything the client sent reached the guest. + self.tcpEOF = true + if self.pendingOut.count - self.pendingOffset == 0 { + print("[BRIDGE] \(connId): TCP complete, closing") + self.close() + } else { + print("[BRIDGE] \(connId): TCP complete, closing after \(self.pendingOut.count - self.pendingOffset) pending bytes flush") + } } else if self.tcpOpen { - self.startReadingFromTCP() + if self.pendingOut.count - self.pendingOffset > ConnectionBridge.pendingHighWater { + self.tcpReadPaused = true + } else { + self.startReadingFromTCP() + } + } + } + } + + /// Write as much of `pendingOut` to the vsock as it accepts without + /// blocking. On EAGAIN, arms a write source that calls back here once the + /// guest drains its buffer. When fully flushed: resumes paused TCP reads, or + /// closes if TCP already hit EOF. Returns false if the bridge was closed on + /// error. Runs only on `queue`. + @discardableResult + private func flushPendingTоVsоck() -> Bool { + guard vsockOpen else { return false } + let fd = vsockConnection.fileDescriptor + let connId = id.uuidString.prefix(8) + var writeErrno: Int32 = 0 + let written: Int = pendingOut.withUnsafeBytes { raw -> Int in + guard let base = raw.baseAddress else { return 0 } + var off = pendingOffset + while off < raw.count { + let n = Darwin.write(fd, base.advanced(by: off), raw.count - off) + if n > 0 { + off += n + continue + } + let e = errno + if n < 0 && e == EINTR { continue } + if n < 0 && (e == EAGAIN || e == EWOULDBLOCK) { break } + writeErrno = n < 0 ? e : EPIPE + break + } + return off - pendingOffset + } + pendingOffset += written + if writeErrno != 0 { + print("[BRIDGE] \(connId): Vsock write error: errno=\(writeErrno) (\(String(cString: strerror(writeErrno))))") + close() + return false + } + + if pendingOffset >= pendingOut.count { + // Fully flushed. + pendingOut.removeAll(keepingCapacity: pendingOut.count <= ConnectionBridge.pendingHighWater) + pendingOffset = 0 + vsockWriteSource?.cancel() + vsockWriteSource = nil + if tcpEOF { + print("[BRIDGE] \(connId): TCP complete and pending flushed, closing") + close() + return false + } + if tcpReadPaused && tcpOpen { + tcpReadPaused = false + startReadingFromTCP() + } + return true + } + + // Compact once the consumed prefix dominates, so the buffer stays bounded. + if pendingOffset > 65536 && pendingOffset * 2 > pendingOut.count { + pendingOut.removeSubrange(0.. Bool { + return data.withUnsafeBytes { raw -> Bool in + guard var ptr = raw.baseAddress else { return true } + var remaining = raw.count + while remaining > 0 { + let n = Darwin.write(fd, ptr, remaining) + if n > 0 { + ptr = ptr.advanced(by: n) + remaining -= n + } else if n < 0 && errno == EINTR { + continue + } else if n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK) { + var pfd = pollfd(fd: fd, events: Int16(POLLOUT), revents: 0) + let r = poll(&pfd, 1, 30_000) + if r == 0 { errno = ETIMEDOUT; return false } + if r < 0 { + if errno == EINTR { continue } + return false + } + } else { + return false + } } + return true } } @@ -719,6 +1072,7 @@ class ConnectionBridge { var buffer = [UInt8](repeating: 0, count: 65536) let bytesRead = Darwin.read(fd, &buffer, buffer.count) + let readErrno = errno if bytesRead > 0 { if ConnectionBridge.verboseLogs { @@ -735,9 +1089,9 @@ class ConnectionBridge { // EOF print("[BRIDGE] \(connId): Vsock EOF, closing") self.close() - } else if errno != EAGAIN && errno != EWOULDBLOCK { + } else if readErrno != EAGAIN && readErrno != EWOULDBLOCK && readErrno != EINTR { // Real error - print("[BRIDGE] \(connId): Vsock read error: errno=\(errno)") + print("[BRIDGE] \(connId): Vsock read error: errno=\(readErrno)") self.close() } } diff --git a/desktop/scripts/drift-allowlist.json b/desktop/scripts/drift-allowlist.json index 0b4fdc3b..74f9d3ca 100644 --- a/desktop/scripts/drift-allowlist.json +++ b/desktop/scripts/drift-allowlist.json @@ -29,6 +29,7 @@ "WHATSAPP_BUSINESS_PHONE": "External service — see WHATSAPP_ACCESS_TOKEN.", "WHATSAPP_PHONE_NUMBER_ID": "External service — see WHATSAPP_ACCESS_TOKEN.", "WHATSAPP_VERIFY_TOKEN": "External service — see WHATSAPP_ACCESS_TOKEN.", + "WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX": "Cloud-only — only read when provisioning Fly machines (sandboxInternalTokenEnv). The desktop VM is booted by main.rs and never receives INTERNAL_API_TOKEN.", "DISCORD_BOT_TOKEN": "External service — Discord interaction webhook needs a public URL.", "DISCORD_PUBLIC_KEY": "External service — see DISCORD_BOT_TOKEN.", "SLACK_SIGNING_SECRET": "External service — Slack events API needs a public URL.", diff --git a/desktop/tests/regression-smoke.sh b/desktop/tests/regression-smoke.sh index 8fcd147a..ee50ba0b 100755 --- a/desktop/tests/regression-smoke.sh +++ b/desktop/tests/regression-smoke.sh @@ -154,10 +154,20 @@ d2="$(X 'pgrep -c bash')"; z="$(X 'ps -eo stat,comm | grep -c "^Z.*bash"')" # P # ---- invariants: path traversal + output redaction ------------------------ echo "[inv] file-API path traversal blocked" ITEM8="$(new_term)"; sleep 3 -SSID="$(curl -s "${AUTH[@]}" "$CP/dashboards/$DID" | jq_py "import json,sys;d=json.load(sys.stdin);print(next(s['sandboxSessionId'] for s in d['sessions'] if s['itemId']=='$ITEM8'))")" -out="$(curl -s -m6 -H 'X-Internal-Token: dev-sandbox-token' "$SB/sessions/$SSID/file?path=../../../../etc/passwd")" +read -r CSID SSID <<<"$(curl -s "${AUTH[@]}" "$CP/dashboards/$DID" | jq_py "import json,sys;d=json.load(sys.stdin);s=next(s for s in d['sessions'] if s['itemId']=='$ITEM8');print(s['id'],s['sandboxSessionId'])")" +# Through the control plane's file proxy, which signs the request — so this +# reaches the sandbox's path guard rather than stopping at auth. +okc="$(code "${AUTH[@]}" "$CP/sessions/$CSID/files")" +[ "$okc" = "200" ] && ok "signed file proxy reaches the sandbox (200)" || bad "signed file proxy -> $okc (want 200)" +out="$(curl -s -m6 "${AUTH[@]}" "$CP/sessions/$CSID/file?path=../../../../etc/passwd")" echo "$out" | grep -q "root:" && bad "path traversal LEAKED /etc/passwd" || ok "traversal blocked" +echo "[inv] sandbox accepts only signed requests (no baked token)" +c1="$(code "$SB/sessions/$SSID/files")" +c2="$(code -H 'X-Internal-Token: dev-sandbox-token' "$SB/sessions/$SSID/files")" +{ [ "$c1" = "401" ] && [ "$c2" = "401" ]; } && ok "unsigned -> 401, old dev-sandbox-token -> 401" \ + || bad "sandbox accepted an unsigned request (none=$c1, dev-sandbox-token=$c2)" + echo "[inv] broker-secret output redaction" SEC="REDACTME_smoke_55512345" curl -s -o /dev/null -X POST "${AUTH[@]}" -H 'Content-Type: application/json' \ diff --git a/desktop/vm/scripts/build-images.sh b/desktop/vm/scripts/build-images.sh index 932601dc..3b939bdc 100755 --- a/desktop/vm/scripts/build-images.sh +++ b/desktop/vm/scripts/build-images.sh @@ -391,7 +391,7 @@ chmod +x /mnt/rootfs/etc/init.d/orcabot mkdir -p /mnt/rootfs/etc/rc.d cat > /mnt/rootfs/etc/rc.local << "RCLOCAL" #!/bin/sh -echo "Starting Orcabot sandbox (rc.local v6-reverse-vsock-cp)..." > /dev/console +echo "Starting Orcabot sandbox (rc.local v7-guest-timesync)..." > /dev/console # Ensure log/run dirs exist mkdir -p /var/log /run @@ -417,6 +417,22 @@ else echo "/dev/vsock missing" > /dev/console fi +# REVISION: rclocal-v3-guest-timesync +# Guest clock sync. The VM has no NTP and the kernel reads the RTC only at boot, +# so after the host sleeps the guest clock lags by the sleep time and every signed +# control-plane request fails ("issued in the future"). The host (vz-helper / +# the Linux QEMU backend) pushes its UTC time over vsock port 1023; this root +# daemon steps CLOCK_REALTIME when drift > 2s. Reserved vsock port + host-CID-only +# peer check: agent processes cannot set the time. Started before the server so the +# first host push (right after boot) lands as early as possible. +echo "[rc.local] REVISION: rclocal-v3-guest-timesync" > /dev/console +if [ -x /usr/local/bin/orcabot-guest-timesync ]; then + /usr/local/bin/orcabot-guest-timesync -port 1023 > /dev/console 2>&1 & + echo $! > /run/guest-timesync.pid +else + echo "orcabot-guest-timesync missing; guest clock will drift after host sleep" > /dev/console +fi + # Bring up outbound networking. systemd boots here but ifupdown/networking.service # is not installed, so nothing actions /etc/network/interfaces. The Virtualization. # framework NAT attachment runs a DHCP server; lease an address on eth0 (forced by @@ -467,16 +483,45 @@ fi export PORT=${PORT:-8080} export WORKSPACE_BASE=/workspace export ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-*} -export SANDBOX_INTERNAL_TOKEN=${SANDBOX_INTERNAL_TOKEN:-dev-sandbox-token} -# Control-plane callback config. CONTROLPLANE_URL points at the reverse vsock bridge -# above; INTERNAL_API_TOKEN must match what the host workerd expects (main.rs default -# is dev-internal-token). These reach only the server process — PTY env filters them, -# so agents never see INTERNAL_API_TOKEN. +# REVISION: rclocal-v2-signed-auth-no-baked-tokens +# Inbound auth: the host signs every request with a per-boot Ed25519 key and passes +# only the PUBLIC key on the kernel command line (agents can read /proc/cmdline; +# fine for a public key). No static token is baked into the image — without +# the key the server rejects every request (fail closed). +# REVISION: rclocal-v4-egress-flag +# orcabot.egress_proxy_enabled=1 means EGRESS_PROXY_ENABLED=true on the host (the +# guest never sees the VM env set on the host): the egress proxy + kernel +# default-deny for pool UIDs, and terminals run as those UIDs instead of root. +# (No apostrophes anywhere in here: this whole script is inside bash -c single quotes.) +for arg in $(cat /proc/cmdline); do + case "$arg" in + orcabot.sandbox_auth_public_key=*) export SANDBOX_AUTH_PUBLIC_KEY="${arg#*=}" ;; + orcabot.egress_proxy_enabled=1) export EGRESS_PROXY_ENABLED=true ;; + esac +done +[ -n "${SANDBOX_AUTH_PUBLIC_KEY:-}" ] || echo "WARNING: no orcabot.sandbox_auth_public_key on the kernel command line; sandbox will reject all requests" > /dev/console +echo "[rc.local] REVISION: rclocal-v4-egress-flag (egress proxy: ${EGRESS_PROXY_ENABLED:-off})" > /dev/console +# Control-plane callbacks go through the reverse vsock bridge above and authenticate +# with the dashboard-scoped tokens the control plane hands each session — the VM +# holds no INTERNAL_API_TOKEN. export CONTROLPLANE_URL=${CONTROLPLANE_URL:-http://127.0.0.1:8787} -export INTERNAL_API_TOKEN=${INTERNAL_API_TOKEN:-dev-internal-token} # Enable the debug-exec endpoint (POST /debug/exec) for the desktop VM only. Fly # starts orcabot-server via its own path, never this rc.local, so it stays off there. export ORCABOT_DEBUG_EXEC=1 +# With egress control on, the server allows pool UIDs DNS only to the nameservers +# in /etc/resolv.conf at startup, and DHCP (above) runs in the background: start +# the server before the lease and terminals can resolve nothing. Wait for a +# nameserver (bounded; the lease normally lands within a second of boot). +if [ "${EGRESS_PROXY_ENABLED:-}" = "true" ]; then + n=0 + while [ "$n" -lt 40 ] && ! grep -q "^nameserver" /etc/resolv.conf 2>/dev/null; do + sleep 0.5 + n=$((n + 1)) + done + grep -q "^nameserver" /etc/resolv.conf 2>/dev/null \ + && echo "[rc.local] nameserver ready for egress DNS rules after ${n} waits" > /dev/console \ + || echo "WARNING: no nameserver after 20s; pool UIDs will have no DNS" > /dev/console +fi touch /var/log/orcabot.log /var/log/vsock-bridge.log /usr/local/bin/orcabot-server >> /var/log/orcabot.log 2>&1 & echo $! > /run/orcabot.pid @@ -692,7 +737,7 @@ if [ -f "$OUTPUT_DIR/custom-modules.tar.gz" ] && [ -f "$OUTPUT_DIR/sandbox.img" /bin/bash -c ' set -euo pipefail apt-get update -qq -apt-get install -y -qq e2fsprogs tar kmod > /dev/null +apt-get install -y -qq e2fsprogs tar kmod binutils > /dev/null # Extract full module tree to /tmp first, strip there, then copy only needed modules to disk. # The full tarball is ~1.2GB which would overflow the disk if extracted directly. @@ -707,14 +752,25 @@ KVER=$(basename "$KVER_DIR") mkdir -p /tmp/keep-modules/$KVER/kernel if [ -d "$KVER_DIR/kernel" ]; then cd "$KVER_DIR/kernel" + # REVISION: vm-modules-v2-netfilter + # Netfilter (iptables/ip6tables, nft + legacy, nat, owner/conntrack/multiport + # matches, REDIRECT/REJECT): the sandbox installs the pool-UID default-deny + # egress chain, the transparent-proxy REDIRECT and the browser port block with + # them. Without these modules iptables fails, the PTY pool never activates and + # every terminal runs as root in the VM. find . -type f \( -name "virtio*" -o -name "vsock*" -o -name "vmw_vsock*" \ -o -name "ext4*" -o -name "fuse*" -o -name "virtiofs*" \ -o -name "jbd2*" -o -name "mbcache*" -o -name "crc16*" \ - -o -name "crc32*" \) | while read f; do + -o -name "crc32*" -o -name "libcrc32c*" \ + -o -path "./net/netfilter/*" -o -path "./net/ipv4/netfilter/*" \ + -o -path "./net/ipv6/netfilter/*" \) | while read f; do mkdir -p "/tmp/keep-modules/$KVER/kernel/$(dirname "$f")" cp "$f" "/tmp/keep-modules/$KVER/kernel/$f" done cd / + # The custom kernel ships unstripped modules (netfilter alone is ~150MB); + # debug info is not needed to load them. + find /tmp/keep-modules -name "*.ko" -exec strip --strip-debug {} + fi # Copy module metadata for meta in modules.builtin modules.builtin.modinfo modules.order; do diff --git a/desktop/workerd/config/workerd.desktop.capnp b/desktop/workerd/config/workerd.desktop.capnp index 9e2b1093..5d90111e 100644 --- a/desktop/workerd/config/workerd.desktop.capnp +++ b/desktop/workerd/config/workerd.desktop.capnp @@ -38,8 +38,15 @@ const config :Workerd.Config = ( bindings = [ # Core (name = "SANDBOX_URL", fromEnvironment = "SANDBOX_URL"), + # Per-boot Ed25519 key that signs every request to the sandbox VM; the VM + # gets only the public key. SANDBOX_INTERNAL_TOKEN is the legacy static + # token, set only when the host env explicitly provides one. + (name = "SANDBOX_AUTH_SIGNING_KEY", fromEnvironment = "SANDBOX_AUTH_SIGNING_KEY"), (name = "SANDBOX_INTERNAL_TOKEN", fromEnvironment = "SANDBOX_INTERNAL_TOKEN"), + # Per-boot, host-only; the VM uses dashboard-scoped tokens instead. (name = "INTERNAL_API_TOKEN", fromEnvironment = "INTERNAL_API_TOKEN"), + # Host-only HMAC key for PTY/dashboard tokens (never given to the VM). + (name = "PTY_TOKEN_SECRET", fromEnvironment = "PTY_TOKEN_SECRET"), (name = "D1_HTTP_URL", fromEnvironment = "D1_HTTP_URL"), (name = "D1_SHIM", service = "d1-shim"), (name = "D1_SHIM_DEBUG", fromEnvironment = "D1_SHIM_DEBUG"), diff --git a/e2e/fixtures/base.ts b/e2e/fixtures/base.ts index ba04940b..06a63ec5 100644 --- a/e2e/fixtures/base.ts +++ b/e2e/fixtures/base.ts @@ -1,4 +1,4 @@ -// REVISION: e2e-base-v2-diagnostics-env +// REVISION: e2e-base-v3-desktop-surface-token import { test as base, expect } from "@playwright/test"; import { login, devModeLoginViaUI, logout } from "./auth"; import { @@ -17,7 +17,7 @@ import { CONTROLPLANE_URL } from "../helpers/controlplane-url"; import { createDiagnostics, type E2EDiagnostics } from "../helpers/diagnostics"; import { getEnv, requiredEnvReport } from "../helpers/env"; -const MODULE_REVISION = "e2e-base-v2-diagnostics-env"; +const MODULE_REVISION = "e2e-base-v3-desktop-surface-token"; console.log( `[e2e-base] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}` ); @@ -68,7 +68,37 @@ export const test = base.extend<{ terminal: TerminalFixture; api: APIFixture; diagnostics: DiagnosticsFixture; + desktopSurface: void; }>({ + /** + * Desktop builds gate dev-auth behind a per-boot surface token. The real app + * hands it to the frontend as ?surface= on the loading-screen redirect, and the + * frontend keeps it only in memory, so every full page load needs it again — + * HTTP calls send it as X-Orcabot-Surface and the dashboard WebSocket as + * ?surface=. Mirror that handoff before any page script runs. No-op unless + * E2E_SURFACE_TOKEN is set (it is unset on cloud/local-dev targets). + */ + desktopSurface: [ + async ({ page }, use) => { + const token = getEnv("E2E_SURFACE_TOKEN"); + if (token) { + await page.addInitScript((t) => { + try { + const url = new URL(window.location.href); + if (url.protocol.startsWith("http") && !url.searchParams.has("surface")) { + url.searchParams.set("surface", t); + window.history.replaceState(window.history.state, "", url.toString()); + } + } catch { + /* about:blank and friends */ + } + }, token); + } + await use(); + }, + { auto: true }, + ], + diagnostics: [ async ({ page }, use, testInfo) => { const collector = await createDiagnostics(page); diff --git a/e2e/fixtures/terminal.ts b/e2e/fixtures/terminal.ts index 02a2896d..1435bf73 100644 --- a/e2e/fixtures/terminal.ts +++ b/e2e/fixtures/terminal.ts @@ -1,5 +1,11 @@ +// REVISION: e2e-terminal-v2-wrapped-prompt import { type Page, expect } from "@playwright/test"; +const MODULE_REVISION = "e2e-terminal-v2-wrapped-prompt"; +console.log( + `[e2e-terminal] REVISION: ${MODULE_REVISION} loaded at ${new Date().toISOString()}` +); + /** * Add a terminal block to the current dashboard canvas. * Clicks the toolbar button identified by data-guidance-target. @@ -58,8 +64,13 @@ export async function readTerminalText(page: Page): Promise { * the old `content.trim().length > 0` check green-lit that stuck state, hiding * the flaky-first-connect bug. Requiring the `user@host:…[#$]` shape is what * makes the stuck state fail a test instead of passing it. + * + * The cwd is `/workspace/`, so a prompt is ~70 columns and can + * wrap onto a second row (desktop terminals run as `pty-NNN`, longer than + * `root`). readTerminalText joins rows with `\n`, so the `#`/`$` may sit on the + * next row: allow one continuation row of unbroken path characters before it. */ -export const SHELL_PROMPT_RE = /[\w.-]+@[\w.-]+:[^\n]*[#$]/; +export const SHELL_PROMPT_RE = /[\w.-]+@[\w.-]+:[^\n]*(?:\n\S*)?[#$]/; /** * Wait for the terminal to connect and show a real shell prompt. diff --git a/frontend/src/app/(app)/dashboards/[id]/page.tsx b/frontend/src/app/(app)/dashboards/[id]/page.tsx index 4e647d3c..709a1cce 100644 --- a/frontend/src/app/(app)/dashboards/[id]/page.tsx +++ b/frontend/src/app/(app)/dashboards/[id]/page.tsx @@ -3,8 +3,8 @@ "use client"; -// REVISION: dashboard-v56-mobile-fullscreen -console.log(`[dashboard] REVISION: dashboard-v56-mobile-fullscreen loaded at ${new Date().toISOString()}`); +// REVISION: dashboard-v58-cancel-pending-edge +console.log(`[dashboard] REVISION: dashboard-v58-cancel-pending-edge loaded at ${new Date().toISOString()}`); import * as React from "react"; import { useParams, useRouter } from "next/navigation"; @@ -680,6 +680,18 @@ export default function DashboardPage() { // Track recently deleted edge IDs (prevents sync effect from re-adding edges // when a query refetch arrives before the server-side DELETE completes) const recentlyDeletedEdgeIdsRef = React.useRef>(new Set()); + // Temp edge IDs whose create is in flight (it may be awaiting a pending block + // create, for any length of time). Deleting one records it in + // cancelledTempEdgeIdsRef — not time-limited like recentlyDeletedEdgeIdsRef — + // so the POST is skipped, or the server edge deleted, instead of re-adding it. + const pendingTempEdgeIdsRef = React.useRef>(new Set()); + const cancelledTempEdgeIdsRef = React.useRef>(new Set()); + const markEdgeDeleted = React.useCallback((edgeId: string) => { + recentlyDeletedEdgeIdsRef.current.add(edgeId); + if (pendingTempEdgeIdsRef.current.has(edgeId)) { + cancelledTempEdgeIdsRef.current.add(edgeId); + } + }, []); // Track previous collaboration items to detect what actually changed const prevCollabItemsRef = React.useRef([]); const prevCollabEdgesRef = React.useRef([]); @@ -832,6 +844,26 @@ export default function DashboardPage() { return map; }, [items]); + // Node IDs are stable keys (the optimistic temp-… ID survives the swap to the + // server ID), but the server only knows real item IDs and rejects edges whose + // endpoints aren't items on this dashboard (E79391). Edge creation translates + // node IDs back through these refs; a block whose create is still in flight + // has a pending promise that settles with its real ID. + const realIdToStableKeyRef = React.useRef(realIdToStableKey); + realIdToStableKeyRef.current = realIdToStableKey; + const itemsForIdRef = React.useRef(items); + itemsForIdRef.current = items; + const pendingItemIdPromisesRef = React.useRef(new Map>()); + const pendingItemResolversRef = React.useRef( + new Map void; reject: (err: Error) => void }>() + ); + const resolveItemId = React.useCallback(async (nodeId: string): Promise => { + const pending = pendingItemIdPromisesRef.current.get(nodeId); + if (pending) return pending; + const item = itemsForIdRef.current.find((i) => i._stableKey === nodeId); + return item ? item.id : nodeId; + }, []); + // Build a map from (itemId, provider) → securityLevel from integration labels const integrationLabelMap = React.useMemo(() => { const map = new Map(); @@ -1062,6 +1094,11 @@ export default function DashboardPage() { const now = new Date().toISOString(); const tempId = item.clientTempId || `temp-${generateId()}`; + const pendingId = new Promise((resolve, reject) => { + pendingItemResolversRef.current.set(tempId, { resolve, reject }); + }); + pendingId.catch(() => {}); // only edge creation awaits it; a failed create is reported elsewhere + pendingItemIdPromisesRef.current.set(tempId, pendingId); const optimisticItem: DashboardItem = { id: tempId, dashboardId, @@ -1188,6 +1225,11 @@ export default function DashboardPage() { return; } + if (context?.tempId) { + pendingItemResolversRef.current.get(context.tempId)?.resolve(createdItem.id); + pendingItemResolversRef.current.delete(context.tempId); + } + // Track this item so the WebSocket echo doesn't trigger a refetch // (which would lose the _stableKey and cause the node to remount) recentlyCreatedItemsRef.current.add(createdItem.id); @@ -1250,6 +1292,12 @@ export default function DashboardPage() { // Decrement mutations in-flight counter mutationsInFlightRef.current--; + if (context?.tempId) { + pendingItemResolversRef.current.get(context.tempId)?.reject(error); + pendingItemResolversRef.current.delete(context.tempId); + pendingItemIdPromisesRef.current.delete(context.tempId); + } + if (context?.previous) { queryClient.setQueryData(["dashboard", dashboardId], context.previous); } @@ -1266,7 +1314,7 @@ export default function DashboardPage() { }); const createEdgeMutation = useMutation({ - mutationFn: (edge: { + mutationFn: async (edge: { sourceItemId: string; targetItemId: string; sourceHandle?: string; @@ -1274,15 +1322,36 @@ export default function DashboardPage() { _tempEdgeId?: string; }) => { const { _tempEdgeId, ...payload } = edge; - return createEdge(dashboardId, payload); + if (_tempEdgeId) pendingTempEdgeIdsRef.current.add(_tempEdgeId); + const [sourceItemId, targetItemId] = await Promise.all([ + resolveItemId(payload.sourceItemId), + resolveItemId(payload.targetItemId), + ]); + // Deleted while waiting for a block create — don't POST it at all. + if (_tempEdgeId && cancelledTempEdgeIdsRef.current.has(_tempEdgeId)) return null; + return createEdge(dashboardId, { ...payload, sourceItemId, targetItemId }); }, onSuccess: (createdEdge, variables) => { // If this edge (or its temp ID) was deleted while the create was in-flight, - // don't re-add it — just clean up the temp edge and bail out. + // don't re-add it — clean up the temp edge, and delete the server edge if + // the POST already went out (the user's delete only knew the temp ID). + const tempEdgeId = variables._tempEdgeId; const deletedIds = recentlyDeletedEdgeIdsRef.current; - if (deletedIds.has(createdEdge.id) || (variables._tempEdgeId && deletedIds.has(variables._tempEdgeId))) { - if (variables._tempEdgeId) { - setEdges((prev) => prev.filter((e) => e.id !== variables._tempEdgeId)); + const cancelled = + !createdEdge || + deletedIds.has(createdEdge.id) || + (!!tempEdgeId && (deletedIds.has(tempEdgeId) || cancelledTempEdgeIdsRef.current.has(tempEdgeId))); + if (cancelled) { + if (tempEdgeId) { + setEdges((prev) => prev.filter((e) => e.id !== tempEdgeId)); + } + if (createdEdge && !deletedIds.has(createdEdge.id)) { + const serverEdgeId = createdEdge.id; + deletedIds.add(serverEdgeId); + setTimeout(() => recentlyDeletedEdgeIdsRef.current.delete(serverEdgeId), 5000); + deleteEdge(dashboardId, serverEdgeId).catch((err) => { + console.warn("[createEdgeMutation] failed to delete edge removed while pending:", err); + }); } return; } @@ -1296,7 +1365,17 @@ export default function DashboardPage() { const cleaned = prev.filter( (e) => e.id !== variables._tempEdgeId && e.id !== createdEdge.id ); - return [...cleaned, { ...toFlowEdge(createdEdge), data: tempEdge?.data }]; + const flowEdge = toFlowEdge(createdEdge); + const stableKeys = realIdToStableKeyRef.current; + return [ + ...cleaned, + { + ...flowEdge, + source: stableKeys.get(flowEdge.source) || flowEdge.source, + target: stableKeys.get(flowEdge.target) || flowEdge.target, + data: tempEdge?.data, + }, + ]; }); } @@ -1360,6 +1439,12 @@ export default function DashboardPage() { } toast.error(`Failed to save connection: ${error.message}`); }, + onSettled: (_data, _error, variables) => { + if (variables._tempEdgeId) { + pendingTempEdgeIdsRef.current.delete(variables._tempEdgeId); + cancelledTempEdgeIdsRef.current.delete(variables._tempEdgeId); + } + }, }); // Handle integration-to-terminal edge creation/deletion @@ -2243,6 +2328,11 @@ export default function DashboardPage() { // Track as recently deleted to prevent refetches from bringing it back recentlyDeletedItemsRef.current.add(itemId); + // Drop its pending-ID entry (kept after the create resolves to map stale + // temp IDs) so deleted items don't accumulate. Awaiters already hold the promise. + pendingItemIdPromisesRef.current.delete(itemId); + const deletedStableKey = realIdToStableKey.get(itemId); + if (deletedStableKey) pendingItemIdPromisesRef.current.delete(deletedStableKey); // Always cancel in-flight queries to prevent stale data overwriting our updates await queryClient.cancelQueries({ queryKey: ["dashboard", dashboardId] }); @@ -2280,7 +2370,12 @@ export default function DashboardPage() { const idsToRemove = new Set([itemId]); if (stableKey) idsToRemove.add(stableKey); setEdges((prev) => - prev.filter((edge) => !idsToRemove.has(edge.source) && !idsToRemove.has(edge.target)) + prev.filter((edge) => { + if (!idsToRemove.has(edge.source) && !idsToRemove.has(edge.target)) return true; + // An edge to this block still awaiting its create must not be POSTed. + if (pendingTempEdgeIdsRef.current.has(edge.id)) cancelledTempEdgeIdsRef.current.add(edge.id); + return false; + }) ); return { previous, previousEdges }; @@ -2464,7 +2559,7 @@ export default function DashboardPage() { // All state and cache mutations happen here, BEFORE any awaits, so the // edge disappears from the UI immediately. mutationsInFlightRef.current++; - recentlyDeletedEdgeIdsRef.current.add(edgeId); + markEdgeDeleted(edgeId); const data = queryClient.getQueryData<{ dashboard: Dashboard; @@ -2495,7 +2590,7 @@ export default function DashboardPage() { e.sourceItemId === edgeToDelete.targetItemId && e.targetItemId === edgeToDelete.sourceItemId ) { - recentlyDeletedEdgeIdsRef.current.add(e.id); + markEdgeDeleted(e.id); edgeIdsToRemove.push(e.id); } } @@ -2513,7 +2608,7 @@ export default function DashboardPage() { if (edgeRef && e.id !== edgeId && e.source === edgeRef.source && e.target === edgeRef.target && e.sourceHandle === edgeRef.sourceHandle && e.targetHandle === edgeRef.targetHandle) { - recentlyDeletedEdgeIdsRef.current.add(e.id); + markEdgeDeleted(e.id); return false; } return true; @@ -2573,7 +2668,7 @@ export default function DashboardPage() { }, 5000); } }, - [dashboardId, queryClient, handleIntegrationEdge, setEdges] + [dashboardId, queryClient, handleIntegrationEdge, setEdges, markEdgeDeleted] ); // Callback to send UI command results back to DashboardDO for broadcast diff --git a/frontend/src/app/(app)/layout.tsx b/frontend/src/app/(app)/layout.tsx index 5186d5ad..ee501da0 100644 --- a/frontend/src/app/(app)/layout.tsx +++ b/frontend/src/app/(app)/layout.tsx @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + import { UpdateProgressOverlay } from "@/components/UpdateProgressOverlay"; // REVISION: app-layout-v1-update-overlay diff --git a/frontend/src/components/DesktopVersionBadge.tsx b/frontend/src/components/DesktopVersionBadge.tsx index 8cded226..a880f6de 100644 --- a/frontend/src/components/DesktopVersionBadge.tsx +++ b/frontend/src/components/DesktopVersionBadge.tsx @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + // REVISION: desktop-version-badge-v1 "use client"; diff --git a/frontend/src/components/UpdateProgressOverlay.tsx b/frontend/src/components/UpdateProgressOverlay.tsx index d5462971..0aa63d60 100644 --- a/frontend/src/components/UpdateProgressOverlay.tsx +++ b/frontend/src/components/UpdateProgressOverlay.tsx @@ -1,3 +1,6 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + // REVISION: update-progress-overlay-v1 "use client"; diff --git a/frontend/src/components/blocks/TerminalBlock.tsx b/frontend/src/components/blocks/TerminalBlock.tsx index da851a7b..86ebc069 100644 --- a/frontend/src/components/blocks/TerminalBlock.tsx +++ b/frontend/src/components/blocks/TerminalBlock.tsx @@ -3,8 +3,8 @@ "use client"; -// REVISION: terminal-block-v12-fullscreen-actions -const TERMINAL_BLOCK_REVISION = "terminal-block-v12-fullscreen-actions"; +// REVISION: terminal-block-v13-no-connected-banner +const TERMINAL_BLOCK_REVISION = "terminal-block-v13-no-connected-banner"; console.log(`[TerminalBlock] REVISION: ${TERMINAL_BLOCK_REVISION} loaded at ${new Date().toISOString()}`); @@ -2450,7 +2450,9 @@ export function TerminalBlock({ terminalRef.current?.write("\x1b[2J\x1b[H"); // Clear screen } hasConnectedOnceRef.current = true; - terminalRef.current?.write("\x1b[32m$ Connected to sandbox\x1b[0m\r\n\r\n"); + // No "Connected" banner: the replayed prompt arrives before this effect + // runs, so the banner landed mid-line after it. The prompt itself shows + // the connection is up. // Control is requested separately once connected. } diff --git a/sandbox/CLAUDE.md b/sandbox/CLAUDE.md index 91866421..8c40328d 100644 --- a/sandbox/CLAUDE.md +++ b/sandbox/CLAUDE.md @@ -283,9 +283,16 @@ Defined in `internal/egress/defaults.json` (embedded into `allowlist.go` via `// Glob matching: `*.example.com` matches subdomains but NOT `example.com` itself. +### Control Plane Credentials +- Inbound (`internal/auth`): requests from the control plane carry `X-Sandbox-Auth`, verified with `SANDBOX_AUTH_PUBLIC_KEY` and required to name this machine (`FLY_MACHINE_ID`/`FLY_ALLOC_ID`), the method and the raw request target. The static `SANDBOX_INTERNAL_TOKEN` is accepted only while set; a present-but-invalid signature is always rejected. +- Calls to the control plane's `/internal/*` go through `internal/cpauth`: `cpauth.Apply(req, dashboardID)` attaches the dashboard-scoped token (`X-Dashboard-Token`) the control plane sent with the session, refreshing it before its 7-day expiry. Tokens are owned by the session that brought them (`SetSessionToken`, dropped by `ForgetSession` on session delete); the one used is the newest sandbox generation (`sbx_gen`) a live session still holds, so the session of a desktop provisioning request that lost the claim (deleted by the control plane) never supplies the token. A refresh refused as permanently invalid — E79398 (replaced sandbox), E79395/E79393 (bad signature, e.g. rotated `PTY_TOKEN_SECRET`, or expired past the grace) — drops that generation and marks its sessions dropped, so `Session.DashboardToken()` does not fall back to the session's original copy (`cpauth.SessionTokenDropped`). Network errors/5xx/E79399 keep the token. After any failed refresh no new one is attempted for 30s (`refreshBackoff`), so callers with an expired token don't each block on a doomed round trip. The control plane sends a fresh token as `mcp_token` with every `POST /sessions/:id/ptys`; the handler hands it to cpauth for the session's own dashboard (`Session.SetDashboardToken`), which is how a VM recovers from a dropped token. Never read `INTERNAL_API_TOKEN` directly — it is only a legacy fallback and is absent once the control plane sets `WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX=true`. + ### Feature Flag - `EGRESS_PROXY_ENABLED=true` env var: single gate — enables globally for all sessions - Without this, proxy runs but PTYs don't route through it (iptables rules not installed, UID pool not activated) +- With it (and CAP_NET_ADMIN), pool UIDs are **default-deny** via the `ORCABOT_POOL_EGRESS` chain (`cmd/server/egress_linux.go`): loopback (incl. REDIRECTed 80/443), established replies, and DNS to resolv.conf nameservers pass; all other ports are rejected. `EGRESS_EXTRA_TCP_PORTS=22,...` opts specific raw TCP ports back in (each is an uninspected channel). +- **Fail closed**: if the proxy can't start, or (with CAP_NET_ADMIN) any kernel layer fails — NAT REDIRECT, transparent listener, `sandbox` group, MCPLocal/browser-port blocks, default-deny chain, privileged socket — the pool doesn't activate **and all PTY creation is refused** with `E79770` (`pty.RefuseSpawns`, `setupKernelEgress` in `cmd/server/main.go`) instead of spawning root shells outside every rule. `/health` stays 200 and reports the reason in `pty_spawn_refused`. Missing CAP_NET_ADMIN with `EGRESS_PROXY_ENABLED=true` is refused the same way (E79770): no kernel layer can be installed, and root PTYs behind `HTTP_PROXY` alone could ignore it +- Integration test (mutates netns; run in a disposable container): `ORCABOT_IPTABLES_IT=1 go test -run TestPoolEgress ./cmd/server/` with `--cap-add NET_ADMIN` + iptables installed - Per-session opt-in (`egress_enabled`) has been removed; the env var is the only control ### Localhost Bypass @@ -293,6 +300,8 @@ Localhost traffic (`localhost`, `127.0.0.1`, `::1`) always bypasses the proxy: - `NO_PROXY=localhost,127.0.0.1` env var (client-side hint) - Server-side `isLocalhost()` check in proxy (catches tools that ignore NO_PROXY) +**Except protected loopback services** (`internal/egress/loopback_guard.go`): the proxy, transparent proxy and secrets broker run as root, outside the iptables rules that block pool UIDs from the browser ports (6100-6299) and MCPLocal (8081), so they refuse to relay there. `decide()` denies recognisable targets up front (localhost names, 127/8, ::1, IPv4-mapped, 0.0.0.0/::, decimal/octal/hex IPv4), and every root-run forwarder dials through `egress.GuardedDialer`, whose `Control` hook checks the address actually connected after DNS (names resolving or rebinding to loopback). New loopback services pool UIDs must not reach: `egress.ProtectLoopbackPort` + an iptables rule. New root-run forwarders: dial via `GuardedDialer`. + ### Key Files - `internal/egress/proxy.go` — HTTP/HTTPS forward proxy with connection holding - `internal/egress/allowlist.go` — Default + runtime allowlist with glob matching @@ -385,7 +394,7 @@ curl -X POST "http://localhost:8082/broker/{sessionID}/custom/MY_API_KEY?target= -d '{"message": "test"}' # Expected response (403): domain requires approval -# {"error":"domain_approval_required","domain":"api.example.com","secret":"MY_API_KEY","message":"This domain requires owner approval before the secret can be sent."} +# {"error":"domain_approval_required","domain":"api.example.com","secret":"MY_API_KEY","message":"E79130: This domain requires owner approval before the secret can be sent."} # The broker will also notify the control plane, creating a pending approval request # that appears in the frontend for the dashboard owner to approve. @@ -396,21 +405,42 @@ curl -X POST "http://localhost:8082/broker/{sessionID}/custom/MY_API_KEY?target= The sandbox runs an embedded Chromium browser accessible via WebSocket for in-browser testing. **Startup chain (4 processes):** -1. **Xvfb** — X Virtual Framebuffer (display `:N`, 1280x720x24) -2. **x11vnc** — VNC server with clipboard sync -3. **websockify** — VNC→WebSocket proxy on allocated port -4. **Chromium** — Browser with remote debugging enabled +1. **Xvfb** — X Virtual Framebuffer (display `:N`, 1280x720x24) — runs as `orcabot-vnc` +2. **x11vnc** — VNC server with clipboard sync — `orcabot-vnc` +3. **websockify** — VNC→WebSocket proxy on allocated port — `orcabot-vnc` +4. **Chromium** — Browser with remote debugging enabled — `orcabot-browser` **Key behaviors:** -- Auto-cleans Chromium profile lock files + crash state on startup +- **One shared stack per workspace root** (`browserStack`, `stackFor` in `internal/browser/browser.go`): every `Controller` for the same workspace (several sessions in one web VM, or a desktop dashboard reopened while its old session lives) drives ONE Chromium on that workspace's profile instead of each launching its own on it (which corrupted the profile). `Start`/`OpenURL` register the controller as a user; `Stop` only detaches it and tears the stack down when the **last** user leaves. +- **Profile lock**: `SingletonLock`/`SingletonSocket`/`SingletonCookie` are cleared only when the pid the lock names is dead, on another host, or not a Chromium of the browser user. A live Chromium holding the profile fails the start (`errProfileInUse`) rather than having a second one open it. Crash state ("Restore pages?") is cleared on start. +- **Process-wide port reservations**: `freePortInRange` reserves each websockify/x11vnc/DevTools port in a process-wide map until that stack stops or its start fails (`releasePorts`), so two concurrent `Start`s (two dashboards on desktop) can't pick the same port and end up driving each other's Chromium. +- **Navigation is http/https only** (`browser.ValidateNavigationURL`, `internal/browser/navurl.go`): every URL an agent can make the browser load (navigate, `OpenURL`, `/json/new`) must be http(s) — no `file://` (would read other dashboards' profiles), `chrome://`, `devtools://`, `javascript:`, `data:` — and loopback/unspecified hosts on ports 6100-6299 are refused (names are resolved; WHATWG IPv4 forms are understood; whitespace/control/backslash refused outright). - Log filtering suppresses noisy DBus/DPMS/WebSocket messages - `OpenURL()` reuses blank tabs or deduplicates existing URLs via Chrome DevTools Protocol - Port waiting with timeouts (10–20s per process) - Status returns: `Running`, `Ready`, `WSPort`, `Display`, `DebugPort` +- **Liveness**: every `Status()` checks each of the four processes is still alive (`processRunning`, waitid WNOWAIT); if any died the whole stack is torn down and all ports/display are zeroed, so a dead browser never keeps advertising ports an agent may since have bound. `Stop()` zeroes them too. +- **Stopping**: processes are stopped with SIGTERM to each process group, then SIGKILL after a 2s grace (`killProcessTree` in `internal/browser/browser.go`). x11vnc frees its SysV shared-memory framebuffers only on a clean exit; SIGKILL alone leaked them (~3.6 MB each) until the VM hit `kernel.shmmni` (4096) after a few hundred starts, and from then on every x11vnc failed (`shmget: No space left on device` → "browser vnc server failed to start"). The server also sets `kernel.shm_rmid_forced=1` at startup (`browser.EnableOrphanShmCleanup`) so segments of an x11vnc that crashes are freed too; that sysctl is read-only inside a Docker container (logged warning), and applies on Fly and in the desktop VM. - **Readiness re-probe**: `Ready` is re-evaluated on every `Status()` poll (cold chromium boot can exceed the short startup budget), so the browser flips to ready as soon as DevTools answers instead of latching `false` until an `OpenURL`. - **Pre-warm**: chromium is started in the background at sandbox session creation (`Manager.Create`) so the browser is ready (~instant) when first opened, instead of paying the ~25s cold boot on demand. One sandbox session per VM ⇒ one pre-warmed browser per VM (~250–350 MB idle on the 4 GB VMs). **On by default; set `BROWSER_PREWARM=false` to disable.** Future: idle auto-stop to reclaim RAM when unused. -**Key file:** `internal/browser/browser.go`, pre-warm hook in `internal/sessions/manager.go` +**Isolation (security-critical):** +- **Two unprivileged users, never root** (`internal/browser/isolation.go`, created in `docker/Dockerfile`; the desktop VM image is exported from the same image, so `build-images.sh` needs no change): Chromium runs as `orcabot-browser` (uid 1990), Xvfb/x11vnc/websockify as `orcabot-vnc` (uid 1991). Neither is in group `sandbox`. Chromium still needs `--no-sandbox` in the VM, so a root Chromium with an open DevTools port was a root escalation for any agent. +- **Why the split**: page JavaScript, links and an agent's `browser_evaluate` run as Chromium's user, and websockify serves noVNC with no password (`x11vnc -nopw`). When the whole stack was one user, a page in dashboard A could open dashboard B's websockify/noVNC (view and drive B's screen) or read B's DevTools `/json/list` by navigating to it — desktop runs every dashboard's stack in one VM. (CDP's websocket was already safe from pages: Chromium 111+ answers 403 to any `Origin` without `--remote-allow-origins`, which we don't pass.) Now `setupBrowserUserPortBlock` (`cmd/server/egress_linux.go`) installs `-d 127.0.0.0/8 / ::1 --dport 6100:6299 -m owner --uid-owner orcabot-browser -j REJECT` at the top of OUTPUT. Chromium only *listens* in the range (DevTools; its replies have a source port there, which a `--dport` rule never matches) and websockify → x11vnc is `orcabot-vnc`, so nothing legitimate is blocked; the server (root) stays the only other client. It is installed whenever CAP_NET_ADMIN is present, egress or not (the threat is a hostile page, not only an agent); with egress on, a failure refuses PTYs (E79770). +- Each user has its own 0700 home (`checkBrowserHome` checks both). Xvfb's cookie is written to `orcabot-vnc`'s home; Chromium gets its own copy of the same cookie in `orcabot-browser`'s home (`chromeXauth`). Both are removed on teardown. +- Server dials verify the expected owner **per port**: `browser.DialBrowser` (DevTools → `orcabot-browser`) and `browser.DialVNC` (websockify/x11vnc → `orcabot-vnc`; `NewVNCTransport` backs the noVNC reverse proxy). `waitForPort` takes the dialer. +- x11vnc (`-localhost`), websockify (`127.0.0.1:`) and DevTools listen on loopback only, on ports from 6100-6299. For pool UIDs, `setupPoolBrowserPortBlock` installs two OUTPUT rules per family (iptables + ip6tables): `--dport` REJECT so pool UIDs can't *connect* to the range, and `--sport` DROP so a pool UID's own *listener* in the range can't answer anyone (owner match applies to its SYN-ACK). The second rule is what protects websockify → x11vnc: websockify can't check who accepted, so an agent that bound x11vnc's port after it died would otherwise get every viewer's VNC session. The server (root) is the only client; the frontend reaches VNC through its authenticated proxy. +- x11vnc stays on TCP rather than `-unixsock` + websockify `--unix-target`: x11vnc 0.9.16 never accepts on its unix socket in `-threads` mode (tested), and `-threads` is kept for frame rate. +- Pool UIDs can still *bind* those ports first (iptables can't stop bind(); binding all 200 makes Start fail with "no free port", fail-closed — an availability-only DoS). So every server connection to a browser port (`waitForPort`, the DevTools HTTP clients, the CDP websocket, the noVNC reverse proxy) goes through `browser.DialBrowser` (`internal/browser/portverify.go`), which checks in `/proc/net/tcp` that the socket that accepted it is owned by the port's stack user (`DialBrowser` / `DialVNC`, above). A connection still in the accept queue has no inode (older kernels print uid 0 for it), so the check falls back to the listener's owner. Never dial a browser port with a plain `net.Dial`/`http.Client`. +- Xvfb runs with `-auth` (per-start MIT cookie in the browser user's 0700 home), so agents can't inject keystrokes into the display. It is started with `-displayfd`: it picks a free display (skipping squatted sockets) and reports it once ready. x11vnc and Chromium are started only while Xvfb is still running. +- Each workspace gets its own profile at `/home/orcabot-browser/profiles/` (`profilePath`; browser-owned 0700, in the browser user's 0700 home). Desktop runs every dashboard in one VM, so a shared profile would let one dashboard read another's cookies and local storage. Profiles live **never in `/workspace`**: `/workspace` is group-writable by every pool UID with no sticky bit, so an agent could rename a profile there and mkdir a replacement while Chromium ran (cookie theft, planted prefs/extensions). `Start()` refuses to run unless the home is browser-owned 0700 and every parent is a root-owned dir nobody else can write (`checkBrowserHome`). Root still prepares the profile through an `O_NOFOLLOW` dir fd and moves aside anything not owned by the browser user or root. Logins don't survive a rebuilt root filesystem. +- The old `/workspace/.browser` profile is deleted, not migrated (adopting a tree from the agent-writable workspace was the hand-over attack surface): only if it is a real dir owned by the browser user, via `O_NOFOLLOW` fds, and unlinked only if the entry still names the same inode (`removeLegacyProfile`). +- **Screenshots** are written by root into `/workspace`, so `writeWorkspaceFile` (`internal/browser/safewrite.go`) never follows a symlink: each parent is opened relative to the last with `O_NOFOLLOW|O_DIRECTORY`, the data goes to an `O_CREAT|O_EXCL|O_NOFOLLOW` temp file, and `renameat` replaces the target (a symlinked target is replaced, not written through; a symlinked parent is refused). Any new root write into `/workspace` must go through it. +- With egress control on, both browser users are also held to the pool's default-deny egress chain, so nothing Chromium loads can bypass the egress proxy (and websockify/x11vnc get loopback only). On an image without a user, its jump is skipped with a warning instead of failing the chain. +- Not covered by the split: `orcabot-browser` still owns every dashboard's profile and Chromium cookie copy, so a renderer exploit (not page JS) in one dashboard's Chromium could read another's; and if x11vnc dies, websockify can't check who accepts its port (the pool `--sport` rule covers pool UIDs, not `orcabot-browser`; `Status()` tears the stack down on the next poll). +- Chromium background downloads are off via managed policy (`docker/chromium-policies.json` → `/etc/chromium/policies/managed/orcabot.json`): `--disable-component-update` alone does not stop Chromium 155 fetching the on-device AI model manifest from `edgedl.me.gvt1.com` (it showed up as an egress prompt); `ComponentUpdatesEnabled=false` does. + +**Key file:** `internal/browser/browser.go`, `internal/browser/isolation.go`, `internal/browser/safewrite.go`, pre-warm hook in `internal/sessions/manager.go` --- diff --git a/sandbox/cmd/guest-timesync/main.go b/sandbox/cmd/guest-timesync/main.go new file mode 100644 index 00000000..e4938a82 --- /dev/null +++ b/sandbox/cmd/guest-timesync/main.go @@ -0,0 +1,91 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v2-error-codes +// +// orcabot-guest-timesync keeps the desktop sandbox VM's wall clock in step with +// the host. The desktop VM (Apple Virtualization.framework / QEMU) has no NTP +// client and the kernel reads the RTC only at boot, so after the host sleeps the +// guest clock falls behind by the sleep duration. Control-plane -> sandbox +// requests are Ed25519-signed with iat/exp and the verifier rejects an iat more +// than 60s in the guest's future, so a lagging guest clock fails every request +// ("signed request: issued in the future"). +// +// The HOST pushes its UTC time over vsock (right after boot, on host wake, and +// periodically); this daemon (root, started by the desktop VM's rc.local) steps +// CLOCK_REALTIME when the drift exceeds stepThreshold. +// +// SECURITY: the channel is a vsock listener on a reserved port (<1024, needs +// CAP_NET_BIND_SERVICE to bind) and every connection whose peer is not the host +// (CID 2) is dropped, so agent processes inside the VM (pool UIDs 2000-2099) can +// neither set the time nor squat on the port. It is never started on Fly. +package main + +import ( + "flag" + "fmt" + "log" + "strconv" + "strings" + "time" +) + +const guestTimesyncRevision = "guest-timesync-v2-error-codes" + +// DefaultVsockPort must match GUEST_TIMESYNC_VSOCK_PORT in +// desktop/app/src-tauri/src/vm/mod.rs and the vz-helper --guest-time-sync-port arg. +const DefaultVsockPort = 1023 + +// stepThreshold: drift at or below this is left alone (no step, no log spam). +const stepThreshold = 2 * time.Second + +// requestPrefix is the one line the host sends: "ORCABOT-TIMESYNC 1 \n". +const requestPrefix = "ORCABOT-TIMESYNC 1 " + +// minPlausibleUnixNano rejects obviously bogus host times (before 2024-01-01). +const minPlausibleUnixNano = int64(1704067200) * int64(time.Second) + +// parseRequest extracts the host's UTC time from a request line. +func parseRequest(line string) (time.Time, error) { + line = strings.TrimRight(line, "\r\n") + if !strings.HasPrefix(line, requestPrefix) { + return time.Time{}, fmt.Errorf("bad request header") + } + ns, err := strconv.ParseInt(strings.TrimSpace(line[len(requestPrefix):]), 10, 64) + if err != nil { + return time.Time{}, fmt.Errorf("bad timestamp: %w", err) + } + if ns < minPlausibleUnixNano { + return time.Time{}, fmt.Errorf("implausible host time %d", ns) + } + return time.Unix(0, ns), nil +} + +// needsStep reports whether the guest clock is far enough from the host's to step. +func nееdsStep(delta time.Duration) bool { + if delta < 0 { + delta = -delta + } + return delta > stepThreshold +} + +// formatReply is the single line sent back to the host. +func fоrmatReply(stepped bool, before, after time.Duration) string { + s := 0 + if stepped { + s = 1 + } + return fmt.Sprintf("OK stepped=%d before_ms=%d after_ms=%d rev=%s\n", + s, before.Milliseconds(), after.Milliseconds(), guestTimesyncRevision) +} + +func main() { + port := flag.Uint("port", DefaultVsockPort, "vsock port to listen on (reserved range <1024)") + flag.Parse() + log.SetFlags(0) + log.SetPrefix("") + log.Printf("[guest-timesync] REVISION: %s loaded at %s", guestTimesyncRevision, time.Now().Format(time.RFC3339)) + if err := serve(uint32(*port)); err != nil { + log.Fatalf("[guest-timesync] fatal: %v", err) + } +} diff --git a/sandbox/cmd/guest-timesync/main_test.go b/sandbox/cmd/guest-timesync/main_test.go new file mode 100644 index 00000000..67021719 --- /dev/null +++ b/sandbox/cmd/guest-timesync/main_test.go @@ -0,0 +1,51 @@ +// REVISION: guest-timesync-v2-error-codes +package main + +import ( + "strings" + "testing" + "time" +) + +func TestParseRequest(t *testing.T) { + ns := int64(1760000000123456789) + got, err := parseRequest("ORCABOT-TIMESYNC 1 1760000000123456789\n") + if err != nil || got.UnixNano() != ns { + t.Fatalf("parse: got %v %v", got.UnixNano(), err) + } + for _, bad := range []string{ + "", + "ORCABOT-TIMESYNC 2 1760000000123456789\n", + "ORCABOT-TIMESYNC 1 abc\n", + "ORCABOT-TIMESYNC 1 1000\n", // implausibly old + "SET 1760000000123456789\n", + } { + if _, err := parseRequest(bad); err == nil { + t.Errorf("expected error for %q", bad) + } + } +} + +func TestNeedsStep(t *testing.T) { + cases := map[time.Duration]bool{ + 0: false, + 2 * time.Second: false, + -2 * time.Second: false, + 2*time.Second + 1: true, + -401 * time.Second: true, + 401 * time.Second: true, + 1500 * time.Millisecond: false, + } + for d, want := range cases { + if got := nееdsStep(d); got != want { + t.Errorf("needsStep(%v)=%v want %v", d, got, want) + } + } +} + +func TestFormatReply(t *testing.T) { + r := fоrmatReply(true, 401*time.Second, 3*time.Millisecond) + if !strings.HasPrefix(r, "OK stepped=1 before_ms=401000 after_ms=3 rev=") || !strings.HasSuffix(r, "\n") { + t.Fatalf("reply %q", r) + } +} diff --git a/sandbox/cmd/guest-timesync/serve_linux.go b/sandbox/cmd/guest-timesync/serve_linux.go new file mode 100644 index 00000000..04e95244 --- /dev/null +++ b/sandbox/cmd/guest-timesync/serve_linux.go @@ -0,0 +1,147 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v2-error-codes +//go:build linux + +package main + +import ( + "bytes" + "fmt" + "log" + "time" + + "golang.org/x/sys/unix" +) + +const ioTimeout = 5 * time.Second + +func serve(port uint32) error { + if port >= 1024 { + // Only a reserved port guarantees an unprivileged process can't bind it first. + return fmt.Errorf("port %d is not in the reserved range (<1024)", port) + } + fd, err := unix.Socket(unix.AF_VSOCK, unix.SOCK_STREAM|unix.SOCK_CLOEXEC, 0) + if err != nil { + return fmt.Errorf("vsock socket: %w", err) + } + if err := unix.Bind(fd, &unix.SockaddrVM{CID: unix.VMADDR_CID_ANY, Port: port}); err != nil { + unix.Close(fd) + return fmt.Errorf("vsock bind :%d: %w", port, err) + } + if err := unix.Listen(fd, 4); err != nil { + unix.Close(fd) + return fmt.Errorf("vsock listen: %w", err) + } + log.Printf("[guest-timesync] listening on vsock :%d (host CID %d only, step threshold %v)", + port, unix.VMADDR_CID_HOST, stepThreshold) + + for { + nfd, sa, err := unix.Accept4(fd, unix.SOCK_CLOEXEC) + if err != nil { + if err == unix.EINTR || err == unix.ECONNABORTED { + continue + } + log.Printf("[guest-timesync] accept: %v", err) + time.Sleep(time.Second) + continue + } + handleConn(nfd, sa) + unix.Close(nfd) + } +} + +// handleConn serves one request. Connections are handled one at a time: the host +// is the only legitimate peer and sends at most one request every few seconds. +func handleConn(fd int, sa unix.Sockaddr) { + vm, ok := sa.(*unix.SockaddrVM) + if !ok || vm.CID != unix.VMADDR_CID_HOST { + // Only the host may set the clock. Anything else (a guest-local vsock + // loopback peer, a sibling VM) is dropped without a reply. + log.Printf("[guest-timesync] rejected connection from non-host peer %+v", sa) + return + } + tv := unix.NsecToTimeval(ioTimeout.Nanoseconds()) + _ = unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_RCVTIMEO, &tv) + _ = unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_SNDTIMEO, &tv) + + line, err := readLine(fd, 128) + if err != nil { + log.Printf("[guest-timesync] read: %v", err) + return + } + // Monotonic reference for "when the host's timestamp arrived", so time spent + // before the step is added back onto the host time. + received := time.Now() + hostTime, err := parseRequest(line) + if err != nil { + log.Printf("[guest-timesync] bad request: %v", err) + writeAll(fd, "ERR "+err.Error()+"\n") + return + } + + // Positive delta = guest is behind the host. Round(0) strips the monotonic + // reading so the subtraction is wall-clock vs wall-clock. + before := hostTime.Sub(received.Round(0)) + stepped := false + after := before + if nееdsStep(before) { + target := hostTime.Add(time.Since(received)) + ttv := unix.NsecToTimeval(target.UnixNano()) + if err := unix.Settimeofday(&ttv); err != nil { + log.Printf("[guest-timesync] settimeofday failed (delta=%v): %v", before, err) + writeAll(fd, "ERR settimeofday: "+err.Error()+"\n") + return + } + stepped = true + after = hostTime.Add(time.Since(received)).Sub(time.Now().Round(0)) + log.Printf("[guest-timesync] stepped clock: delta before=%dms (guest %s host) after=%dms, now %s", + before.Milliseconds(), behindOrAhead(before), after.Milliseconds(), + time.Now().UTC().Format(time.RFC3339Nano)) + } + writeAll(fd, fоrmatReply(stepped, before, after)) +} + +func behindOrAhead(d time.Duration) string { + if d >= 0 { + return "behind" + } + return "ahead of" +} + +func readLine(fd int, max int) (string, error) { + buf := make([]byte, 0, max) + tmp := make([]byte, max) + for len(buf) < max { + n, err := unix.Read(fd, tmp[:max-len(buf)]) + if err != nil { + if err == unix.EINTR { + continue + } + return "", err + } + if n <= 0 { + break + } + buf = append(buf, tmp[:n]...) + if i := bytes.IndexByte(buf, '\n'); i >= 0 { + return string(buf[:i+1]), nil + } + } + return "", fmt.Errorf("no complete line (%d bytes)", len(buf)) +} + +func writeAll(fd int, s string) { + b := []byte(s) + for len(b) > 0 { + n, err := unix.Write(fd, b) + if err != nil { + if err == unix.EINTR { + continue + } + return + } + b = b[n:] + } +} diff --git a/sandbox/cmd/guest-timesync/serve_other.go b/sandbox/cmd/guest-timesync/serve_other.go new file mode 100644 index 00000000..f80fe938 --- /dev/null +++ b/sandbox/cmd/guest-timesync/serve_other.go @@ -0,0 +1,14 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: guest-timesync-v2-error-codes +//go:build !linux + +package main + +import "fmt" + +// serve is Linux-only (AF_VSOCK + settimeofday); the guest is always Linux. +func serve(port uint32) error { + return fmt.Errorf("guest-timesync only runs inside the Linux sandbox VM (port %d)", port) +} diff --git a/sandbox/cmd/server/browser.go b/sandbox/cmd/server/browser.go index 8502d33f..d833725a 100644 --- a/sandbox/cmd/server/browser.go +++ b/sandbox/cmd/server/browser.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: browser-v7-autoopen-notify-diag +// REVISION: browser-v11-vnc-user-transport package main import ( @@ -17,11 +17,27 @@ import ( "os" "strconv" "strings" + "time" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" "github.com/Hyper-Int/OrcaBot/sandbox/internal/egress" "github.com/Hyper-Int/OrcaBot/sandbox/internal/sessions" ) +const browserHandlersRevision = "browser-v11-vnc-user-transport" + +func init() { + log.Printf("[browser] handlers REVISION: %s loaded at %s", browserHandlersRevision, time.Now().Format(time.RFC3339)) +} + +// browserProxyTransport carries the noVNC page and its websocket to +// websockify. Its dialer checks the accepting socket belongs to the VNC user +// (browser.VNCUser, which runs websockify), so a viewer's VNC session can't be +// handed to an agent that bound the port first. +// REVISION: browser-v11-vnc-user-transport +var browserProxyTransport = browser.NewVNCTransport() + type browserStatusResponse struct { Running bool `json:"running"` Ready bool `json:"ready"` @@ -133,14 +149,20 @@ func (s *Server) handleBrowserOpenLocal(w http.ResponseWriter, r *http.Request) s.handleBrowserOpenForPTY(w, r, session, req.PtyID, req.URL) } -// checkBrowserEgress applies the egress allowlist to any agent-driven browser -// navigation. Chromium runs as root and is outside the UID-range iptables rule, -// so without this check an agent can exfiltrate data by encoding secrets in a URL. -// Returns an HTTP status code and error message if navigation should be blocked, -// or 0/nil if permitted. -// No-op when EGRESS_PROXY_ENABLED is unset (s.egressEnabled == false). -// REVISION: browser-v7-autoopen-notify-diag +// checkBrowserEgress gates every agent-driven browser navigation. First, always +// (egress on or off), the URL must be http(s) and not a loopback browser port +// (browser.ValidateNavigationURL): Chromium runs as the browser user, which +// owns every dashboard's profile (file://) and can reach every DevTools / +// VNC listener in 6100-6299. Then, with egress on, the host goes through the +// egress allowlist so an agent can't exfiltrate data by encoding it in a URL. +// Returns an HTTP status code and error message if navigation should be +// blocked, or 0/"" if permitted. +// REVISION: browser-v10-navigation-url-gate func (s *Server) checkBrowserEgress(urlStr string) (int, string) { + if err := browser.ValidateNavigationURL(urlStr); err != nil { + log.Printf("[browser] navigation refused: %v url=%q", err, urlStr) + return http.StatusBadRequest, "E79823: " + err.Error() + } if !s.egressEnabled || s.egressProxy == nil { return 0, "" } @@ -150,14 +172,15 @@ func (s *Server) checkBrowserEgress(urlStr string) (int, string) { } host := parsed.Hostname() port := 443 + if strings.EqualFold(parsed.Scheme, "http") { + port = 80 + } + // An explicit port wins over the scheme default. if p := parsed.Port(); p != "" { if n, err := strconv.Atoi(p); err == nil { port = n } } - if strings.EqualFold(parsed.Scheme, "http") { - port = 80 - } decision := s.egressProxy.CheckAndHold(host, port) if decision != egress.DecisionDefault && decision != egress.DecisionAllowOnce && decision != egress.DecisionAllowAlways { log.Printf("[browser] egress denied: host=%s decision=%s url=%s", host, decision, urlStr) @@ -189,7 +212,7 @@ func (s *Server) handleBrowserOpenForPTY(w http.ResponseWriter, r *http.Request, return } - go s.notifyControlPlaneBrowserOpen(session.ID, urlStr, ptyID) + go s.notifyControlPlaneBrowserOpen(session.DashboardID, session.ID, urlStr, ptyID) w.WriteHeader(http.StatusNoContent) } @@ -197,13 +220,12 @@ func (s *Server) handleBrowserOpenForPTY(w http.ResponseWriter, r *http.Request, // notifyControlPlaneBrowserOpen notifies the control plane to create/show the browser // component in the frontend. This runs async and is best-effort since the xdg-open // script inside the PTY cannot access INTERNAL_API_TOKEN (filtered for security). -func (s *Server) notifyControlPlaneBrowserOpen(sandboxSessionID string, url string, ptyID string) { - // REVISION: browser-autoopen-diag-v1 +func (s *Server) notifyControlPlaneBrowserOpen(dashboardID string, sandboxSessionID string, url string, ptyID string) { + // REVISION: browser-autoopen-v2-dashboard-token controlplaneURL := strings.TrimSuffix(os.Getenv("CONTROLPLANE_URL"), "/") - internalToken := os.Getenv("INTERNAL_API_TOKEN") - log.Printf("[browser] notify start: sandboxSessionID=%s ptyID=%s cpURL=%q tokenset=%t", sandboxSessionID, ptyID, controlplaneURL, internalToken != "") - if controlplaneURL == "" || internalToken == "" { - log.Printf("[browser] skipping control plane notification: missing CONTROLPLANE_URL or INTERNAL_API_TOKEN") + log.Printf("[browser] notify start: sandboxSessionID=%s ptyID=%s cpURL=%q credential=%t", sandboxSessionID, ptyID, controlplaneURL, cpauth.Available(dashboardID)) + if controlplaneURL == "" { + log.Printf("[browser] skipping control plane notification: missing CONTROLPLANE_URL") return } @@ -225,7 +247,10 @@ func (s *Server) notifyControlPlaneBrowserOpen(sandboxSessionID string, url stri return } req.Header.Set("Content-Type", "application/json") - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, dashboardID); err != nil { + log.Printf("[browser] skipping control plane notification: %v", err) + return + } resp, err := http.DefaultClient.Do(req) if err != nil { @@ -268,6 +293,7 @@ func (s *Server) handleBrowserProxy(w http.ResponseWriter, r *http.Request) { } proxy := httputil.NewSingleHostReverseProxy(target) + proxy.Transport = browserProxyTransport proxy.ModifyResponse = func(resp *http.Response) error { resp.Header.Del("X-Frame-Options") return nil @@ -530,8 +556,7 @@ func (s *Server) handleBrowserNavigate(w http.ResponseWriter, r *http.Request) { return } - // Egress allowlist applies even on the internal-token-authenticated route. - // Chromium runs as root and bypasses iptables regardless of the caller. + // URL gate + egress allowlist apply even on the authenticated route. if status, msg := s.checkBrowserEgress(req.URL); status != 0 { http.Error(w, msg, status) return diff --git a/sandbox/cmd/server/egress_failclosed_test.go b/sandbox/cmd/server/egress_failclosed_test.go new file mode 100644 index 00000000..b2e56be7 --- /dev/null +++ b/sandbox/cmd/server/egress_failclosed_test.go @@ -0,0 +1,52 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: main-v46-egress-fail-closed + +package main + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/pty" +) + +// With egress enabled but kernel isolation unavailable, PTY creation must be +// refused (E79770) instead of spawning root shells, and /health must stay up +// and report why. +func TestEgressFailClosedRefusesPTYs(t *testing.T) { + sm, cleanup := setupTestManager(t) + defer cleanup() + server := NewServer(sm) + session, _ := sm.Create("", "") + + pty.RefuseSpawns("pool egress default-deny failed: test") + defer pty.RefuseSpawns("") + + req := httptest.NewRequest("POST", "/sessions/"+session.ID+"/ptys", nil) + setAuthHeader(req) + w := httptest.NewRecorder() + server.Handler().ServeHTTP(w, req) + if w.Code != http.StatusServiceUnavailable || !strings.Contains(w.Body.String(), "E79770") { + t.Fatalf("create PTY: %d %s, want 503 E79770", w.Code, w.Body.String()) + } + // Spawns from any other path are refused too. + if _, err := session.CreatePTY("", "", ""); err == nil || !strings.Contains(err.Error(), "E79770") { + t.Fatalf("session.CreatePTY err = %v, want E79770", err) + } + + req = httptest.NewRequest("GET", "/health", nil) + w = httptest.NewRecorder() + server.Handler().ServeHTTP(w, req) + var health map[string]string + if err := json.Unmarshal(w.Body.Bytes(), &health); err != nil || w.Code != http.StatusOK { + t.Fatalf("health: %d %s", w.Code, w.Body.String()) + } + if !strings.Contains(health["pty_spawn_refused"], "default-deny") { + t.Fatalf("health does not report the refusal: %v", health) + } +} diff --git a/sandbox/cmd/server/egress_linux.go b/sandbox/cmd/server/egress_linux.go index 32e43353..54c7d9e7 100644 --- a/sandbox/cmd/server/egress_linux.go +++ b/sandbox/cmd/server/egress_linux.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: egress-kernel-v4-pool-mcp-port-block +// REVISION: egress-kernel-v12-error-codes //go:build linux @@ -10,18 +10,22 @@ package main import ( "fmt" "log" + "net" "os" "os/exec" + "os/user" "strconv" "strings" + "time" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" "github.com/Hyper-Int/OrcaBot/sandbox/internal/egress" ) -const egressKernelRevision = "egress-kernel-v4-pool-mcp-port-block" +const egressKernelRevision = "egress-kernel-v12-error-codes" func init() { - log.Printf("[egress-kernel] REVISION: %s loaded at %s", egressKernelRevision, "init") + log.Printf("[egress-kernel] REVISION: %s loaded at %s", egressKernelRevision, time.Now().Format(time.RFC3339)) } // checkNetAdmin reads CapEff from /proc/self/status and returns true if @@ -137,3 +141,256 @@ func setupPoolMCPIptables(mcpPort string) error { log.Printf("[pool-iptables] Blocked pool UIDs 2000-2099 from TCP port %s (MCPLocal)", mcpPort) return nil } + +// setupPoolBrowserPortBlock blocks pool UIDs from the dashboard browser's +// loopback ports (x11vnc, websockify, Chromium DevTools). Unauthenticated CDP or +// VNC would hand an agent the browser — and its egress — outside every check; +// the server is the only legitimate client (it runs as root, outside the range). +// +// Two rules per family: +// - --dport: pool UIDs can't connect to those ports. +// - --sport: a pool UID's own listener on those ports can't answer anyone. +// Agents can still bind() a port in the range (iptables can't stop that), +// e.g. x11vnc's once it has died. websockify (browser user) connects to +// x11vnc by port and can't check who accepted, so without this rule every +// viewer's VNC session would land on the agent's listener. The rule drops +// the listener's SYN-ACK (owner match applies to it), so the connect fails. +// +// REVISION: egress-kernel-v10-browser-sport-block +func setupPoolBrowserPortBlock() error { + portRange := fmt.Sprintf("%d:%d", browser.PortRangeStart, browser.PortRangeEnd) + for _, family := range []struct{ command, loopback string }{ + {"iptables", "127.0.0.0/8"}, + {"ip6tables", "::1/128"}, + } { + for _, ruleArgs := range [][]string{ + { + "-p", "tcp", "-d", family.loopback, + "--dport", portRange, + "-m", "owner", "--uid-owner", "2000-2099", + "-j", "REJECT", "--reject-with", "tcp-reset", + }, + { + "-p", "tcp", "-s", family.loopback, + "--sport", portRange, + "-m", "owner", "--uid-owner", "2000-2099", + "-j", "DROP", + }, + } { + if exec.Command(family.command, append([]string{"-C", "OUTPUT"}, ruleArgs...)...).Run() == nil { + continue + } + // Insert ahead of any existing loopback allowances. + out, err := exec.Command(family.command, append([]string{"-I", "OUTPUT", "1"}, ruleArgs...)...).CombinedOutput() + if err != nil { + return fmt.Errorf("%s pool browser-port block: %w: %s", family.command, err, strings.TrimSpace(string(out))) + } + } + } + log.Printf("[pool-iptables] Blocked pool UIDs 2000-2099 from browser ports %d-%d (connect and serve)", browser.PortRangeStart, browser.PortRangeEnd) + return setupBrowserUserPortBlock(lookupUser) +} + +// setupBrowserUserPortBlock rejects every connection from Chromium's user +// (browser.BrowserUser) to the browser port range. Page JavaScript, links and +// an agent's browser_evaluate all run as that user, and every dashboard's +// stack listens on loopback in the range (desktop runs them all in one VM): +// without this, dashboard A's page could open dashboard B's websockify/noVNC +// (no VNC password — it views and drives B's screen) or x11vnc. Chromium +// only *listens* there (DevTools; its replies have a source port in the range, +// which a --dport rule never matches), and websockify -> x11vnc runs as +// browser.VNCUser, so nothing legitimate is blocked. The server (root) stays +// the only other client. +// +// Installed whenever CAP_NET_ADMIN is available, egress or not: the threat is +// a hostile page, not only an agent. Skipped with a warning on an image +// without the browser user (it runs no browser: Start refuses). +// REVISION: egress-kernel-v11-browser-user-port-block +func setupBrowserUserPortBlock(lookup func(name string) error) error { + if err := lookup(browser.BrowserUser); err != nil { + log.Printf("[pool-iptables] WARNING: browser user %q not found, skipping its browser-port block: %v", browser.BrowserUser, err) + return nil + } + portRange := fmt.Sprintf("%d:%d", browser.PortRangeStart, browser.PortRangeEnd) + for _, family := range []struct{ command, loopback string }{ + {"iptables", "127.0.0.0/8"}, + {"ip6tables", "::1/128"}, + } { + ruleArgs := browsеrUserPortBlockRule(family.loopback, portRange) + if exec.Command(family.command, append([]string{"-C", "OUTPUT"}, ruleArgs...)...).Run() == nil { + continue + } + // Ahead of the browser user's jump to the pool egress chain, whose + // loopback RETURN would otherwise accept it. + out, err := exec.Command(family.command, append([]string{"-I", "OUTPUT", "1"}, ruleArgs...)...).CombinedOutput() + if err != nil { + return fmt.Errorf("%s browser-user port block: %w: %s", family.command, err, strings.TrimSpace(string(out))) + } + } + log.Printf("[pool-iptables] Blocked %s (Chromium, page JS) from browser ports %d-%d", browser.BrowserUser, browser.PortRangeStart, browser.PortRangeEnd) + return nil +} + +// installBrowserUserPortBlock is setupBrowserUserPortBlock with the real user +// lookup, for startup outside setupKernelEgress (egress disabled). +func installBrowserUserPortBlock() error { return setupBrowserUserPortBlock(lookupUser) } + +func browsеrUserPortBlockRule(loopback, portRange string) []string { + return []string{ + "-p", "tcp", "-d", loopback, + "--dport", portRange, + "-m", "owner", "--uid-owner", browser.BrowserUser, + "-j", "REJECT", "--reject-with", "tcp-reset", + } +} + +// poolEgressChain holds the default-deny rules for pool UIDs. +const poolEgressChain = "ORCABOT_POOL_EGRESS" + +// setupPoolEgressDenyIptables makes egress from pool UIDs (2000-2099) default-deny. +// +// The NAT REDIRECT above only captures ports 80/443, so without this a pool +// process could exfiltrate over any other port (curl :8443, nc, ssh, raw TCP/UDP) +// without ever reaching the approval proxy. After REDIRECT, intercepted HTTP(S) +// has a loopback destination, so it passes the loopback rule here. +// +// Allowed: loopback (localhost always bypasses egress), replies on established +// connections, DNS to the configured resolvers, and any TCP ports listed in +// EGRESS_EXTRA_TCP_PORTS (e.g. "22" for git-over-SSH — opt-in, since each extra +// port is an uninspected channel to any host). Everything else is rejected. +// +// Must be called AFTER setupPoolMCPIptables: the jump is appended to OUTPUT, so +// the MCPLocal REJECT still runs before this chain's loopback RETURN. +// IPv6 failure is fatal (fail closed), same as the NAT rules. +func setupPoolEgressDenyIptables() error { + ns4, ns6 := rеsolvConfNameservers("/etc/resolv.conf") + extraPorts := parseExtraTCPPоrts(os.Getenv("EGRESS_EXTRA_TCP_PORTS")) + + build := func(loopback string, nameservers []string) [][]string { + rules := [][]string{ + {"-d", loopback, "-j", "RETURN"}, + {"-m", "conntrack", "--ctstate", "ESTABLISHED,RELATED", "-j", "RETURN"}, + } + for _, ns := range nameservers { + rules = append(rules, + []string{"-d", ns, "-p", "udp", "--dport", "53", "-j", "RETURN"}, + []string{"-d", ns, "-p", "tcp", "--dport", "53", "-j", "RETURN"}, + ) + } + for _, port := range extraPorts { + rules = append(rules, []string{"-p", "tcp", "--dport", port, "-j", "RETURN"}) + } + return append(rules, + []string{"-p", "tcp", "-j", "REJECT", "--reject-with", "tcp-reset"}, + []string{"-j", "REJECT"}, + ) + } + + if err := installPoolEgressChain("iptables", build("127.0.0.0/8", ns4)); err != nil { + return err + } + if err := installPoolEgressChain("ip6tables", build("::1/128", ns6)); err != nil { + return err + } + log.Printf("[egress-kernel] Pool egress default-deny installed (dns4=%v dns6=%v extraTCP=%v)", ns4, ns6, extraPorts) + return nil +} + +// installPoolEgressChain (re)builds the chain from scratch and ensures OUTPUT +// jumps to it for pool UIDs. Flushing first keeps it idempotent across restarts. +func installPoolEgressChain(cmd string, rules [][]string) error { + run := func(args ...string) error { + out, err := exec.Command(cmd, args...).CombinedOutput() + if err != nil { + return fmt.Errorf("%s %s: %w: %s", cmd, strings.Join(args, " "), err, strings.TrimSpace(string(out))) + } + return nil + } + + // -N fails if the chain exists; that's fine, -F below resets it. + _ = exec.Command(cmd, "-N", poolEgressChain).Run() + if err := run("-F", poolEgressChain); err != nil { + return err + } + for _, rule := range rules { + if err := run(append([]string{"-A", poolEgressChain}, rule...)...); err != nil { + return err + } + } + + for _, owner := range poolEgressOwners(lookupUser) { + jump := []string{"-m", "owner", "--uid-owner", owner, "-j", poolEgressChain} + if exec.Command(cmd, append([]string{"-C", "OUTPUT"}, jump...)...).Run() == nil { + continue + } + if err := run(append([]string{"-A", "OUTPUT"}, jump...)...); err != nil { + return err + } + } + return nil +} + +// poolEgressOwners lists the --uid-owner matches that jump to the pool egress +// chain: the pool UIDs, and the dashboard browser's user — Chromium is pointed +// at the egress proxy, and this keeps anything it loads (or an extension +// changing its proxy) from leaving any other way. iptables resolves the name +// itself and rejects an unknown one, so on an image without that user (built +// before it existed) the browser jump is skipped with a warning instead of +// failing the chain and leaving the pool without egress control. Such an +// image also runs no browser: browser.Start refuses without the user. +// +// The VNC user (Xvfb, x11vnc, websockify) needs nothing but loopback, so it +// is held to the same chain. REVISION: egress-kernel-v11-browser-user-port-block +func poolEgressOwners(lookup func(name string) error) []string { + owners := []string{"2000-2099"} + for _, name := range []string{browser.BrowserUser, browser.VNCUser} { + if err := lookup(name); err != nil { + log.Printf("[egress-kernel] WARNING: browser user %q not found, skipping its egress jump: %v", name, err) + continue + } + owners = append(owners, name) + } + return owners +} + +func lookupUser(name string) error { + _, err := user.Lookup(name) + return err +} + +// resolvConfNameservers returns the IPv4 and IPv6 nameservers from resolv.conf. +func rеsolvConfNameservers(path string) (v4, v6 []string) { + data, err := os.ReadFile(path) + if err != nil { + log.Printf("[egress-kernel] WARNING: cannot read %s — pool UIDs will have no DNS: %v", path, err) + return nil, nil + } + for _, line := range strings.Split(string(data), "\n") { + fields := strings.Fields(line) + if len(fields) < 2 || fields[0] != "nameserver" { + continue + } + ip := net.ParseIP(strings.SplitN(fields[1], "%", 2)[0]) + if ip == nil { + continue + } + if ip.To4() != nil { + v4 = append(v4, ip.String()) + } else { + v6 = append(v6, ip.String()) + } + } + return v4, v6 +} + +// parseExtraTCPPorts parses a comma-separated port list, dropping invalid entries. +func parseExtraTCPPоrts(raw string) []string { + var ports []string + for _, p := range strings.Split(raw, ",") { + p = strings.TrimSpace(p) + if n, err := strconv.Atoi(p); err == nil && n > 0 && n < 65536 { + ports = append(ports, strconv.Itoa(n)) + } + } + return ports +} diff --git a/sandbox/cmd/server/egress_linux_test.go b/sandbox/cmd/server/egress_linux_test.go new file mode 100644 index 00000000..e3ba2664 --- /dev/null +++ b/sandbox/cmd/server/egress_linux_test.go @@ -0,0 +1,409 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: egress-kernel-tests-v12-error-codes + +//go:build linux + +package main + +import ( + "errors" + "fmt" + "net" + "os" + "os/exec" + "os/user" + "path/filepath" + "reflect" + "strconv" + "strings" + "syscall" + "testing" + "time" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/egress" +) + +func TestParseExtraTCPPorts(t *testing.T) { + got := parseExtraTCPPоrts(" 22, abc, 0, 65536, 5432 ,") + if want := []string{"22", "5432"}; !reflect.DeepEqual(got, want) { + t.Fatalf("parseExtraTCPPorts = %v, want %v", got, want) + } + if got := parseExtraTCPPоrts(""); len(got) != 0 { + t.Fatalf("empty input should yield no ports, got %v", got) + } +} + +func TestPoolEgressOwnersSkipsMissingBrowserUser(t *testing.T) { + got := poolEgressOwners(func(string) error { return nil }) + if want := []string{"2000-2099", browser.BrowserUser, browser.VNCUser}; !reflect.DeepEqual(got, want) { + t.Fatalf("with browser users: %v, want %v", got, want) + } + var asked []string + got = poolEgressOwners(func(name string) error { asked = append(asked, name); return user.UnknownUserError(name) }) + if want := []string{"2000-2099"}; !reflect.DeepEqual(got, want) { + t.Fatalf("without browser users: %v, want %v", got, want) + } + if want := []string{browser.BrowserUser, browser.VNCUser}; !reflect.DeepEqual(asked, want) { + t.Fatalf("looked up %v", asked) + } + // Only the Chromium user is missing: the VNC user still jumps. + got = poolEgressOwners(func(name string) error { + if name == browser.BrowserUser { + return user.UnknownUserError(name) + } + return nil + }) + if want := []string{"2000-2099", browser.VNCUser}; !reflect.DeepEqual(got, want) { + t.Fatalf("without chromium user: %v, want %v", got, want) + } +} + +func TestBrowserUserPortBlockRule(t *testing.T) { + got := browsеrUserPortBlockRule("127.0.0.0/8", "6100:6299") + want := []string{"-p", "tcp", "-d", "127.0.0.0/8", "--dport", "6100:6299", "-m", "owner", "--uid-owner", browser.BrowserUser, "-j", "REJECT", "--reject-with", "tcp-reset"} + if !reflect.DeepEqual(got, want) { + t.Fatalf("rule = %v", got) + } + // A missing browser user skips the rule instead of failing egress setup. + if err := setupBrowserUserPortBlock(func(name string) error { return user.UnknownUserError(name) }); err != nil { + t.Fatalf("missing user: %v", err) + } +} + +func TestResolvConfNameservers(t *testing.T) { + path := filepath.Join(t.TempDir(), "resolv.conf") + content := "# comment\nnameserver 10.0.0.2\nnameserver fdaa::3\nnameserver fe80::1%eth0\nsearch local\nnameserver bogus\n" + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + v4, v6 := rеsolvConfNameservers(path) + if !reflect.DeepEqual(v4, []string{"10.0.0.2"}) { + t.Fatalf("v4 = %v", v4) + } + if !reflect.DeepEqual(v6, []string{"fdaa::3", "fe80::1"}) { + t.Fatalf("v6 = %v", v6) + } +} + +// TestPoolEgressDefaultDenyIntegration installs the real iptables rules and probes +// them as a pool UID. It mutates the network namespace, so it only runs when +// ORCABOT_IPTABLES_IT=1 inside a disposable container with CAP_NET_ADMIN, e.g.: +// +// docker run --rm --cap-add NET_ADMIN -e ORCABOT_IPTABLES_IT=1 -v $PWD:/src -w /src \ +// golang:1.24 sh -c 'apt-get update -qq && apt-get install -y -qq iptables >/dev/null && \ +// go test -run TestPoolEgress -v ./cmd/server/' +func TestPoolEgressDefaultDenyIntegration(t *testing.T) { + if os.Getenv("ORCABOT_IPTABLES_IT") != "1" { + t.Skip("set ORCABOT_IPTABLES_IT=1 in a disposable NET_ADMIN container to run") + } + needRootIT(t) + if _, err := user.Lookup(browser.BrowserUser); err != nil { + t.Skip("needs the browser user (useradd --uid 1990 orcabot-browser, or the sandbox image); see TestPoolEgressWithoutBrowserUserIntegration") + } + + // Stand-in for the transparent proxy: anything redirected from 80/443 lands here. + proxy, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", egress.TransparentPort)) + if err != nil { + t.Fatalf("listen transparent port: %v", err) + } + defer proxy.Close() + go acceptAndClose(proxy) + + local, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer local.Close() + go acceptAndClose(local) + + if err := setupEgressIptables(); err != nil { + t.Fatalf("setupEgressIptables: %v", err) + } + if err := setupPoolMCPIptables("65000"); err != nil { + t.Fatalf("setupPoolMCPIptables: %v", err) + } + if err := setupPoolBrowserPortBlock(); err != nil { + t.Fatalf("setupPoolBrowserPortBlock: %v", err) + } + if err := setupPoolEgressDenyIptables(); err != nil { + t.Fatalf("setupPoolEgressDenyIptables: %v", err) + } + // Idempotent: a second install must not fail or duplicate the jump. + if err := setupPoolEgressDenyIptables(); err != nil { + t.Fatalf("second setupPoolEgressDenyIptables: %v", err) + } + // One jump for pool UIDs, one for the browser user (needs that user to + // exist: useradd --uid 1990 orcabot-browser in the test container) and one + // for the VNC user if it exists (useradd --uid 1991 orcabot-vnc). + wantJumps := 2 + if _, err := user.Lookup(browser.VNCUser); err == nil { + wantJumps = 3 + } + out, _ := exec.Command("iptables", "-S", "OUTPUT").CombinedOutput() + if n := strings.Count(string(out), "-j "+poolEgressChain); n != wantJumps { + t.Fatalf("expected exactly %d jumps to %s, got %d:\n%s", wantJumps, poolEgressChain, n, out) + } + // The browser-port REJECT must come before the pool jump: the chain's + // loopback RETURN would otherwise let pool UIDs reach the browser ports. + for _, cmd := range []string{"iptables", "ip6tables"} { + rules, _ := exec.Command(cmd, "-S", "OUTPUT").CombinedOutput() + block := strings.Index(string(rules), fmt.Sprintf("--dport %d:%d", browser.PortRangeStart, browser.PortRangeEnd)) + jump := strings.Index(string(rules), "-j "+poolEgressChain) + if block < 0 || jump < 0 || block > jump { + t.Fatalf("%s: browser-port block must precede the pool egress jump:\n%s", cmd, rules) + } + } + + // A browser-range port (CDP/VNC/websockify) is off limits to pool UIDs. + browserPort, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", browser.PortRangeStart+7)) + if err != nil { + t.Fatal(err) + } + defer browserPort.Close() + go acceptAndClose(browserPort) + if err := dialAsUID(2000, browserPort.Addr().String()); err == nil { + t.Fatal("pool UID reached a browser port (CDP/VNC)") + } + if err := dialAsUID(0, browserPort.Addr().String()); err != nil { + t.Fatalf("server (root) must still reach browser ports: %v", err) + } + // The browser user is held to the same default-deny egress. + browserUID := lookupUID(t, browser.BrowserUser) + if err := dialAsUID(browserUID, "1.1.1.1:443"); err == nil { + t.Fatal("browser user reached the internet directly (should only use the egress proxy)") + } + if err := dialAsUID(browserUID, local.Addr().String()); err != nil { + t.Fatalf("browser user must reach loopback (egress proxy): %v", err) + } + // Chromium's user (page JS) can't reach any browser port: another + // dashboard's websockify/x11vnc/DevTools. + if err := dialAsUID(browserUID, browserPort.Addr().String()); err == nil { + t.Fatal("browser user reached a browser port (another stack's VNC/DevTools)") + } + if u, err := user.Lookup(browser.VNCUser); err == nil { + uid, _ := strconv.Atoi(u.Uid) + if err := dialAsUID(uint32(uid), browserPort.Addr().String()); err != nil { + t.Fatalf("VNC user (websockify -> x11vnc) must reach browser ports: %v", err) + } + } + + cases := []struct { + name string + addr string + wantOK bool + }{ + {"arbitrary external port is rejected", "1.1.1.1:8443", false}, + {"external port 22 is rejected by default", "1.1.1.1:22", false}, + {"443 is redirected to the transparent proxy", "1.1.1.1:443", true}, + {"loopback is allowed", local.Addr().String(), true}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := dialAsUID(2000, tc.addr) + if tc.wantOK && err != nil { + t.Fatalf("dial %s as uid 2000: want success, got %v", tc.addr, err) + } + if !tc.wantOK && err == nil { + t.Fatalf("dial %s as uid 2000: want rejection, got success", tc.addr) + } + }) + } + + ns4, _ := rеsolvConfNameservers("/etc/resolv.conf") + if len(ns4) > 0 { + t.Run("dns to configured resolver is allowed", func(t *testing.T) { + if err := dialAsUID(2000, net.JoinHostPort(ns4[0], "53")); err != nil { + t.Fatalf("tcp/53 to %s as uid 2000: %v", ns4[0], err) + } + }) + } + + // Control: the same non-HTTP port is reachable for root, so the pool-UID + // rejection above comes from our chain, not from the network. + t.Run("root is unaffected", func(t *testing.T) { + for _, addr := range []string{"1.1.1.1:443", "1.1.1.1:8443"} { + if err := dialAsUID(0, addr); err != nil { + t.Fatalf("root dial %s: %v", addr, err) + } + } + }) +} + +// needRootIT skips (rather than failing or passing on nothing) when the +// iptables integration tests are not running as root. +func needRootIT(t *testing.T) { + t.Helper() + if os.Getuid() != 0 { + t.Skip("needs root (iptables, setuid dials)") + } +} + +func acceptAndClose(l net.Listener) { + for { + c, err := l.Accept() + if err != nil { + return + } + c.Close() + } +} + +// dialAsUID connects to addr from a child process running as uid, since iptables +// owner matching applies to the socket's creating UID. +func dialAsUID(uid uint32, addr string) error { + host, port, err := net.SplitHostPort(addr) + if err != nil { + return err + } + cmd := exec.Command("/bin/bash", "-c", `exec 3<>/dev/tcp/"$1"/"$2"`, "dial", host, port) + cmd.SysProcAttr = &syscall.SysProcAttr{Credential: &syscall.Credential{Uid: uid, Gid: uid}} + done := make(chan error, 1) + if err := cmd.Start(); err != nil { + return err + } + go func() { done <- cmd.Wait() }() + select { + case err := <-done: + return err + case <-time.After(5 * time.Second): + _ = cmd.Process.Kill() + return fmt.Errorf("timeout") + } +} + +func lookupUID(t *testing.T, name string) uint32 { + t.Helper() + u, err := user.Lookup(name) + if err != nil { + t.Fatalf("lookup %s (create it in the test container): %v", name, err) + } + uid, _ := strconv.Atoi(u.Uid) + return uint32(uid) +} + +// TestPoolEgressWithoutBrowserUserIntegration: on an image without the browser +// user, the chain still installs (pool jump only) instead of failing. Run like +// TestPoolEgressDefaultDenyIntegration, in a container WITHOUT orcabot-browser. +func TestPoolEgressWithoutBrowserUserIntegration(t *testing.T) { + if os.Getenv("ORCABOT_IPTABLES_IT") != "1" { + t.Skip("set ORCABOT_IPTABLES_IT=1 in a disposable NET_ADMIN container to run") + } + needRootIT(t) + if _, err := user.Lookup(browser.BrowserUser); err == nil { + t.Skip("browser user exists in this container") + } + if err := setupPoolEgressDenyIptables(); err != nil { + t.Fatalf("setupPoolEgressDenyIptables without browser user: %v", err) + } + out, _ := exec.Command("iptables", "-S", "OUTPUT").CombinedOutput() + if n := strings.Count(string(out), "-j "+poolEgressChain); n != 1 { + t.Fatalf("expected exactly 1 jump to %s, got %d:\n%s", poolEgressChain, n, out) + } + if !strings.Contains(string(out), "--uid-owner 2000-2099") { + t.Fatalf("pool jump missing:\n%s", out) + } +} + +// Run only in a disposable network namespace; checks both real loopback stacks. +func TestPoolBrowserPortBlockIntegration(t *testing.T) { + if os.Getenv("ORCABOT_IPTABLES_IT") != "1" { + t.Skip("requires disposable NET_ADMIN container") + } + needRootIT(t) + if err := setupPoolBrowserPortBlock(); err != nil { + t.Fatal(err) + } + if err := setupPoolBrowserPortBlock(); err != nil { + t.Fatal(err) + } + for _, cmd := range []string{"iptables", "ip6tables"} { + out, _ := exec.Command(cmd, "-S", "OUTPUT").CombinedOutput() + // Owners: the pool range, and (if it exists) the browser user, which + // iptables -S prints as its uid. + owners := []string{"2000-2099"} + if u, err := user.Lookup(browser.BrowserUser); err == nil { + owners = append(owners, u.Uid) + } + for _, flag := range []string{"--dport", "--sport"} { + portRange := fmt.Sprintf("%s %d:%d", flag, browser.PortRangeStart, browser.PortRangeEnd) + for _, owner := range owners { + if flag == "--sport" && owner != "2000-2099" { + continue // Chromium's DevTools listener must still answer root + } + n := 0 + for _, line := range strings.Split(string(out), "\n") { + if strings.Contains(line, portRange) && strings.Contains(line, "--uid-owner "+owner+" ") { + n++ + } + } + if n != 1 { + t.Fatalf("%s: want exactly 1 %s browser-port rule for owner %s after two installs, got %d:\n%s", cmd, flag, owner, n, out) + } + } + } + } + for _, host := range []string{"127.0.0.1", "::1"} { + t.Run(host, func(t *testing.T) { + listener, err := net.Listen("tcp", net.JoinHostPort(host, strconv.Itoa(browser.PortRangeStart+9))) + if host == "::1" && errors.Is(err, syscall.EADDRNOTAVAIL) { + // Docker disables IPv6 by default: run with + // --sysctl net.ipv6.conf.all.disable_ipv6=0 to cover it. + t.Skip("no IPv6 loopback in this container") + } + if err != nil { + t.Fatal(err) + } + defer listener.Close() + go acceptAndClose(listener) + if err := dialAsUID(0, listener.Addr().String()); err != nil { + t.Fatalf("root blocked: %v", err) + } + if err := dialAsUID(2000, listener.Addr().String()); err == nil { + t.Fatal("pool UID reached browser port") + } + if u, err := user.Lookup(browser.BrowserUser); err == nil { + uid, _ := strconv.Atoi(u.Uid) + if err := dialAsUID(uint32(uid), listener.Addr().String()); err == nil { + t.Fatal("browser user (page JS) reached a browser port") + } + } + }) + } + + // A pool UID that binds a browser port (e.g. x11vnc's, after it died) + // must not be able to answer the browser user's websockify, or root. + t.Run("pool listener can't serve", func(t *testing.T) { + addr := fmt.Sprintf("127.0.0.1:%d", browser.PortRangeStart+11) + stop := listenAsUID(t, 2000, addr) + defer stop() + if err := dialAsUID(0, addr); err == nil { + t.Fatal("pool UID's listener on a browser port answered") + } + }) +} + +// listenAsUID runs a listener on addr as uid (python3) until stop is called. +func listenAsUID(t *testing.T, uid uint32, addr string) (stop func()) { + t.Helper() + host, port, _ := net.SplitHostPort(addr) + cmd := exec.Command("python3", "-c", `import socket,sys,time +s=socket.socket(); s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) +s.bind((sys.argv[1], int(sys.argv[2]))); s.listen(8) +print("ok", flush=True) +while True: + c,_=s.accept(); c.close()`, host, port) + cmd.SysProcAttr = &syscall.SysProcAttr{Credential: &syscall.Credential{Uid: uid, Gid: uid}} + out, _ := cmd.StdoutPipe() + if err := cmd.Start(); err != nil { + t.Skipf("python3: %v", err) + } + buf := make([]byte, 3) + if _, err := out.Read(buf); err != nil { + _ = cmd.Process.Kill() + t.Fatalf("listener as uid %d failed to start", uid) + } + return func() { _ = cmd.Process.Kill(); _ = cmd.Wait() } +} diff --git a/sandbox/cmd/server/egress_other.go b/sandbox/cmd/server/egress_other.go index 7e50745f..f063faeb 100644 --- a/sandbox/cmd/server/egress_other.go +++ b/sandbox/cmd/server/egress_other.go @@ -1,6 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: egress-kernel-v11-browser-user-port-block + //go:build !linux package main @@ -13,3 +15,12 @@ func setupEgressIptables() error { return nil } // setupPoolMCPIptables is a no-op on non-Linux platforms. func setupPoolMCPIptables(_ string) error { return nil } + +// setupPoolBrowserPortBlock is a no-op on non-Linux platforms. +func setupPoolBrowserPortBlock() error { return nil } + +// setupPoolEgressDenyIptables is a no-op on non-Linux platforms. +func setupPoolEgressDenyIptables() error { return nil } + +// installBrowserUserPortBlock is a no-op on non-Linux platforms. +func installBrowserUserPortBlock() error { return nil } diff --git a/sandbox/cmd/server/env.go b/sandbox/cmd/server/env.go index dc9b543c..95484692 100644 --- a/sandbox/cmd/server/env.go +++ b/sandbox/cmd/server/env.go @@ -1,16 +1,19 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: env-v3-unset-broker-base-urls package main import ( "encoding/json" "fmt" + "log" "net/http" "os" "path/filepath" "regexp" "strings" + "time" "github.com/Hyper-Int/OrcaBot/sandbox/internal/broker" "github.com/Hyper-Int/OrcaBot/sandbox/internal/id" @@ -45,6 +48,12 @@ type envUpdateResponse struct { var envNamePattern = regexp.MustCompile(`^[A-Z_][A-Z0-9_]*$`) +const envRevision = "env-v3-unset-broker-base-urls" + +func init() { + log.Printf("[env] REVISION: %s loaded at %s", envRevision, time.Now().Format(time.RFC3339)) +} + func (s *Server) handleSessionEnv(w http.ResponseWriter, r *http.Request) { session := s.getSessiоnOrErrоr(w, r.PathValue("sessionId")) if session == nil { @@ -57,7 +66,8 @@ func (s *Server) handleSessionEnv(w http.ResponseWriter, r *http.Request) { return } - if len(req.Set) == 0 && len(req.Secrets) == 0 && len(req.Unset) == 0 { + // An approvals-only update is valid: it replaces the session's approvals. + if len(req.Set) == 0 && len(req.Secrets) == 0 && len(req.Unset) == 0 && req.ApprovedDomains == nil { http.Error(w, "E79745: No environment variables provided", http.StatusBadRequest) return } @@ -191,7 +201,7 @@ func (s *Server) handleSessionEnv(w http.ResponseWriter, r *http.Request) { // Write to .env file root := session.Wоrkspace().Root() - if err := updateDоtEnv(filepath.Join(root, ".env"), effectiveEnvVars, req.Unset); err != nil { + if err := updateDоtEnv(filepath.Join(root, ".env"), effectiveEnvVars, withBrokerEnvKeys(req.Unset, effectiveEnvVars)); err != nil { http.Error(w, "E79747: Failed to update .env", http.StatusInternalServerError) return } @@ -207,6 +217,33 @@ func (s *Server) handleSessionEnv(w http.ResponseWriter, r *http.Request) { json.NewEncoder(w).Encode(envUpdateResponse{Status: "ok"}) } +// withBrokerEnvKeys adds, for each unset built-in key, the base-URL variable +// that pointed its harness at the broker (e.g. ANTHROPIC_BASE_URL). Left +// behind, new PTYs would send requests to a broker route that no longer +// exists. A base URL this same request sets (another secret, or the user's own +// env var) is kept. +func withBrokerEnvKeys(unset []string, set map[string]string) []string { + out := append([]string(nil), unset...) + seen := map[string]bool{} + for _, name := range unset { + seen[name] = true + } + for _, name := range unset { + for _, spec := range broker.GetAllProvidersByEnvKey(name) { + key := spec.BrokerEnvKey + if key == "" || seen[key] { + continue + } + if _, kept := set[key]; kept { + continue + } + seen[key] = true + out = append(out, key) + } + } + return out +} + // envQuote wraps a value in double quotes for .env file, escaping special chars. func envQuote(value string) string { // Escape backslashes, double quotes, dollar signs, and backticks diff --git a/sandbox/cmd/server/env_unset_test.go b/sandbox/cmd/server/env_unset_test.go new file mode 100644 index 00000000..03d32205 --- /dev/null +++ b/sandbox/cmd/server/env_unset_test.go @@ -0,0 +1,51 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: env-v3-unset-broker-base-urls +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestUnsetBuiltinKeyDropsBrokerBaseURL(t *testing.T) { + path := filepath.Join(t.TempDir(), ".env") + os.WriteFile(path, []byte(strings.Join([]string{ + `export ANTHROPIC_API_KEY="dummy"`, + `export ANTHROPIC_BASE_URL="http://127.0.0.1:8082/broker/s1/anthropic"`, + `export OPENAI_API_KEY="dummy"`, + `export OPENAI_BASE_URL="http://127.0.0.1:8082/broker/s1/openai"`, + `export MY_KEY="dummy"`, + `export MY_KEY_BROKER="http://127.0.0.1:8082/broker/s1/custom/MY_KEY"`, + }, "\n")), 0o644) + + // ANTHROPIC and the custom key are removed; OPENAI is still applied. + set := map[string]string{"OPENAI_API_KEY": "dummy", "OPENAI_BASE_URL": "http://127.0.0.1:8082/broker/s1/openai"} + unset := withBrokerEnvKeys([]string{"ANTHROPIC_API_KEY", "MY_KEY", "MY_KEY_BROKER"}, set) + if err := updateDоtEnv(path, set, unset); err != nil { + t.Fatal(err) + } + out, _ := os.ReadFile(path) + got := string(out) + for _, gone := range []string{"ANTHROPIC_API_KEY", "ANTHROPIC_BASE_URL", "MY_KEY"} { + if strings.Contains(got, gone) { + t.Errorf("%s left in .env:\n%s", gone, got) + } + } + if !strings.Contains(got, "OPENAI_BASE_URL") { + t.Errorf("OPENAI_BASE_URL removed:\n%s", got) + } +} + +func TestUnsetKeepsBaseURLSetInSameRequest(t *testing.T) { + // The user's own ANTHROPIC_BASE_URL env var, sent alongside the unset. + unset := withBrokerEnvKeys([]string{"ANTHROPIC_API_KEY"}, map[string]string{"ANTHROPIC_BASE_URL": "https://proxy.example"}) + for _, k := range unset { + if k == "ANTHROPIC_BASE_URL" { + t.Fatal("base URL set in the same request was unset") + } + } +} diff --git a/sandbox/cmd/server/main.go b/sandbox/cmd/server/main.go index 25f125f6..8e907026 100644 --- a/sandbox/cmd/server/main.go +++ b/sandbox/cmd/server/main.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: main-v40-import-gzip-bomb-cap +// REVISION: main-v49-shm-rmid-forced package main @@ -31,6 +31,8 @@ import ( "github.com/Hyper-Int/OrcaBot/sandbox/internal/auth" "github.com/Hyper-Int/OrcaBot/sandbox/internal/broker" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" "github.com/Hyper-Int/OrcaBot/sandbox/internal/debug" "github.com/Hyper-Int/OrcaBot/sandbox/internal/drive" "github.com/Hyper-Int/OrcaBot/sandbox/internal/egress" @@ -40,7 +42,7 @@ import ( "github.com/Hyper-Int/OrcaBot/sandbox/internal/ws" ) -const mainRevision = "main-v40-import-gzip-bomb-cap" +const mainRevision = "main-v49-shm-rmid-forced" // debugExecToken is a random per-boot secret guarding POST /debug/exec. Generated // once at startup when ORCABOT_DEBUG_EXEC=1 and delivered to the host ONLY via the @@ -105,7 +107,7 @@ func init() { // token is checked first thing in the handler, constant-time. func (s *Server) handleDebugExec(w http.ResponseWriter, r *http.Request) { if os.Getenv("ORCABOT_DEBUG_EXEC") != "1" { - http.Error(w, "debug exec disabled (set ORCABOT_DEBUG_EXEC=1)", http.StatusForbidden) + http.Error(w, "E79113: debug exec disabled (set ORCABOT_DEBUG_EXEC=1)", http.StatusForbidden) return } // Auth via the random per-boot token (X-Debug-Exec-Token), NOT the internal @@ -114,7 +116,7 @@ func (s *Server) handleDebugExec(w http.ResponseWriter, r *http.Request) { // agent can neither read nor guess it. Constant-time compare; empty = reject. if debugExecToken == "" || subtle.ConstantTimeCompare([]byte(r.Header.Get("X-Debug-Exec-Token")), []byte(debugExecToken)) != 1 { - http.Error(w, "forbidden: missing or invalid X-Debug-Exec-Token", http.StatusForbidden) + http.Error(w, "E79114: forbidden: missing or invalid X-Debug-Exec-Token", http.StatusForbidden) return } var req struct { @@ -122,7 +124,7 @@ func (s *Server) handleDebugExec(w http.ResponseWriter, r *http.Request) { TimeoutMs int `json:"timeout_ms"` } if err := json.NewDecoder(r.Body).Decode(&req); err != nil || strings.TrimSpace(req.Cmd) == "" { - http.Error(w, "bad request: need JSON {\"cmd\": \"...\"}", http.StatusBadRequest) + http.Error(w, "E79115: bad request: need JSON {\"cmd\": \"...\"}", http.StatusBadRequest) return } timeout := 15 * time.Second @@ -258,9 +260,18 @@ func main() { if egressGlobalEnabled { sessionManager.SetEgressProxyPort(egress.DefaultPort) } + // The proxy, transparent proxy and broker run as root: they must refuse to + // relay to loopback services pool UIDs are blocked from (MCPLocal here; the + // browser port range is built in). REVISION: main-v46-egress-fail-closed + if n, err := strconv.Atoi(mcpLocalPort); err == nil { + egress.ProtectLoopbackPort(n) + } if err := egressProxy.Start(); err != nil { _ = os.Unsetenv("ORCABOT_EGRESS_PROXY_URL") log.Printf("[egress-proxy] WARNING: Failed to start egress proxy: %v (PTY processes will not have HTTP_PROXY)", err) + if egressGlobalEnabled { + refusePTYSpawns(fmt.Sprintf("egress proxy failed to start: %v", err)) + } } else { proxyURL := "http://127.0.0.1:8083" if egressGlobalEnabled { @@ -279,71 +290,49 @@ func main() { // Kernel-level egress enforcement: iptables UID-range rules + transparent proxy. // Requires CAP_NET_ADMIN (privileged container) and EGRESS_PROXY_ENABLED=true. // - // IMPORTANT: InitPool is intentionally deferred until AFTER both iptables and the - // transparent listener are confirmed working. If either fails, the pool is never - // initialized and PTYs fall back to root-UID execution with HTTP_PROXY enforcement - // (bypassable but not broken). Initializing the pool before enforcement is ready - // would create pool-UID PTYs whose traffic is not intercepted — false security. - // REVISION: main-v27-pool-proxy-token-restore + // IMPORTANT: InitPool is intentionally deferred until AFTER every enforcement + // layer is confirmed working: initializing the pool earlier would create + // pool-UID PTYs whose traffic is not intercepted — false security. + // + // FAIL CLOSED: if any step fails, PTY creation is refused (E79770) rather + // than falling back to root PTYs, which would run outside every kernel rule + // (egress default-deny, MCPLocal and browser-port blocks). /health stays up + // and reports the reason so the failure is diagnosable. + // REVISION: main-v46-egress-fail-closed if egressGlobalEnabled && checkNetAdmin() { - // Step 1: Install iptables NAT REDIRECT rules. - if err := setupEgressIptables(); err != nil { - log.Printf("[egress-kernel] ERROR: iptables setup failed — kernel enforcement unavailable. "+ - "PTYs will run without UID isolation (HTTP_PROXY enforcement only): %v", err) - } else if err := egressProxy.StartTransparent(); err != nil { - // Step 2: Start transparent proxy listener. - // iptables rules are installed but the listener is absent — redirected - // connections to port %d would fail closed (connection refused), which - // breaks PTY egress. Log the error; PTYs run as root (rules don't match). - log.Printf("[egress-kernel] ERROR: transparent proxy failed — kernel enforcement unavailable. "+ - "PTYs will run without UID isolation (HTTP_PROXY enforcement only): %v", err) - } else { - // Both enforcement layers confirmed. Check sandbox group before pool activation. - sandboxGID := pty.LookupSandboxGID() - if sandboxGID == 0 { - log.Printf("[egress-kernel] ERROR: 'sandbox' group not found — kernel enforcement unavailable. " + - "PTYs will run without UID isolation. Ensure the 'sandbox' group exists in the image.") - } else { - // Startup order is critical for client fallback correctness. - // REVISION: main-v29-socket-init-order - // - // mcp-bridge and xdg-open detect pool mode by testing whether the socket - // file exists (os.Stat / test -S). The invariant they rely on: - // - // socket file exists → pool is initialised (GetPool() != nil) - // - // To guarantee this we must not create the socket file until ALL - // preconditions are satisfied. Order: - // 1. iptables — no disk side-effect; safe to try first - // 2. socket — creates the file; only reached if iptables succeeded - // 3. InitPool — only reached if socket is ready - // - // If iptables fails → socket never created → clients fall back to TCP ✓ - // If socket fails → socket file absent → clients fall back to TCP ✓ - // If all succeed → socket exists AND pool is active ✓ - // - // The HTTP servers have not started yet at this point, so the brief - // window between socket creation (step 2) and InitPool (step 3) is safe: - // no PTY-creation requests can arrive until ListenAndServe is called. - if err := setupPoolMCPIptables(mcpLocalPort); err != nil { - log.Printf("[pool-iptables] ERROR: Failed to block pool UIDs from MCPLocal port — kernel enforcement unavailable: %v", err) - } else { - privSock := NewPrivilegedSocket(server) - if err := privSock.Start(); err != nil { - log.Printf("[privileged-socket] ERROR: Failed to start — kernel enforcement unavailable. "+ - "PTYs will run without UID isolation: %v", err) - } else { - pty.InitPool(sandboxGID) - log.Printf("[egress-kernel] Kernel enforcement active: uid-owner 2000-2099 → transparent proxy port %d, MCPLocal port %s blocked", egress.TransparentPort, mcpLocalPort) - } - } - } + if err := setupKernelEgress(server, egressProxy, mcpLocalPort); err != nil { + refusePTYSpawns(err.Error()) } } else if egressGlobalEnabled { - log.Printf("[egress-kernel] Skipping kernel enforcement: CAP_NET_ADMIN not available (not a privileged container)") + // Without CAP_NET_ADMIN no kernel layer can be installed: PTYs would + // run as root behind HTTP_PROXY only, which any process can ignore. + // Fail closed like every other kernel-layer failure. + // REVISION: main-v47-no-netadmin-fail-closed + refusePTYSpawns("CAP_NET_ADMIN not available (not a privileged container): kernel egress enforcement cannot be installed") } } + // Free the shared memory of an x11vnc that dies uncleanly, or enough + // browser restarts exhaust kernel.shmmni and the browser never starts + // again. REVISION: main-v49-shm-rmid-forced + browser.EnableOrphanShmCleanup() + + // Chromium's user (page JS, browser_evaluate) must not reach any browser + // port: on desktop every dashboard's websockify/x11vnc/DevTools listens in + // one VM. setupKernelEgress installs this too; it is installed here as well + // so it holds with egress off (the threat is a hostile page, not only an + // agent). Idempotent. REVISION: main-v47-no-netadmin-fail-closed + if checkNetAdmin() { + if err := installBrowserUserPortBlock(); err != nil { + log.Printf("[pool-iptables] WARNING: browser-user port block failed: %v", err) + if egressGlobalEnabled { + refusePTYSpawns(fmt.Sprintf("browser-user port block failed: %v", err)) + } + } + } else { + log.Printf("[pool-iptables] WARNING: CAP_NET_ADMIN not available — Chromium pages can reach other dashboards' browser ports") + } + httpServer := &http.Server{ Addr: ":" + port, Handler: server.Handler(), @@ -443,7 +432,7 @@ type Server struct { func NewServer(sm *sessions.Manager) *Server { authMiddleware := auth.NewMiddleware() if !authMiddleware.IsEnabled() { - log.Println("WARNING: SANDBOX_INTERNAL_TOKEN not set - authentication is disabled, all requests will be rejected") + log.Println("WARNING: neither SANDBOX_AUTH_PUBLIC_KEY nor SANDBOX_INTERNAL_TOKEN set - all requests will be rejected") } return &Server{ sessions: sm, @@ -590,7 +579,7 @@ func (s *Server) requireLocalMCPAuth(next http.HandlerFunc) http.HandlerFunc { func (s *Server) poolModeGuard(next http.HandlerFunc) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { if pty.GetPool() != nil { - http.Error(w, "use Unix socket in pool mode", http.StatusServiceUnavailable) + http.Error(w, "E79116: use Unix socket in pool mode", http.StatusServiceUnavailable) return } next(w, r) @@ -699,10 +688,78 @@ func (s *Server) Handler() http.Handler { return mux } +// refusePTYSpawns makes every PTY spawn fail with E79770 and logs loudly. +// REVISION: main-v46-egress-fail-closed +func refusePTYSpawns(reason string) { + pty.RefuseSpawns(reason) + for i := 0; i < 3; i++ { + log.Printf("[egress-kernel] !!! FAIL CLOSED: EGRESS_PROXY_ENABLED=true but egress isolation is unavailable — ALL PTY CREATION IS REFUSED (E79770): %s", reason) + } +} + +// setupKernelEgress installs every kernel egress layer and activates the UID +// pool. It returns the first failure; the pool is only initialised when all +// layers succeeded. +// +// Startup order is critical for client fallback correctness. +// REVISION: main-v29-socket-init-order +// +// mcp-bridge and xdg-open detect pool mode by testing whether the socket +// file exists (os.Stat / test -S). The invariant they rely on: +// +// socket file exists → pool is initialised (GetPool() != nil) +// +// To guarantee this we must not create the socket file until ALL +// preconditions are satisfied. Order: +// 1. iptables — no disk side-effect; safe to try first +// 2. socket — creates the file; only reached if iptables succeeded +// 3. InitPool — only reached if socket is ready +// +// The HTTP servers have not started yet at this point, so the brief +// window between socket creation (step 2) and InitPool (step 3) is safe: +// no PTY-creation requests can arrive until ListenAndServe is called. +func setupKernelEgress(server *Server, egressProxy *egress.EgressProxy, mcpLocalPort string) error { + // Step 1: Install iptables NAT REDIRECT rules. + if err := setupEgressIptables(); err != nil { + return fmt.Errorf("iptables NAT setup failed: %w", err) + } + // Step 2: Start transparent proxy listener (redirected conns need it). + if err := egressProxy.StartTransparent(); err != nil { + return fmt.Errorf("transparent proxy failed: %w", err) + } + sandboxGID := pty.LookupSandboxGID() + if sandboxGID == 0 { + return fmt.Errorf("'sandbox' group not found — ensure it exists in the image") + } + if err := setupPoolMCPIptables(mcpLocalPort); err != nil { + return fmt.Errorf("failed to block pool UIDs from MCPLocal port: %w", err) + } + if err := setupPoolBrowserPortBlock(); err != nil { + return fmt.Errorf("failed to block pool UIDs from browser ports: %w", err) + } + if err := setupPoolEgressDenyIptables(); err != nil { + // Without default-deny, pool UIDs could bypass the proxy on non-80/443 ports. + return fmt.Errorf("pool egress default-deny failed: %w", err) + } + privSock := NewPrivilegedSocket(server) + if err := privSock.Start(); err != nil { + return fmt.Errorf("privileged socket failed to start: %w", err) + } + pty.InitPool(sandboxGID) + log.Printf("[egress-kernel] Kernel enforcement active: uid-owner 2000-2099 → transparent proxy port %d, MCPLocal port %s blocked", egress.TransparentPort, mcpLocalPort) + return nil +} + func (s *Server) handleHеalth(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusOK) - fmt.Fprintf(w, `{"status":"ok","revision":"%s"}`, mainRevision) + // Stays 200 when PTY spawns are refused (fail-closed egress): the server is up + // and this is where the reason is surfaced. REVISION: main-v46-egress-fail-closed + resp := map[string]any{"status": "ok", "revision": mainRevision} + if reason := pty.SpawnRefusal(); reason != "" { + resp["pty_spawn_refused"] = "E79770: " + reason + } + _ = json.NewEncoder(w).Encode(resp) } // handleListSessions returns active session IDs (for agent discovery on localhost). @@ -739,15 +796,17 @@ func (s *Server) handleCreateSessiоn(w http.ResponseWriter, r *http.Request) { req.MCPToken = "" } } - if err := s.ensureEgressAllowlistLoaded(req.DashboardID); err != nil { - log.Printf("[egress-proxy] Allowlist hydration skipped for dashboard %q: %v", req.DashboardID, err) - } - + // Creating the session registers its dashboard-scoped token for this VM's + // /internal/* calls, so it must precede the allowlist fetch. + // REVISION: main-v45-session-owned-token session, err := s.sessions.Create(req.DashboardID, req.MCPToken) if err != nil { http.Error(w, "E79706: "+err.Error(), http.StatusInternalServerError) return } + if err := s.ensureEgressAllowlistLoaded(req.DashboardID); err != nil { + log.Printf("[egress-proxy] Allowlist hydration skipped for dashboard %q: %v", req.DashboardID, err) + } w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusCreated) json.NewEncoder(w).Encode(map[string]string{ @@ -776,9 +835,8 @@ func (s *Server) ensureEgressAllowlistLoaded(dashboardID string) error { s.egressAllowlistMu.Unlock() controlplaneURL := strings.TrimSuffix(os.Getenv("CONTROLPLANE_URL"), "/") - internalToken := strings.TrimSpace(os.Getenv("INTERNAL_API_TOKEN")) - if controlplaneURL == "" || internalToken == "" { - return errors.New("missing CONTROLPLANE_URL or INTERNAL_API_TOKEN") + if controlplaneURL == "" { + return errors.New("missing CONTROLPLANE_URL") } url := controlplaneURL + "/internal/dashboards/" + dashboardID + "/egress/allowlist" @@ -786,7 +844,9 @@ func (s *Server) ensureEgressAllowlistLoaded(dashboardID string) error { if err != nil { return err } - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, dashboardID); err != nil { + return err + } client := &http.Client{Timeout: 10 * time.Second} resp, err := client.Do(req) @@ -796,7 +856,7 @@ func (s *Server) ensureEgressAllowlistLoaded(dashboardID string) error { defer resp.Body.Close() if resp.StatusCode != http.StatusOK { body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096)) - return errors.New("allowlist fetch failed: " + resp.Status + " " + string(body) + " token=" + tokenFingerprint(internalToken)) + return errors.New("allowlist fetch failed: " + resp.Status + " " + string(body) + " auth=" + crеdentialKind(req)) } var payload struct { @@ -834,16 +894,16 @@ func (s *Server) ensureEgressAllowlistLoaded(dashboardID string) error { return nil } -func tokenFingerprint(token string) string { - token = strings.TrimSpace(token) - if token == "" { - return "(none)" - } - n := len(token) - if n <= 8 { - return fmt.Sprintf("%s(len=%d)", token, n) +// credentialKind names which control plane credential a request carries, for logs. +func crеdentialKind(req *http.Request) string { + switch { + case req.Header.Get("X-Dashboard-Token") != "": + return "dashboard-token" + case req.Header.Get("X-Internal-Token") != "": + return "internal-token" + default: + return "none" } - return fmt.Sprintf("%s...%s(len=%d)", token[:4], token[n-4:], n) } func (s *Server) handleDeleteSessiоn(w http.ResponseWriter, r *http.Request) { @@ -878,6 +938,11 @@ func (s *Server) handleListPTYs(w http.ResponseWriter, r *http.Request) { // REVISION: working-dir-v2-handler-logging func (s *Server) handleCreatePTY(w http.ResponseWriter, r *http.Request) { + // Fail closed: egress isolation failed at startup. REVISION: main-v46-egress-fail-closed + if reason := pty.SpawnRefusal(); reason != "" { + http.Error(w, pty.ErrSpawnRefused.Error()+": "+reason, http.StatusServiceUnavailable) + return + } session := s.getSessiоnOrErrоr(w, r.PathValue("sessionId")) if session == nil { return @@ -891,6 +956,7 @@ func (s *Server) handleCreatePTY(w http.ResponseWriter, r *http.Request) { IntegrationToken string `json:"integration_token"` // JWT for policy gateway auth WorkingDir string `json:"working_dir"` // Relative path within workspace ExecutionID string `json:"execution_id"` // Schedule execution tracking ID + MCPToken string `json:"mcp_token"` // Fresh dashboard token (replaces a dropped one) ModelSelection *struct { Provider string `json:"provider"` Model string `json:"model"` @@ -904,6 +970,13 @@ func (s *Server) handleCreatePTY(w http.ResponseWriter, r *http.Request) { if r.Body != nil { json.NewDecoder(r.Body).Decode(&req) // Ignore errors - all fields are optional } + // A fresh dashboard token rides along with every PTY creation so a VM whose + // token cpauth dropped (E79393/E79395/E79398) gets a working one back. Bound + // to the session's own dashboard, never one named by the request. + // REVISION: main-v46-egress-fail-closed + if req.MCPToken != "" { + session.SetDashboardToken(req.MCPToken) + } // Use CreatePTYWithOptions when any optional field is present var ptyInfo *sessions.PTYInfo @@ -940,6 +1013,10 @@ func (s *Server) handleCreatePTY(w http.ResponseWriter, r *http.Request) { http.Error(w, "E79755: "+err.Error(), http.StatusBadRequest) return } + if errors.Is(err, pty.ErrSpawnRefused) { + http.Error(w, err.Error(), http.StatusServiceUnavailable) + return + } http.Error(w, "E79708: "+err.Error(), http.StatusInternalServerError) return } @@ -955,7 +1032,7 @@ func (s *Server) handleCreatePTY(w http.ResponseWriter, r *http.Request) { // Use captured execID, not a live lookup, to avoid the reuse problem if eid := session.GetExecutionID(ptyID); eid == execID { session.SetExecutionID(ptyID, "") // Clear to prevent duplicate from agent-stopped - go s.notifyExecutionPtyCompleted(execID, ptyID, "process_exit", "") + go s.notifyExecutionPtyCompleted(session.DashboardID, execID, ptyID, "process_exit", "") } }) } @@ -1534,12 +1611,11 @@ func (s *Server) handleToolsChanged(w http.ResponseWriter, r *http.Request) { // notifyExecutionPtyCompleted calls the control plane to report that a schedule-triggered PTY has completed. // Called asynchronously (goroutine) when agent_stopped fires for a PTY with execution context. // REVISION: server-side-cron-v1-agent-stopped-callback -func (s *Server) notifyExecutionPtyCompleted(executionId, ptyId, reason, lastMessage string) { +func (s *Server) notifyExecutionPtyCompleted(dashboardID, executionId, ptyId, reason, lastMessage string) { controlplaneURL := os.Getenv("CONTROLPLANE_URL") - internalToken := os.Getenv("INTERNAL_API_TOKEN") - if controlplaneURL == "" || internalToken == "" { - log.Printf("[execution-callback] Cannot notify control plane: missing CONTROLPLANE_URL or INTERNAL_API_TOKEN") + if controlplaneURL == "" { + log.Printf("[execution-callback] Cannot notify control plane: missing CONTROLPLANE_URL") return } @@ -1572,7 +1648,10 @@ func (s *Server) notifyExecutionPtyCompleted(executionId, ptyId, reason, lastMes return } req.Header.Set("Content-Type", "application/json") - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, dashboardID); err != nil { + log.Printf("[execution-callback] Cannot notify control plane for execution %s: %v", executionId, err) + return + } resp, err := http.DefaultClient.Do(req) if err != nil { @@ -1722,7 +1801,7 @@ func (s *Server) handleAgentStoppedForPTY(w http.ResponseWriter, session *sessio if execId := session.GetExecutionID(ptyID); execId != "" { session.SetExecutionID(ptyID, "") - go s.notifyExecutionPtyCompleted(execId, ptyID, req.Reason, req.LastMessage) + go s.notifyExecutionPtyCompleted(session.DashboardID, execId, ptyID, req.Reason, req.LastMessage) } w.WriteHeader(http.StatusNoContent) @@ -1827,8 +1906,7 @@ func (s *Server) forwardEgressAudit(event egress.AuditEvent) { } controlplaneURL := strings.TrimSuffix(os.Getenv("CONTROLPLANE_URL"), "/") - internalToken := strings.TrimSpace(os.Getenv("INTERNAL_API_TOKEN")) - if controlplaneURL == "" || internalToken == "" { + if controlplaneURL == "" { return } @@ -1849,7 +1927,9 @@ func (s *Server) forwardEgressAudit(event egress.AuditEvent) { return } req.Header.Set("Content-Type", "application/json") - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, dashboardID); err != nil { + return + } resp, err := http.DefaultClient.Do(req) if err != nil { diff --git a/sandbox/cmd/server/mcp.go b/sandbox/cmd/server/mcp.go index a58c8bf0..7193b6af 100644 --- a/sandbox/cmd/server/mcp.go +++ b/sandbox/cmd/server/mcp.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: mcp-v31-direct-dispatch +// REVISION: mcp-v33-error-codes package main @@ -23,7 +23,7 @@ import ( ) func init() { - log.Printf("[mcp-server] REVISION: mcp-v31-direct-dispatch loaded at %s", time.Now().Format(time.RFC3339)) + log.Printf("[mcp-server] REVISION: mcp-v33-error-codes loaded at %s", time.Now().Format(time.RFC3339)) } // batchIntegrationCache holds a dashboard-level cache of all terminal integrations. @@ -267,8 +267,8 @@ func (s *Server) handleMCPListToolsForPTY(w http.ResponseWriter, r *http.Request targetURL := fmt.Sprintf("%s/internal/mcp/ui/tools", controlplaneURL) req, err := http.NewRequestWithContext(r.Context(), "GET", targetURL, nil) if err == nil { - if session.MCPToken != "" { - req.Header.Set("X-Dashboard-Token", session.MCPToken) + if token := session.DashboardToken(); token != "" { + req.Header.Set("X-Dashboard-Token", token) } else if internalToken := os.Getenv("INTERNAL_API_TOKEN"); internalToken != "" { req.Header.Set("X-Internal-Token", internalToken) } @@ -385,7 +385,7 @@ func (s *Server) handleMCPCallToolForPTY(w http.ResponseWriter, r *http.Request, json.NewEncoder(w).Encode(map[string]interface{}{ "error": true, "code": resp.Error, - "message": fmt.Sprintf("Browser policy error (%s): %s", resp.Error, resp.Reason), + "message": fmt.Sprintf("E79128: Browser policy error (%s): %s", resp.Error, resp.Reason), }) return } @@ -394,7 +394,7 @@ func (s *Server) handleMCPCallToolForPTY(w http.ResponseWriter, r *http.Request, w.WriteHeader(http.StatusForbidden) json.NewEncoder(w).Encode(map[string]interface{}{ "error": true, - "message": fmt.Sprintf("Browser policy denied: %s", resp.Reason), + "message": fmt.Sprintf("E79129: Browser policy denied: %s", resp.Reason), "decision": resp.Decision, "policyId": resp.PolicyID, }) @@ -457,8 +457,8 @@ func (s *Server) handleMCPCallToolForPTY(w http.ResponseWriter, r *http.Request, http.Error(w, "E79817: "+err.Error(), http.StatusInternalServerError) return } - if session.MCPToken != "" { - req.Header.Set("X-Dashboard-Token", session.MCPToken) + if token := session.DashboardToken(); token != "" { + req.Header.Set("X-Dashboard-Token", token) } else if internalToken := os.Getenv("INTERNAL_API_TOKEN"); internalToken != "" { req.Header.Set("X-Internal-Token", internalToken) } else { @@ -511,8 +511,8 @@ func (s *Server) handleMCPListItemsForPTY(w http.ResponseWriter, r *http.Request http.Error(w, "E79819: "+err.Error(), http.StatusInternalServerError) return } - if session.MCPToken != "" { - req.Header.Set("X-Dashboard-Token", session.MCPToken) + if token := session.DashboardToken(); token != "" { + req.Header.Set("X-Dashboard-Token", token) } else if internalToken := os.Getenv("INTERNAL_API_TOKEN"); internalToken != "" { req.Header.Set("X-Internal-Token", internalToken) } else { @@ -698,8 +698,8 @@ func (s *Server) handleAgentStateToolCallForPTY(w http.ResponseWriter, r *http.R } if authToken == "" { - if session.MCPToken != "" { - authToken = session.MCPToken + if token := session.DashboardToken(); token != "" { + authToken = token authType = mcp.AuthTypeDashboardToken if sessionScoped, ok := args["sessionScoped"].(bool); ok && sessionScoped { log.Printf("[mcp] WARNING: sessionScoped=true requested but no PTY token proof; creating dashboard-wide instead") diff --git a/sandbox/cmd/server/privileged_socket_linux.go b/sandbox/cmd/server/privileged_socket_linux.go index a8072436..3a62637b 100644 --- a/sandbox/cmd/server/privileged_socket_linux.go +++ b/sandbox/cmd/server/privileged_socket_linux.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: privileged-socket-v11-cleanup-on-start-err +// REVISION: privileged-socket-v12-error-codes //go:build linux @@ -22,7 +22,7 @@ import ( "github.com/Hyper-Int/OrcaBot/sandbox/internal/pty" ) -const privilegedSocketRevision = "privileged-socket-v11-cleanup-on-start-err" +const privilegedSocketRevision = "privileged-socket-v12-error-codes" func init() { log.Printf("[privileged-socket] REVISION: %s loaded at %s", privilegedSocketRevision, time.Now().Format(time.RFC3339)) @@ -171,21 +171,21 @@ func (ps *PrivilegedSocket) handleConn(conn net.Conn) { pool := pty.GetPool() if pool == nil { - writeHTTP403(conn, "pool not initialised") + writeHTTP403(conn, "E79125: pool not initialised") return } entry, ok := pool.Lookup(uid) if !ok { log.Printf("[privileged-socket] Rejected: uid %d not in pool registry", uid) - writeHTTP403(conn, "uid not in pool") + writeHTTP403(conn, "E79126: uid not in pool") return } // Get the session to serve requests for this PTY. session, err := ps.server.sessions.Get(entry.SessionID) if err != nil { - writeHTTP403(conn, "session not found") + writeHTTP403(conn, "E79127: session not found") return } @@ -198,7 +198,7 @@ func (ps *PrivilegedSocket) handleConn(conn net.Conn) { mux.HandleFunc("/anthropic-key", func(w http.ResponseWriter, r *http.Request) { key := session.Broker().GetAnthropicKey(entry.SessionID) if key == "" { - http.Error(w, "no anthropic key", http.StatusNotFound) + http.Error(w, "E79110: no anthropic key", http.StatusNotFound) return } w.Header().Set("Content-Type", "application/json") @@ -218,7 +218,7 @@ func (ps *PrivilegedSocket) handleConn(conn net.Conn) { case "items": ps.server.handleMCPListItemsForPTY(w, r, session) default: - http.Error(w, "not found", http.StatusNotFound) + http.Error(w, "E79111: not found", http.StatusNotFound) } }) @@ -255,7 +255,7 @@ func (ps *PrivilegedSocket) handleConn(conn net.Conn) { var bodyMap map[string]interface{} if r.Body != nil { if err := json.NewDecoder(r.Body).Decode(&bodyMap); err != nil { - http.Error(w, "invalid JSON body", http.StatusBadRequest) + http.Error(w, "E79112: invalid JSON body", http.StatusBadRequest) return } } diff --git a/sandbox/docker/Dockerfile b/sandbox/docker/Dockerfile index 896bfd1a..fc80859d 100644 --- a/sandbox/docker/Dockerfile +++ b/sandbox/docker/Dockerfile @@ -19,6 +19,10 @@ COPY . . RUN CGO_ENABLED=0 GOOS=linux go build -o /build/orcabot-server ./cmd/server RUN CGO_ENABLED=0 GOOS=linux go build -o /build/browser-ctl ./cmd/browser-ctl RUN CGO_ENABLED=0 GOOS=linux go build -o /build/mcp-bridge ./cmd/mcp-bridge +# REVISION: guest-timesync-v1-vsock-step +# Desktop VM only: rc.local starts it (root) so the host can step the guest clock +# over vsock after host sleep. Fly never starts it. +RUN CGO_ENABLED=0 GOOS=linux go build -o /build/orcabot-guest-timesync ./cmd/guest-timesync # ============================================================================= # Stage 2: Sandbox runtime image @@ -214,6 +218,27 @@ RUN for i in $(seq 0 99); do \ $(printf 'pty-%03d' $i); \ done +# Dedicated unprivileged user for the dashboard browser stack (Xvfb, x11vnc, +# websockify, Chromium) — see internal/browser. Outside the pool UID range and +# NOT in the sandbox group, so agents can't write its profile or home, and +# iptables can block pool UIDs from its ports (6100-6299). Never run it as root: +# root Chromium + an open DevTools port was a root escalation for any agent. +RUN groupadd --system --gid 1990 orcabot-browser \ + && useradd --system --uid 1990 --gid 1990 --create-home \ + --home-dir /home/orcabot-browser --shell /usr/sbin/nologin orcabot-browser \ + && chmod 700 /home/orcabot-browser \ + && mkdir -p /tmp/.X11-unix && chmod 1777 /tmp/.X11-unix + +# REVISION: browser-vnc-user-v1-split +# The display side of the browser stack (Xvfb, x11vnc, websockify) runs as a +# second user, so Chromium (orcabot-browser) runs none of it: iptables rejects +# every connection from orcabot-browser to 6100-6299, so a page in one +# dashboard can't open another dashboard's noVNC/x11vnc/DevTools. +RUN groupadd --system --gid 1991 orcabot-vnc \ + && useradd --system --uid 1991 --gid 1991 --create-home \ + --home-dir /home/orcabot-vnc --shell /usr/sbin/nologin orcabot-vnc \ + && chmod 700 /home/orcabot-vnc + # Set umask 0002 for all PTY shell sessions. # Without this, files created by pty-000 land at 0644/0755 (umask 0022) so # pty-001 can read them but not modify them — breaking the shared-workspace model. @@ -238,6 +263,7 @@ ENV PATH="/home/sandbox/.local/bin:${PATH}" COPY --from=builder /build/orcabot-server /usr/local/bin/orcabot-server COPY --from=builder /build/browser-ctl /usr/local/bin/browser-ctl COPY --from=builder /build/mcp-bridge /usr/local/bin/mcp-bridge +COPY --from=builder /build/orcabot-guest-timesync /usr/local/bin/orcabot-guest-timesync # uv (fast Python package manager) — baked in so benchmark harnesses # (e.g. slop-code-bench `uv sync`) can build a venv without a network install. @@ -260,6 +286,15 @@ RUN echo "chromium-refresh ${CHROMIUM_CACHEBUST}" \ && rm -rf /var/lib/apt/lists/* \ && (chromium.real --version 2>/dev/null || chromium --version) +# REVISION: chromium-policies-v1-no-component-downloads +# Managed Chromium policies (root-owned, so agents can't change them). Chromium 155 +# still fetches the on-device AI "Model Manifest" component from edgedl.me.gvt1.com +# at startup despite --disable-component-update, which surfaced as an unexplained +# egress approval prompt; only ComponentUpdatesEnabled=false stops it (verified with +# a net-log probe). GenAILocalFoundationalModelSettings=1 also keeps a page calling +# the on-device AI APIs from triggering the multi-GB model download. +COPY docker/chromium-policies.json /etc/chromium/policies/managed/orcabot.json + # Working directory WORKDIR /workspace diff --git a/sandbox/docker/chromium-policies.json b/sandbox/docker/chromium-policies.json new file mode 100644 index 00000000..a70d2f35 --- /dev/null +++ b/sandbox/docker/chromium-policies.json @@ -0,0 +1,4 @@ +{ + "ComponentUpdatesEnabled": false, + "GenAILocalFoundationalModelSettings": 1 +} diff --git a/sandbox/internal/auth/auth.go b/sandbox/internal/auth/auth.go index 717497ef..f3d6bb0f 100644 --- a/sandbox/internal/auth/auth.go +++ b/sandbox/internal/auth/auth.go @@ -4,14 +4,33 @@ package auth import ( + "crypto/ed25519" + "crypto/subtle" + "encoding/base64" + "encoding/json" "log" "net/http" "os" "strings" + "time" ) -// REVISION: auth-v2-debug-logging -const authRevision = "auth-v2-debug-logging" +// REVISION: auth-v5-error-codes +const authRevision = "auth-v5-error-codes" + +// SignedRequestHeader carries the control plane's per-machine Ed25519 signature. +const SignedRequestHeader = "X-Sandbox-Auth" + +// signingContext must match controlplane/src/sandbox/request-signing.ts. +const signingContext = "orcabot-sandbox-auth-v1." + +// maxClockSkew tolerates small clock differences between control plane and VM. +// A signature is accepted until exp+maxClockSkew, so the replay window is the +// control plane's TTL (30s) plus this: 60s in all. +const maxClockSkew = 30 * time.Second + +// maxSignatureTTL caps the lifetime a signature may claim for itself. +const maxSignatureTTL = 60 * time.Second func init() { log.Printf("[auth] REVISION: %s loaded", authRevision) @@ -19,22 +38,106 @@ func init() { // Middleware provides authentication middleware for HTTP handlers type Middleware struct { - token string + token string // legacy static SANDBOX_INTERNAL_TOKEN (shared by every VM) + publicKey ed25519.PublicKey // SANDBOX_AUTH_PUBLIC_KEY: verifies control plane signatures + machineID string // this machine; signed requests must name it + now func() time.Time } -// NewMiddleware creates a new auth middleware -// Token is read from SANDBOX_INTERNAL_TOKEN environment variable +// NewMiddleware creates a new auth middleware from the environment: +// - SANDBOX_AUTH_PUBLIC_KEY (base64url Ed25519): accept requests the control plane +// signed for this machine (FLY_MACHINE_ID). A VM holding only the public key +// cannot authenticate to any other machine. +// - SANDBOX_INTERNAL_TOKEN: legacy static token, accepted while configured. +// +// With neither configured, every request is rejected (fail-closed). func NewMiddleware() *Middleware { - token := os.Getenv("SANDBOX_INTERNAL_TOKEN") - if token == "" { - log.Printf("[auth] WARNING: SANDBOX_INTERNAL_TOKEN is empty — all requests will be rejected (fail-closed)") + m := &Middleware{ + token: os.Getenv("SANDBOX_INTERNAL_TOKEN"), + machineID: machineIDFrоmEnv(), + now: time.Now, + } + if raw := strings.TrimSpace(os.Getenv("SANDBOX_AUTH_PUBLIC_KEY")); raw != "" { + key, err := base64.RawURLEncoding.DecodeString(strings.TrimRight(raw, "=")) + if err != nil || len(key) != ed25519.PublicKeySize { + log.Printf("[auth] ERROR: SANDBOX_AUTH_PUBLIC_KEY is invalid (err=%v len=%d) — signed requests will be rejected", err, len(key)) + } else { + m.publicKey = ed25519.PublicKey(key) + log.Printf("[auth] signed requests enabled for machine %q", m.machineID) + } + } + if m.token == "" { + log.Printf("[auth] SANDBOX_INTERNAL_TOKEN not set — static token auth disabled") } else { log.Printf("[auth] SANDBOX_INTERNAL_TOKEN configured (len=%d, first4=%q, last4=%q)", - len(token), safePrefix(token, 4), safeSuffix(token, 4)) + len(m.token), safePrefix(m.token, 4), safeSuffix(m.token, 4)) } - return &Middleware{ - token: token, + if !m.IsEnabled() { + log.Printf("[auth] WARNING: no SANDBOX_AUTH_PUBLIC_KEY or SANDBOX_INTERNAL_TOKEN — all requests will be rejected (fail-closed)") } + return m +} + +// machineIDFromEnv mirrors sandboxMachineID() in cmd/server. +func machineIDFrоmEnv() string { + if id := os.Getenv("FLY_MACHINE_ID"); id != "" { + return id + } + return os.Getenv("FLY_ALLOC_ID") +} + +type signedClaims struct { + MachineID string `json:"mid"` + Method string `json:"m"` + Path string `json:"p"` + IssuedAt int64 `json:"iat"` + ExpiresAt int64 `json:"exp"` +} + +// verifySigned checks an X-Sandbox-Auth value: a valid control plane signature, +// unexpired, and addressed to this machine, method and request target. +func (m *Middleware) verifySigned(r *http.Request, value string) (bool, string) { + payloadB64, sigB64, ok := strings.Cut(value, ".") + if !ok { + return false, "malformed" + } + sig, err := base64.RawURLEncoding.DecodeString(sigB64) + if err != nil || !ed25519.Verify(m.publicKey, []byte(signingContext+payloadB64), sig) { + return false, "bad signature" + } + payload, err := base64.RawURLEncoding.DecodeString(payloadB64) + if err != nil { + return false, "malformed payload" + } + var c signedClaims + if err := json.Unmarshal(payload, &c); err != nil { + return false, "malformed payload" + } + now := m.now() + if c.ExpiresAt <= c.IssuedAt || time.Duration(c.ExpiresAt-c.IssuedAt)*time.Second > maxSignatureTTL { + return false, "bad lifetime" + } + if now.After(time.Unix(c.ExpiresAt, 0).Add(maxClockSkew)) { + return false, "expired" + } + if time.Unix(c.IssuedAt, 0).After(now.Add(maxClockSkew)) { + return false, "issued in the future" + } + if c.MachineID != m.machineID { + return false, "signed for machine " + c.MachineID + ", this is " + m.machineID + } + if c.Method != r.Method { + return false, "method mismatch" + } + // The raw request target exactly as sent (signed as URL.pathname + URL.search). + target := r.RequestURI + if target == "" { + target = r.URL.RequestURI() + } + if c.Path != target { + return false, "path mismatch" + } + return true, "" } // RequireAuth wraps an http.Handler and requires valid authentication @@ -61,15 +164,30 @@ func (m *Middleware) RequireAuthFunc(next http.HandlerFunc) http.HandlerFunc { // isAuthenticated checks if the request has valid authentication func (m *Middleware) isAuthenticated(r *http.Request) bool { - // If no token is configured, reject all requests (fail secure) + // If nothing is configured, reject all requests (fail secure) + if !m.IsEnabled() { + log.Printf("[auth] REJECT %s %s — no credentials configured (fail-closed)", r.Method, r.URL.Path) + return false + } + + // Per-machine signature from the control plane + if signed := r.Header.Get(SignedRequestHeader); signed != "" && m.publicKey != nil { + ok, reason := m.verifySigned(r, signed) + if ok { + return true + } + log.Printf("[auth] REJECT %s %s — signed request: %s", r.Method, r.URL.Path, reason) + return false + } + if m.token == "" { - log.Printf("[auth] REJECT %s %s — no token configured (fail-closed)", r.Method, r.URL.Path) + log.Printf("[auth] REJECT %s %s — no valid %s (static token auth disabled)", r.Method, r.URL.Path, SignedRequestHeader) return false } // Check X-Internal-Token header first (for internal service-to-service calls) if token := r.Header.Get("X-Internal-Token"); token != "" { - if token == m.token { + if cоnstantTimeEqual(token, m.token) { return true } log.Printf("[auth] REJECT %s %s — X-Internal-Token mismatch (got len=%d first4=%q last4=%q, want len=%d first4=%q last4=%q)", @@ -93,7 +211,7 @@ func (m *Middleware) isAuthenticated(r *http.Request) bool { return false } - if parts[1] == m.token { + if cоnstantTimeEqual(parts[1], m.token) { return true } log.Printf("[auth] REJECT %s %s — Bearer token mismatch (got len=%d, want len=%d)", @@ -103,7 +221,11 @@ func (m *Middleware) isAuthenticated(r *http.Request) bool { // IsEnabled returns true if authentication is configured func (m *Middleware) IsEnabled() bool { - return m.token != "" + return m.token != "" || m.publicKey != nil +} + +func cоnstantTimeEqual(a, b string) bool { + return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1 } func safePrefix(s string, n int) string { diff --git a/sandbox/internal/auth/auth_test.go b/sandbox/internal/auth/auth_test.go new file mode 100644 index 00000000..59bc990a --- /dev/null +++ b/sandbox/internal/auth/auth_test.go @@ -0,0 +1,180 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: auth-v4-60s-replay-window +package auth + +import ( + "crypto/ed25519" + "crypto/rand" + "encoding/base64" + "encoding/json" + "net/http/httptest" + "testing" + "time" +) + +// sign mirrors controlplane/src/sandbox/request-signing.ts. +func sign(t *testing.T, priv ed25519.PrivateKey, c signedClaims) string { + t.Helper() + payload, err := json.Marshal(c) + if err != nil { + t.Fatal(err) + } + p := base64.RawURLEncoding.EncodeToString(payload) + sig := ed25519.Sign(priv, []byte(signingContext+p)) + return p + "." + base64.RawURLEncoding.EncodeToString(sig) +} + +func newSignedMiddleware(t *testing.T, staticToken string) (*Middleware, ed25519.PrivateKey) { + t.Helper() + pub, priv, err := ed25519.GenerateKey(rand.Reader) + if err != nil { + t.Fatal(err) + } + t.Setenv("SANDBOX_AUTH_PUBLIC_KEY", base64.RawURLEncoding.EncodeToString(pub)) + t.Setenv("SANDBOX_INTERNAL_TOKEN", staticToken) + t.Setenv("FLY_MACHINE_ID", "machine-b") + t.Setenv("FLY_ALLOC_ID", "") + return NewMiddleware(), priv +} + +func validClaims(method, path string) signedClaims { + now := time.Now().Unix() + return signedClaims{MachineID: "machine-b", Method: method, Path: path, IssuedAt: now, ExpiresAt: now + 30} +} + +func TestSignedRequestAccepted(t *testing.T) { + m, priv := newSignedMiddleware(t, "") + r := httptest.NewRequest("POST", "/sessions/s1/ptys/p1/write?x=a%2Bb", nil) + r.Header.Set(SignedRequestHeader, sign(t, priv, validClaims("POST", "/sessions/s1/ptys/p1/write?x=a%2Bb"))) + if !m.isAuthenticated(r) { + t.Fatal("valid signed request rejected") + } +} + +func TestSignedRequestRejections(t *testing.T) { + m, priv := newSignedMiddleware(t, "") + _, otherPriv, _ := ed25519.GenerateKey(rand.Reader) + now := time.Now().Unix() + + cases := []struct { + name string + method string + path string + token func() string + }{ + {"signed for another machine", "GET", "/sessions/s1/ptys", func() string { + c := validClaims("GET", "/sessions/s1/ptys") + c.MachineID = "machine-a" + return sign(t, priv, c) + }}, + {"no machine id while this VM has one", "GET", "/sessions/s1/ptys", func() string { + c := validClaims("GET", "/sessions/s1/ptys") + c.MachineID = "" + return sign(t, priv, c) + }}, + {"method swapped", "DELETE", "/sessions/s1", func() string { + return sign(t, priv, validClaims("GET", "/sessions/s1")) + }}, + {"path swapped", "POST", "/sessions/s2/ptys/p9/write", func() string { + return sign(t, priv, validClaims("POST", "/sessions/s1/ptys/p1/write")) + }}, + {"query swapped", "GET", "/sessions/s1/file?path=/etc/shadow", func() string { + return sign(t, priv, validClaims("GET", "/sessions/s1/file?path=notes.txt")) + }}, + {"expired", "GET", "/sessions/s1/ptys", func() string { + c := validClaims("GET", "/sessions/s1/ptys") + c.IssuedAt, c.ExpiresAt = now-600, now-300 + return sign(t, priv, c) + }}, + {"replayed 61s after signing", "GET", "/sessions/s1/ptys", func() string { + c := validClaims("GET", "/sessions/s1/ptys") + c.IssuedAt, c.ExpiresAt = now-61, now-31 + return sign(t, priv, c) + }}, + {"claims a lifetime over 60s", "GET", "/sessions/s1/ptys", func() string { + c := validClaims("GET", "/sessions/s1/ptys") + c.ExpiresAt = now + 3600 + return sign(t, priv, c) + }}, + {"issued in the future", "GET", "/sessions/s1/ptys", func() string { + c := validClaims("GET", "/sessions/s1/ptys") + c.IssuedAt, c.ExpiresAt = now+600, now+660 + return sign(t, priv, c) + }}, + {"signed by a different key", "GET", "/sessions/s1/ptys", func() string { + return sign(t, otherPriv, validClaims("GET", "/sessions/s1/ptys")) + }}, + {"garbage", "GET", "/sessions/s1/ptys", func() string { return "not-a-token" }}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + r := httptest.NewRequest(tc.method, tc.path, nil) + r.Header.Set(SignedRequestHeader, tc.token()) + if m.isAuthenticated(r) { + t.Fatal("request accepted, want rejection") + } + }) + } +} + +// Once SANDBOX_INTERNAL_TOKEN is withheld from VMs, a token copied from some +// older VM must not work. +func TestStaticTokenRejectedWhenNotConfigured(t *testing.T) { + m, _ := newSignedMiddleware(t, "") + r := httptest.NewRequest("GET", "/sessions/s1/ptys", nil) + r.Header.Set("X-Internal-Token", "old-shared-token") + if m.isAuthenticated(r) { + t.Fatal("static token accepted with static auth disabled") + } +} + +func TestStaticTokenStillAcceptedDuringMigration(t *testing.T) { + m, _ := newSignedMiddleware(t, "shared") + r := httptest.NewRequest("GET", "/sessions/s1/ptys", nil) + r.Header.Set("X-Internal-Token", "shared") + if !m.isAuthenticated(r) { + t.Fatal("static token rejected while configured") + } + r = httptest.NewRequest("GET", "/sessions/s1/ptys", nil) + r.Header.Set("Authorization", "Bearer shared") + if !m.isAuthenticated(r) { + t.Fatal("static bearer rejected while configured") + } +} + +// An invalid signature must not fall through to the static token. +func TestBadSignatureDoesNotFallBackToStatic(t *testing.T) { + m, _ := newSignedMiddleware(t, "shared") + r := httptest.NewRequest("GET", "/sessions/s1/ptys", nil) + r.Header.Set(SignedRequestHeader, "bogus.sig") + r.Header.Set("X-Internal-Token", "shared") + if m.isAuthenticated(r) { + t.Fatal("bad signature fell back to static token") + } +} + +func TestNothingConfiguredFailsClosed(t *testing.T) { + t.Setenv("SANDBOX_AUTH_PUBLIC_KEY", "") + t.Setenv("SANDBOX_INTERNAL_TOKEN", "") + m := NewMiddleware() + r := httptest.NewRequest("GET", "/sessions", nil) + r.Header.Set("X-Internal-Token", "") + if m.IsEnabled() || m.isAuthenticated(r) { + t.Fatal("auth should be disabled and reject everything") + } +} + +// Desktop / local docker: no FLY_MACHINE_ID, control plane signs with mid "". +func TestSignedRequestWithoutFlyMachine(t *testing.T) { + m, priv := newSignedMiddleware(t, "") + m.machineID = "" + c := validClaims("GET", "/sessions/s1/ptys") + c.MachineID = "" + r := httptest.NewRequest("GET", "/sessions/s1/ptys", nil) + r.Header.Set(SignedRequestHeader, sign(t, priv, c)) + if !m.isAuthenticated(r) { + t.Fatal("signed request without machine id rejected on a machine without one") + } +} diff --git a/sandbox/internal/broker/secrets_broker.go b/sandbox/internal/broker/secrets_broker.go index 5395acb3..972bbb47 100644 --- a/sandbox/internal/broker/secrets_broker.go +++ b/sandbox/internal/broker/secrets_broker.go @@ -15,15 +15,26 @@ import ( "strings" "sync" "time" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/egress" ) -// REVISION: broker-v7-evict-logged-routes -const brokerRevision = "broker-v7-evict-logged-routes" +// REVISION: broker-v9-error-codes +const brokerRevision = "broker-v9-error-codes" func init() { log.Printf("[broker] REVISION: %s loaded at %s", brokerRevision, time.Now().Format(time.RFC3339)) } +// brokerTransport is DefaultTransport with a guarded dial: the broker runs as +// root, so a custom target or approved domain resolving to loopback must not +// reach a service pool UIDs are blocked from (browser ports, MCPLocal). +var brokerTransport = func() *http.Transport { + t := http.DefaultTransport.(*http.Transport).Clone() + t.DialContext = egress.GuardеdDialContеxt(30 * time.Second) + return t +}() + // ProviderConfig holds runtime configuration for a specific provider. type ProviderConfig struct { Name string // Provider name (e.g., "anthropic") or "custom/SECRET_NAME" @@ -367,7 +378,7 @@ func (b *SecretsBroker) ServeHTTP(w http.ResponseWriter, r *http.Request) { "error": "domain_approval_required", "domain": domain, "secret": secretName, - "message": "This domain requires owner approval before the secret can be sent.", + "message": "E79130: This domain requires owner approval before the secret can be sent.", }) // Notify owner asynchronously with the session ID from the config if b.onApprovalNeeded != nil && config.SessionID != "" { @@ -475,7 +486,8 @@ func (b *SecretsBroker) ServeHTTP(w http.ResponseWriter, r *http.Request) { // Forward request with timeout client := &http.Client{ - Timeout: 120 * time.Second, + Transport: brokerTransport, + Timeout: 120 * time.Second, CheckRedirect: func(req *http.Request, via []*http.Request) error { // Prevent redirects to different domains for security if len(via) > 0 { diff --git a/sandbox/internal/browser/browser.go b/sandbox/internal/browser/browser.go index 20f0e4f4..e071a30c 100644 --- a/sandbox/internal/browser/browser.go +++ b/sandbox/internal/browser/browser.go @@ -1,16 +1,18 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: browser-v12-process-group-kill +// REVISION: browser-v22-graceful-stop package browser import ( + "bufio" "bytes" + "context" "encoding/json" + "errors" "fmt" "io" "log" - "net" "net/http" "net/url" "os" @@ -19,13 +21,12 @@ import ( "strconv" "strings" "sync" - "sync/atomic" "syscall" "time" ) -// killProcessTree kills a browser process and the process group it leads, then -// reaps it. +// killProcessTree stops a browser process and the process group it leads, +// then reaps it. // // Each browser process is started with Setpgid so it leads its own group. // Killing only the direct process leaves Chromium's children (zygote, @@ -36,17 +37,60 @@ import ( // process stays a zombie holding its PID. Grandchildren are reaped by tini // (see the Dockerfile ENTRYPOINT), which cannot be done from here. func killProcessTree(cmd *exec.Cmd) { - if cmd == nil || cmd.Process == nil { - return + stоpProcessGroups(cmd) +} + +// processStopGrace is how long stоpProcessGroups waits after SIGTERM before SIGKILL. +const processStopGrace = 2 * time.Second + +// stоpProcessGroups sends SIGTERM to each process group, waits up to +// processStopGrace for the leaders to exit, SIGKILLs every group, then reaps +// the leaders. +// +// SIGTERM first because x11vnc frees its SysV shared-memory framebuffers only +// on a clean exit. Under SIGKILL the segments outlive it (several MB each): +// after a few hundred browser starts the VM hits kernel.shmmni (4096) and +// every later x11vnc fails ("shmget: No space left on device"), so the browser +// never starts again until the VM restarts. kernel.shm_rmid_forced +// (EnableOrphanShmCleanup) is the backstop for processes that die uncleanly. +// +// Leaders are not reaped until after the SIGKILL, so the pid each group is +// signalled by cannot have been reused. +func stоpProcessGroups(cmds ...*exec.Cmd) { + var live []*exec.Cmd + for _, cmd := range cmds { + if cmd == nil || cmd.Process == nil { + continue + } + live = append(live, cmd) + if err := syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM); err != nil { + _ = cmd.Process.Signal(syscall.SIGTERM) + } + } + deadline := time.Now().Add(processStopGrace) + for time.Now().Before(deadline) { + running := false + for _, cmd := range live { + if processRunning(cmd) { + running = true + break + } + } + if !running { + break + } + time.Sleep(20 * time.Millisecond) + } + for _, cmd := range live { + // Best-effort: the group may already be gone, and children that + // called setsid() escape it (tini still reaps those once they exit). + if err := syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL); err != nil { + _ = cmd.Process.Kill() + } } - pid := cmd.Process.Pid - // Negative pid = the process group led by pid. Best-effort: the group may - // already be gone, and children that called setsid() escape it (tini still - // reaps those once they exit). - if err := syscall.Kill(-pid, syscall.SIGKILL); err != nil { - _ = cmd.Process.Kill() + for _, cmd := range live { + _, _ = cmd.Process.Wait() } - _, _ = cmd.Process.Wait() } type Status struct { @@ -57,30 +101,73 @@ type Status struct { DebugPort int `json:"debug_port"` } +// Controller is one session's handle on the browser for its workspace. +// Every controller for the same workspace root shares one browserStack, so +// two sessions on one workspace (several sessions in one web dashboard VM, or +// a desktop dashboard reopened while its old session lives) drive ONE +// Chromium instead of each launching their own on the same profile, which +// corrupted it. The stack stops when its last user stops. type Controller struct { - mu sync.Mutex workspace string egressProxyPort int // when >0, Chromium is started with --proxy-server pointing here - display int - wsPort int - vncPort int - debugPort int - running bool - ready bool - processes []*exec.Cmd + *browserStack } -// REVISION: browser-v12-process-group-kill -const browserRevision = "browser-v12-process-group-kill" +// browserStack is the browser (Xvfb, x11vnc, websockify, Chromium) for one +// workspace root, and so for one profile. +type browserStack struct { + mu sync.Mutex + display int + wsPort int + vncPort int + debugPort int + xauth string // the display's cookie file (VNCUser's), removed on stop + chromeXauth string // Chromium's copy of the cookie (BrowserUser's), removed on stop + running bool + ready bool + processes []*exec.Cmd + legacyCleanup sync.Once + // users are the controllers that started (or opened a URL in) the + // running browser; Stop tears it down only when the last one leaves. + users map[*Controller]struct{} +} + +var ( + stacksMu sync.Mutex + stacks = map[string]*browserStack{} +) + +// stackFor returns the shared stack for a workspace root. Keyed the same way +// as profileName, so one stack per profile. +func stackFor(workspace string) *browserStack { + key := filepath.Clean(workspace) + stacksMu.Lock() + defer stacksMu.Unlock() + st, ok := stacks[key] + if !ok { + st = &browserStack{} + stacks[key] = st + } + return st +} + +// attachLocked records c as a user of the running stack. Needs c.mu. +func (c *Controller) attachLocked() { + if c.users == nil { + c.users = make(map[*Controller]struct{}) + } + c.users[c] = struct{}{} +} + +// REVISION: browser-v20-vnc-user-split +const browserRevision = "browser-v22-graceful-stop" func init() { log.Printf("[browser] REVISION: %s loaded at %s", browserRevision, time.Now().Format(time.RFC3339)) } -var displayCounter uint32 = 90 - func NewController(workspace string) *Controller { - return &Controller{workspace: workspace} + return &Controller{workspace: workspace, browserStack: stackFor(workspace)} } // NewControllerWithEgress creates a Controller that routes all Chromium network @@ -90,13 +177,15 @@ func NewController(workspace string) *Controller { // Loopback addresses bypass the proxy so the CDP debug connection is unaffected. // REVISION: browser-v10-no-sandbox-warning func NewControllerWithEgress(workspace string, egressProxyPort int) *Controller { - return &Controller{workspace: workspace, egressProxyPort: egressProxyPort} + return &Controller{workspace: workspace, egressProxyPort: egressProxyPort, browserStack: stackFor(workspace)} } func (c *Controller) Start() (Status, error) { c.mu.Lock() defer c.mu.Unlock() + // Another session on this workspace may already run the browser: share it. + c.attachLocked() if c.running { return c.statusLocked(), nil } @@ -117,47 +206,114 @@ func (c *Controller) Start() (Status, error) { return Status{}, err } - display := int(atomic.AddUint32(&displayCounter, 1)) - wsPort, err := freePort() + // Chromium runs as the unprivileged browser user, Xvfb / x11vnc / + // websockify as the VNC user (isolation.go): iptables keeps the browser + // user (and so every page it loads) off the whole browser port range. + identity, err := lookupBrowserIdentity() if err != nil { return Status{}, err } - vncPort, err := freePort() + vncIdentity, err := lookupVNCIdentity() if err != nil { return Status{}, err } - debugPort, err := freePort() - if err != nil { + + // The profile and X cookies live in these users' homes; refuse to start + // if anyone but that user and root could reach them. + if err := checkBrowserHome(identity); err != nil { + return Status{}, err + } + if err := checkBrowserHome(vncIdentity); err != nil { return Status{}, err } - userDataDir := filepath.Join(c.workspace, ".browser") - if err := os.MkdirAll(userDataDir, 0o755); err != nil { + // Reserved process-wide until teardown (or failure below), so no other + // controller's Chromium can be started on them meanwhile. + wsPort, vncPort, debugPort, err := rеserveBrowserPorts() + if err != nil { return Status{}, err } + started := false + defer func() { + if !started { + releasePorts(wsPort, vncPort, debugPort) + } + }() + + // The profile lives outside /workspace (an agent could rename a profile + // there and swap in its own directory while the browser runs). Make it a + // browser-owned 0700 dir and clear stale locks / crash state (no + // concurrent writer in this process: every controller for this workspace + // shares this stack and Start holds its lock; a live Chromium from + // elsewhere holding the lock fails the start instead). + // One profile per workspace: on desktop each dashboard has its own. + userDataDir, err := prepareProfile(identity, c.workspace) + if err != nil { + return Status{}, err + } + c.legacyCleanup.Do(func() { + workspace := c.workspace + go func() { + if err := removeLegacyProfile(workspace, identity); err != nil { + log.Printf("[browser] legacy profile cleanup: %v", err) + } + // The briefly shared profile mixes every dashboard's state. Only + // root and the browser user can write the home, and RemoveAll + // unlinks symlinks rather than following them. + if err := os.RemoveAll(filepath.Join(identity.home, sharedProfileDirName)); err != nil { + log.Printf("[browser] shared profile cleanup: %v", err) + } + }() + }) - // Clean stale Chromium profile lock files left by previous container runs. - // These persist on the Docker volume and cause "profile in use" errors. - for _, lockFile := range []string{"SingletonLock", "SingletonSocket", "SingletonCookie"} { - _ = os.Remove(filepath.Join(userDataDir, lockFile)) + if err := os.MkdirAll("/tmp/.X11-unix", 0o777); err == nil { + _ = os.Chmod("/tmp/.X11-unix", 0o777|os.ModeSticky) } - // Clean crash state so Chromium won't show "Restore pages?" bubble. - // Set exit_type to Normal and exited_cleanly to true in Preferences. - // Safe to call here: Chromium is not yet launched (Start holds c.mu and - // launches chromiumCmd below), so there is no concurrent writer. - cleanCrashState(userDataDir) + // Xvfb picks a free display itself (-displayfd), so an agent squatting + // /tmp/.X11-unix/X can't make it fail or hand our clients its server. + cookie, err := nеwXCookie() + if err != nil { + return Status{}, err + } + xvfbCmd, display, xauth, err := startXvfbWithCookie(vncIdentity, cookie, xvfbReadyTimeout) + if err != nil { + return Status{}, err + } + if testHookXvfbStarted != nil { + testHookXvfbStarted(xvfbCmd) + } + // Chromium's own copy of the cookie, in its own home: VNCUser's 0700 home + // is closed to it. + chromeXauth, err := xauthorityPath(identity) + if err == nil { + err = writeXauthority(chromeXauth, display, cookie, identity) + } + if err != nil { + killProcessTree(xvfbCmd) + _ = os.Remove(xauth) + return Status{}, fmt.Errorf("browser X auth: %w", err) + } displayVar := fmt.Sprintf(":%d", display) - env := append(os.Environ(), "DISPLAY="+displayVar) - - xvfbCmd := exec.Command("Xvfb", displayVar, "-screen", "0", "1280x720x24", "-nolisten", "tcp") - xvfbCmd.Env = env + env := vncIdentity.env(displayVar, xauth) + chromeEnv := identity.env(displayVar, chromeXauth) vncCmd := exec.Command( "x11vnc", "-display", displayVar, + "-auth", xauth, "-rfbport", strconv.Itoa(vncPort), + // Loopback only: websockify (below) is the one client, reached through + // the server's authenticated proxy. Pool UIDs are blocked from the port + // range. websockify can't verify who it connects to, so if x11vnc dies + // and an agent binds its port, the kernel stops the agent's listener + // from answering (setupPoolBrowserPortBlock's source-port rule) and + // Status() tears the stack down on the next poll. + // + // Not -unixsock: x11vnc 0.9.16 never accepts on its unix socket in + // -threads mode (tested), and -threads is kept for frame rate. + "-localhost", "-forever", "-shared", "-nopw", @@ -176,7 +332,8 @@ func (c *Controller) Start() (Status, error) { webArgs := []string{ "--heartbeat", "30", - strconv.Itoa(wsPort), + // Loopback only; the server proxies authenticated clients to it. + fmt.Sprintf("127.0.0.1:%d", wsPort), // 127.0.0.1, not "localhost": the VM's /etc/hosts may lack a localhost // entry, in which case websockify's name resolution of "localhost:" // fails and it closes every VNC client with 1011 "Failed to connect to @@ -193,8 +350,9 @@ func (c *Controller) Start() (Status, error) { "--no-sandbox", // Suppress chromium's yellow "You are using an unsupported command-line // flag: --no-sandbox" infobar (--disable-infobars does NOT hide this one in - // current chromium). --no-sandbox is required here (chromium runs as root in - // the VM), so the warning is noise that just clutters the browser-block view. + // current chromium). --no-sandbox is still required in the VM (no user + // namespaces for chromium's sandbox), so Chromium runs as the unprivileged + // browser user instead of root; the warning is just noise in the block. "--test-type", "--disable-dev-shm-usage", "--no-first-run", @@ -214,9 +372,9 @@ func (c *Controller) Start() (Status, error) { "--user-data-dir=" + userDataDir, "--window-size=1280,720", } - // Route all Chromium traffic through the egress proxy when enabled. - // Chromium runs as root and is outside the iptables UID range, so without - // this every redirect, XHR, and subresource load bypasses kernel egress controls. + // Route all Chromium traffic through the egress proxy when enabled. The + // browser user is also held to the pool's default-deny egress chain + // (cmd/server/egress_linux.go), so nothing it loads can go around the proxy. // Loopback is bypassed so the CDP debug connection on 127.0.0.1 is unaffected. // REVISION: browser-v10-no-sandbox-warning if c.egressProxyPort > 0 { @@ -227,15 +385,22 @@ func (c *Controller) Start() (Status, error) { } chromiumArgs = append(chromiumArgs, "about:blank") chromiumCmd := exec.Command("chromium", chromiumArgs...) - chromiumCmd.Env = env + chromiumCmd.Env = chromeEnv processes := []*exec.Cmd{xvfbCmd, vncCmd, websockifyCmd, chromiumCmd} killAll := func() { - for _, cmd := range processes { - killProcessTree(cmd) - } + stоpProcessGroups(processes...) + _ = os.Remove(xauth) + _ = os.Remove(chromeXauth) } - for i, cmd := range processes { + for _, cmd := range processes[1:] { + // x11vnc and Chromium connect to the display by name; if Xvfb has + // died since it reported ready, whatever now owns :N would get them. + if (cmd == vncCmd || cmd == chromiumCmd) && !processRunning(xvfbCmd) { + log.Printf("browser Xvfb on display %d exited before %s started", display, cmd.Path) + killAll() + return Status{}, fmt.Errorf("browser display server exited") + } log.Printf("browser starting %s", cmd.Path) procName := filepath.Base(cmd.Path) logWriter := newProcessLogWriter(procName) @@ -244,29 +409,29 @@ func (c *Controller) Start() (Status, error) { // Lead a process group so teardown can signal the whole tree, not just // this pid — Chromium's children would otherwise survive and be // reparented to PID 1. - cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + if cmd == chromiumCmd { + cmd.SysProcAttr = identity.sysProcAttr() + } else { + cmd.SysProcAttr = vncIdentity.sysProcAttr() + } if err := cmd.Start(); err != nil { log.Printf("browser failed to start %s: %v", cmd.Path, err) killAll() return Status{}, err } - if i == 0 { - // Wait for Xvfb to be ready before starting x11vnc - time.Sleep(500 * time.Millisecond) - } } // Wait for x11vnc's VNC port — websockify proxies to it on each client connect. // Without this, the frontend gets "Connection refused" if x11vnc is slow to start. - if !waitForPort(vncPort, 10*time.Second) { + if !waitForPort(DialVNC, vncPort, 10*time.Second) { log.Printf("browser x11vnc port %d did not open", vncPort) killAll() return Status{}, fmt.Errorf("browser vnc server failed to start") } - if !waitForPort(wsPort, 20*time.Second) { + if !waitForPort(DialVNC, wsPort, 20*time.Second) { log.Printf("browser websockify port %d did not open (retrying)", wsPort) - if !waitForPort(wsPort, 10*time.Second) { + if !waitForPort(DialVNC, wsPort, 10*time.Second) { log.Printf("browser websockify port %d did not open", wsPort) killAll() return Status{}, fmt.Errorf("browser proxy failed to start") @@ -277,8 +442,11 @@ func (c *Controller) Start() (Status, error) { c.wsPort = wsPort c.vncPort = vncPort c.debugPort = debugPort + c.xauth = xauth + c.chromeXauth = chromeXauth c.processes = processes c.running = true + started = true spawnElapsed := time.Since(startTime) readyStart := time.Now() // Short readiness budget under the lock — warm/fast browsers report ready right @@ -288,8 +456,8 @@ func (c *Controller) Start() (Status, error) { // Perf: spawn = time to launch all 4 processes; debugReady = time waiting for // chromium's DevTools endpoint to answer; total = full cold bring-up. - log.Printf("[browser][perf] started display=%d wsPort=%d vncPort=%d spawn=%dms debugReady=%dms total=%dms ready=%t", - display, wsPort, vncPort, + log.Printf("[browser][perf] started display=%d wsPort=%d vncPort=%d debugPort=%d spawn=%dms debugReady=%dms total=%dms ready=%t", + display, wsPort, vncPort, debugPort, spawnElapsed.Milliseconds(), time.Since(readyStart).Milliseconds(), time.Since(startTime).Milliseconds(), c.ready) return c.statusLocked(), nil } @@ -364,25 +532,70 @@ func shouldSuppressProcessLine(proc, line string) bool { return false } +// Stop detaches this controller from the browser and stops it unless another +// session on the same workspace still uses it. func (c *Controller) Stop() { c.mu.Lock() defer c.mu.Unlock() + delete(c.users, c) if !c.running { return } - - for i := len(c.processes) - 1; i >= 0; i-- { - killProcessTree(c.processes[i]) + if n := len(c.users); n > 0 { + log.Printf("[browser] stop: browser still used by %d other session(s) on this workspace; leaving it running", n) + return } + c.teardownLocked() +} +// teardownLocked kills every browser process and clears all state, ports +// included: a stopped or dead browser must not keep reporting the ports it +// had, which an agent may have bound since. +func (c *Controller) teardownLocked() { + // Stop every group together so the grace period is paid once, not per + // process. REVISION: browser-v22-graceful-stop + stоpProcessGroups(c.processes...) c.processes = nil c.running = false c.ready = false + releasePorts(c.wsPort, c.vncPort, c.debugPort) + c.wsPort = 0 + c.vncPort = 0 + c.debugPort = 0 + c.display = 0 + if c.xauth != "" { + _ = os.Remove(c.xauth) + c.xauth = "" + } + if c.chromeXauth != "" { + _ = os.Remove(c.chromeXauth) + c.chromeXauth = "" + } +} + +// deadProcessLocked returns the name of a browser process that has exited, +// or "" if all are alive. x11vnc, Xvfb or Chromium can die while websockify +// keeps listening, so the websockify port check alone would keep reporting a +// broken stack as running — and, with x11vnc gone, keep proxying viewers to +// whatever binds its port next. +func (c *Controller) deadProcessLocked() string { + for _, p := range c.processes { + if !processRunning(p) { + return filepath.Base(p.Path) + } + } + return "" } func (c *Controller) Status() Status { c.mu.Lock() + if c.running { + if dead := c.deadProcessLocked(); dead != "" { + log.Printf("[browser] %s exited unexpectedly — tearing down the browser stack", dead) + c.teardownLocked() + } + } running := c.running wsPort := c.wsPort c.mu.Unlock() @@ -390,21 +603,21 @@ func (c *Controller) Status() Status { // If we think the browser is running, verify websockify is still listening. // If it's gone the processes died unexpectedly and we need to allow a restart. if running && wsPort > 0 { - conn, err := net.DialTimeout("tcp", fmt.Sprintf("127.0.0.1:%d", wsPort), 200*time.Millisecond) + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + conn, err := DialVNC(ctx, "tcp", fmt.Sprintf("127.0.0.1:%d", wsPort)) + cancel() if err == nil { conn.Close() + } else if !websockifyGone(err) { + // A slow /proc lookup or a timeout under load is not evidence the + // stack died; tearing it down would drop every viewer's session. + log.Printf("[browser] websockify port %d check inconclusive, keeping state: %v", wsPort, err) } else { c.mu.Lock() // Only clear if no concurrent Start() has already replaced the state. if c.running && c.wsPort == wsPort { log.Printf("[browser] websockify port %d gone, processes exited unexpectedly — clearing state", wsPort) - for _, p := range c.processes { - // Kill the group and reap, same as Stop(). - killProcessTree(p) - } - c.processes = nil - c.running = false - c.ready = false + c.teardownLocked() } c.mu.Unlock() } @@ -443,20 +656,31 @@ func (c *Controller) statusLocked() Status { } func (c *Controller) OpenURL(target string) error { + // Only http(s), never a browser port (ValidateNavigationURL): this also + // reaches /json/new, which loads whatever URL it is given. + if err := ValidateNavigationURL(target); err != nil { + return err + } + c.mu.Lock() if !c.running || c.debugPort == 0 { + c.mu.Unlock() return fmt.Errorf("browser not running") } - if !strings.HasPrefix(target, "http://") && !strings.HasPrefix(target, "https://") { - return fmt.Errorf("invalid url") - } - if !waitForDebugReady(c.debugPort, 10*time.Second) { + // Opening a URL makes this session a user of the shared browser too, so + // another session's Stop doesn't close it under it. + c.attachLocked() + debugPort := c.debugPort + c.mu.Unlock() + if !waitForDebugReady(debugPort, 10*time.Second) { return fmt.Errorf("debug port unavailable") } c.mu.Lock() - c.ready = true + if c.running && c.debugPort == debugPort { + c.ready = true + } c.mu.Unlock() - client := &http.Client{Timeout: 5 * time.Second} + client := newBrowserHTTPClient(5 * time.Second) // If a tab already has this URL, just activate it — don't open a duplicate. // This happens when the frontend BrowserBlock echoes the URL back after @@ -618,29 +842,47 @@ func trimForLog(value string) string { return trimmed } -func freePort() (int, error) { - listener, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - return 0, err - } - defer listener.Close() - addr := listener.Addr().(*net.TCPAddr) - return addr.Port, nil -} - -func waitForPort(port int, timeout time.Duration) bool { +// waitForPort waits until the port's expected owner (DialBrowser: Chromium, +// DialVNC: websockify / x11vnc) is accepting on port. A port served by anyone +// else fails at once: the real process can't bind it any more. +func waitForPort(dial dialFunc, port int, timeout time.Duration) bool { deadline := time.Now().Add(timeout) + addr := fmt.Sprintf("127.0.0.1:%d", port) for time.Now().Before(deadline) { - conn, err := net.DialTimeout("tcp", fmt.Sprintf("127.0.0.1:%d", port), 200*time.Millisecond) + ctx, cancel := context.WithDeadline(context.Background(), minTime(deadline, time.Now().Add(time.Second))) + conn, err := dial(ctx, "tcp", addr) + cancel() if err == nil { _ = conn.Close() return true } + if errors.Is(err, errNotBrowserSocket) { + log.Printf("[browser] SECURITY: %v", err) + return false + } time.Sleep(200 * time.Millisecond) } return false } +// websockifyGone reports whether a failed liveness dial means the browser +// stack is really gone: nothing listens on the port any more, or another user +// now does (so websockify must have exited and the port was taken). +func websockifyGone(err error) bool { + if errors.Is(err, errNotBrowserSocket) { + log.Printf("[browser] SECURITY: %v", err) + return true + } + return errors.Is(err, syscall.ECONNREFUSED) +} + +func minTime(a, b time.Time) time.Time { + if a.Before(b) { + return a + } + return b +} + // probeDebugReady does a single quick check of chromium's DevTools endpoint. // Unlike waitForDebugReady (which loops until a deadline), this returns immediately, // so it's safe to call on every Status() poll without blocking. @@ -648,86 +890,137 @@ func probeDebugReady(port int) bool { if port == 0 { return false } - client := &http.Client{Timeout: 300 * time.Millisecond} + return debugVersionOK(newBrowserHTTPClient(300*time.Millisecond), port) +} + +func debugVersionOK(client *http.Client, port int) bool { resp, err := client.Get(fmt.Sprintf("http://127.0.0.1:%d/json/version", port)) if err != nil { return false } + _, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 64<<10)) _ = resp.Body.Close() return resp.StatusCode < 300 } +// waitForDebugReady waits for DevTools to answer, within `timeout` overall. +// Each request is bounded too: Start() calls this holding c.mu, so a listener +// that accepts and never answers must not hang it. func waitForDebugReady(port int, timeout time.Duration) bool { - if !waitForPort(port, timeout) { + deadline := time.Now().Add(timeout) + if !waitForPort(DialBrowser, port, timeout) { return false } - deadline := time.Now().Add(timeout) - for time.Now().Before(deadline) { - resp, err := http.Get(fmt.Sprintf("http://127.0.0.1:%d/json/version", port)) - if err == nil { - _ = resp.Body.Close() - if resp.StatusCode < 300 { - return true - } + for { + remaining := time.Until(deadline) + if remaining <= 0 { + return false + } + if debugVersionOK(newBrowserHTTPClient(min(remaining, 2*time.Second)), port) { + return true + } + if time.Until(deadline) <= 200*time.Millisecond { + return false } time.Sleep(200 * time.Millisecond) } - return false } -func ensureBinary(name string) error { - if _, err := exec.LookPath(name); err != nil { - return fmt.Errorf("missing %s binary", name) - } - return nil -} +const xvfbReadyTimeout = 10 * time.Second -// cleanCrashState resets Chromium's crash/restore state in the profile so it -// won't show a "Restore pages?" bubble on next launch. Chromium stores this in -// Default/Preferences as exit_type and exited_cleanly. -func cleanCrashState(userDataDir string) { - prefsPath := filepath.Join(userDataDir, "Default", "Preferences") +// testHookXvfbStarted, when set by a test, runs right after Xvfb reports its +// display, so a test can kill it before x11vnc and Chromium start. +var testHookXvfbStarted func(*exec.Cmd) - info, statErr := os.Stat(prefsPath) - data, err := os.ReadFile(prefsPath) +// startXvfb starts Xvfb as id (the VNC user) with -displayfd, so the server +// chooses a free display and reports it only once it accepts connections (no +// fixed sleep, and an agent squatting /tmp/.X11-unix/X can't make it fail +// or stand in for it). The cookie exists before Xvfb starts so -auth is in +// force from the first connection; the server ignores an entry's display +// number, so the file is rewritten with the real one once Xvfb reports it. +func startXvfb(id browserIdentity, timeout time.Duration) (*exec.Cmd, int, string, error) { + cookie, err := nеwXCookie() if err != nil { - return // No preferences file yet — fresh profile + return nil, 0, "", err } + return startXvfbWithCookie(id, cookie, timeout) +} - var prefs map[string]interface{} - if err := json.Unmarshal(data, &prefs); err != nil { - log.Printf("[browser] cleanCrashState: failed to parse %s: %v", prefsPath, err) - return +// startXvfbWithCookie is startXvfb with the caller's cookie, so Start can give +// Chromium (another user) its own copy. +func startXvfbWithCookie(id browserIdentity, cookie []byte, timeout time.Duration) (*exec.Cmd, int, string, error) { + xauth, err := xauthorityPath(id) + if err != nil { + return nil, 0, "", err } - - // Set profile.exit_type = "Normal" and profile.exited_cleanly = true - profile, ok := prefs["profile"].(map[string]interface{}) - if !ok { - profile = make(map[string]interface{}) - prefs["profile"] = profile + if err := writeXauthority(xauth, 0, cookie, id); err != nil { + return nil, 0, "", fmt.Errorf("browser X auth: %w", err) } - profile["exit_type"] = "Normal" - profile["exited_cleanly"] = true - updated, err := json.Marshal(prefs) + r, w, err := os.Pipe() if err != nil { - log.Printf("[browser] cleanCrashState: failed to marshal prefs: %v", err) - return + os.Remove(xauth) + return nil, 0, "", err + } + defer r.Close() + + // ExtraFiles[0] is fd 3 in the child. + cmd := exec.Command("Xvfb", "-displayfd", "3", "-screen", "0", "1280x720x24", "-nolisten", "tcp", "-auth", xauth) + cmd.Env = id.env("", xauth) + cmd.ExtraFiles = []*os.File{w} + logWriter := newProcessLogWriter("Xvfb") + cmd.Stdout = logWriter + cmd.Stderr = logWriter + cmd.SysProcAttr = id.sysProcAttr() + log.Printf("browser starting %s", cmd.Path) + err = cmd.Start() + w.Close() // only the child holds the write end now, so its exit reads as EOF + if err != nil { + os.Remove(xauth) + return nil, 0, "", err } - // Preserve existing file permissions - perm := os.FileMode(0o644) - if statErr == nil { - perm = info.Mode().Perm() + fail := func(err error) (*exec.Cmd, int, string, error) { + killProcessTree(cmd) + os.Remove(xauth) + return nil, 0, "", err } + display, err := readDisplayFD(r, timeout) + if err != nil { + return fail(fmt.Errorf("browser display server failed to start: %w", err)) + } + if !processRunning(cmd) { + return fail(fmt.Errorf("browser display server exited after reporting display %d", display)) + } + if err := writeXauthority(xauth, display, cookie, id); err != nil { + return fail(fmt.Errorf("browser X auth: %w", err)) + } + return cmd, display, xauth, nil +} - // Atomic write: temp file + rename to avoid corruption on crash - tmpPath := prefsPath + ".tmp" - if err := os.WriteFile(tmpPath, updated, perm); err != nil { - log.Printf("[browser] cleanCrashState: failed to write %s: %v", tmpPath, err) - return +// readDisplayFD reads the newline-terminated display number Xvfb writes to its +// -displayfd once it is ready. EOF means Xvfb exited without becoming ready. +func readDisplayFD(r *os.File, timeout time.Duration) (int, error) { + if err := r.SetReadDeadline(time.Now().Add(timeout)); err != nil { + return 0, err + } + line, err := bufio.NewReader(io.LimitReader(r, 32)).ReadString('\n') + if err != nil { + if errors.Is(err, os.ErrDeadlineExceeded) { + return 0, fmt.Errorf("no display reported within %s", timeout) + } + return 0, fmt.Errorf("Xvfb exited before reporting a display (%v)", err) + } + n, err := strconv.Atoi(strings.TrimSpace(line)) + if err != nil || n < 0 { + return 0, fmt.Errorf("bad display %q from Xvfb", strings.TrimSpace(line)) } - if err := os.Rename(tmpPath, prefsPath); err != nil { - log.Printf("[browser] cleanCrashState: failed to rename %s → %s: %v", tmpPath, prefsPath, err) + return n, nil +} + +func ensureBinary(name string) error { + if _, err := exec.LookPath(name); err != nil { + return fmt.Errorf("missing %s binary", name) } + return nil } diff --git a/sandbox/internal/browser/cdp.go b/sandbox/internal/browser/cdp.go index f20dface..ffd00d56 100644 --- a/sandbox/internal/browser/cdp.go +++ b/sandbox/internal/browser/cdp.go @@ -1,17 +1,16 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: browser-cdp-v4-navigate-http-only package browser import ( "encoding/base64" "encoding/json" + "errors" "fmt" "io" - "net/http" - "os" - "path/filepath" - "strings" + "log" "sync" "sync/atomic" "time" @@ -19,6 +18,12 @@ import ( "github.com/gorilla/websocket" ) +const cdpRevision = "browser-cdp-v4-navigate-http-only" + +func init() { + log.Printf("[browser-cdp] REVISION: %s loaded at %s", cdpRevision, time.Now().Format(time.RFC3339)) +} + // CDPClient provides Chrome DevTools Protocol access to control the browser type CDPClient struct { debugPort int @@ -59,7 +64,9 @@ func (c *CDPClient) connectToTarget(preferredTargetID string) error { } // Get list of targets to find a page target - httpClient := &http.Client{Timeout: 10 * time.Second} + // Both the target list and the websocket go through DialBrowser, so a + // listener squatting the debug port never sees our commands. + httpClient := newBrowserHTTPClient(10 * time.Second) resp, err := httpClient.Get(fmt.Sprintf("http://127.0.0.1:%d/json/list", c.debugPort)) if err != nil { return fmt.Errorf("failed to list targets: %w", err) @@ -104,10 +111,7 @@ func (c *CDPClient) connectToTarget(preferredTargetID string) error { } // Connect via WebSocket - dialer := websocket.Dialer{ - HandshakeTimeout: 5 * time.Second, - } - conn, _, err := dialer.Dial(wsURL, nil) + conn, _, err := browserWSDialer().Dial(wsURL, nil) if err != nil { return fmt.Errorf("failed to connect to CDP: %w", err) } @@ -338,9 +342,9 @@ func (c *CDPClient) Type(selector string, text string) error { // Clear existing content _, _ = c.call("Input.dispatchKeyEvent", map[string]interface{}{ - "type": "keyDown", - "key": "a", - "modifiers": 2, // Ctrl + "type": "keyDown", + "key": "a", + "modifiers": 2, // Ctrl "windowsVirtualKeyCode": 65, }) _, _ = c.call("Input.dispatchKeyEvent", map[string]interface{}{ @@ -348,8 +352,8 @@ func (c *CDPClient) Type(selector string, text string) error { "key": "a", }) _, _ = c.call("Input.dispatchKeyEvent", map[string]interface{}{ - "type": "keyDown", - "key": "Backspace", + "type": "keyDown", + "key": "Backspace", "windowsVirtualKeyCode": 8, }) _, _ = c.call("Input.dispatchKeyEvent", map[string]interface{}{ @@ -488,8 +492,13 @@ func (c *CDPClient) WaitForSelector(selector string, timeout time.Duration) erro return fmt.Errorf("timeout waiting for selector: %s", selector) } -// Navigate navigates to a URL and waits for load +// Navigate navigates to a URL and waits for load. Page.navigate loads any +// scheme (file:// reads every profile in the browser user's home, chrome:// +// is browser-internal), so the URL must pass ValidateNavigationURL. func (c *CDPClient) Navigate(url string) error { + if err := ValidateNavigationURL(url); err != nil { + return err + } _, err := c.call("Page.navigate", map[string]interface{}{ "url": url, }) @@ -543,39 +552,19 @@ func (c *Controller) Screenshot(outputPath string) (string, error) { return "", err } - // Determine output path - always resolve within workspace + // Root writes this into the agent-writable workspace, so the write never + // follows a symlink at the target or any parent (writeWorkspaceFile). if outputPath == "" { - outputPath = filepath.Join(c.workspace, fmt.Sprintf("screenshot-%d.png", time.Now().Unix())) - } else if filepath.IsAbs(outputPath) { - // Reject absolute paths to prevent writes outside workspace - return "", fmt.Errorf("absolute paths are not allowed; use a relative path") - } else { - outputPath = filepath.Join(c.workspace, outputPath) - } - - // Verify resolved path is within workspace (prevent ../ traversal) - resolved, err := filepath.Abs(outputPath) - if err != nil { - return "", fmt.Errorf("failed to resolve path: %w", err) + outputPath = fmt.Sprintf("screenshot-%d.png", time.Now().Unix()) } - workspaceAbs, err := filepath.Abs(c.workspace) + path, err := writeWorkspaceFile(c.workspace, outputPath, data, 0o644) if err != nil { - return "", fmt.Errorf("failed to resolve workspace: %w", err) - } - if !strings.HasPrefix(resolved, workspaceAbs+string(filepath.Separator)) && resolved != workspaceAbs { - return "", fmt.Errorf("path escapes workspace directory") - } - - // Ensure directory exists - if err := os.MkdirAll(filepath.Dir(outputPath), 0755); err != nil { - return "", fmt.Errorf("failed to create directory: %w", err) - } - - if err := os.WriteFile(outputPath, data, 0644); err != nil { + if errors.Is(err, errSymlinkInPath) { + log.Printf("[browser] SECURITY: screenshot path %q goes through a symlink — refused", outputPath) + } return "", fmt.Errorf("failed to write screenshot: %w", err) } - - return outputPath, nil + return path, nil } // Click clicks an element @@ -698,8 +687,13 @@ func (c *Controller) WaitForSelector(selector string, timeout time.Duration) err return cdp.WaitForSelector(selector, timeout) } -// Navigate navigates to a URL using CDP (alternative to OpenURL) +// Navigate navigates to a URL using CDP (alternative to OpenURL). Agents +// reach this through MCP and the HTTP API, so the URL is checked before +// anything else (ValidateNavigationURL; CDPClient.Navigate checks again). func (c *Controller) Navigate(url string) error { + if err := ValidateNavigationURL(url); err != nil { + return err + } if !c.running || c.debugPort == 0 { return fmt.Errorf("browser not running") } diff --git a/sandbox/internal/browser/isolation.go b/sandbox/internal/browser/isolation.go new file mode 100644 index 00000000..df5ad456 --- /dev/null +++ b/sandbox/internal/browser/isolation.go @@ -0,0 +1,717 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-isolation-v8-error-codes +package browser + +import ( + "crypto/rand" + "crypto/sha256" + "encoding/binary" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "log" + "net" + "os" + "os/user" + "path/filepath" + "strconv" + "strings" + "sync" + "syscall" + "time" + + "golang.org/x/sys/unix" +) + +const isolationRevision = "browser-isolation-v8-error-codes" + +func init() { + log.Printf("[browser] REVISION: %s loaded at %s", isolationRevision, time.Now().Format(time.RFC3339)) +} + +// Chromium runs as this unprivileged user (docker/Dockerfile), never root; the +// display side of the stack runs as VNCUser. Chromium needs --no-sandbox in the VM, so a root Chromium with a +// DevTools port reachable over loopback let any agent read and write files as +// root (file:// pages, Browser.setDownloadBehavior). As this user, CDP access +// is worth no more than the browser itself. +const BrowserUser = "orcabot-browser" + +// VNCUser runs the display side of the stack (Xvfb, x11vnc, websockify), so +// Chromium does not. Chromium runs whatever pages load, and page JavaScript +// (or an agent's browser_evaluate) can fetch or navigate to any loopback port +// its user can connect to: as one user, dashboard A's page could open +// dashboard B's unauthenticated websockify/noVNC (view and drive its screen) +// on desktop, where one VM runs every dashboard's browser. With the split, +// iptables rejects every connection from BrowserUser to the browser port range +// (setupBrowserUserPortBlock, cmd/server/egress_linux.go): Chromium only +// listens there (DevTools) and never connects; websockify (this user) is the +// one client of x11vnc. Not in the sandbox group, like BrowserUser. +const VNCUser = "orcabot-vnc" + +// Ports for x11vnc, websockify and Chromium's DevTools come from this range so +// iptables can block pool UIDs from all of them (cmd/server/egress_linux.go). +const ( + PortRangeStart = 6100 + PortRangeEnd = 6299 +) + +type browserIdentity struct { + uid, gid int + home string + name string // user name; empty in tests that build an identity by hand +} + +// lookupBrowserIdentity is the Chromium user. +func lookupBrowserIdentity() (browserIdentity, error) { + return lооkupStackIdentity(BrowserUser) +} + +// lookupVNCIdentity is the Xvfb / x11vnc / websockify user. +func lookupVNCIdentity() (browserIdentity, error) { + return lооkupStackIdentity(VNCUser) +} + +func lооkupStackIdentity(name string) (browserIdentity, error) { + u, err := user.Lookup(name) + if err != nil { + return browserIdentity{}, fmt.Errorf("browser user %q missing (rebuild the sandbox image): %w", name, err) + } + uid, err := strconv.Atoi(u.Uid) + if err != nil { + return browserIdentity{}, err + } + gid, err := strconv.Atoi(u.Gid) + if err != nil { + return browserIdentity{}, err + } + if uid == 0 || gid == 0 { + return browserIdentity{}, fmt.Errorf("browser user %q must not be root", name) + } + return browserIdentity{uid: uid, gid: gid, home: u.HomeDir, name: name}, nil +} + +func (id browserIdentity) sysProcAttr() *syscall.SysProcAttr { + return &syscall.SysProcAttr{ + Setpgid: true, + Credential: &syscall.Credential{Uid: uint32(id.uid), Gid: uint32(id.gid), NoSetGroups: false}, + } +} + +// env for the browser processes: the server's env minus anything secret-bearing +// it doesn't need, with the browser user's HOME and the X auth cookie. +func (id browserIdentity) env(display, xauthority string) []string { + return []string{ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "HOME=" + id.home, + "USER=" + id.userName(), + "DISPLAY=" + display, + "XAUTHORITY=" + xauthority, + "LANG=C.UTF-8", + } +} + +func (id browserIdentity) userName() string { + if id.name == "" { + return BrowserUser + } + return id.name +} + +// Ports handed out by freePortInRange stay reserved process-wide until the +// stack that owns them stops (releasePorts). The probe below closes its +// listener before the browser process binds the port, so without this two +// concurrent Starts (two dashboards on desktop) could pick the same port and +// one controller would end up driving the other's Chromium: DialBrowser only +// checks the listener belongs to the browser user, and both stacks do. +var ( + portReserveMu sync.Mutex + reservedPorts = map[int]struct{}{} +) + +// freePortInRange reserves a free 127.0.0.1 port in [PortRangeStart, +// PortRangeEnd] that no other controller holds, skipping ports already chosen +// for this start. The caller must releasePorts it when the stack stops or its +// start fails. +func freePortInRange(skip ...int) (int, error) { + portReserveMu.Lock() + defer portReserveMu.Unlock() + var b [2]byte + _, _ = rand.Read(b[:]) + span := PortRangeEnd - PortRangeStart + 1 + offset := int(binary.BigEndian.Uint16(b[:])) % span + for i := 0; i < span; i++ { + port := PortRangeStart + (offset+i)%span + taken := false + for _, s := range skip { + taken = taken || s == port + } + if _, held := reservedPorts[port]; taken || held { + continue + } + l, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", port)) + if err != nil { + continue + } + _ = l.Close() + reservedPorts[port] = struct{}{} + return port, nil + } + return 0, fmt.Errorf("no free port in %d-%d", PortRangeStart, PortRangeEnd) +} + +// releasePorts returns ports to the pool (zero and unreserved ports are ignored). +func releasePorts(ports ...int) { + portReserveMu.Lock() + defer portReserveMu.Unlock() + for _, p := range ports { + delete(reservedPorts, p) + } +} + +// reserveBrowserPorts reserves the websockify, x11vnc and DevTools ports, +// releasing any it got if it can't get all three. +func rеserveBrowserPorts() (wsPort, vncPort, debugPort int, err error) { + if wsPort, err = freePortInRange(); err != nil { + return 0, 0, 0, err + } + if vncPort, err = freePortInRange(wsPort); err != nil { + releasePorts(wsPort) + return 0, 0, 0, err + } + if debugPort, err = freePortInRange(wsPort, vncPort); err != nil { + releasePorts(wsPort, vncPort) + return 0, 0, 0, err + } + return wsPort, vncPort, debugPort, nil +} + +func nеwXCookie() ([]byte, error) { + cookie := make([]byte, 16) + _, err := rand.Read(cookie) + return cookie, err +} + +// writeXauthority writes an MIT-MAGIC-COOKIE-1 entry for `display` (any host) +// to a 0600 file owned by the browser user, replacing it atomically. Xvfb is +// started with -auth on it, so only processes holding the cookie (the browser +// stack) can open the display — not agents injecting keystrokes into the +// browser window. The file lives in the browser user's home, so the temp file +// is created O_EXCL|O_NOFOLLOW: a symlink planted there is never followed. +func writeXauthority(path string, display int, cookie []byte, id browserIdentity) error { + field := func(buf []byte, v []byte) []byte { + buf = binary.BigEndian.AppendUint16(buf, uint16(len(v))) + return append(buf, v...) + } + var entry []byte + entry = binary.BigEndian.AppendUint16(entry, 0xffff) // FamilyWild: any address + entry = field(entry, nil) + entry = field(entry, []byte(strconv.Itoa(display))) + entry = field(entry, []byte("MIT-MAGIC-COOKIE-1")) + entry = field(entry, cookie) + + tmp := path + ".tmp" + _ = os.Remove(tmp) + f, err := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_EXCL|syscall.O_NOFOLLOW, 0o600) + if err != nil { + return err + } + _, werr := f.Write(entry) + cerr := f.Chown(id.uid, id.gid) + f.Close() + if werr != nil || cerr != nil { + os.Remove(tmp) + return errors.Join(werr, cerr) + } + return os.Rename(tmp, path) +} + +// profilesDirName holds one Chromium profile per workspace in the browser +// user's home. Desktop runs one VM for every dashboard, each with its own +// workspace root (/workspace/) and browser controller, so a single +// profile would let one dashboard's pages read another's cookies and storage. +// It used to be /workspace/.browser (see removeLegacyProfile), and briefly one +// shared /profile (see sharedProfileDirName). +const profilesDirName = "profiles" + +// sharedProfileDirName is the single profile every dashboard used to share; +// it is deleted rather than handed to any one of them. +const sharedProfileDirName = "profile" + +func profilesPath(id browserIdentity) string { + return filepath.Join(id.home, profilesDirName) +} + +// profilePath is the profile for one workspace: a hash of its root, so the +// name is a fixed-length, path-safe component whatever the dashboard ID holds. +func profilePath(id browserIdentity, workspace string) string { + return filepath.Join(profilesPath(id), profileName(workspace)) +} + +func profileName(workspace string) string { + sum := sha256.Sum256([]byte(filepath.Clean(workspace))) + return hex.EncodeToString(sum[:16]) +} + +// checkBrowserHome verifies the browser user's home is a place only that user +// (and root) can reach: a real directory owned by the browser user with no +// group/other bits, every parent a real, root-owned directory nobody else can +// write. The profile and the X cookie live there; if an agent +// could rename or replace any directory on the way, it could swap in its own +// profile (cookie theft, planted prefs/extensions) as it could under +// /workspace. Fails closed. +func checkBrowserHome(id browserIdentity) error { + if id.home == "" || !filepath.IsAbs(id.home) { + return fmt.Errorf("browser user home %q is not an absolute path", id.home) + } + home := filepath.Clean(id.home) + fi, err := os.Lstat(home) + if err != nil { + return fmt.Errorf("browser user home: %w", err) + } + st, ok := fi.Sys().(*syscall.Stat_t) + if !ok || !fi.IsDir() || fi.Mode()&os.ModeSymlink != 0 { + return fmt.Errorf("browser user home %s is not a directory", home) + } + if int(st.Uid) != id.uid || st.Mode&0o077 != 0 { + return fmt.Errorf("browser user home %s must be owned by %s with mode 0700 (uid %d mode %o)", home, id.userName(), st.Uid, st.Mode&0o7777) + } + for dir := filepath.Dir(home); ; dir = filepath.Dir(dir) { + fi, err := os.Lstat(dir) + if err != nil { + return fmt.Errorf("browser user home parent: %w", err) + } + st, ok := fi.Sys().(*syscall.Stat_t) + if !ok || !fi.IsDir() || fi.Mode()&os.ModeSymlink != 0 || st.Uid != 0 || st.Mode&0o022 != 0 { + return fmt.Errorf("browser user home parent %s must be a root-owned directory only root can write", dir) + } + if dir == "/" { + return nil + } + } +} + +// prepareProfile makes the workspace's profile ready for Chromium and returns +// its path. Every step works through directory fds: the home is opened by +// walking each component from / with O_NOFOLLOW|O_DIRECTORY, and profiles/ +// and the profile itself are created, checked and adopted relative to their +// parent's fd. Swapping profiles/ for a symlink between steps therefore can't +// make root mkdir or chown anywhere else. +func prepareProfile(id browserIdentity, workspace string) (string, error) { + homeFd, err := openDirNoFollow(id.home) + if err != nil { + return "", fmt.Errorf("browser home: %w", err) + } + defer unix.Close(homeFd) + profilesFd, err := prepareDirAt(homeFd, profilesDirName, profilesPath(id), id) + if err != nil { + return "", err + } + defer unix.Close(profilesFd) + path := profilePath(id, workspace) + fd, err := prepareDirAt(profilesFd, profileName(workspace), path, id) + if err != nil { + return "", err + } + defer unix.Close(fd) + if err := finishProfileDir(fd, id); err != nil { + return "", fmt.Errorf("profile %s: %w", path, err) + } + return path, nil +} + +// prepareProfileDir prepares the profile at an absolute path (tests, and any +// caller outside prepareProfile): its parent is reached with openDirNoFollow. +func prepareProfileDir(path string, id browserIdentity) error { + parentFd, err := openDirNoFollow(filepath.Dir(path)) + if err != nil { + return err + } + defer unix.Close(parentFd) + fd, err := prepareDirAt(parentFd, filepath.Base(path), path, id) + if err != nil { + return err + } + defer unix.Close(fd) + return finishProfileDir(fd, id) +} + +// openDirNoFollow opens an absolute directory path component by component +// from /, each with O_NOFOLLOW|O_DIRECTORY, so a symlink anywhere on the path +// fails the open instead of being followed. +func openDirNoFollow(path string) (int, error) { + if !filepath.IsAbs(path) { + return -1, fmt.Errorf("%s is not an absolute path", path) + } + fd, err := unix.Open("/", unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0) + if err != nil { + return -1, err + } + for _, comp := range strings.Split(filepath.Clean(path), "/") { + if comp == "" { + continue + } + next, err := unix.Openat(fd, comp, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + unix.Close(fd) + if err != nil { + return -1, fmt.Errorf("open %s (at %q): %w", path, comp, err) + } + fd = next + } + return fd, nil +} + +// prepareDirAt makes name (in parentFd, inside the browser user's private +// home, see checkBrowserHome) a real directory owned by the browser user +// (0700) and returns an fd for it. Only root and the browser user can write +// the home, so the one party that could have planted a symlink or a foreign +// directory there is the browser itself; root still works only through fds +// opened with O_NOFOLLOW and moves anything that isn't a browser- or +// root-owned directory aside rather than adopting it. A root-owned directory +// can only be the one root just created. display is the full path, for logs. +func prepareDirAt(parentFd int, name, display string, id browserIdentity) (int, error) { + for attempt := 0; ; attempt++ { + fd, err := openProfileDirAt(parentFd, name, display) + if err != nil { + return -1, err + } + var st unix.Stat_t + if err := unix.Fstat(fd, &st); err != nil { + unix.Close(fd) + return -1, err + } + if int(st.Uid) != id.uid && st.Uid != 0 { + unix.Close(fd) + why := fmt.Sprintf("owned by uid %d", st.Uid) + if attempt > 0 { + return -1, fmt.Errorf("profile %s replaced while preparing it (%s)", display, why) + } + if err := moveAsideAt(parentFd, name, display, why); err != nil { + return -1, err + } + continue + } + if err := unix.Fchown(fd, id.uid, id.gid); err != nil { + unix.Close(fd) + return -1, err + } + if err := unix.Fchmod(fd, 0o700); err != nil { + unix.Close(fd) + return -1, err + } + return fd, nil + } +} + +// openProfileDirAt moves a non-directory (or symlink) named name in parentFd +// aside, creates the directory if missing, and opens it without following +// symlinks — all relative to parentFd. +func openProfileDirAt(parentFd int, name, display string) (int, error) { + var st unix.Stat_t + err := unix.Fstatat(parentFd, name, &st, unix.AT_SYMLINK_NOFOLLOW) + switch { + case errors.Is(err, unix.ENOENT): + case err != nil: + return -1, fmt.Errorf("stat %s: %w", display, err) + case st.Mode&unix.S_IFMT != unix.S_IFDIR: + if err := moveAsideAt(parentFd, name, display, fmt.Sprintf("not a directory (mode %o)", st.Mode)); err != nil { + return -1, err + } + } + if err := unix.Mkdirat(parentFd, name, 0o700); err != nil && !errors.Is(err, unix.EEXIST) { + return -1, fmt.Errorf("mkdir %s: %w", display, err) + } + fd, err := unix.Openat(parentFd, name, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + if err != nil { + return -1, fmt.Errorf("open profile dir %s: %w", display, err) + } + return fd, nil +} + +// moveAsideAt renames name in parentFd (whatever it is; rename never follows a +// symlink) out of the way so a fresh profile can be created. +func moveAsideAt(parentFd int, name, display, why string) error { + var suffix [4]byte + _, _ = rand.Read(suffix[:]) + aside := fmt.Sprintf("%s.invalid-%d-%s", name, time.Now().Unix(), hex.EncodeToString(suffix[:])) + log.Printf("[browser] SECURITY: %s is %s — moving it to %s", display, why, filepath.Join(filepath.Dir(display), aside)) + if err := unix.Renameat(parentFd, name, parentFd, aside); err != nil { + return fmt.Errorf("move aside %s: %w", display, err) + } + return nil +} + +// singletonFiles are Chromium's profile lock: SingletonLock is a symlink to +// "-" of the Chromium that has the profile open. +var singletonFiles = []string{"SingletonLock", "SingletonSocket", "SingletonCookie"} + +// errProfileInUse: a live Chromium still holds the profile. Its lock files are +// left alone — removing them let a second Chromium open the same profile and +// corrupt it. +var errProfileInUse = errors.New("browser profile is in use by another Chromium") + +// procRoot is /proc; a var so tests can fake process state. +var procRoot = "/proc" + +// finishProfileDir clears a stale profile lock and crash state. A lock that +// names a live Chromium of the browser user on this host is not stale: it is +// another browser on this profile (one left behind by an earlier server, say), +// and starting a second one on it would corrupt the profile, so it fails. +func finishProfileDir(fd int, id browserIdentity) error { + if pid := livеSingletonPid(fd, id); pid > 0 { + return fmt.Errorf("%w (pid %d)", errProfileInUse, pid) + } + for _, name := range singletonFiles { + _ = unix.Unlinkat(fd, name, 0) + } + clеanCrashStateAt(fd, id) + return nil +} + +// liveSingletonPid returns the pid SingletonLock names if that process is +// alive on this host, runs as the browser user and is a Chromium; else 0. +func livеSingletonPid(fd int, id browserIdentity) int { + buf := make([]byte, 256) + n, err := unix.Readlinkat(fd, "SingletonLock", buf) + if err != nil || n <= 0 || n >= len(buf) { + return 0 + } + target := string(buf[:n]) + i := strings.LastIndexByte(target, '-') + if i <= 0 { + return 0 + } + pid, err := strconv.Atoi(target[i+1:]) + if err != nil || pid <= 0 { + return 0 + } + if host, err := os.Hostname(); err == nil && host != target[:i] { + return 0 // another machine's lock (e.g. from an image build) + } + if !isBrowserChromium(pid, id.uid) { + return 0 + } + return pid +} + +// isBrowserChromium reports whether pid is a live process running as uid +// whose name starts with "chrom". /proc//status is world-readable (unlike +// /proc//fd, which needs CAP_SYS_PTRACE the container doesn't have). +func isBrowserChromium(pid, uid int) bool { + data, err := os.ReadFile(filepath.Join(procRoot, strconv.Itoa(pid), "status")) + if err != nil { + return false + } + var name string + ruid := -1 + for _, line := range strings.Split(string(data), "\n") { + k, v, ok := strings.Cut(line, ":") + if !ok { + continue + } + v = strings.TrimSpace(v) + switch k { + case "Name": + name = v + case "State": + if strings.HasPrefix(v, "Z") || strings.HasPrefix(v, "X") { + return false // exited, not yet reaped + } + case "Uid": + if f := strings.Fields(v); len(f) > 0 { + ruid, _ = strconv.Atoi(f[0]) + } + } + } + return ruid == uid && strings.HasPrefix(name, "chrom") +} + +// readDirFd lists dirfd from the start. A dup shares the file offset, so it +// rewinds first. +func readDirFd(dirfd int) ([]os.DirEntry, error) { + dup, err := unix.Dup(dirfd) + if err != nil { + return nil, err + } + d := os.NewFile(uintptr(dup), "") + defer d.Close() + if _, err := d.Seek(0, io.SeekStart); err != nil { + return nil, err + } + return d.ReadDir(-1) +} + +// legacyProfileName is where the profile lived before it moved out of the +// workspace. +const legacyProfileName = ".browser" + +// removeLegacyProfile deletes the old /workspace/.browser profile (cookies and +// sessions the browser no longer uses) rather than migrating it: adopting a +// tree from the agent-writable workspace is exactly the hand-over attack +// surface (planted dirs, hard links, fds held open) that moving the profile +// removes. Only a real directory owned by the browser user is removed, and +// only through fds opened with O_NOFOLLOW: its contents are writable by no one +// but the browser user, and it is unlinked from the workspace only if the +// entry still names the directory that was emptied. Anything else (an agent's +// directory or symlink renamed into place) is left alone. +func removeLegacyProfile(workspace string, id browserIdentity) error { + wsfd, err := unix.Open(workspace, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0) + if err != nil { + return err + } + defer unix.Close(wsfd) + fd, err := unix.Openat(wsfd, legacyProfileName, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + if errors.Is(err, unix.ENOENT) { + return nil + } + if err != nil { + return fmt.Errorf("open legacy profile: %w", err) + } + defer unix.Close(fd) + var st unix.Stat_t + if err := unix.Fstat(fd, &st); err != nil { + return err + } + if int(st.Uid) != id.uid { + return fmt.Errorf("legacy profile %s/%s is owned by uid %d, not %s; leaving it", workspace, legacyProfileName, st.Uid, BrowserUser) + } + if err := rеmoveTreeAt(fd); err != nil { + return err + } + var now unix.Stat_t + if err := unix.Fstatat(wsfd, legacyProfileName, &now, unix.AT_SYMLINK_NOFOLLOW); err != nil { + return err + } + if now.Dev != st.Dev || now.Ino != st.Ino { + return fmt.Errorf("legacy profile replaced while removing it; leaving the new entry") + } + return unix.Unlinkat(wsfd, legacyProfileName, unix.AT_REMOVEDIR) +} + +// removeTreeAt empties the directory dirfd without following symlinks. +func rеmoveTreeAt(dirfd int) error { + entries, err := readDirFd(dirfd) + if err != nil { + return err + } + for _, e := range entries { + name := e.Name() + if e.IsDir() { + sub, err := unix.Openat(dirfd, name, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + if err != nil { + return fmt.Errorf("%s: %w", name, err) + } + err = rеmoveTreeAt(sub) + unix.Close(sub) + if err != nil { + return fmt.Errorf("%s: %w", name, err) + } + if err := unix.Unlinkat(dirfd, name, unix.AT_REMOVEDIR); err != nil { + return fmt.Errorf("%s: %w", name, err) + } + continue + } + if err := unix.Unlinkat(dirfd, name, 0); err != nil && !errors.Is(err, unix.ENOENT) { + return fmt.Errorf("%s: %w", name, err) + } + } + return nil +} + +// cleanCrashStateAt sets profile.exit_type=Normal / exited_cleanly=true in +// Default/Preferences so Chromium doesn't show "Restore pages?", working only +// through fds relative to the profile dir with O_NOFOLLOW. +func clеanCrashStateAt(profileFd int, id browserIdentity) { + defFd, err := unix.Openat(profileFd, "Default", unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + if err != nil { + return // fresh profile + } + defer unix.Close(defFd) + + data, err := readPrefsAt(defFd) + if err != nil { + if !errors.Is(err, unix.ENOENT) { + log.Printf("[browser] cleanCrashState: %v", err) + } + return + } + + var prefs map[string]interface{} + if err := json.Unmarshal(data, &prefs); err != nil { + log.Printf("[browser] cleanCrashState: failed to parse Preferences: %v", err) + return + } + profile, ok := prefs["profile"].(map[string]interface{}) + if !ok { + profile = make(map[string]interface{}) + prefs["profile"] = profile + } + profile["exit_type"] = "Normal" + profile["exited_cleanly"] = true + updated, err := json.Marshal(prefs) + if err != nil { + return + } + + // Atomic replace: temp file + rename, both relative to Default/. + _ = unix.Unlinkat(defFd, "Preferences.tmp", 0) + outFd, err := unix.Openat(defFd, "Preferences.tmp", unix.O_WRONLY|unix.O_CREAT|unix.O_EXCL|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0o600) + if err != nil { + log.Printf("[browser] cleanCrashState: create temp: %v", err) + return + } + out := os.NewFile(uintptr(outFd), "Preferences.tmp") + _, werr := out.Write(updated) + _ = out.Chown(id.uid, id.gid) + out.Close() + if werr != nil { + _ = unix.Unlinkat(defFd, "Preferences.tmp", 0) + return + } + if err := unix.Renameat(defFd, "Preferences.tmp", defFd, "Preferences"); err != nil { + log.Printf("[browser] cleanCrashState: rename: %v", err) + } +} + +// maxPrefsSize caps how much of Preferences root reads. +const maxPrefsSize = 16 << 20 + +// readPrefsAt reads Default/Preferences, which the browser user controls. It +// is opened O_NONBLOCK so a FIFO planted there can't hang Start (which holds +// the controller lock), and only a regular file within maxPrefsSize is read. +func readPrefsAt(defFd int) ([]byte, error) { + inFd, err := unix.Openat(defFd, "Preferences", unix.O_RDONLY|unix.O_NOFOLLOW|unix.O_NONBLOCK|unix.O_NOCTTY|unix.O_CLOEXEC, 0) + if err != nil { + return nil, err + } + var st unix.Stat_t + if err := unix.Fstat(inFd, &st); err != nil { + unix.Close(inFd) + return nil, err + } + if st.Mode&unix.S_IFMT != unix.S_IFREG { + unix.Close(inFd) + return nil, fmt.Errorf("Preferences is not a regular file (mode %o)", st.Mode) + } + if st.Size > maxPrefsSize { + unix.Close(inFd) + return nil, fmt.Errorf("Preferences is too large (%d bytes)", st.Size) + } + in := os.NewFile(uintptr(inFd), "Preferences") + defer in.Close() + return io.ReadAll(io.LimitReader(in, maxPrefsSize)) +} + +// xauthorityPath picks a fresh path for a display's cookie in the browser +// user's home, which is 0700 and outside /workspace. It is chosen before Xvfb +// picks the display, so it is random rather than named after it. +func xauthorityPath(id browserIdentity) (string, error) { + var b [8]byte + if _, err := rand.Read(b[:]); err != nil { + return "", err + } + return filepath.Join(id.home, ".Xauthority-"+hex.EncodeToString(b[:])), nil +} diff --git a/sandbox/internal/browser/isolation_test.go b/sandbox/internal/browser/isolation_test.go new file mode 100644 index 00000000..9c949fe3 --- /dev/null +++ b/sandbox/internal/browser/isolation_test.go @@ -0,0 +1,634 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-isolation-v7-error-codes + +//go:build linux + +package browser + +import ( + "bytes" + "encoding/binary" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "syscall" + "testing" + "time" +) + +// nobody: an unprivileged identity to hand files to. These tests need root +// (they chown), which is how the sandbox server runs. +var testID = browserIdentity{uid: 65534, gid: 65534} + +func needRoot(t *testing.T) { + t.Helper() + if os.Geteuid() != 0 { + t.Skip("needs root (chown)") + } +} + +func owner(t *testing.T, path string) int { + t.Helper() + fi, err := os.Lstat(path) + if err != nil { + t.Fatal(err) + } + return int(fi.Sys().(*syscall.Stat_t).Uid) +} + +func TestProfileSymlinkIsMovedAsideNotFollowed(t *testing.T) { + needRoot(t) + ws := t.TempDir() + victim := t.TempDir() // e.g. /etc/cron.d + os.WriteFile(filepath.Join(victim, "SingletonLock"), []byte("keep"), 0o644) + profile := filepath.Join(ws, "profile") + if err := os.Symlink(victim, profile); err != nil { + t.Fatal(err) + } + + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + fi, _ := os.Lstat(profile) + if !fi.IsDir() || fi.Mode()&os.ModeSymlink != 0 { + t.Fatalf("profile should be a real directory now, got %v", fi.Mode()) + } + if owner(t, profile) != testID.uid || fi.Mode().Perm() != 0o700 { + t.Fatalf("profile owner/mode = %d/%v", owner(t, profile), fi.Mode().Perm()) + } + if _, err := os.Stat(filepath.Join(victim, "SingletonLock")); err != nil { + t.Fatal("root followed the symlink and deleted a file in the target") + } + if owner(t, victim) == testID.uid { + t.Fatal("root chowned the symlink target") + } + moved, _ := filepath.Glob(profile + ".invalid-*") + if len(moved) != 1 { + t.Fatalf("symlink not moved aside: %v", moved) + } +} + +func TestCrashStateCleanedWithoutFollowingDefaultSymlink(t *testing.T) { + needRoot(t) + ws := t.TempDir() + profile := filepath.Join(ws, "profile") + os.MkdirAll(profile, 0o700) + elsewhere := t.TempDir() + prefs := filepath.Join(elsewhere, "Preferences") + os.WriteFile(prefs, []byte(`{"profile":{"exit_type":"Crashed"}}`), 0o644) + os.Symlink(elsewhere, filepath.Join(profile, "Default")) + + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + if data, _ := os.ReadFile(prefs); !strings.Contains(string(data), "Crashed") { + t.Fatal("rewrote Preferences through a Default symlink") + } +} + +func TestCrashStateCleaned(t *testing.T) { + needRoot(t) + profile := filepath.Join(t.TempDir(), "profile") + os.MkdirAll(filepath.Join(profile, "Default"), 0o700) + os.WriteFile(filepath.Join(profile, "Default", "Preferences"), []byte(`{"profile":{"exit_type":"Crashed","name":"p"}}`), 0o600) + + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + data, _ := os.ReadFile(filepath.Join(profile, "Default", "Preferences")) + var got struct { + Profile map[string]interface{} `json:"profile"` + } + json.Unmarshal(data, &got) + if got.Profile["exit_type"] != "Normal" || got.Profile["exited_cleanly"] != true || got.Profile["name"] != "p" { + t.Fatalf("prefs = %s", data) + } + if owner(t, filepath.Join(profile, "Default", "Preferences")) != testID.uid { + t.Fatal("rewritten Preferences not owned by the browser user") + } +} + +func TestXauthorityCookie(t *testing.T) { + needRoot(t) + path := filepath.Join(t.TempDir(), "xauth") + cookie, _ := nеwXCookie() + if err := writeXauthority(path, 91, cookie, testID); err != nil { + t.Fatal(err) + } + fi, _ := os.Stat(path) + if fi.Mode().Perm() != 0o600 || owner(t, path) != testID.uid { + t.Fatalf("xauth mode/owner = %v/%d", fi.Mode().Perm(), owner(t, path)) + } + data, _ := os.ReadFile(path) + if binary.BigEndian.Uint16(data[0:2]) != 0xffff { + t.Fatal("expected FamilyWild") + } + if !strings.Contains(string(data), "91") || !strings.Contains(string(data), "MIT-MAGIC-COOKIE-1") { + t.Fatalf("unexpected xauth entry %q", data) + } + if len(data) != 2+2+0+2+2+2+18+2+16 || !bytes.HasSuffix(data, cookie) { + t.Fatalf("entry length %d", len(data)) + } + + // Rewritten with the display Xvfb picked: same cookie, new number. + if err := writeXauthority(path, 7, cookie, testID); err != nil { + t.Fatal(err) + } + data, _ = os.ReadFile(path) + if !bytes.HasSuffix(data, cookie) || strings.Contains(string(data), "91") || !strings.Contains(string(data), "\x00\x017") { + t.Fatalf("rewritten entry %q", data) + } +} + +func TestXauthorityTempSymlinkNotFollowed(t *testing.T) { + needRoot(t) + dir := t.TempDir() + victim := filepath.Join(t.TempDir(), "victim") + os.WriteFile(victim, []byte("keep"), 0o644) + path := filepath.Join(dir, "xauth") + // The browser user owns its home, so it could plant this. + os.Symlink(victim, path+".tmp") + cookie, _ := nеwXCookie() + if err := writeXauthority(path, 1, cookie, testID); err != nil { + t.Fatal(err) + } + if data, _ := os.ReadFile(victim); string(data) != "keep" || owner(t, victim) != 0 { + t.Fatal("writeXauthority wrote through a symlink") + } +} + +func TestForeignOwnedProfileIsMovedAsideNotHandedOver(t *testing.T) { + needRoot(t) + ws := t.TempDir() + profile := filepath.Join(ws, "profile") + // Something other than the browser user or root owns the profile (it + // can't happen in the 0700 home, but if it did): seeded with a setgid, + // group-writable dir its owner could keep writing to. + os.MkdirAll(filepath.Join(profile, "Default"), 0o777) + os.WriteFile(filepath.Join(profile, "Default", "Preferences"), []byte(`{}`), 0o666) + for _, p := range []string{profile, filepath.Join(profile, "Default"), filepath.Join(profile, "Default", "Preferences")} { + os.Chown(p, 2000, 2000) + } + os.Chmod(filepath.Join(profile, "Default"), 0o2777) + + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + fi, _ := os.Lstat(profile) + if owner(t, profile) != testID.uid || fi.Mode().Perm() != 0o700 { + t.Fatalf("profile owner/mode = %d/%v", owner(t, profile), fi.Mode().Perm()) + } + if entries, _ := os.ReadDir(profile); len(entries) != 0 { + t.Fatalf("foreign profile contents carried over: %v", entries) + } + moved, _ := filepath.Glob(profile + ".invalid-*") + if len(moved) != 1 { + t.Fatalf("foreign profile not moved aside: %v", moved) + } + if owner(t, moved[0]) != 2000 || owner(t, filepath.Join(moved[0], "Default", "Preferences")) != 2000 { + t.Fatal("foreign profile was handed over before being moved aside") + } +} + +func TestBrowserOwnedProfileKept(t *testing.T) { + needRoot(t) + profile := filepath.Join(t.TempDir(), "profile") + os.MkdirAll(filepath.Join(profile, "Default"), 0o700) + os.WriteFile(filepath.Join(profile, "Default", "Cookies"), []byte("c"), 0o600) + os.Chown(profile, testID.uid, testID.gid) + + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(profile, "Default", "Cookies")); err != nil { + t.Fatal("browser-owned profile lost its contents") + } + if moved, _ := filepath.Glob(profile + ".invalid-*"); len(moved) != 0 { + t.Fatalf("browser-owned profile moved aside: %v", moved) + } +} + +func TestPortsComeFromTheBlockedRange(t *testing.T) { + a, err := freePortInRange() + if err != nil { + t.Fatal(err) + } + b, _ := freePortInRange(a) + defer releasePorts(a, b) + if a < PortRangeStart || a > PortRangeEnd || b < PortRangeStart || b > PortRangeEnd || a == b { + t.Fatalf("ports %d, %d", a, b) + } +} + +// A root-owned profile directory can only be one root itself created (the +// home is 0700, owned by the browser user): it is given to the browser user. +func TestRootOwnedProfileDirIsChowned(t *testing.T) { + needRoot(t) + profile := filepath.Join(t.TempDir(), "profile") + os.Mkdir(profile, 0o755) + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + fi, _ := os.Lstat(profile) + if owner(t, profile) != testID.uid || fi.Mode().Perm() != 0o700 { + t.Fatalf("profile owner/mode = %d/%v", owner(t, profile), fi.Mode().Perm()) + } +} + +// The profile must never live in the agent-writable workspace again: an agent +// could rename it and swap in its own directory while the browser runs. +func TestProfileIsInBrowserHomeNotWorkspace(t *testing.T) { + id := browserIdentity{uid: 1990, gid: 1990, home: "/home/orcabot-browser"} + for _, ws := range []string{"/workspace", "/workspace/dash-a"} { + got := profilePath(id, ws) + if !strings.HasPrefix(got, "/home/orcabot-browser/profiles/") { + t.Fatalf("profilePath(%s) = %s", ws, got) + } + if strings.HasPrefix(got, "/workspace") { + t.Fatalf("profile %s is under the workspace", got) + } + } +} + +// Desktop runs every dashboard in one VM, each with its own browser +// controller: they must not share cookies or local storage. +func TestEachDashboardWorkspaceGetsItsOwnProfile(t *testing.T) { + id := browserIdentity{uid: 1990, gid: 1990, home: "/home/orcabot-browser"} + a, b := profilePath(id, "/workspace/dash-a"), profilePath(id, "/workspace/dash-b") + if a == b { + t.Fatalf("dashboards share profile %s", a) + } + if profilePath(id, "/workspace/dash-a/") != a { + t.Fatal("profile depends on a trailing slash") + } + if filepath.Dir(a) != "/home/orcabot-browser/profiles" || strings.ContainsAny(filepath.Base(a), "/.") { + t.Fatalf("profile %s is not one fixed component under profiles/", a) + } +} + +func TestCheckBrowserHome(t *testing.T) { + needRoot(t) + // Parents must be root-owned and not group/other-writable, so build the + // chain under / rather than the sticky, world-writable /tmp. + base, err := os.MkdirTemp("/", "orcabot-home-test-") + if err != nil { + t.Fatal(err) + } + defer os.RemoveAll(base) + os.Chmod(base, 0o755) + home := filepath.Join(base, "home") + os.Mkdir(home, 0o700) + os.Chown(home, testID.uid, testID.gid) + id := browserIdentity{uid: testID.uid, gid: testID.gid, home: home} + if err := checkBrowserHome(id); err != nil { + t.Fatalf("good home rejected: %v", err) + } + + os.Chmod(home, 0o750) + if checkBrowserHome(id) == nil { + t.Fatal("group-readable home accepted") + } + os.Chmod(home, 0o700) + + os.Chmod(base, 0o775) + if checkBrowserHome(id) == nil { + t.Fatal("home under a group-writable parent accepted") + } + os.Chmod(base, 0o755) + + os.Chown(base, 2000, 2000) + if checkBrowserHome(id) == nil { + t.Fatal("home under an agent-owned parent accepted") + } + os.Chown(base, 0, 0) + + link := filepath.Join(base, "link") + os.Symlink(home, link) + if checkBrowserHome(browserIdentity{uid: testID.uid, gid: testID.gid, home: link}) == nil { + t.Fatal("symlinked home accepted") + } + + tmpHome := t.TempDir() // under /tmp: sticky, world-writable parent + os.Chown(tmpHome, testID.uid, testID.gid) + os.Chmod(tmpHome, 0o700) + if checkBrowserHome(browserIdentity{uid: testID.uid, gid: testID.gid, home: tmpHome}) == nil { + t.Fatal("home under /tmp accepted") + } +} + +func TestLegacyWorkspaceProfileRemoved(t *testing.T) { + needRoot(t) + ws := t.TempDir() + outside := filepath.Join(t.TempDir(), "keep") + os.WriteFile(outside, []byte("keep"), 0o600) + legacy := filepath.Join(ws, ".browser") + os.MkdirAll(filepath.Join(legacy, "Default", "Cache"), 0o700) + os.WriteFile(filepath.Join(legacy, "Default", "Cookies"), []byte("c"), 0o600) + os.Symlink(filepath.Dir(outside), filepath.Join(legacy, "Default", "link")) + filepath.Walk(legacy, func(p string, _ os.FileInfo, _ error) error { + os.Lchown(p, testID.uid, testID.gid) + return nil + }) + + if err := removeLegacyProfile(ws, testID); err != nil { + t.Fatal(err) + } + if _, err := os.Lstat(legacy); !os.IsNotExist(err) { + t.Fatal("legacy profile not removed") + } + if data, _ := os.ReadFile(outside); string(data) != "keep" { + t.Fatal("removal followed a symlink out of the profile") + } +} + +func TestLegacyProfileNotBrowserOwnedIsLeftAlone(t *testing.T) { + needRoot(t) + ws := t.TempDir() + // An agent's directory renamed to .browser: not ours to delete. + agentDir := filepath.Join(ws, ".browser") + os.MkdirAll(agentDir, 0o775) + os.WriteFile(filepath.Join(agentDir, "work"), []byte("w"), 0o664) + os.Chown(agentDir, 2000, 2000) + if err := removeLegacyProfile(ws, testID); err == nil { + t.Fatal("expected a refusal for a foreign-owned .browser") + } + if _, err := os.Stat(filepath.Join(agentDir, "work")); err != nil { + t.Fatal("agent's files deleted") + } + + // A symlink to somewhere else: never followed. + os.RemoveAll(agentDir) + target := t.TempDir() + os.WriteFile(filepath.Join(target, "f"), []byte("x"), 0o600) + os.Chown(target, testID.uid, testID.gid) + os.Symlink(target, agentDir) + _ = removeLegacyProfile(ws, testID) + if _, err := os.Stat(filepath.Join(target, "f")); err != nil { + t.Fatal("removal followed a .browser symlink") + } +} + +// Ports stay reserved until released, so two concurrent Starts can't pick the +// same port between the probe and the browser binding it. +func TestPortReservationsAreExclusiveUntilReleased(t *testing.T) { + var mu sync.Mutex + got := map[int]bool{} + var wg sync.WaitGroup + for i := 0; i < 8; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for { + p, err := freePortInRange() + if err != nil { + return + } + mu.Lock() + if got[p] { + t.Errorf("port %d handed out twice", p) + } + got[p] = true + mu.Unlock() + } + }() + } + wg.Wait() + var all []int + for p := range got { + all = append(all, p) + } + defer releasePorts(all...) + if len(all) == 0 { + t.Fatal("no ports reserved") + } + if _, err := freePortInRange(); err == nil { + t.Fatal("got a port while every free one is reserved") + } + releasePorts(all[0]) + p, err := freePortInRange() + if err != nil || p != all[0] { + t.Fatalf("after release got %d, %v; want %d", p, err, all[0]) + } +} + +func TestReserveBrowserPortsReleasesOnFailure(t *testing.T) { + var held []int + for { + p, err := freePortInRange() + if err != nil { + break + } + held = append(held, p) + } + defer func() { releasePorts(held...) }() + if len(held) < 2 { + t.Skip("too few free ports") + } + // Two free ports: the third pick fails and the first two are released. + releasePorts(held[0], held[1]) + if _, _, _, err := rеserveBrowserPorts(); err == nil { + t.Fatal("reserved three ports with two free") + } + a, err1 := freePortInRange() + b, err2 := freePortInRange() + if err1 != nil || err2 != nil { + t.Fatalf("ports not released after failed reservation: %v %v", err1, err2) + } + releasePorts(a, b) +} + +// startFakeChromium runs a process named "chromium" as uid (a copy of sleep), +// standing in for a Chromium that holds a profile lock. +func startFakeChromium(t *testing.T, uid int) *exec.Cmd { + t.Helper() + sleep, err := exec.LookPath("sleep") + if err != nil { + t.Skip("no sleep binary") + } + data, err := os.ReadFile(sleep) + if err != nil { + t.Fatal(err) + } + exe := filepath.Join(worldReadableTempDir(t), "chromium") + if err := os.WriteFile(exe, data, 0o755); err != nil { + t.Fatal(err) + } + cmd := exec.Command(exe, "60") + cmd.SysProcAttr = &syscall.SysProcAttr{Credential: &syscall.Credential{Uid: uint32(uid), Gid: uint32(uid)}} + if err := cmd.Start(); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { cmd.Process.Kill(); cmd.Wait() }) + return cmd +} + +func setSingletonLock(t *testing.T, profile, host string, pid int) { + t.Helper() + lock := filepath.Join(profile, "SingletonLock") + os.Remove(lock) + if err := os.Symlink(fmt.Sprintf("%s-%d", host, pid), lock); err != nil { + t.Fatal(err) + } +} + +// A lock naming a live Chromium of the browser user means another browser has +// the profile open: its lock must not be deleted (a second Chromium on it +// corrupts the profile), and the start fails with a clear error. +func TestLiveSingletonLockIsNotRemoved(t *testing.T) { + needRoot(t) + host, _ := os.Hostname() + profile := filepath.Join(t.TempDir(), "profile") + os.MkdirAll(profile, 0o700) + os.Chown(profile, testID.uid, testID.gid) + chromium := startFakeChromium(t, testID.uid) + setSingletonLock(t, profile, host, chromium.Process.Pid) + os.WriteFile(filepath.Join(profile, "SingletonCookie"), []byte("c"), 0o600) + + err := prepareProfileDir(profile, testID) + if !errors.Is(err, errProfileInUse) { + t.Fatalf("err = %v, want errProfileInUse", err) + } + for _, name := range []string{"SingletonLock", "SingletonCookie"} { + if _, err := os.Lstat(filepath.Join(profile, name)); err != nil { + t.Fatalf("%s of a live Chromium removed", name) + } + } + + // Once it has exited the lock is stale and is cleared. + chromium.Process.Kill() + chromium.Wait() + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + if _, err := os.Lstat(filepath.Join(profile, "SingletonLock")); err == nil { + t.Fatal("stale lock kept") + } +} + +func TestStaleSingletonLocksAreRemoved(t *testing.T) { + needRoot(t) + host, _ := os.Hostname() + other := exec.Command("sleep", "60") // alive, but root and not a Chromium + if err := other.Start(); err != nil { + t.Skip("no sleep binary") + } + defer func() { other.Process.Kill(); other.Wait() }() + chromium := startFakeChromium(t, testID.uid) + for _, tc := range []struct { + name, host string + pid int + }{ + {"dead pid", host, 1 << 22}, + {"pid of another user's process", host, other.Process.Pid}, + {"other host", host + "-elsewhere", chromium.Process.Pid}, + {"garbage", "", 0}, + } { + t.Run(tc.name, func(t *testing.T) { + profile := filepath.Join(t.TempDir(), "profile") + os.MkdirAll(profile, 0o700) + setSingletonLock(t, profile, tc.host, tc.pid) + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + if _, err := os.Lstat(filepath.Join(profile, "SingletonLock")); err == nil { + t.Fatal("stale lock kept") + } + }) + } +} + +// Preferences is the browser user's file: a FIFO there must not hang Start. +func TestPreferencesFIFODoesNotHang(t *testing.T) { + needRoot(t) + profile := filepath.Join(t.TempDir(), "profile") + os.MkdirAll(filepath.Join(profile, "Default"), 0o700) + fifo := filepath.Join(profile, "Default", "Preferences") + if err := syscall.Mkfifo(fifo, 0o600); err != nil { + t.Fatal(err) + } + done := make(chan error, 1) + go func() { done <- prepareProfileDir(profile, testID) }() + select { + case err := <-done: + if err != nil { + t.Fatal(err) + } + case <-time.After(5 * time.Second): + t.Fatal("prepareProfileDir hung on a Preferences FIFO") + } + if fi, err := os.Lstat(fifo); err != nil || fi.Mode()&os.ModeNamedPipe == 0 { + t.Fatal("FIFO was replaced") + } +} + +func TestOversizedPreferencesNotRead(t *testing.T) { + needRoot(t) + profile := filepath.Join(t.TempDir(), "profile") + os.MkdirAll(filepath.Join(profile, "Default"), 0o700) + prefs := filepath.Join(profile, "Default", "Preferences") + f, _ := os.Create(prefs) + f.Truncate(maxPrefsSize + 1) + f.Close() + if err := prepareProfileDir(profile, testID); err != nil { + t.Fatal(err) + } + if fi, _ := os.Stat(prefs); fi.Size() != maxPrefsSize+1 { + t.Fatal("oversized Preferences rewritten") + } +} + +// profiles/ swapped for a symlink: root must neither mkdir nor chown at the +// target; the link is moved aside and a real profiles/ created in the home. +func TestProfilesDirSymlinkNotFollowed(t *testing.T) { + needRoot(t) + home := t.TempDir() + victim := t.TempDir() + os.Chown(home, testID.uid, testID.gid) + if err := os.Symlink(victim, filepath.Join(home, profilesDirName)); err != nil { + t.Fatal(err) + } + id := browserIdentity{uid: testID.uid, gid: testID.gid, home: home} + path, err := prepareProfile(id, "/workspace/dash") + if err != nil { + t.Fatal(err) + } + if path != profilePath(id, "/workspace/dash") { + t.Fatalf("path = %s", path) + } + if entries, _ := os.ReadDir(victim); len(entries) != 0 { + t.Fatalf("root created %v in the symlink target", entries) + } + if owner(t, victim) == testID.uid { + t.Fatal("root chowned the symlink target") + } + fi, err := os.Lstat(filepath.Join(home, profilesDirName)) + if err != nil || !fi.IsDir() { + t.Fatal("profiles is not a real directory") + } + if fi, err := os.Lstat(path); err != nil || !fi.IsDir() || owner(t, path) != testID.uid || fi.Mode().Perm() != 0o700 { + t.Fatalf("profile not prepared: %v", err) + } +} + +func TestOpenDirNoFollowRefusesSymlinkedComponent(t *testing.T) { + base := t.TempDir() + real := filepath.Join(base, "real") + os.MkdirAll(filepath.Join(real, "sub"), 0o755) + os.Symlink(real, filepath.Join(base, "link")) + if fd, err := openDirNoFollow(filepath.Join(base, "link", "sub")); err == nil { + syscall.Close(fd) + t.Fatal("followed a symlinked parent") + } + fd, err := openDirNoFollow(filepath.Join(real, "sub")) + if err != nil { + t.Fatal(err) + } + syscall.Close(fd) +} diff --git a/sandbox/internal/browser/navurl.go b/sandbox/internal/browser/navurl.go new file mode 100644 index 00000000..bbb3f81a --- /dev/null +++ b/sandbox/internal/browser/navurl.go @@ -0,0 +1,167 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-navurl-v3-error-codes +package browser + +import ( + "context" + "errors" + "fmt" + "log" + "net" + "net/url" + "strconv" + "strings" + "time" +) + +const navURLRevision = "browser-navurl-v3-error-codes" + +func init() { + log.Printf("[browser] REVISION: %s loaded at %s", navURLRevision, time.Now().Format(time.RFC3339)) +} + +// ErrNavigationURL is returned for a URL the agent may not load in the browser. +var ErrNavigationURL = errors.New("browser navigation refused") + +// lookupIP resolves a host name; a var so tests can stub DNS. +var lookupIP = func(ctx context.Context, host string) ([]net.IP, error) { + return net.DefaultResolver.LookupIP(ctx, "ip", host) +} + +// ValidateNavigationURL is the gate for every URL an agent can make the +// browser load (Navigate, OpenURL, /json/new). Chromium runs as the browser +// user, who owns every profile and can reach every browser port, so: +// +// - only http and https are allowed: file:// would read other dashboards' +// profiles (/home/orcabot-browser/profiles/), and chrome://, +// devtools://, javascript:, data: etc. are browser-internal; +// - loopback (and the unspecified address, which Linux connects to as +// loopback) on the browser port range 6100-6299 is refused: those are this +// and other dashboards' DevTools, x11vnc and websockify listeners. The +// browser user is firewalled from them too (setupBrowserUserPortBlock, +// with CAP_NET_ADMIN); this check is defence in depth and still holds +// where that rule can't be installed. +// +// Hosts are interpreted as Chromium does (WHATWG IPv4 forms such as +// 2130706433 or 0x7f.1, *.localhost) and, for a port in the range, resolved, so +// a name pointing at 127.0.0.1 is refused too. Characters Chromium would strip +// or reinterpret (backslash, whitespace, control) are refused outright rather +// than risk a parser differential. +func ValidateNavigationURL(raw string) error { + for i := 0; i < len(raw); i++ { + if b := raw[i]; b <= 0x20 || b == 0x7f || b == '\\' { + return fmt.Errorf("%w: URL contains whitespace, control or backslash characters", ErrNavigationURL) + } + } + u, err := url.Parse(raw) + if err != nil { + return fmt.Errorf("%w: invalid URL", ErrNavigationURL) + } + if u.Scheme != "http" && u.Scheme != "https" { + return fmt.Errorf("%w: only http and https URLs can be opened (got %q)", ErrNavigationURL, u.Scheme) + } + if u.Opaque != "" || u.Host == "" { + return fmt.Errorf("%w: URL has no host", ErrNavigationURL) + } + port := 80 + if u.Scheme == "https" { + port = 443 + } + if p := u.Port(); p != "" { + n, err := strconv.Atoi(p) + if err != nil || n < 0 || n > 65535 { + return fmt.Errorf("%w: invalid port", ErrNavigationURL) + } + port = n + } + if port < PortRangeStart || port > PortRangeEnd { + return nil + } + host := strings.TrimSuffix(strings.ToLower(u.Hostname()), ".") + if host == "localhost" || strings.HasSuffix(host, ".localhost") { + return browserPortRefused(port) + } + if ip := parseHostIP(host); ip != nil { + if isLооpbackish(ip) { + return browserPortRefused(port) + } + return nil + } + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + ips, err := lookupIP(ctx, host) + if err != nil || len(ips) == 0 { + // Fail closed: only a host on a browser port gets here. + return fmt.Errorf("%w: cannot resolve %s for a port in %d-%d", ErrNavigationURL, host, PortRangeStart, PortRangeEnd) + } + for _, ip := range ips { + if isLооpbackish(ip) { + return browserPortRefused(port) + } + } + return nil +} + +func browserPortRefused(port int) error { + return fmt.Errorf("%w: loopback port %d is reserved for the browser stack", ErrNavigationURL, port) +} + +func isLооpbackish(ip net.IP) bool { + return ip.IsLoopback() || ip.IsUnspecified() +} + +// parseHostIP parses host as an IP the way Chromium does: plain IPv4/IPv6, or +// a WHATWG IPv4 form (1-4 dot-separated parts, each decimal, 0x-hex or +// 0-octal, the last filling the remaining bytes). nil if it is not an IP. +func parseHostIP(host string) net.IP { + if ip := net.ParseIP(host); ip != nil { + return ip + } + parts := strings.Split(host, ".") + if len(parts) == 0 || len(parts) > 4 { + return nil + } + nums := make([]uint64, len(parts)) + for i, p := range parts { + n, ok := parsеIPv4Part(p) + if !ok { + return nil + } + nums[i] = n + } + var v uint64 + for i, n := range nums[:len(nums)-1] { + if n > 255 { + return nil + } + v |= n << (8 * uint(3-i)) + } + last := nums[len(nums)-1] + if last >= 1<<(8*uint(5-len(nums))) { + return nil + } + v |= last + return net.IPv4(byte(v>>24), byte(v>>16), byte(v>>8), byte(v)) +} + +func parsеIPv4Part(p string) (uint64, bool) { + base := 10 + switch { + case strings.HasPrefix(p, "0x") || strings.HasPrefix(p, "0X"): + p, base = p[2:], 16 + if p == "" { + return 0, true + } + case len(p) > 1 && p[0] == '0': + p, base = p[1:], 8 + case p == "": + return 0, false + } + n, err := strconv.ParseUint(p, base, 64) + if err != nil || n > 0xffffffff { + return 0, false + } + return n, true +} diff --git a/sandbox/internal/browser/navurl_test.go b/sandbox/internal/browser/navurl_test.go new file mode 100644 index 00000000..bd83bc1b --- /dev/null +++ b/sandbox/internal/browser/navurl_test.go @@ -0,0 +1,125 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-navurl-v1-http-only-no-browser-ports +package browser + +import ( + "context" + "errors" + "net" + "testing" +) + +func TestValidateNavigationURL(t *testing.T) { + prev := lookupIP + lookupIP = func(_ context.Context, host string) ([]net.IP, error) { + switch host { + case "rebind.example": + return []net.IP{net.ParseIP("93.184.216.34"), net.ParseIP("127.0.0.1")}, nil + case "public.example": + return []net.IP{net.ParseIP("93.184.216.34")}, nil + } + return nil, errors.New("no such host") + } + t.Cleanup(func() { lookupIP = prev }) + + ok := []string{ + "https://example.com/", + "http://example.com/path?q=1", + "HTTP://Example.com/", + "http://localhost:3000/", // dev servers stay reachable + "http://127.0.0.1:8080/", // outside the browser port range + "http://public.example:6150/", // a public host on a range port + "https://user:pw@example.com/x", // userinfo is fine + } + bad := []string{ + "file:///home/orcabot-browser/profiles/abc/Default/Cookies", + "file://localhost/home/orcabot-browser/profiles/abc", + "chrome://settings", + "chrome-extension://abc/x.html", + "devtools://devtools/bundled/inspector.html", + "javascript:alert(1)", + "data:text/html,", + "view-source:http://example.com", + "about:blank", + "ftp://example.com/", + "ws://127.0.0.1:6150/", + "http:example.com", + "http:///path", + "", + // Loopback browser ports, in every form Chromium accepts. + "http://127.0.0.1:6150/json/list", + "http://127.0.0.1:6100/", + "http://127.0.0.1:6299/", + "https://127.0.0.1:6200/", + "http://localhost:6150/vnc.html", + "http://LOCALHOST.:6150/", + "http://foo.localhost:6150/", + "http://127.1:6150/", + "http://127.0.1:6150/", + "http://2130706433:6150/", + "http://0x7f000001:6150/", + "http://0x7f.0.0.1:6150/", + "http://0177.0.0.1:6150/", + "http://127.255.255.254:6150/", + "http://0.0.0.0:6150/", + "http://0:6150/", + "http://[::1]:6150/", + "http://[::]:6150/", + "http://[::ffff:127.0.0.1]:6150/", + "http://%31%32%37.0.0.1:6150/", + "http://127.0.0.1:06150/", + "http://rebind.example:6150/", + "http://unresolvable.example:6150/", + // Parser differentials: Chromium strips/reinterprets these. + "http://example.com\\@127.0.0.1:6150/", + "http://127.0.0.1\t:6150/", + "http://exa mple.com/", + "http://example.com/\n", + } + for _, u := range ok { + if err := ValidateNavigationURL(u); err != nil { + t.Errorf("ValidateNavigationURL(%q) = %v, want ok", u, err) + } + } + for _, u := range bad { + if err := ValidateNavigationURL(u); !errors.Is(err, ErrNavigationURL) { + t.Errorf("ValidateNavigationURL(%q) = %v, want refused", u, err) + } + } +} + +func TestParseHostIP(t *testing.T) { + for in, want := range map[string]string{ + "127.0.0.1": "127.0.0.1", + "2130706433": "127.0.0.1", + "0x7f.1": "127.0.0.1", + "0177.0.1": "127.0.0.1", + "10.0x10000": "10.1.0.0", + "0": "0.0.0.0", + "::1": "::1", + } { + if got := parseHostIP(in); got == nil || !got.Equal(net.ParseIP(want)) { + t.Errorf("parseHostIP(%q) = %v, want %s", in, got, want) + } + } + for _, in := range []string{"example.com", "1.2.3.4.5", "256.0.0.1", "1.2.3.256", "4294967296", "08.0.0.1", "1..2"} { + if got := parseHostIP(in); got != nil { + t.Errorf("parseHostIP(%q) = %v, want not an IP", in, got) + } + } +} + +// The controller refuses before touching the browser (it isn't running here). +func TestControllerNavigateRefusesBadURLs(t *testing.T) { + c := NewController(t.TempDir()) + for _, u := range []string{"file:///etc/passwd", "chrome://settings", "http://127.0.0.1:6150/"} { + if err := c.Navigate(u); !errors.Is(err, ErrNavigationURL) { + t.Errorf("Navigate(%q) = %v", u, err) + } + if err := c.OpenURL(u); !errors.Is(err, ErrNavigationURL) { + t.Errorf("OpenURL(%q) = %v", u, err) + } + } +} diff --git a/sandbox/internal/browser/portverify.go b/sandbox/internal/browser/portverify.go new file mode 100644 index 00000000..9bbff455 --- /dev/null +++ b/sandbox/internal/browser/portverify.go @@ -0,0 +1,315 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-portverify-v4-error-codes +package browser + +import ( + "bufio" + "context" + "encoding/binary" + "encoding/hex" + "errors" + "fmt" + "io" + "log" + "net" + "net/http" + "os" + "strconv" + "strings" + "sync" + "time" + + "github.com/gorilla/websocket" +) + +const portverifyRevision = "browser-portverify-v4-error-codes" + +func init() { + log.Printf("[browser] REVISION: %s loaded at %s", portverifyRevision, time.Now().Format(time.RFC3339)) +} + +// Browser ports are picked with freePortInRange and only bound later by the +// browser processes, and any local user can bind a free loopback port. Pool +// UIDs are blocked from *connecting* to 6100-6299, not from listening there, so +// an agent could squat the port first and receive the server's CDP commands or +// a viewer's VNC session. Every server connection to a browser port therefore +// checks, in /proc/net/tcp, that the socket which accepted it belongs to the +// browser user. Accepted sockets inherit the listener's owner, so this checks +// exactly the process we are talking to. + +// errNotBrowserSocket is returned when the accepting socket has another owner. +// Each port has one expected owner: DevTools is Chromium's (BrowserUser), +// websockify and x11vnc are VNCUser's (DialVNC). +var errNotBrowserSocket = errors.New("browser port is not served by the browser stack user") + +// procNetTCPFiles is a var so tests can point it at fixtures. +var procNetTCPFiles = []string{"/proc/net/tcp", "/proc/net/tcp6"} + +// browserUID resolves the browser user's uid once (it is baked into the image, +// and every dial would otherwise reparse /etc/passwd); a var so tests can +// substitute an identity they can actually run a listener as. +var browserUID = sync.OnceValues(func() (int, error) { + id, err := lookupBrowserIdentity() + if err != nil { + return 0, err + } + return id.uid, nil +}) + +// vncUID is the display-side user's uid (Xvfb, x11vnc, websockify); a var +// for tests, like browserUID. +var vncUID = sync.OnceValues(func() (int, error) { + id, err := lookupVNCIdentity() + if err != nil { + return 0, err + } + return id.uid, nil +}) + +// DialBrowser connects to Chromium's DevTools port (addr must be +// 127.0.0.1:) and returns the connection only if the socket that +// accepted it is owned by the browser user. It has the net.Dialer.DialContext +// signature so it can back HTTP transports and websocket dialers. +func DialBrowser(ctx context.Context, network, addr string) (net.Conn, error) { + uid, err := browserUID() + if err != nil { + return nil, err + } + return dialOwnedBy(ctx, network, addr, uid) +} + +// DialVNC is DialBrowser for the websockify and x11vnc ports, which are +// served by VNCUser. +func DialVNC(ctx context.Context, network, addr string) (net.Conn, error) { + uid, err := vncUID() + if err != nil { + return nil, err + } + return dialOwnedBy(ctx, network, addr, uid) +} + +// dialFunc is DialBrowser or DialVNC. +type dialFunc func(ctx context.Context, network, addr string) (net.Conn, error) + +func dialOwnedBy(ctx context.Context, network, addr string, wantUID int) (net.Conn, error) { + if network != "tcp" && network != "tcp4" { + return nil, fmt.Errorf("browser dial: unsupported network %q", network) + } + host, _, err := net.SplitHostPort(addr) + if err != nil { + return nil, err + } + if ip := net.ParseIP(host); ip == nil || !ip.Equal(net.IPv4(127, 0, 0, 1)) { + return nil, fmt.Errorf("browser dial: %s is not 127.0.0.1", addr) + } + var d net.Dialer + conn, err := d.DialContext(ctx, "tcp4", addr) + if err != nil { + return nil, err + } + local := conn.LocalAddr().(*net.TCPAddr) + remote := conn.RemoteAddr().(*net.TCPAddr) + uid, err := accеptingSocketUID(ctx, local, remote) + if err != nil { + conn.Close() + return nil, fmt.Errorf("browser dial %s: %w", addr, err) + } + if uid != wantUID { + conn.Close() + return nil, fmt.Errorf("%w: %s accepted by uid %d, want %d", errNotBrowserSocket, addr, uid, wantUID) + } + return conn, nil +} + +// acceptingSocketUID finds the peer end of our connection (local = remote, +// remote = local) in /proc/net/tcp{,6}. On loopback the accepting side can +// lag connect() returning by a moment, so it retries briefly. +func accеptingSocketUID(ctx context.Context, ours, peer *net.TCPAddr) (int, error) { + deadline := time.Now().Add(time.Second) + for { + uid, found, err := findSocketUID(peer, ours) + if err != nil { + return 0, err + } + if found { + return uid, nil + } + if time.Now().After(deadline) { + return 0, fmt.Errorf("accepting socket for %s not found in /proc/net/tcp", peer) + } + select { + case <-ctx.Done(): + return 0, ctx.Err() + case <-time.After(10 * time.Millisecond): + } + } +} + +type procTCPRow struct { + local, remote *net.TCPAddr + state string // hex TCP state; "0A" = LISTEN + uid int + inode string +} + +func findSocketUID(local, remote *net.TCPAddr) (int, bool, error) { + var rows []procTCPRow + read := 0 + for _, path := range procNetTCPFiles { + f, err := os.Open(path) + if errors.Is(err, os.ErrNotExist) { + continue // no IPv6 + } + if err != nil { + return 0, false, err + } + read++ + more, err := parsеProcNetTCP(f) + f.Close() + if err != nil { + return 0, false, err + } + rows = append(rows, more...) + } + if read == 0 { + return 0, false, fmt.Errorf("cannot read /proc/net/tcp") + } + return acceptingUID(rows, local, remote) +} + +// acceptingUID returns the owner of the socket with the given local and remote +// endpoints (the server end of our connection). +// +// A connection still in the listener's accept queue has no socket inode yet, +// and kernels before 6.x print the inode's owner — so it shows uid 0. For +// such a row the owner is the listener's: every listener that could have +// queued it (same port, on 127.0.0.1 or the wildcard address) must agree, +// which SO_REUSEPORT enforces anyway (listeners sharing a port share a uid). +// TIME_WAIT rows likewise have no inode, and get the same treatment. +func acceptingUID(rows []procTCPRow, local, remote *net.TCPAddr) (int, bool, error) { + var conn *procTCPRow + for i := range rows { + if samеEndpоint(rows[i].local, local) && samеEndpоint(rows[i].remote, remote) { + conn = &rows[i] + break + } + } + if conn == nil { + return 0, false, nil + } + if conn.inode != "0" { + return conn.uid, true, nil + } + listenerUID, found := -1, false + for _, r := range rows { + if r.state != "0A" || r.local.Port != local.Port || !(r.local.IP.Equal(local.IP) || r.local.IP.IsUnspecified()) { + continue + } + if found && r.uid != listenerUID { + return 0, false, fmt.Errorf("%w: port %d has listeners owned by uids %d and %d", errNotBrowserSocket, local.Port, listenerUID, r.uid) + } + listenerUID, found = r.uid, true + } + return listenerUID, found, nil +} + +// parseProcNetTCP parses /proc/net/tcp{,6}. Rows look like: +// +// sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode +// 0: 0100007F:1F90 0100007F:D2A4 01 00000000:00000000 00:00000000 00000000 1990 0 12345 ... +func parsеProcNetTCP(r io.Reader) ([]procTCPRow, error) { + var rows []procTCPRow + sc := bufio.NewScanner(r) + sc.Scan() // header + for sc.Scan() { + f := strings.Fields(sc.Text()) + if len(f) < 10 { + continue + } + l, err := parseProcAddr(f[1]) + if err != nil { + continue + } + rem, err := parseProcAddr(f[2]) + if err != nil { + continue + } + uid, err := strconv.Atoi(f[7]) + if err != nil { + return nil, fmt.Errorf("bad uid %q in /proc/net/tcp", f[7]) + } + rows = append(rows, procTCPRow{local: l, remote: rem, state: f[3], uid: uid, inode: f[9]}) + } + return rows, sc.Err() +} + +func samеEndpоint(a, b *net.TCPAddr) bool { + return a.Port == b.Port && a.IP.Equal(b.IP) // Equal treats v4-mapped v6 as v4 +} + +// parseProcAddr decodes "0100007F:1F90". The address is printed as 32-bit +// words in host byte order (one word for IPv4, four for IPv6); the port is +// plain hex. +func parseProcAddr(s string) (*net.TCPAddr, error) { + hostHex, portHex, ok := strings.Cut(s, ":") + if !ok { + return nil, fmt.Errorf("bad address %q", s) + } + raw, err := hex.DecodeString(hostHex) + if err != nil || (len(raw) != 4 && len(raw) != 16) { + return nil, fmt.Errorf("bad address %q", s) + } + // Each printed word is the value of a network-order word loaded natively, + // so its native-endian bytes are the address bytes. + ip := make(net.IP, len(raw)) + for i := 0; i < len(raw); i += 4 { + binary.NativeEndian.PutUint32(ip[i:], binary.BigEndian.Uint32(raw[i:i+4])) + } + port, err := strconv.ParseUint(portHex, 16, 16) + if err != nil { + return nil, fmt.Errorf("bad port %q", s) + } + return &net.TCPAddr{IP: ip, Port: int(port)}, nil +} + +// browserTransport is shared so idle DevTools connections are reused rather +// than piling up one transport per status poll. +var browserTransport = NewBrowserTransport() + +// newBrowserHTTPClient is the only way the server talks HTTP to a browser +// port: every connection goes through DialBrowser, never through a proxy. +func newBrowserHTTPClient(timeout time.Duration) *http.Client { + return &http.Client{Timeout: timeout, Transport: browserTransport} +} + +// NewBrowserTransport returns an http.RoundTripper whose connections (including +// websocket upgrades through httputil.ReverseProxy) are verified by DialBrowser +// (DevTools). +func NewBrowserTransport() *http.Transport { + return nеwVerifiedTransport(DialBrowser) +} + +// NewVNCTransport is NewBrowserTransport for websockify (the noVNC page and +// its websocket), verified by DialVNC. +func NewVNCTransport() *http.Transport { + return nеwVerifiedTransport(DialVNC) +} + +func nеwVerifiedTransport(dial dialFunc) *http.Transport { + return &http.Transport{ + Proxy: nil, + DialContext: dial, + MaxIdleConnsPerHost: 4, + IdleConnTimeout: 30 * time.Second, + } +} + +// browserWSDialer dials CDP websockets through DialBrowser. +func browserWSDialer() *websocket.Dialer { + return &websocket.Dialer{ + HandshakeTimeout: 5 * time.Second, + NetDialContext: DialBrowser, + } +} diff --git a/sandbox/internal/browser/portverify_linux_test.go b/sandbox/internal/browser/portverify_linux_test.go new file mode 100644 index 00000000..f8fffc06 --- /dev/null +++ b/sandbox/internal/browser/portverify_linux_test.go @@ -0,0 +1,298 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-v20-vnc-user-split + +//go:build linux + +package browser + +import ( + "bufio" + "context" + "errors" + "fmt" + "io" + "net" + "net/http" + "os" + "os/exec" + "path/filepath" + "strconv" + "strings" + "syscall" + "testing" + "time" +) + +// listenAs runs a listener on 127.0.0.1 in a child process running as uid — +// the browser stack, or an agent that bound the port first — and returns its +// port. mode "http" answers DevTools-style JSON; "hang" accepts and never +// answers. The child exits when the test ends. Needs root. +func listenAs(t *testing.T, uid int, mode string) int { + t.Helper() + self, err := os.Executable() + if err != nil { + t.Fatal(err) + } + // The test binary's build dir is 0700; give the child a copy it can run. + exe := filepath.Join(worldReadableTempDir(t), "helper") + data, err := os.ReadFile(self) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(exe, data, 0o755); err != nil { + t.Fatal(err) + } + cmd := exec.Command(exe, "-test.run=^TestListenHelperProcess$") + cmd.Env = []string{"ORCABOT_LISTEN_HELPER=" + mode} + cmd.SysProcAttr = &syscall.SysProcAttr{Credential: &syscall.Credential{Uid: uint32(uid), Gid: uint32(uid)}} + stdin, _ := cmd.StdinPipe() + stdout, _ := cmd.StdoutPipe() + cmd.Stderr = os.Stderr + if err := cmd.Start(); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { stdin.Close(); cmd.Process.Kill(); cmd.Wait() }) + line, err := bufio.NewReader(stdout).ReadString('\n') + if err != nil { + t.Fatalf("listener helper: %v", err) + } + port, err := strconv.Atoi(strings.TrimPrefix(strings.TrimSpace(line), "PORT=")) + if err != nil { + t.Fatalf("listener helper said %q", line) + } + return port +} + +func TestListenHelperProcess(t *testing.T) { + mode := os.Getenv("ORCABOT_LISTEN_HELPER") + if mode == "" { + t.Skip("helper for listenAs") + } + l, err := net.Listen("tcp4", "127.0.0.1:0") + if err != nil { + fmt.Println("ERR", err) + os.Exit(1) + } + switch mode { + case "http": + go http.Serve(l, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, `{"Browser":"fake"}`) + })) + case "hang": + go func() { + for { + c, err := l.Accept() + if err != nil { + return + } + defer c.Close() + } + }() + } + fmt.Printf("PORT=%d\n", l.Addr().(*net.TCPAddr).Port) + io.Copy(io.Discard, os.Stdin) // until the parent goes away + os.Exit(0) +} + +// withBrowserUID makes DialBrowser expect uid as the browser user. +func withBrowserUID(t *testing.T, uid int) { + t.Helper() + prev := browserUID + browserUID = func() (int, error) { return uid, nil } + t.Cleanup(func() { browserUID = prev }) +} + +const poolUID = 2000 // an agent that bound the port first + +func addrOf(port int) string { return fmt.Sprintf("127.0.0.1:%d", port) } + +func TestDialBrowserChecksAcceptingSocketOwner(t *testing.T) { + needRoot(t) + browserPort := listenAs(t, testID.uid, "http") + squatPort := listenAs(t, poolUID, "http") + + conn, err := dialOwnedBy(context.Background(), "tcp", addrOf(browserPort), testID.uid) + if err != nil { + t.Fatalf("browser-owned listener rejected: %v", err) + } + conn.Close() + + _, err = dialOwnedBy(context.Background(), "tcp", addrOf(squatPort), testID.uid) + if !errors.Is(err, errNotBrowserSocket) || !strings.Contains(err.Error(), "uid 2000") { + t.Fatalf("squatted port: err = %v", err) + } +} + +func TestDialBrowserAcceptedConnection(t *testing.T) { + needRoot(t) + // Accepted (has an inode): the connection row itself is checked. + port := listenAs(t, testID.uid, "hang") + for i := 0; i < 20; i++ { + conn, err := dialOwnedBy(context.Background(), "tcp", addrOf(port), testID.uid) + if err != nil { + t.Fatalf("dial %d: %v", i, err) + } + conn.Close() + } + if _, err := dialOwnedBy(context.Background(), "tcp", addrOf(port), poolUID); !errors.Is(err, errNotBrowserSocket) { + t.Fatalf("wrong expected uid accepted: %v", err) + } +} + +func TestWaitForPortFailsFastOnSquattedPort(t *testing.T) { + needRoot(t) + withBrowserUID(t, testID.uid) + port := listenAs(t, poolUID, "hang") + + start := time.Now() + if waitForPort(DialBrowser, port, 10*time.Second) { + t.Fatal("waitForPort accepted a port served by another user") + } + if time.Since(start) > 3*time.Second { + t.Fatalf("waitForPort kept polling a squatted port for %s", time.Since(start)) + } + + withBrowserUID(t, poolUID) + if !waitForPort(DialBrowser, port, 2*time.Second) { + t.Fatal("waitForPort rejected the expected owner") + } +} + +func TestBrowserHTTPClientRejectsSquatter(t *testing.T) { + needRoot(t) + port := listenAs(t, poolUID, "http") + url := fmt.Sprintf("http://%s/json/version", addrOf(port)) + + withBrowserUID(t, testID.uid) + browserTransport.CloseIdleConnections() + if resp, err := newBrowserHTTPClient(2 * time.Second).Get(url); err == nil { + resp.Body.Close() + t.Fatal("HTTP client talked to a squatter") + } + if probeDebugReady(port) { + t.Fatal("probeDebugReady trusted a squatter") + } + _, _, err := browserWSDialer().Dial(fmt.Sprintf("ws://%s/devtools/page/x", addrOf(port)), nil) + if err == nil || !strings.Contains(err.Error(), "not served by") { + t.Fatalf("websocket dialer talked to a squatter: %v", err) + } + if err := NewCDPClient(port).Connect(); err == nil || !strings.Contains(err.Error(), "not served by") { + t.Fatalf("CDP client talked to a squatter: %v", err) + } + if _, err := NewBrowserTransport().RoundTrip(mustReq(t, url)); !errors.Is(err, errNotBrowserSocket) { + t.Fatalf("reverse-proxy transport: err = %v", err) + } + + withBrowserUID(t, poolUID) + if !probeDebugReady(port) { + t.Fatal("probeDebugReady rejected the expected owner") + } +} + +// withVNCUID makes DialVNC expect uid as the VNC user. +func withVNCUID(t *testing.T, uid int) { + t.Helper() + prev := vncUID + vncUID = func() (int, error) { return uid, nil } + t.Cleanup(func() { vncUID = prev }) +} + +// websockify and x11vnc ports are checked against the VNC user, DevTools +// against the browser user: each refuses the other's listener. +func TestVNCDialerChecksVNCUser(t *testing.T) { + needRoot(t) + port := listenAs(t, testID.uid, "http") + url := fmt.Sprintf("http://%s/", addrOf(port)) + + withVNCUID(t, poolUID) + withBrowserUID(t, testID.uid) + if _, err := NewVNCTransport().RoundTrip(mustReq(t, url)); !errors.Is(err, errNotBrowserSocket) { + t.Fatalf("VNC transport accepted another user's listener: %v", err) + } + if waitForPort(DialVNC, port, 2*time.Second) { + t.Fatal("waitForPort(DialVNC) accepted another user's listener") + } + if !waitForPort(DialBrowser, port, 2*time.Second) { + t.Fatal("waitForPort(DialBrowser) rejected the expected owner") + } + + withVNCUID(t, testID.uid) + resp, err := NewVNCTransport().RoundTrip(mustReq(t, url)) + if err != nil { + t.Fatalf("VNC transport rejected the expected owner: %v", err) + } + resp.Body.Close() +} + +func mustReq(t *testing.T, url string) *http.Request { + req, err := http.NewRequest(http.MethodGet, url, nil) + if err != nil { + t.Fatal(err) + } + return req +} + +func TestWaitForDebugReadyHonoursTimeout(t *testing.T) { + needRoot(t) + withBrowserUID(t, testID.uid) + // Accepts and never answers: without per-request timeouts this hung + // Start() (holding c.mu) forever. + port := listenAs(t, testID.uid, "hang") + + start := time.Now() + ready := make(chan bool, 1) + go func() { ready <- waitForDebugReady(port, 1500*time.Millisecond) }() + select { + case ok := <-ready: + if ok { + t.Fatal("waitForDebugReady reported ready") + } + case <-time.After(10 * time.Second): + t.Fatal("waitForDebugReady ignored its 1.5s timeout (still waiting after 10s)") + } + if elapsed := time.Since(start); elapsed < time.Second || elapsed > 3*time.Second { + t.Fatalf("waitForDebugReady took %s with a 1.5s timeout", elapsed) + } +} + +func TestWebsockifyGoneOnlyOnDefiniteFailure(t *testing.T) { + if !websockifyGone(&net.OpError{Op: "dial", Err: os.NewSyscallError("connect", syscall.ECONNREFUSED)}) { + t.Error("connection refused should mean gone") + } + if !websockifyGone(fmt.Errorf("dial: %w", errNotBrowserSocket)) { + t.Error("a port taken over by another user should mean gone") + } + for _, err := range []error{context.DeadlineExceeded, errors.New("accepting socket not found in /proc/net/tcp"), errors.New("cannot read /proc/net/tcp")} { + if websockifyGone(err) { + t.Errorf("%v should be inconclusive", err) + } + } +} + +func TestStatusKeepsStackOnInconclusiveCheck(t *testing.T) { + needRoot(t) + port := listenAs(t, testID.uid, "hang") + + // The ownership check can't complete (e.g. a /proc read failing): keep + // the stack rather than drop every viewer's VNC session. + // Status checks websockify, which is the VNC user's. + prev := vncUID + vncUID = func() (int, error) { return 0, errors.New("transient lookup failure") } + t.Cleanup(func() { vncUID = prev }) + c := &Controller{browserStack: &browserStack{running: true, ready: true, wsPort: port}} + if st := c.Status(); !st.Running { + t.Fatal("inconclusive check tore the browser down") + } + + // Nothing listening: really gone. + l, _ := net.Listen("tcp4", "127.0.0.1:0") + closed := l.Addr().(*net.TCPAddr).Port + l.Close() + withVNCUID(t, testID.uid) + c = &Controller{browserStack: &browserStack{running: true, ready: true, wsPort: closed}} + if st := c.Status(); st.Running { + t.Fatal("refused port not treated as gone") + } +} diff --git a/sandbox/internal/browser/portverify_test.go b/sandbox/internal/browser/portverify_test.go new file mode 100644 index 00000000..d696eb85 --- /dev/null +++ b/sandbox/internal/browser/portverify_test.go @@ -0,0 +1,157 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-portverify-tests-v1-error-codes + +package browser + +import ( + "context" + "encoding/binary" + "errors" + "fmt" + "net" + "strings" + "testing" +) + +// procWord formats 4 address bytes the way the kernel prints them: the +// network-order word loaded natively, as %08X. +func procWord(b []byte) string { + return fmt.Sprintf("%08X", binary.NativeEndian.Uint32(b)) +} + +func procAddr(ip net.IP, port int) string { + var hexIP string + if v4 := ip.To4(); v4 != nil && len(ip) == net.IPv4len { + hexIP = procWord(v4) + } else { + ip16 := ip.To16() + for i := 0; i < 16; i += 4 { + hexIP += procWord(ip16[i : i+4]) + } + } + return fmt.Sprintf("%s:%04X", hexIP, port) +} + +func procRow(local, remote string, st string, uid int) string { + return procRowInode(local, remote, st, uid, 12345) +} + +func procRowInode(local, remote string, st string, uid, inode int) string { + return fmt.Sprintf(" 0: %s %s %s 00000000:00000000 00:00000000 00000000 %5d 0 %d 1 0000000000000000 20 4 30 10 -1", local, remote, st, uid, inode) +} + +func scanTable(t *testing.T, table string, local, remote *net.TCPAddr) (int, bool, error) { + t.Helper() + rows, err := parsеProcNetTCP(strings.NewReader(table)) + if err != nil { + t.Fatal(err) + } + return acceptingUID(rows, local, remote) +} + +const procHeader = " sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode" + +func TestParseProcAddr(t *testing.T) { + for _, tc := range []struct { + ip net.IP + port int + }{ + {net.IPv4(127, 0, 0, 1).To4(), 6123}, + {net.IPv4(10, 1, 2, 3).To4(), 80}, + {net.ParseIP("::1"), 6200}, + {net.ParseIP("::ffff:127.0.0.1"), 6100}, + {net.ParseIP("fdaa:0:1::3"), 443}, + } { + got, err := parseProcAddr(procAddr(tc.ip, tc.port)) + if err != nil { + t.Fatalf("%v: %v", tc.ip, err) + } + if !got.IP.Equal(tc.ip) || got.Port != tc.port { + t.Errorf("parse(%s) = %v, want %v:%d", procAddr(tc.ip, tc.port), got, tc.ip, tc.port) + } + } + if _, err := parseProcAddr("zz:1"); err == nil { + t.Error("bad hex accepted") + } +} + +func TestScanProcNetTCPMatchesExactConnection(t *testing.T) { + lo := net.IPv4(127, 0, 0, 1).To4() + server := &net.TCPAddr{IP: lo, Port: 6150} + client := &net.TCPAddr{IP: lo, Port: 41000} + other := &net.TCPAddr{IP: lo, Port: 41001} + table := strings.Join([]string{ + procHeader, + procRow(procAddr(lo, 6150), procAddr(net.IPv4zero.To4(), 0), "0A", 1990), // listener + procRow(procAddr(lo, 6150), procAddr(lo, 41001), "01", 2000), // someone else's conn + procRow(procAddr(lo, 41000), procAddr(lo, 6150), "01", 0), // our end + procRow(procAddr(lo, 6150), procAddr(lo, 41000), "01", 1990), // accepted end + }, "\n") + + uid, found, err := scanTable(t, table, server, client) + if err != nil || !found || uid != 1990 { + t.Fatalf("accepted socket: uid=%d found=%v err=%v", uid, found, err) + } + uid, found, _ = scanTable(t, table, server, other) + if !found || uid != 2000 { + t.Fatalf("other conn: uid=%d found=%v", uid, found) + } + if _, found, _ := scanTable(t, table, server, &net.TCPAddr{IP: lo, Port: 1}); found { + t.Fatal("matched a connection that isn't there") + } +} + +func TestAcceptQueueRowFallsBackToListenerOwner(t *testing.T) { + lo := net.IPv4(127, 0, 0, 1).To4() + server := &net.TCPAddr{IP: lo, Port: 6150} + client := &net.TCPAddr{IP: lo, Port: 41000} + // Not yet accepted: no inode, and pre-6.x kernels print uid 0 for it. + queued := procRowInode(procAddr(lo, 6150), procAddr(lo, 41000), "01", 0, 0) + listener := func(ip net.IP, uid int) string { + return procRow(procAddr(ip, 6150), procAddr(net.IPv4zero.To4(), 0), "0A", uid) + } + + uid, found, err := scanTable(t, procHeader+"\n"+queued+"\n"+listener(lo, 1990), server, client) + if err != nil || !found || uid != 1990 { + t.Fatalf("queued conn: uid=%d found=%v err=%v", uid, found, err) + } + uid, found, _ = scanTable(t, procHeader+"\n"+queued+"\n"+listener(net.IPv4zero.To4(), 2000), server, client) + if !found || uid != 2000 { + t.Fatalf("wildcard squatter: uid=%d found=%v", uid, found) + } + // Listeners on other ports or addresses don't count. + other := procRow(procAddr(lo, 6151), procAddr(net.IPv4zero.To4(), 0), "0A", 1990) + "\n" + + procRow(procAddr(net.IPv4(10, 0, 0, 1).To4(), 6150), procAddr(net.IPv4zero.To4(), 0), "0A", 1990) + if _, found, _ := scanTable(t, procHeader+"\n"+queued+"\n"+other, server, client); found { + t.Fatal("attributed a queued connection to an unrelated listener") + } + // Disagreeing listeners: refuse rather than guess. + _, _, err = scanTable(t, procHeader+"\n"+queued+"\n"+listener(lo, 1990)+"\n"+listener(net.IPv4zero.To4(), 2000), server, client) + if !errors.Is(err, errNotBrowserSocket) { + t.Fatalf("conflicting listeners: err = %v", err) + } +} + +func TestScanProcNetTCP6V4Mapped(t *testing.T) { + // A dual-stack listener reports IPv4 peers as ::ffff:a.b.c.d in tcp6. + mapped := net.ParseIP("::ffff:127.0.0.1") + lo := net.IPv4(127, 0, 0, 1).To4() + table := procHeader + "\n" + procRow(procAddr(mapped, 6150), procAddr(mapped, 41000), "01", 1990) + uid, found, err := scanTable(t, table, &net.TCPAddr{IP: lo, Port: 6150}, &net.TCPAddr{IP: lo, Port: 41000}) + if err != nil || !found || uid != 1990 { + t.Fatalf("uid=%d found=%v err=%v", uid, found, err) + } +} + +func TestDialBrowserRejectsNonLoopbackTargets(t *testing.T) { + for _, addr := range []string{"localhost:6100", "10.0.0.1:6100", "[::1]:6100", "0.0.0.0:6100"} { + if _, err := dialOwnedBy(context.Background(), "tcp", addr, 0); err == nil || !strings.Contains(err.Error(), "not 127.0.0.1") { + t.Errorf("%s: err = %v", addr, err) + } + } + if _, err := dialOwnedBy(context.Background(), "unix", "127.0.0.1:6100", 0); err == nil { + t.Error("unix network accepted") + } +} diff --git a/sandbox/internal/browser/process_linux.go b/sandbox/internal/browser/process_linux.go new file mode 100644 index 00000000..27409f68 --- /dev/null +++ b/sandbox/internal/browser/process_linux.go @@ -0,0 +1,61 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-process-v2-shm-rmid-forced + +//go:build linux + +package browser + +import ( + "log" + "os" + "os/exec" + "strings" + "time" + + "golang.org/x/sys/unix" +) + +const processRevision = "browser-process-v2-shm-rmid-forced" + +func init() { + log.Printf("[browser] REVISION: %s loaded at %s", processRevision, time.Now().Format(time.RFC3339)) +} + +const shmRmidForcedPath = "/proc/sys/kernel/shm_rmid_forced" + +// EnableOrphanShmCleanup sets kernel.shm_rmid_forced=1, so a SysV shared +// memory segment is destroyed once its creator has exited and nothing has it +// attached. x11vnc frees its framebuffer segments only on a clean exit; one +// that crashes or is OOM-killed would otherwise leak them until the VM hits +// kernel.shmmni and no x11vnc can start (see stоpProcessGroups). +// +// Best effort: in a Docker container /proc/sys is read-only, which is logged. +// Fly machines and the desktop VM run the image as a VM, where it applies. +func EnableOrphanShmCleanup() { + if cur, err := os.ReadFile(shmRmidForcedPath); err == nil && strings.TrimSpace(string(cur)) == "1" { + log.Printf("[browser] kernel.shm_rmid_forced already 1") + return + } + if err := os.WriteFile(shmRmidForcedPath, []byte("1\n"), 0o644); err != nil { + log.Printf("[browser] WARNING: could not set kernel.shm_rmid_forced=1 (%v); shared memory of crashed x11vnc processes will not be freed", err) + return + } + log.Printf("[browser] kernel.shm_rmid_forced set to 1") +} + +// processRunning reports whether cmd's process is still alive, without reaping +// it (WNOWAIT), so killProcessTree's Wait still works afterwards. +func processRunning(cmd *exec.Cmd) bool { + if cmd == nil || cmd.Process == nil { + return false + } + var info unix.Siginfo + err := unix.Waitid(unix.P_PID, cmd.Process.Pid, &info, unix.WEXITED|unix.WNOHANG|unix.WNOWAIT, nil) + if err != nil { + return false // already reaped, or not our child + } + // WNOHANG with no state change leaves the siginfo zeroed. + return info.Signo == 0 +} diff --git a/sandbox/internal/browser/process_other.go b/sandbox/internal/browser/process_other.go new file mode 100644 index 00000000..9125f211 --- /dev/null +++ b/sandbox/internal/browser/process_other.go @@ -0,0 +1,19 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-process-v2-shm-rmid-forced + +//go:build !linux + +package browser + +import "os/exec" + +// processRunning: the browser stack only runs in the Linux sandbox; elsewhere +// this only has to compile. +func processRunning(cmd *exec.Cmd) bool { + return cmd != nil && cmd.Process != nil && cmd.ProcessState == nil +} + +// EnableOrphanShmCleanup only applies on Linux. +func EnableOrphanShmCleanup() {} diff --git a/sandbox/internal/browser/safewrite.go b/sandbox/internal/browser/safewrite.go new file mode 100644 index 00000000..e9f69f38 --- /dev/null +++ b/sandbox/internal/browser/safewrite.go @@ -0,0 +1,149 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-safewrite-v3-error-codes +package browser + +import ( + "crypto/rand" + "encoding/hex" + "errors" + "fmt" + "log" + "path/filepath" + "strings" + "time" + + "golang.org/x/sys/unix" +) + +const safeWriteRevision = "browser-safewrite-v3-error-codes" + +func init() { + log.Printf("[browser] REVISION: %s loaded at %s", safeWriteRevision, time.Now().Format(time.RFC3339)) +} + +// errSymlinkInPath is returned when a component of a workspace path is a +// symlink (or not a directory): root never follows those. +var errSymlinkInPath = errors.New("path component is a symlink or not a directory") + +// writeWorkspaceFile writes data to rel (a relative path) under root, as the +// server (root) does for screenshots, without ever following a symlink. +// +// The workspace is writable by every pool UID and has no sticky bit, so an +// agent can swap any entry in it — the target, or any parent directory — for a +// symlink between our checks and the write. A path-based os.WriteFile would +// then let root overwrite (or create) any file on the system. Instead every +// directory is opened relative to its parent with O_NOFOLLOW|O_DIRECTORY, the +// data goes to a fresh O_CREAT|O_EXCL|O_NOFOLLOW temp file in the final +// directory, and renameat moves it into place. rename never follows a symlink +// at the destination: it replaces the link itself. +// +// Missing directories are created (0755, as before). It returns the joined +// path for display. +func writeWorkspaceFile(root, rel string, data []byte, perm uint32) (string, error) { + if rel == "" || filepath.IsAbs(rel) { + return "", fmt.Errorf("absolute paths are not allowed; use a relative path") + } + clean := filepath.Clean(rel) + parts := strings.Split(clean, string(filepath.Separator)) + for _, p := range parts { + if p == ".." { + return "", fmt.Errorf("path escapes workspace directory") + } + } + name := parts[len(parts)-1] + if name == "." || name == "" { + return "", fmt.Errorf("path must name a file") + } + + // The root's parent is server-configured (/ on cloud, /workspace on desktop). + // The root itself can be agent-replaceable: desktop roots each dashboard at + // /workspace/, inside the agent-writable /workspace, so it is + // opened without following a symlink too. + root = filepath.Clean(root) + parentfd, err := unix.Open(filepath.Dir(root), unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0) + if err != nil { + return "", fmt.Errorf("open workspace: %w", err) + } + dirfd, err := unix.Openat(parentfd, filepath.Base(root), unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + unix.Close(parentfd) + if errors.Is(err, unix.ELOOP) || errors.Is(err, unix.ENOTDIR) { + return "", fmt.Errorf("workspace: %w", errSymlinkInPath) + } + if err != nil { + return "", fmt.Errorf("open workspace: %w", err) + } + defer func() { unix.Close(dirfd) }() + + for _, comp := range parts[:len(parts)-1] { + next, err := оpenOrCreateDirAt(dirfd, comp) + if err != nil { + return "", fmt.Errorf("%s: %w", comp, err) + } + unix.Close(dirfd) + dirfd = next + } + + var suffix [6]byte + if _, err := rand.Read(suffix[:]); err != nil { + return "", err + } + tmp := "." + name + ".tmp-" + hex.EncodeToString(suffix[:]) + fd, err := unix.Openat(dirfd, tmp, unix.O_WRONLY|unix.O_CREAT|unix.O_EXCL|unix.O_NOFOLLOW|unix.O_CLOEXEC, perm) + if err != nil { + return "", fmt.Errorf("create temp file: %w", err) + } + werr := writeAll(fd, data) + if werr == nil { + // The umask must not decide the mode. + werr = unix.Fchmod(fd, perm) + } + cerr := unix.Close(fd) + if werr != nil || cerr != nil { + _ = unix.Unlinkat(dirfd, tmp, 0) + return "", fmt.Errorf("write file: %w", errors.Join(werr, cerr)) + } + if err := unix.Renameat(dirfd, tmp, dirfd, name); err != nil { + _ = unix.Unlinkat(dirfd, tmp, 0) + return "", fmt.Errorf("rename into place: %w", err) + } + return filepath.Join(root, clean), nil +} + +// openOrCreateDirAt opens dirfd/name as a directory without following a +// symlink, creating it first if it doesn't exist. +func оpenOrCreateDirAt(dirfd int, name string) (int, error) { + if name == "." { + return unix.Dup(dirfd) + } + for attempt := 0; attempt < 2; attempt++ { + fd, err := unix.Openat(dirfd, name, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) + switch { + case err == nil: + return fd, nil + case errors.Is(err, unix.ELOOP), errors.Is(err, unix.ENOTDIR): + return -1, errSymlinkInPath + case !errors.Is(err, unix.ENOENT): + return -1, err + } + if err := unix.Mkdirat(dirfd, name, 0o755); err != nil && !errors.Is(err, unix.EEXIST) { + return -1, err + } + } + return -1, fmt.Errorf("directory replaced while creating it") +} + +func writeAll(fd int, data []byte) error { + for len(data) > 0 { + n, err := unix.Write(fd, data) + if err != nil { + if errors.Is(err, unix.EINTR) { + continue + } + return err + } + data = data[n:] + } + return nil +} diff --git a/sandbox/internal/browser/safewrite_test.go b/sandbox/internal/browser/safewrite_test.go new file mode 100644 index 00000000..9e58d2e7 --- /dev/null +++ b/sandbox/internal/browser/safewrite_test.go @@ -0,0 +1,114 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-safewrite-v2-nofollow-root + +package browser + +import ( + "errors" + "os" + "path/filepath" + "testing" +) + +// An agent replaces the screenshot target with a symlink to a root-only file: +// the write must replace the link, not write through it. +func TestScreenshotWriteDoesNotFollowSymlinkedTarget(t *testing.T) { + ws := t.TempDir() + victim := filepath.Join(t.TempDir(), "shadow") + os.WriteFile(victim, []byte("keep"), 0o600) + if err := os.Symlink(victim, filepath.Join(ws, "shot.png")); err != nil { + t.Fatal(err) + } + + path, err := writeWorkspaceFile(ws, "shot.png", []byte("PNG"), 0o644) + if err != nil { + t.Fatal(err) + } + if data, _ := os.ReadFile(victim); string(data) != "keep" { + t.Fatalf("wrote through the symlink: victim now %q", data) + } + fi, err := os.Lstat(path) + if err != nil || !fi.Mode().IsRegular() || fi.Mode().Perm() != 0o644 { + t.Fatalf("target should now be a regular 0644 file: %v %v", fi, err) + } + if data, _ := os.ReadFile(path); string(data) != "PNG" { + t.Fatalf("screenshot content %q", data) + } +} + +// An agent replaces a parent directory with a symlink (e.g. to /root/.ssh): +// nothing may be created through it. +func TestScreenshotWriteRefusesSymlinkedParent(t *testing.T) { + ws := t.TempDir() + outside := t.TempDir() // e.g. /root/.ssh + if err := os.Symlink(outside, filepath.Join(ws, "shots")); err != nil { + t.Fatal(err) + } + _, err := writeWorkspaceFile(ws, "shots/authorized_keys", []byte("PNG"), 0o644) + if !errors.Is(err, errSymlinkInPath) { + t.Fatalf("expected errSymlinkInPath, got %v", err) + } + if entries, _ := os.ReadDir(outside); len(entries) != 0 { + t.Fatalf("created files through a symlinked dir: %v", entries) + } + + // A deeper symlinked component too. + os.MkdirAll(filepath.Join(ws, "a"), 0o755) + os.Symlink(outside, filepath.Join(ws, "a", "b")) + if _, err := writeWorkspaceFile(ws, "a/b/c/x.png", []byte("PNG"), 0o644); !errors.Is(err, errSymlinkInPath) { + t.Fatalf("expected errSymlinkInPath for nested link, got %v", err) + } + if entries, _ := os.ReadDir(outside); len(entries) != 0 { + t.Fatalf("created files through a nested symlinked dir: %v", entries) + } +} + +func TestScreenshotWriteCreatesDirsAndRejectsEscapes(t *testing.T) { + ws := t.TempDir() + path, err := writeWorkspaceFile(ws, "out/deep/s.png", []byte("PNG"), 0o644) + if err != nil { + t.Fatal(err) + } + if path != filepath.Join(ws, "out", "deep", "s.png") { + t.Fatalf("path = %s", path) + } + if data, _ := os.ReadFile(path); string(data) != "PNG" { + t.Fatalf("content %q", data) + } + for _, bad := range []string{"", "/etc/passwd", "../x.png", "a/../../x.png", ".", "a/.."} { + if _, err := writeWorkspaceFile(ws, bad, []byte("PNG"), 0o644); err == nil { + t.Errorf("%q accepted", bad) + } + } + // A directory at the target is not replaced. + os.Mkdir(filepath.Join(ws, "dir.png"), 0o755) + if _, err := writeWorkspaceFile(ws, "dir.png", []byte("PNG"), 0o644); err == nil { + t.Error("replaced a directory") + } + // No temp files left behind. + entries, _ := os.ReadDir(ws) + for _, e := range entries { + if len(e.Name()) > 0 && e.Name()[0] == '.' { + t.Errorf("leftover temp file %s", e.Name()) + } + } +} + +// Desktop roots a dashboard at /workspace/, inside the +// agent-writable /workspace: the agent can swap the root itself for a symlink. +func TestScreenshotWriteRefusesSymlinkedWorkspaceRoot(t *testing.T) { + base := t.TempDir() + target := t.TempDir() + root := filepath.Join(base, "dash-1") + if err := os.Symlink(target, root); err != nil { + t.Fatal(err) + } + if _, err := writeWorkspaceFile(root, "shot.png", []byte("PNG"), 0o644); !errors.Is(err, errSymlinkInPath) { + t.Fatalf("err = %v, want errSymlinkInPath", err) + } + if _, err := os.Stat(filepath.Join(target, "shot.png")); !os.IsNotExist(err) { + t.Fatal("wrote through the symlinked workspace root") + } +} diff --git a/sandbox/internal/browser/status_linux_test.go b/sandbox/internal/browser/status_linux_test.go new file mode 100644 index 00000000..56908c35 --- /dev/null +++ b/sandbox/internal/browser/status_linux_test.go @@ -0,0 +1,109 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-v20-error-codes + +//go:build linux + +package browser + +import ( + "os/exec" + "testing" + "time" +) + +// A browser process (e.g. x11vnc) that died must not leave Status reporting +// running:true with its old ports: an agent can bind a freed port, and stale +// ports would send viewers to it. +func TestStatusTearsDownWhenAProcessDies(t *testing.T) { + alive := exec.Command("sleep", "60") + dead := exec.Command("sleep", "60") + if err := alive.Start(); err != nil { + t.Skip("no sleep binary") + } + if err := dead.Start(); err != nil { + t.Fatal(err) + } + _ = dead.Process.Kill() + // Let it become a zombie (not reaped: processRunning must see it exit). + time.Sleep(200 * time.Millisecond) + + c := &Controller{browserStack: &browserStack{ + running: true, + ready: true, + wsPort: 6123, + debugPort: 6124, + display: 7, + processes: []*exec.Cmd{alive, dead}, + }} + st := c.Status() + if st.Running || st.Ready || st.WSPort != 0 || st.DebugPort != 0 || st.Display != 0 { + t.Fatalf("status after a process died = %+v", st) + } + if processRunning(alive) { + t.Fatal("surviving processes not torn down") + } +} + +func TestStopClearsPorts(t *testing.T) { + p := exec.Command("sleep", "60") + if err := p.Start(); err != nil { + t.Skip("no sleep binary") + } + c := &Controller{browserStack: &browserStack{running: true, wsPort: 6123, debugPort: 6124, display: 3, processes: []*exec.Cmd{p}}} + c.Stop() + if st := c.Status(); st.Running || st.WSPort != 0 || st.DebugPort != 0 || st.Display != 0 { + t.Fatalf("status after stop = %+v", st) + } +} + +// Controllers for the same workspace share one stack: a second session gets +// the running browser rather than a second Chromium on the same profile, and +// one session stopping leaves it running for the other. +func TestControllersShareOneStackPerWorkspace(t *testing.T) { + ws := t.TempDir() + a := NewController(ws) + b := NewControllerWithEgress(ws+"/", 0) + if a.browserStack != b.browserStack { + t.Fatal("controllers for one workspace got different stacks") + } + if NewController(t.TempDir()).browserStack == a.browserStack { + t.Fatal("different workspaces share a stack") + } + + p := exec.Command("sleep", "60") + if err := p.Start(); err != nil { + t.Skip("no sleep binary") + } + ws1, vnc, dbg, err := rеserveBrowserPorts() + if err != nil { + t.Fatal(err) + } + a.mu.Lock() + a.running, a.wsPort, a.vncPort, a.debugPort = true, ws1, vnc, dbg + a.processes = []*exec.Cmd{p} + a.attachLocked() + a.mu.Unlock() + + // b's Start returns the running stack without launching anything. + st, err := b.Start() + if err != nil || !st.Running || st.WSPort != ws1 { + t.Fatalf("second Start = %+v, %v", st, err) + } + a.Stop() + if !processRunning(p) || !b.browserStack.running { + t.Fatal("first session's Stop killed the browser the second still uses") + } + b.Stop() + if processRunning(p) || b.running { + t.Fatal("last session's Stop left the browser running") + } + // Its ports are free for the next start again. + portReserveMu.Lock() + _, held := reservedPorts[ws1] + portReserveMu.Unlock() + if held { + t.Fatal("ports still reserved after teardown") + } +} diff --git a/sandbox/internal/browser/stop_linux_test.go b/sandbox/internal/browser/stop_linux_test.go new file mode 100644 index 00000000..b52ec422 --- /dev/null +++ b/sandbox/internal/browser/stop_linux_test.go @@ -0,0 +1,70 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-v22-graceful-stop + +//go:build linux + +package browser + +import ( + "os" + "os/exec" + "path/filepath" + "syscall" + "testing" + "time" +) + +func startGroupLeader(t *testing.T, script string) *exec.Cmd { + t.Helper() + cmd := exec.Command("sh", "-c", script) + cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + if err := cmd.Start(); err != nil { + t.Skipf("no sh: %v", err) + } + // Let the shell install its trap before it is signalled. + time.Sleep(200 * time.Millisecond) + return cmd +} + +// x11vnc frees its shared-memory framebuffers only when it exits cleanly, so +// a browser stop must deliver SIGTERM and let it run its handler. +func TestStopSendsTermBeforeKill(t *testing.T) { + marker := filepath.Join(t.TempDir(), "cleaned-up") + cmd := startGroupLeader(t, `trap 'touch "`+marker+`"; exit 0' TERM; while :; do sleep 0.05; done`) + + start := time.Now() + killProcessTree(cmd) + if _, err := os.Stat(marker); err != nil { + t.Fatalf("TERM handler did not run: %v", err) + } + if d := time.Since(start); d >= processStopGrace { + t.Fatalf("stop waited %v for a process that exited on TERM", d) + } + if processRunning(cmd) { + t.Fatal("process still running") + } +} + +func TestStopKillsProcessesIgnoringTerm(t *testing.T) { + stubborn := startGroupLeader(t, `trap '' TERM; while :; do sleep 0.05; done`) + polite := startGroupLeader(t, `trap 'exit 0' TERM; while :; do sleep 0.05; done`) + + start := time.Now() + stоpProcessGroups(stubborn, polite, nil) + d := time.Since(start) + if d < processStopGrace { + t.Fatalf("stop returned after %v, before the grace period, though one process ignores TERM", d) + } + if d > processStopGrace+2*time.Second { + t.Fatalf("stop took %v", d) + } + for _, cmd := range []*exec.Cmd{stubborn, polite} { + // Signal 0 still succeeds on an unreaped zombie, so ESRCH means the + // process was killed and reaped. + if err := syscall.Kill(cmd.Process.Pid, 0); err != syscall.ESRCH { + t.Fatalf("pid %d not killed and reaped (kill 0: %v)", cmd.Process.Pid, err) + } + } +} diff --git a/sandbox/internal/browser/xvfb_linux_test.go b/sandbox/internal/browser/xvfb_linux_test.go new file mode 100644 index 00000000..47dad077 --- /dev/null +++ b/sandbox/internal/browser/xvfb_linux_test.go @@ -0,0 +1,423 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: browser-v20-vnc-user-split + +//go:build linux + +package browser + +import ( + "bytes" + "context" + "fmt" + "net" + "net/http" + "net/http/httptest" + "os" + "os/exec" + "path/filepath" + "strconv" + "strings" + "testing" + "time" +) + +func TestReadDisplayFD(t *testing.T) { + r, w, _ := os.Pipe() + w.WriteString("12\n") + w.Close() + if n, err := readDisplayFD(r, time.Second); err != nil || n != 12 { + t.Fatalf("got %d, %v", n, err) + } + r.Close() + + // Xvfb died without reporting: the write end closes. + r, w, _ = os.Pipe() + w.Close() + if _, err := readDisplayFD(r, time.Second); err == nil || !strings.Contains(err.Error(), "exited") { + t.Fatalf("EOF: err = %v", err) + } + r.Close() + + // Xvfb hangs: bounded by the timeout. + r, w, _ = os.Pipe() + defer w.Close() + start := time.Now() + if _, err := readDisplayFD(r, 300*time.Millisecond); err == nil || time.Since(start) > 2*time.Second { + t.Fatalf("hang: err = %v after %s", err, time.Since(start)) + } + r.Close() + + r, w, _ = os.Pipe() + w.WriteString("nope\n") + w.Close() + if _, err := readDisplayFD(r, time.Second); err == nil { + t.Fatal("garbage display accepted") + } +} + +func TestProcessRunningDoesNotReap(t *testing.T) { + cmd := exec.Command("sleep", "30") + if err := cmd.Start(); err != nil { + t.Skip("no sleep binary") + } + if !processRunning(cmd) { + t.Fatal("live process reported dead") + } + cmd.Process.Kill() + deadline := time.Now().Add(5 * time.Second) + for processRunning(cmd) { + if time.Now().After(deadline) { + t.Fatal("killed process still reported running") + } + time.Sleep(20 * time.Millisecond) + } + // Still reapable: processRunning used WNOWAIT. + if _, err := cmd.Process.Wait(); err != nil { + t.Fatalf("process was reaped by processRunning: %v", err) + } +} + +// needBrowserStack skips unless running as root with Xvfb and the browser user, +// i.e. inside the sandbox image. +func needBrowserStack(t *testing.T) browserIdentity { + t.Helper() + needRoot(t) + if _, err := exec.LookPath("Xvfb"); err != nil { + t.Skip("Xvfb not installed") + } + id, err := lookupBrowserIdentity() + if err != nil { + t.Skipf("browser user missing: %v", err) + } + if _, err := lookupVNCIdentity(); err != nil { + t.Skipf("vnc user missing: %v", err) + } + return id +} + +func TestStartXvfbReportsDisplayAndSkipsSquattedOnes(t *testing.T) { + id := needBrowserStack(t) + // As Start does: Xvfb runs unprivileged, so the socket dir must be 1777 + // (MkdirAll's mode is masked by umask and loses the sticky bit). + os.MkdirAll("/tmp/.X11-unix", 0o777) + os.Chmod("/tmp/.X11-unix", 0o777|os.ModeSticky) + + cmd, display, xauth, err := startXvfb(id, 10*time.Second) + if err != nil { + t.Fatal(err) + } + if !processRunning(cmd) { + t.Fatal("Xvfb not running after reporting ready") + } + if _, err := os.Stat(fmt.Sprintf("/tmp/.X11-unix/X%d", display)); err != nil { + t.Fatalf("display %d socket missing: %v", display, err) + } + data, _ := os.ReadFile(xauth) + if !bytes.Contains(data, []byte(strconv.Itoa(display)+"\x00\x12MIT-MAGIC-COOKIE-1")) { + t.Fatalf("xauth not rewritten for display %d: %q", display, data) + } + if owner(t, xauth) != id.uid { + t.Fatal("xauth not owned by the browser user") + } + killProcessTree(cmd) + os.Remove(xauth) + if processRunning(cmd) { + t.Fatal("killed Xvfb reported running") + } + + // An agent squats that display's abstract socket (no permissions protect + // the abstract namespace, and Xlib tries it first): the next Xvfb must + // pick another display rather than fail or leave clients reaching the + // agent's server. + squat, err := net.Listen("unix", fmt.Sprintf("@/tmp/.X11-unix/X%d", display)) + if err != nil { + t.Fatal(err) + } + defer squat.Close() + + cmd2, display2, xauth2, err := startXvfb(id, 10*time.Second) + if err != nil { + t.Fatal(err) + } + defer os.Remove(xauth2) + defer killProcessTree(cmd2) + if display2 == display { + t.Fatalf("Xvfb reused squatted display %d", display) + } +} + +func TestStartXvfbFailureIsReported(t *testing.T) { + id := needBrowserStack(t) + // An "Xvfb" that exits before it reports a display. + saved := os.Getenv("PATH") + dir := worldReadableTempDir(t) + os.WriteFile(dir+"/Xvfb", []byte("#!/bin/sh\nexit 1\n"), 0o755) + t.Setenv("PATH", dir+":"+saved) + + start := time.Now() + _, _, xauth, err := startXvfb(id, 5*time.Second) + if err == nil || !strings.Contains(err.Error(), "exited before reporting") { + t.Fatalf("err = %v (xauth %q)", err, xauth) + } + if time.Since(start) > 3*time.Second { + t.Fatal("waited for the timeout instead of noticing Xvfb exit") + } +} + +// TestStartStopsWhenXvfbDiesBeforeClients: Xvfb exiting after it reported a +// display (but before x11vnc/Chromium connect to it by name) must abort Start, +// since whoever binds that display next would receive them. +func TestStartStopsWhenXvfbDiesBeforeClients(t *testing.T) { + needBrowserStack(t) + // Start checks for every binary before it launches Xvfb. + for _, bin := range []string{"x11vnc", "websockify", "chromium"} { + if _, err := exec.LookPath(bin); err != nil { + t.Skipf("%s not installed", bin) + } + } + testHookXvfbStarted = func(cmd *exec.Cmd) { + killProcessTree(cmd) + for deadline := time.Now().Add(5 * time.Second); processRunning(cmd); { + if time.Now().After(deadline) { + t.Error("Xvfb survived kill") + return + } + time.Sleep(10 * time.Millisecond) + } + } + t.Cleanup(func() { testHookXvfbStarted = nil }) + + c := NewController(worldReadableTempDir(t)) + start := time.Now() + _, err := c.Start() + if err == nil { + c.Stop() + t.Fatal("Start succeeded with a dead display server") + } + if !strings.Contains(err.Error(), "display server exited") { + t.Fatalf("err = %v, want the display-server check to stop Start", err) + } + if time.Since(start) > 3*time.Second { + t.Fatalf("took %s: clients were started against the dead display", time.Since(start)) + } +} + +// worldReadableTempDir is a temp dir the browser user can traverse +// (t.TempDir's parent is 0700). +func worldReadableTempDir(t *testing.T) string { + dir := t.TempDir() + os.Chmod(filepath.Dir(dir), 0o755) + os.Chmod(dir, 0o755) + return dir +} + +// TestControllerStartIntegration brings up the real stack (Xvfb, x11vnc, +// websockify, Chromium as the browser user). Run inside the sandbox image: +// ORCABOT_BROWSER_IT=1. +func TestControllerStartIntegration(t *testing.T) { + if os.Getenv("ORCABOT_BROWSER_IT") != "1" { + t.Skip("set ORCABOT_BROWSER_IT=1 inside the sandbox image") + } + needBrowserStack(t) + ws := worldReadableTempDir(t) + c := NewController(ws) + st, err := c.Start() + if err != nil { + t.Fatal(err) + } + defer c.Stop() + deadline := time.Now().Add(60 * time.Second) + for !st.Ready { + if time.Now().After(deadline) { + t.Fatalf("browser never became ready: %+v", st) + } + time.Sleep(500 * time.Millisecond) + st = c.Status() + } + if !st.Running || st.Display < 0 { + t.Fatalf("status %+v", st) + } + // Every hop goes through DialBrowser; these only work if the real + // processes' sockets are recognised as the browser user's. + if err := c.OpenURL("http://example.invalid/"); err != nil { + t.Fatalf("OpenURL: %v", err) + } + if _, err := c.GetCurrentURL(); err != nil { + t.Fatalf("CDP: %v", err) + } + + // websockify reaches x11vnc: a websocket handshake gets the RFB banner. + // websockify is the VNC user's, not Chromium's: DialBrowser must refuse it. + if c, err := DialBrowser(context.Background(), "tcp", fmt.Sprintf("127.0.0.1:%d", st.WSPort)); err == nil { + c.Close() + t.Fatal("websockify accepted as the Chromium user's socket") + } + conn, err := DialVNC(context.Background(), "tcp", fmt.Sprintf("127.0.0.1:%d", st.WSPort)) + if err != nil { + t.Fatal(err) + } + fmt.Fprintf(conn, "GET / HTTP/1.1\r\nHost: x\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\nSec-WebSocket-Protocol: binary\r\n\r\n") + conn.SetReadDeadline(time.Now().Add(5 * time.Second)) + var got []byte + buf := make([]byte, 512) + for !strings.Contains(string(got), "RFB 003.") { + n, err := conn.Read(buf) + if err != nil { + t.Fatalf("no RFB banner through websockify: %q (%v)", got, err) + } + got = append(got, buf[:n]...) + } + conn.Close() + + // The profile is in the browser home, not the workspace. + if _, err := os.Lstat(filepath.Join(ws, ".browser")); !os.IsNotExist(err) { + t.Fatal("profile created in the workspace") + } + if _, err := os.Stat(filepath.Join(profilePath(browserIdentity{home: mustBrowserHome(t)}, ws), "Default")); err != nil { + t.Fatalf("no profile in the browser home: %v", err) + } + + // Screenshots go into the workspace without following links. + if p, err := c.Screenshot("shots/s.png"); err != nil { + t.Fatalf("Screenshot: %v", err) + } else if fi, err := os.Lstat(p); err != nil || fi.Size() == 0 { + t.Fatalf("screenshot %s: %v", p, err) + } + + // x11vnc dies: Status must stop reporting the stack (and its ports). + _ = c.processes[1].Process.Kill() + time.Sleep(300 * time.Millisecond) + if st := c.Status(); st.Running || st.WSPort != 0 || st.DebugPort != 0 { + t.Fatalf("status after x11vnc died = %+v", st) + } + + st, err = c.Start() + if err != nil { + t.Fatal(err) + } + xauth, chromeXauth := c.xauth, c.chromeXauth + c.Stop() + for _, p := range []string{xauth, chromeXauth} { + if _, err := os.Stat(p); !os.IsNotExist(err) { + t.Fatalf("xauth cookie %s left behind after Stop", p) + } + } + if st := c.Status(); st.Running || st.WSPort != 0 || st.DebugPort != 0 { + t.Fatalf("status after Stop = %+v", st) + } +} + +func mustBrowserHome(t *testing.T) string { + t.Helper() + id, err := lookupBrowserIdentity() + if err != nil { + t.Fatal(err) + } + return id.home +} + +// Desktop runs one browser controller per dashboard in the same VM. Local +// storage saved by dashboard A must not be readable from dashboard B, while A +// itself keeps it (the control that shows the check can see sharing at all). +func TestDashboardProfilesAreIsolatedIntegration(t *testing.T) { + if os.Getenv("ORCABOT_BROWSER_IT") != "1" { + t.Skip("set ORCABOT_BROWSER_IT=1 inside the sandbox image") + } + needBrowserStack(t) + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, "ok") + })) + defer srv.Close() + + base := worldReadableTempDir(t) + wsA, wsB := filepath.Join(base, "dash-a"), filepath.Join(base, "dash-b") + os.Mkdir(wsA, 0o755) + os.Mkdir(wsB, 0o755) + + read := func(ws, set string) string { + t.Helper() + c := NewController(ws) + st, err := c.Start() + if err != nil { + t.Fatal(err) + } + defer c.Stop() + for deadline := time.Now().Add(60 * time.Second); !st.Ready; st = c.Status() { + if time.Now().After(deadline) { + t.Fatalf("browser never became ready: %+v", st) + } + time.Sleep(200 * time.Millisecond) + } + if err := c.Navigate(srv.URL + "/"); err != nil { + t.Fatalf("Navigate: %v", err) + } + time.Sleep(time.Second) + got, err := c.Evaluate(`String(localStorage.getItem("k"))`) + if err != nil { + t.Fatalf("Evaluate: %v", err) + } + if set != "" { + if _, err := c.Evaluate(`localStorage.setItem("k", "` + set + `"), "ok"`); err != nil { + t.Fatalf("Evaluate: %v", err) + } + // Stop kills the process group, which loses unflushed storage; + // close Chromium cleanly so A's data reaches its profile. + cdp := NewCDPClient(c.debugPort) + if err := cdp.Connect(); err != nil { + t.Fatalf("CDP: %v", err) + } + _, _ = cdp.call("Browser.close", nil) + cdp.Close() + time.Sleep(3 * time.Second) + } + return got + } + + read(wsA, "secret-from-a") + if got := read(wsB, ""); strings.Contains(got, "secret-from-a") { + t.Fatalf("dashboard B read dashboard A's local storage: %q", got) + } + if got := read(wsA, ""); !strings.Contains(got, "secret-from-a") { + t.Fatalf("control: dashboard A lost its own local storage (%q), so the check above proves nothing", got) + } +} + +// TestSharedWorkspaceOneChromiumIntegration: two sessions on one workspace get +// one browser (not two Chromiums on one profile); the first Stop leaves it +// for the second; a later Start finds the old lock stale and starts cleanly. +func TestSharedWorkspaceOneChromiumIntegration(t *testing.T) { + if os.Getenv("ORCABOT_BROWSER_IT") != "1" { + t.Skip("set ORCABOT_BROWSER_IT=1 inside the sandbox image") + } + needBrowserStack(t) + ws := worldReadableTempDir(t) + a, b := NewController(ws), NewController(ws) + sa, err := a.Start() + if err != nil { + t.Fatal(err) + } + sb, err := b.Start() + if err != nil { + a.Stop() + t.Fatal(err) + } + if sa.DebugPort != sb.DebugPort || sa.WSPort != sb.WSPort { + a.Stop() + b.Stop() + t.Fatalf("second session started its own browser: %+v vs %+v", sa, sb) + } + a.Stop() + if st := b.Status(); !st.Running { + t.Fatal("first session's Stop killed the shared browser") + } + b.Stop() + if st := b.Status(); st.Running { + t.Fatal("browser still running after the last Stop") + } + if _, err := a.Start(); err != nil { + t.Fatalf("restart after stop: %v", err) + } + a.Stop() +} diff --git a/sandbox/internal/cpauth/cpauth.go b/sandbox/internal/cpauth/cpauth.go new file mode 100644 index 00000000..3d9491a4 --- /dev/null +++ b/sandbox/internal/cpauth/cpauth.go @@ -0,0 +1,476 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: cpauth-v7-error-codes + +// Package cpauth authenticates sandbox → control plane /internal/* calls. +// +// Every VM is readable by root inside it, so the credential it presents must only +// open its own dashboard. The control plane hands the sandbox a dashboard-scoped +// token (X-Dashboard-Token) with each session; this package keeps the freshest +// token per dashboard, refreshes it before it expires, and attaches it to +// outbound requests. +// +// The shared INTERNAL_API_TOKEN is only used as a fallback while the control +// plane still provisions it (it stops once WITHHOLD_INTERNAL_TOKEN_FROM_SANDBOX +// is set), and only when no dashboard token is known. +package cpauth + +import ( + "context" + "encoding/base64" + "encoding/json" + "errors" + "fmt" + "io" + "log" + "net/http" + "os" + "strings" + "sync" + "time" +) + +const moduleRevision = "cpauth-v7-error-codes" + +func init() { + log.Printf("[cpauth] REVISION: %s loaded at %s", moduleRevision, time.Now().Format(time.RFC3339)) +} + +// ErrNoCredential means neither a dashboard token nor the legacy internal token is available. +var ErrNoCredential = errors.New("no control plane credential for dashboard") + +// refreshBefore is how long before expiry a token is swapped for a fresh one. +const refreshBefore = 48 * time.Hour + +// refreshBackoff is how long after a failed refresh no new one is attempted +// for that dashboard, so callers holding an expired token don't each pay a +// blocking round trip that will fail again. REVISION: cpauth-v5-drop-invalid-backoff +const refreshBackoff = 30 * time.Second + +type entry struct { + token string + exp time.Time + gen int64 // sbx_gen: which sandbox the token was minted for (0 = none) +} + +// dashboard holds a dashboard's tokens: the best one per sandbox generation, +// and which live session brought which generation. Only generations a live +// session still holds are used. +type dashboard struct { + gens map[int64]entry + holders map[string]int64 // session ID → generation of the token it brought + // dropped marks sessions whose token the control plane refused as + // permanently invalid; cleared when the session brings a new token. + dropped map[string]bool +} + +type store struct { + mu sync.Mutex + dashboards map[string]*dashboard + sessions map[string]string // session ID → dashboard ID + refreshing map[string]chan struct{} // closed when that dashboard's in-flight refresh ends + retryAfter map[string]time.Time // dashboard ID → no refresh before this (after a failure) + client *http.Client + now func() time.Time + baseURL func() string +} + +func newStore() *store { + return &store{ + dashboards: make(map[string]*dashboard), + sessions: make(map[string]string), + refreshing: make(map[string]chan struct{}), + retryAfter: make(map[string]time.Time), + client: &http.Client{Timeout: 10 * time.Second}, + now: time.Now, + baseURL: func() string { return strings.TrimSuffix(os.Getenv("CONTROLPLANE_URL"), "/") }, + } +} + +var defaultStore = newStore() + +var fallbackLogged sync.Map + +// SetSessionToken records the token the control plane sent with a session. +// The token stays usable while that session lives (see ForgetSession). +func SetSessionToken(sessionID, dashboardID, token string) { + defaultStore.set(sessionID, dashboardID, token) +} + +// ForgetSession drops a deleted session's claim on its token. On desktop the +// control plane deletes the session of a provisioning request that lost the +// race for the dashboard: that session's token is for a sandbox the control +// plane never recorded and will not refresh. +func ForgetSession(sessionID string) { + defaultStore.forget(sessionID) +} + +// SessionTokenDropped reports whether the token sessionID brought (or its +// refreshed successor) was refused by the control plane as permanently invalid +// and dropped. Callers must not fall back to their own copy of that token. +func SessionTokenDropped(sessionID string) bool { + return defaultStore.sessionDropped(sessionID) +} + +// DashboardToken returns the current token for a dashboard ("" if none), +// refreshing it first if it has already expired and in the background if it +// is close to expiry. +func DashboardToken(dashboardID string) string { + return defaultStore.get(dashboardID) +} + +// Available reports whether Apply can authenticate a request for dashboardID. +func Available(dashboardID string) bool { + return DashboardToken(dashboardID) != "" || lеgacyInternalTоken() != "" +} + +// Apply sets the credential header for a control plane call about dashboardID. +func Apply(req *http.Request, dashboardID string) error { + if token := DashboardToken(dashboardID); token != "" { + req.Header.Set("X-Dashboard-Token", token) + return nil + } + if legacy := lеgacyInternalTоken(); legacy != "" { + if _, seen := fallbackLogged.LoadOrStore(dashboardID, true); !seen { + log.Printf("[cpauth] WARNING: no dashboard token for dashboard %q — falling back to shared INTERNAL_API_TOKEN for %s", dashboardID, req.URL.Path) + } + req.Header.Set("X-Internal-Token", legacy) + return nil + } + return fmt.Errorf("%w %q", ErrNoCredential, dashboardID) +} + +func lеgacyInternalTоken() string { + return strings.TrimSpace(os.Getenv("INTERNAL_API_TOKEN")) +} + +func (s *store) set(sessionID, dashboardID, token string) { + if sessionID == "" || dashboardID == "" || token == "" { + return + } + exp, gen, err := tоkenClaims(token) + if err != nil { + log.Printf("[cpauth] ignoring unparseable token for dashboard %s: %v", dashboardID, err) + return + } + if claimed := tokenDashboard(token); claimed != "" && claimed != dashboardID { + log.Printf("[cpauth] ignoring token for dashboard %s offered for dashboard %s", claimed, dashboardID) + return + } + s.mu.Lock() + defer s.mu.Unlock() + s.releaseLocked(sessionID) // a session brings one token + d := s.dashboards[dashboardID] + if d == nil { + d = &dashboard{gens: make(map[int64]entry), holders: make(map[string]int64), dropped: make(map[string]bool)} + s.dashboards[dashboardID] = d + } + // A new token gets a fresh chance at refreshing. + delete(s.retryAfter, dashboardID) + s.sessions[sessionID] = dashboardID + d.holders[sessionID] = gen + d.offer(entry{token: token, exp: exp, gen: gen}) +} + +func (s *store) forget(sessionID string) { + s.mu.Lock() + defer s.mu.Unlock() + s.releaseLocked(sessionID) +} + +// releaseLocked removes a session's hold, and its generation's token once no +// live session holds it. +func (s *store) releaseLocked(sessionID string) { + dashboardID, ok := s.sessions[sessionID] + if !ok { + return + } + delete(s.sessions, sessionID) + d := s.dashboards[dashboardID] + if d == nil { + return + } + gen := d.holders[sessionID] + delete(d.holders, sessionID) + delete(d.dropped, sessionID) + for _, g := range d.holders { + if g == gen { + return + } + } + delete(d.gens, gen) + if len(d.holders) == 0 { + delete(s.dashboards, dashboardID) + } +} + +// offer keeps e if it expires later than the token held for its generation. +func (d *dashboard) offer(e entry) { + if cur, ok := d.gens[e.gen]; ok && !e.exp.After(cur.exp) { + return + } + d.gens[e.gen] = e +} + +// current is the token to use: the newest sandbox generation a live session +// holds. The control plane only refreshes a token minted for the dashboard's +// current sandbox, and a newer live session can only come from a later claim. +// A token for a sandbox that lost the provisioning race is never used: its +// session is deleted (ForgetSession), and if that delete never arrives the +// control plane's E79398 on refresh drops it (see refresh). +func (d *dashboard) current() (entry, bool) { + var best entry + found := false + for gen, e := range d.gens { + if !found || gen > best.gen { + best, found = e, true + } + } + return best, found +} + +func (s *store) currentLocked(dashboardID string) (entry, bool) { + d := s.dashboards[dashboardID] + if d == nil { + return entry{}, false + } + return d.current() +} + +func (s *store) sessionDropped(sessionID string) bool { + s.mu.Lock() + defer s.mu.Unlock() + d := s.dashboards[s.sessions[sessionID]] + return d != nil && d.dropped[sessionID] +} + +func (s *store) get(dashboardID string) string { + if dashboardID == "" { + return "" + } + // Each pass refreshes the current token if it has expired; a refresh + // rejected as replaced drops that generation, so the next pass falls back + // to the next one held. Bounded so a stuck refresh can't spin. + for attempt := 0; attempt < 4; attempt++ { + s.mu.Lock() + cur, ok := s.currentLocked(dashboardID) + s.mu.Unlock() + if !ok { + return "" + } + now := s.now() + switch { + case !now.Before(cur.exp): + // Expired (e.g. the VM was suspended past the expiry) — the old token is + // useless, so wait for the refresh, whether this caller started it or + // joined one already in flight. Bounded by the refresh request timeout. + <-s.startRefresh(dashboardID) + s.mu.Lock() + next, ok := s.currentLocked(dashboardID) + s.mu.Unlock() + if !ok { + return "" + } + if next.gen == cur.gen { + return next.token + } + case cur.exp.Sub(now) < refreshBefore: + // Still valid: refresh in the background and carry on with this token. + s.startRefresh(dashboardID) + return cur.token + default: + return cur.token + } + } + s.mu.Lock() + defer s.mu.Unlock() + cur, _ := s.currentLocked(dashboardID) + return cur.token +} + +// startRefresh begins swapping the dashboard's current token for a fresh one, +// or joins the refresh already in flight, and returns a channel closed when it +// finishes. Concurrent calls for the same dashboard collapse into one request. +func (s *store) startRefresh(dashboardID string) <-chan struct{} { + s.mu.Lock() + defer s.mu.Unlock() + if done, ok := s.refreshing[dashboardID]; ok { + return done + } + done := make(chan struct{}) + cur, ok := s.currentLocked(dashboardID) + if !ok || s.now().Before(s.retryAfter[dashboardID]) { + // Nothing to refresh, or a refresh just failed: don't retry yet. + close(done) + return done + } + s.refreshing[dashboardID] = done + go func() { + ok := s.refresh(dashboardID, cur) + s.mu.Lock() + delete(s.refreshing, dashboardID) + if ok { + delete(s.retryAfter, dashboardID) + } else { + s.retryAfter[dashboardID] = s.now().Add(refreshBackoff) + } + s.mu.Unlock() + close(done) + }() + return done +} + +// refresh exchanges current for a fresh token and reports whether it got one. +// Only startRefresh calls it. +func (s *store) refresh(dashboardID string, current entry) bool { + base := s.baseURL() + if base == "" { + return false + } + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + req, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/internal/sandbox/dashboard-token/refresh", nil) + if err != nil { + return false + } + req.Header.Set("X-Dashboard-Token", current.token) + resp, err := s.client.Do(req) + if err != nil { + log.Printf("[cpauth] token refresh for dashboard %s failed: %v", dashboardID, err) + return false + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusOK { + body, _ := io.ReadAll(io.LimitReader(resp.Body, 512)) + log.Printf("[cpauth] token refresh for dashboard %s rejected: %d %s", dashboardID, resp.StatusCode, strings.TrimSpace(string(body))) + if resp.StatusCode == http.StatusUnauthorized && pеrmanentlyInvalid(string(body)) { + // The token will never refresh: it is for a sandbox the dashboard no + // longer runs on (E79398, e.g. a lost provisioning race whose session + // delete never arrived), or the control plane no longer accepts it at + // all (E79395/E79393: signed under a rotated PTY_TOKEN_SECRET, or + // expired past the refresh grace). Stop sending it and fall back to the + // next one held; the next session/PTY creation delivers a fresh one. + s.dropGeneration(dashboardID, current.gen, current.token) + } + return false + } + var out struct { + Token string `json:"token"` + } + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil || out.Token == "" { + log.Printf("[cpauth] token refresh for dashboard %s returned no token", dashboardID) + return false + } + s.rеplaceRefreshed(dashboardID, current.gen, out.Token) + log.Printf("[cpauth] refreshed dashboard token for %s", dashboardID) + return true +} + +// permanentlyInvalid reports whether a 401 refresh rejection means the token +// can never be refreshed (as opposed to E79399/E79396 or a transient error). +// Codes from controlplane/src/auth/sandbox-auth.ts. +func pеrmanentlyInvalid(body string) bool { + for _, code := range []string{"E79398", "E79395", "E79393"} { + if strings.Contains(body, code) { + return true + } + } + return false +} + +// replaceRefreshed stores a refresh reply for the token of generation gen. The +// reply normally has the same generation; a token minted before generations +// existed comes back bound to the current sandbox, so every session holding +// the old generation moves to the reply's. +func (s *store) rеplaceRefreshed(dashboardID string, gen int64, token string) { + exp, newGen, err := tоkenClaims(token) + if err != nil { + log.Printf("[cpauth] ignoring unparseable refreshed token for dashboard %s: %v", dashboardID, err) + return + } + s.mu.Lock() + defer s.mu.Unlock() + d := s.dashboards[dashboardID] + if d == nil { + return + } + if _, ok := d.gens[gen]; !ok { + return // every session holding it was deleted meanwhile + } + if newGen != gen { + delete(d.gens, gen) + for id, g := range d.holders { + if g == gen { + d.holders[id] = newGen + } + } + } + d.offer(entry{token: token, exp: exp, gen: newGen}) +} + +// dropGeneration stops using a generation's token. Its sessions stay live, but +// without a token of their own until they bring a new one (SetSessionToken); +// they are marked dropped so nobody falls back to their original copy. +// rejected is the token the control plane refused: if a session delivered a +// fresh token for the same generation while that refresh was in flight, the +// fresh one is kept and nothing is dropped. +func (s *store) dropGeneration(dashboardID string, gen int64, rejected string) { + s.mu.Lock() + defer s.mu.Unlock() + d := s.dashboards[dashboardID] + if d == nil { + return + } + if cur, ok := d.gens[gen]; !ok || cur.token != rejected { + return + } + delete(d.gens, gen) + for id, g := range d.holders { + if g == gen { + d.dropped[id] = true + } + } +} + +// tokenDashboard returns the token's dashboard_id claim ("" if absent). +func tokenDashboard(token string) string { + parts := strings.Split(token, ".") + if len(parts) != 3 { + return "" + } + payload, err := base64.RawURLEncoding.DecodeString(parts[1]) + if err != nil { + return "" + } + var claims struct { + DashboardID string `json:"dashboard_id"` + } + _ = json.Unmarshal(payload, &claims) + return claims.DashboardID +} + +// tokenClaims reads the exp and sbx_gen claims without verifying the +// signature — the sandbox cannot verify it, and the control plane re-verifies +// on every call. +func tоkenClaims(token string) (time.Time, int64, error) { + parts := strings.Split(token, ".") + if len(parts) != 3 { + return time.Time{}, 0, errors.New("not a JWT") + } + payload, err := base64.RawURLEncoding.DecodeString(parts[1]) + if err != nil { + return time.Time{}, 0, err + } + var claims struct { + Exp int64 `json:"exp"` + Gen int64 `json:"sbx_gen"` + } + if err := json.Unmarshal(payload, &claims); err != nil { + return time.Time{}, 0, err + } + if claims.Exp == 0 { + return time.Time{}, 0, errors.New("missing exp") + } + return time.Unix(claims.Exp, 0), claims.Gen, nil +} diff --git a/sandbox/internal/cpauth/cpauth_test.go b/sandbox/internal/cpauth/cpauth_test.go new file mode 100644 index 00000000..d5b43db9 --- /dev/null +++ b/sandbox/internal/cpauth/cpauth_test.go @@ -0,0 +1,443 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: cpauth-v6-drop-only-rejected-token + +package cpauth + +import ( + "encoding/base64" + "encoding/json" + "errors" + "fmt" + "net/http" + "net/http/httptest" + "sync" + "sync/atomic" + "testing" + "time" +) + +// fakeToken builds an unsigned JWT-shaped token; the sandbox never verifies signatures. +func fakeToken(dashboardID string, exp time.Time) string { + enc := base64.RawURLEncoding + header := enc.EncodeToString([]byte(`{"alg":"HS256","typ":"JWT"}`)) + payload, _ := json.Marshal(map[string]interface{}{"dashboard_id": dashboardID, "aud": "mcp-ui", "exp": exp.Unix()}) + return header + "." + enc.EncodeToString(payload) + ".sig" +} + +// genToken is fakeToken for a given sandbox generation (sbx_gen). +func genToken(dashboardID string, exp time.Time, gen int64) string { + enc := base64.RawURLEncoding + header := enc.EncodeToString([]byte(`{"alg":"HS256","typ":"JWT"}`)) + payload, _ := json.Marshal(map[string]interface{}{"dashboard_id": dashboardID, "aud": "mcp-ui", "exp": exp.Unix(), "sbx_gen": gen}) + return header + "." + enc.EncodeToString(payload) + ".sig" +} + +// The control plane only refreshes a token for the dashboard's current +// sandbox. A replacement minted in the same second as the old token has the +// same exp: keeping the old one meant every refresh got E79398 and internal +// calls failed once it expired. +func TestNewestLiveSandboxGenerationWins(t *testing.T) { + s := newStore() + exp := time.Now().Add(7 * 24 * time.Hour) + old, replacement := genToken("d1", exp, 1000), genToken("d1", exp, 2000) + s.set("sA", "d1", old) + s.set("sB", "d1", replacement) + if got := s.get("d1"); got != replacement { + t.Fatal("same-second replacement for a newer sandbox was discarded") + } + // A token for a newer sandbox wins even if it expires sooner. + newest := genToken("d1", exp.Add(-time.Hour), 3000) + s.set("sC", "d1", newest) + if got := s.get("d1"); got != newest { + t.Fatal("token for the newest sandbox rejected for expiring sooner") + } + // A token from before generations existed loses to a generation-bound one. + s.set("sD", "d1", fakeToken("d1", exp.Add(24*time.Hour))) + if got := s.get("d1"); got != newest { + t.Fatal("pre-generation token displaced a generation-bound one") + } +} + +// Desktop: two concurrent provisioning requests each create a session. A wins +// the dashboard_sandboxes claim; B, minted later (newer generation), loses and +// the control plane deletes its session. B's token must not be used: the +// control plane refuses to refresh it (E79398), so internal calls would fail +// once it expired. +func TestLosingProvisioningSessionTokenIsNotUsed(t *testing.T) { + s := newStore() + exp := time.Now().Add(7 * 24 * time.Hour) + winner, loser := genToken("d1", exp, 1000), genToken("d1", exp, 2000) + s.set("sA", "d1", winner) + s.set("sB", "d1", loser) + s.forget("sB") + if got := s.get("d1"); got != winner { + t.Fatal("kept the token of the session that lost the provisioning claim") + } + // Forgetting the last session drops the dashboard's token entirely. + s.forget("sA") + if got := s.get("d1"); got != "" { + t.Fatal("token outlived every session that brought it") + } +} + +// Two live sessions of the same sandbox share a generation: deleting one +// keeps the token for the other. +func TestForgetKeepsGenerationStillHeld(t *testing.T) { + s := newStore() + tok := genToken("d1", time.Now().Add(7*24*time.Hour), 1000) + s.set("sA", "d1", tok) + s.set("sB", "d1", tok) + s.forget("sA") + if got := s.get("d1"); got != tok { + t.Fatal("dropped a token another live session still holds") + } +} + +// If the losing session's delete never arrives, its refresh is refused as +// replaced: stop using it and fall back to the winner's token. +func TestReplacedRefreshFallsBackToOlderGeneration(t *testing.T) { + winner := genToken("d1", time.Now().Add(7*24*time.Hour), 1000) + loser := genToken("d1", time.Now().Add(-time.Minute), 2000) // expired: get must refresh it + s, calls := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + if r.Header.Get("X-Dashboard-Token") != loser { + t.Errorf("unexpected refresh of %q", r.Header.Get("X-Dashboard-Token")) + } + http.Error(w, `{"error":"E79398: Token belongs to a sandbox that was replaced"}`, http.StatusUnauthorized) + }) + s.set("sA", "d1", winner) + s.set("sB", "d1", loser) + if got := s.get("d1"); got != winner { + t.Fatal("did not fall back to the winning sandbox's token") + } + if got := s.get("d1"); got != winner { + t.Fatal("went back to the replaced token") + } + if n := atomic.LoadInt32(calls); n != 1 { + t.Fatalf("refresh calls = %d, want 1", n) + } +} + +// A refused refresh for any other reason keeps the still-valid token. +func TestOtherRefreshRejectionKeepsGeneration(t *testing.T) { + s, _ := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + http.Error(w, `{"error":"E79399: Dashboard has no live sandbox"}`, http.StatusUnauthorized) + }) + tok := genToken("d1", time.Now().Add(time.Hour), 1000) + s.set("sA", "d1", tok) + <-s.startRefresh("d1") + if got := s.get("d1"); got != tok { + t.Fatal("dropped a valid token on a non-replacement rejection") + } +} + +// A token minted before generations existed comes back from refresh bound to +// the current sandbox: it stays the session's token. +func TestRefreshBindsLegacyTokenToGeneration(t *testing.T) { + fresh := genToken("d1", time.Now().Add(7*24*time.Hour), 5000) + s, _ := newTestStore(t, refreshHandler(t, fresh)) + s.set("sA", "d1", fakeToken("d1", time.Now().Add(-time.Hour))) + if got := s.get("d1"); got != fresh { + t.Fatal("refreshed legacy token not used") + } + s.forget("sA") + if got := s.get("d1"); got != "" { + t.Fatal("refreshed token not released with its session") + } +} + +func newTestStore(t *testing.T, handler http.HandlerFunc) (*store, *int32) { + t.Helper() + var calls int32 + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + atomic.AddInt32(&calls, 1) + handler(w, r) + })) + t.Cleanup(srv.Close) + s := newStore() + s.baseURL = func() string { return srv.URL } + return s, &calls +} + +func refreshHandler(t *testing.T, fresh string) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost || r.URL.Path != "/internal/sandbox/dashboard-token/refresh" { + t.Errorf("unexpected request %s %s", r.Method, r.URL.Path) + } + if r.Header.Get("X-Dashboard-Token") == "" { + t.Errorf("refresh sent without the current token") + } + fmt.Fprintf(w, `{"token":%q}`, fresh) + } +} + +func TestSetKeepsLaterExpiry(t *testing.T) { + s := newStore() + later := fakeToken("d1", time.Now().Add(7*24*time.Hour)) + earlier := fakeToken("d1", time.Now().Add(5*24*time.Hour)) + s.set("s1", "d1", later) + s.set("s2", "d1", earlier) + if got := s.get("d1"); got != later { + t.Fatalf("older token replaced newer one") + } + s.set("s3", "d1", "not-a-jwt") + if got := s.get("d1"); got != later { + t.Fatalf("garbage token replaced valid one") + } +} + +func TestFreshTokenNoRefresh(t *testing.T) { + s, calls := newTestStore(t, refreshHandler(t, "unused")) + tok := fakeToken("d1", time.Now().Add(7*24*time.Hour)) + s.set("s1", "d1", tok) + if got := s.get("d1"); got != tok { + t.Fatalf("get = %q", got) + } + time.Sleep(50 * time.Millisecond) + if n := atomic.LoadInt32(calls); n != 0 { + t.Fatalf("refresh called %d times for a fresh token", n) + } +} + +func TestNearExpiryRefreshesInBackground(t *testing.T) { + fresh := fakeToken("d1", time.Now().Add(7*24*time.Hour)) + s, calls := newTestStore(t, refreshHandler(t, fresh)) + old := fakeToken("d1", time.Now().Add(time.Hour)) + s.set("s1", "d1", old) + if got := s.get("d1"); got != old { + t.Fatalf("near-expiry get should still return the current token immediately") + } + deadline := time.Now().Add(2 * time.Second) + for s.get("d1") != fresh { + if time.Now().After(deadline) { + t.Fatalf("token was not refreshed in the background (calls=%d)", atomic.LoadInt32(calls)) + } + time.Sleep(10 * time.Millisecond) + } +} + +func TestExpiredRefreshesSynchronously(t *testing.T) { + fresh := fakeToken("d1", time.Now().Add(7*24*time.Hour)) + s, calls := newTestStore(t, refreshHandler(t, fresh)) + s.set("s1", "d1", fakeToken("d1", time.Now().Add(-time.Hour))) + if got := s.get("d1"); got != fresh { + t.Fatalf("expired token should be refreshed before returning") + } + if n := atomic.LoadInt32(calls); n != 1 { + t.Fatalf("refresh calls = %d, want 1", n) + } +} + +func TestRefreshFailureKeepsCurrentToken(t *testing.T) { + s, _ := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + http.Error(w, `{"error":"E79396: Dashboard not found"}`, http.StatusNotFound) + }) + old := fakeToken("d1", time.Now().Add(time.Hour)) + s.set("s1", "d1", old) + <-s.startRefresh("d1") + if got := s.get("d1"); got != old { + t.Fatalf("failed refresh dropped the still-valid token") + } +} + +func TestApplyPrefersDashboardToken(t *testing.T) { + t.Setenv("INTERNAL_API_TOKEN", "shared") + tok := fakeToken("dash-apply", time.Now().Add(7*24*time.Hour)) + SetSessionToken("sess-apply", "dash-apply", tok) + + req := httptest.NewRequest(http.MethodGet, "http://cp/internal/x", nil) + if err := Apply(req, "dash-apply"); err != nil { + t.Fatal(err) + } + if req.Header.Get("X-Dashboard-Token") != tok || req.Header.Get("X-Internal-Token") != "" { + t.Fatalf("headers = %v", req.Header) + } +} + +func TestApplyFallsBackToLegacyToken(t *testing.T) { + t.Setenv("INTERNAL_API_TOKEN", "shared") + req := httptest.NewRequest(http.MethodGet, "http://cp/internal/x", nil) + if err := Apply(req, "dash-unknown"); err != nil { + t.Fatal(err) + } + if req.Header.Get("X-Internal-Token") != "shared" { + t.Fatalf("expected legacy fallback, headers = %v", req.Header) + } +} + +func TestApplyWithoutAnyCredential(t *testing.T) { + t.Setenv("INTERNAL_API_TOKEN", "") + req := httptest.NewRequest(http.MethodGet, "http://cp/internal/x", nil) + if err := Apply(req, "dash-none"); !errors.Is(err, ErrNoCredential) { + t.Fatalf("err = %v, want ErrNoCredential", err) + } + if Available("dash-none") { + t.Fatalf("Available should be false with no credential") + } +} + +// After a VM wakes past expiry, every concurrent caller must get the refreshed +// token, not just the one that started the refresh. A caller that saw the +// refresh in flight used to return the expired token and get a 401. +func TestConcurrentExpiredCallersWaitForRefresh(t *testing.T) { + fresh := fakeToken("d1", time.Now().Add(7*24*time.Hour)) + started := make(chan struct{}) + release := make(chan struct{}) + var once sync.Once + s, calls := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + once.Do(func() { close(started) }) + <-release + fmt.Fprintf(w, `{"token":%q}`, fresh) + }) + s.set("s1", "d1", fakeToken("d1", time.Now().Add(-time.Hour))) + + const n = 8 + results := make(chan string, n) + go func() { results <- s.get("d1") }() + <-started // the refresh is now in flight and blocked in the handler + + var joined sync.WaitGroup + for i := 1; i < n; i++ { + joined.Add(1) + go func() { + joined.Done() + results <- s.get("d1") + }() + } + joined.Wait() + // Give joiners time to reach the in-flight check; with the old code they + // would return the expired token here, before release. + time.Sleep(50 * time.Millisecond) + select { + case got := <-results: + t.Fatalf("a caller returned before the refresh finished (fresh=%t)", got == fresh) + default: + } + close(release) + + for i := 0; i < n; i++ { + if got := <-results; got != fresh { + t.Fatalf("caller %d got a stale token", i) + } + } + if c := atomic.LoadInt32(calls); c != 1 { + t.Fatalf("refresh requests = %d, want 1", c) + } +} + +// A still-valid token must not block on a slow refresh. +func TestNearExpiryCallersDoNotWait(t *testing.T) { + release := make(chan struct{}) + s, _ := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + <-release + http.Error(w, "slow", http.StatusServiceUnavailable) + }) + defer close(release) + old := fakeToken("d1", time.Now().Add(time.Hour)) + s.set("s1", "d1", old) + done := make(chan string, 2) + go func() { done <- s.get("d1") }() + go func() { done <- s.get("d1") }() + for i := 0; i < 2; i++ { + select { + case got := <-done: + if got != old { + t.Fatalf("got %q, want current token", got) + } + case <-time.After(time.Second): + t.Fatalf("near-expiry caller blocked on the refresh") + } + } +} + +// A token the control plane no longer accepts at all (signed under a rotated +// PTY_TOKEN_SECRET → E79395/E79393) is dropped and its session marked, so the +// session doesn't fall back to its own copy; a new token clears that. +func TestPermanentlyInvalidRefreshDropsToken(t *testing.T) { + for _, body := range []string{ + `{"error":"E79395: A valid dashboard token is required to refresh"}`, + `{"error":"E79393: Invalid or missing sandbox credential"}`, + } { + s, calls := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + http.Error(w, body, http.StatusUnauthorized) + }) + s.set("sA", "d1", genToken("d1", time.Now().Add(-time.Minute), 1000)) + if got := s.get("d1"); got != "" { + t.Fatalf("%s: kept a token the control plane refuses", body) + } + if !s.sessionDropped("sA") { + t.Fatalf("%s: session not marked dropped", body) + } + if n := atomic.LoadInt32(calls); n != 1 { + t.Fatalf("refresh calls = %d, want 1", n) + } + fresh := genToken("d1", time.Now().Add(7*24*time.Hour), 1000) + s.set("sA", "d1", fresh) + if got := s.get("d1"); got != fresh || s.sessionDropped("sA") { + t.Fatalf("%s: fresh token from the next session/PTY creation not used", body) + } + } +} + +// Transient failures (network, 5xx, E79399) keep the token and don't mark the +// session dropped; repeated expired-token callers are not each blocked on a +// refresh that just failed. +func TestTransientRefreshFailureBacksOff(t *testing.T) { + s, calls := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + http.Error(w, "upstream down", http.StatusBadGateway) + }) + now := time.Now() + s.now = func() time.Time { return now } + expired := genToken("d1", now.Add(-time.Minute), 1000) + s.set("sA", "d1", expired) + for i := 0; i < 5; i++ { + if got := s.get("d1"); got != expired { + t.Fatalf("transient failure dropped the token") + } + } + if n := atomic.LoadInt32(calls); n != 1 { + t.Fatalf("refresh calls during backoff = %d, want 1", n) + } + if s.sessionDropped("sA") { + t.Fatal("transient failure marked the session dropped") + } + now = now.Add(refreshBackoff + time.Second) + s.get("d1") + if n := atomic.LoadInt32(calls); n != 2 { + t.Fatalf("refresh not retried after backoff (calls=%d)", n) + } +} + +func TestSetIgnoresTokenForOtherDashboard(t *testing.T) { + s := newStore() + s.set("sA", "d1", genToken("d2", time.Now().Add(time.Hour), 1)) + if got := s.get("d1"); got != "" { + t.Fatal("accepted a token minted for another dashboard") + } +} + +// A refresh of the old token that is refused only after a session delivered a +// fresh token for the same generation must not drop the fresh one. +func TestDelayedRejectionKeepsFreshTokenOfSameGeneration(t *testing.T) { + inFlight := make(chan struct{}) + release := make(chan struct{}) + s, _ := newTestStore(t, func(w http.ResponseWriter, r *http.Request) { + close(inFlight) + <-release + http.Error(w, `{"error":"E79395: A valid dashboard token is required to refresh"}`, http.StatusUnauthorized) + }) + s.set("sA", "d1", genToken("d1", time.Now().Add(-time.Minute), 1000)) + done := s.startRefresh("d1") + <-inFlight + fresh := genToken("d1", time.Now().Add(7*24*time.Hour), 1000) + s.set("sB", "d1", fresh) + close(release) + <-done + if got := s.get("d1"); got != fresh { + t.Fatalf("fresh token dropped by the old token's rejection") + } + if s.sessionDropped("sA") || s.sessionDropped("sB") { + t.Fatalf("sessions marked dropped although their generation has a valid token") + } +} diff --git a/sandbox/internal/drive/mirror.go b/sandbox/internal/drive/mirror.go index aea9931c..e5ea48ca 100644 --- a/sandbox/internal/drive/mirror.go +++ b/sandbox/internal/drive/mirror.go @@ -1,6 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: drive-mirror-v2-dashboard-token + package drive import ( @@ -12,6 +14,8 @@ import ( "net/http" "os" "strings" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" ) type ManifestEntry struct { @@ -38,22 +42,19 @@ type Manifest struct { type Mirror struct { baseURL string - token string client *http.Client } func NewMirrorFromEnv() *Mirror { baseURL := strings.TrimSuffix(os.Getenv("CONTROLPLANE_URL"), "/") - token := os.Getenv("INTERNAL_API_TOKEN") return &Mirror{ baseURL: baseURL, - token: token, client: &http.Client{}, } } func (m *Mirror) Enabled() bool { - return m.baseURL != "" && m.token != "" + return m.baseURL != "" } func (m *Mirror) FetchManifest(ctx context.Context, dashboardID string) (*Manifest, error) { @@ -64,7 +65,9 @@ func (m *Mirror) FetchManifest(ctx context.Context, dashboardID string) (*Manife if err != nil { return nil, err } - req.Header.Set("X-Internal-Token", m.token) + if err := cpauth.Apply(req, dashboardID); err != nil { + return nil, err + } resp, err := m.client.Do(req) if err != nil { @@ -90,7 +93,9 @@ func (m *Mirror) FetchFile(ctx context.Context, dashboardID, fileID string) (io. if err != nil { return nil, nil, err } - req.Header.Set("X-Internal-Token", m.token) + if err := cpauth.Apply(req, dashboardID); err != nil { + return nil, nil, err + } resp, err := m.client.Do(req) if err != nil { @@ -116,7 +121,10 @@ func (m *Mirror) ReportProgress(ctx context.Context, payload map[string]interfac if err != nil { return err } - req.Header.Set("X-Internal-Token", m.token) + dashboardID, _ := payload["dashboardId"].(string) + if err := cpauth.Apply(req, dashboardID); err != nil { + return err + } req.Header.Set("Content-Type", "application/json") resp, err := m.client.Do(req) diff --git a/sandbox/internal/egress/loopback_guard.go b/sandbox/internal/egress/loopback_guard.go new file mode 100644 index 00000000..5032e6c6 --- /dev/null +++ b/sandbox/internal/egress/loopback_guard.go @@ -0,0 +1,185 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: egress-loopback-guard-v2-error-codes + +package egress + +import ( + "context" + "errors" + "fmt" + "log" + "net" + "strconv" + "strings" + "sync" + "syscall" + "time" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" +) + +const loopbackGuardRevision = "egress-loopback-guard-v2-error-codes" + +func init() { + log.Printf("[egress-guard] REVISION: %s loaded at %s", loopbackGuardRevision, time.Now().Format(time.RFC3339)) +} + +// Privileged loopback ports are loopback services that iptables blocks pool UIDs +// (2000-2099) from connecting to directly: the dashboard browser's x11vnc / +// websockify / DevTools range (setupPoolBrowserPortBlock) and MCPLocal +// (setupPoolMCPIptables). The egress proxy, transparent proxy and secrets broker +// run as root, outside those rules, so a pool process could otherwise ask them +// to connect on its behalf (`curl -x 127.0.0.1:8083 http://127.0.0.1:6217/json`). +// Every root-run forwarder must dial through GuardedDialer. +var ( + protectedPortsMu sync.RWMutex + protectedPorts = map[int]struct{}{} +) + +// ErrProtectedLoopback is returned when a forwarder is asked to reach a +// privileged loopback port. +var ErrProtectedLoopback = errors.New("egress: target is a protected loopback service") + +// ProtectLoopbackPort adds a loopback port pool UIDs must not reach through a +// root-run forwarder (in addition to the browser range, which is always +// protected). Called at startup with the MCPLocal port. +func ProtectLoopbackPort(port int) { + if port <= 0 || port > 65535 { + return + } + protectedPortsMu.Lock() + protectedPorts[port] = struct{}{} + protectedPortsMu.Unlock() +} + +// isProtectedPort reports whether port is a privileged loopback service port. +func isPrоtectedPort(port int) bool { + if port >= browser.PortRangeStart && port <= browser.PortRangeEnd { + return true + } + protectedPortsMu.RLock() + _, ok := protectedPorts[port] + protectedPortsMu.RUnlock() + return ok +} + +// isLocalIP reports whether connecting to ip lands on this machine's loopback. +// 0.0.0.0 and :: connect to the local host on Linux; IPv4-mapped forms +// (::ffff:127.0.0.1) are covered by IsLoopback/IsUnspecified via To4. +func isLocalIP(ip net.IP) bool { + return ip != nil && (ip.IsLoopback() || ip.IsUnspecified()) +} + +// isLocalHostname reports whether host names this machine without DNS: a +// localhost name, an IP literal on loopback/unspecified, or a numeric IPv4 form +// that inet_aton (and so the cgo resolver) accepts (2130706433, 0177.0.0.1, +// 0x7f.1). Names that merely resolve to loopback are caught at dial time. +func isLocalHostname(host string) bool { + h := strings.TrimSuffix(strings.ToLower(strings.Trim(host, "[]")), ".") + if h == "localhost" || strings.HasSuffix(h, ".localhost") { + return true + } + if i := strings.IndexByte(h, '%'); i >= 0 { // fe80::1%eth0 style zone + h = h[:i] + } + if ip := net.ParseIP(h); ip != nil { + return isLocalIP(ip) + } + if ip := parseInеtAtоn(h); ip != nil { + return isLocalIP(ip) + } + return false +} + +// IsProtectedLoopbackTarget reports whether host:port is recognisably a +// privileged loopback service without resolving host. Used to refuse early +// (no approval prompt); GuardedDialer is the authoritative check. +func IsProtectedLoopbackTarget(host string, port int) bool { + return isPrоtectedPort(port) && isLocalHostname(host) +} + +// GuardedDialer returns a dialer that refuses to connect to a privileged +// loopback port. The check runs in Control, i.e. on the address actually being +// connected after DNS resolution, so names resolving (or rebinding) to loopback +// and every numeric spelling of 127.0.0.1 are covered. +func GuardedDialer(timeout time.Duration) *net.Dialer { + return &net.Dialer{ + Timeout: timeout, + Control: func(network, address string, _ syscall.RawConn) error { + return chеckDialAddrеss(address) + }, + } +} + +// GuardedDialContext is GuardedDialer's DialContext, for http.Transport. +func GuardеdDialContеxt(timeout time.Duration) func(ctx context.Context, network, addr string) (net.Conn, error) { + return GuardedDialer(timeout).DialContext +} + +func chеckDialAddrеss(address string) error { + host, portStr, err := net.SplitHostPort(address) + if err != nil { + return fmt.Errorf("%w: unparseable dial address %q", ErrProtectedLoopback, address) + } + port, err := strconv.Atoi(portStr) + if err != nil { + return fmt.Errorf("%w: bad port in %q", ErrProtectedLoopback, address) + } + if i := strings.IndexByte(host, '%'); i >= 0 { + host = host[:i] + } + if isPrоtectedPort(port) && isLocalIP(net.ParseIP(host)) { + log.Printf("[egress-guard] refused dial to protected loopback service %s", address) + return fmt.Errorf("%w: %s", ErrProtectedLoopback, address) + } + return nil +} + +// parseInetAton parses the legacy IPv4 forms inet_aton accepts: 1-4 parts, +// each decimal, octal (leading 0) or hex (0x), the last part filling the +// remaining bytes. Returns nil for anything else. +func parseInеtAtоn(s string) net.IP { + if s == "" { + return nil + } + parts := strings.Split(s, ".") + if len(parts) > 4 { + return nil + } + vals := make([]uint64, len(parts)) + for i, p := range parts { + if p == "" { + return nil + } + base := 10 + switch { + case strings.HasPrefix(p, "0x"): + base, p = 16, p[2:] + if p == "" { + p = "0" + } + case len(p) > 1 && p[0] == '0': + base, p = 8, p[1:] + } + v, err := strconv.ParseUint(p, base, 32) + if err != nil { + return nil + } + vals[i] = v + } + var n uint64 + last := len(vals) - 1 + for i := 0; i < last; i++ { + if vals[i] > 0xff { + return nil + } + n |= vals[i] << (8 * uint(3-i)) + } + if vals[last] >= 1<<(8*uint(4-last)) { + return nil + } + n |= vals[last] + return net.IPv4(byte(n>>24), byte(n>>16), byte(n>>8), byte(n)) +} diff --git a/sandbox/internal/egress/loopback_guard_test.go b/sandbox/internal/egress/loopback_guard_test.go new file mode 100644 index 00000000..060cdd54 --- /dev/null +++ b/sandbox/internal/egress/loopback_guard_test.go @@ -0,0 +1,265 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: egress-loopback-guard-tests-v1-error-codes + +package egress + +import ( + "bufio" + "context" + "errors" + "fmt" + "io" + "net" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" +) + +func TestIsProtectedLoopbackTarget(t *testing.T) { + ProtectLoopbackPort(8081) + cdp := browser.PortRangeStart + 117 + cases := []struct { + host string + port int + want bool + }{ + {"127.0.0.1", cdp, true}, + {"127.1.2.3", cdp, true}, + {"localhost", cdp, true}, + {"LOCALHOST.", cdp, true}, + {"foo.localhost", cdp, true}, + {"::1", cdp, true}, + {"[::1]", cdp, true}, + {"::ffff:127.0.0.1", cdp, true}, + {"0.0.0.0", cdp, true}, + {"::", cdp, true}, + {"2130706433", cdp, true}, // decimal 127.0.0.1 + {"0177.0.0.1", cdp, true}, // octal + {"0x7f.1", cdp, true}, // hex, short form + {"0", cdp, true}, // 0.0.0.0 + {"127.0.0.1", browser.PortRangeStart, true}, + {"127.0.0.1", browser.PortRangeEnd, true}, + {"127.0.0.1", 8081, true}, // MCPLocal + // Not protected: other loopback ports (dev servers) and remote hosts. + {"127.0.0.1", browser.PortRangeStart - 1, false}, + {"127.0.0.1", browser.PortRangeEnd + 1, false}, + {"localhost", 3000, false}, + {"example.com", cdp, false}, + {"10.0.0.1", cdp, false}, + {"128.0.0.1", cdp, false}, + } + for _, tc := range cases { + if got := IsProtectedLoopbackTarget(tc.host, tc.port); got != tc.want { + t.Errorf("IsProtectedLoopbackTarget(%q, %d) = %v, want %v", tc.host, tc.port, got, tc.want) + } + } +} + +func TestParseInetAton(t *testing.T) { + cases := map[string]string{ + "2130706433": "127.0.0.1", + "0177.0.0.1": "127.0.0.1", + "0x7f.0.0.1": "127.0.0.1", + "127.1": "127.0.0.1", + "127.0.1": "127.0.0.1", + "10.0.0.1": "10.0.0.1", + "example.com": "", + "1.2.3.4.5": "", + "256.0.0.1": "", + "08.0.0.1": "", // invalid octal + } + for in, want := range cases { + got := "" + if ip := parseInеtAtоn(in); ip != nil { + got = ip.String() + } + if got != want { + t.Errorf("parseInetAton(%q) = %q, want %q", in, got, want) + } + } +} + +// listenProtected starts a loopback listener, marks its port protected, and +// counts accepted connections. +func listenProtected(t *testing.T) (port int, accepted *int32) { + t.Helper() + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { ln.Close() }) + var n int32 + go func() { + for { + c, err := ln.Accept() + if err != nil { + return + } + atomic.AddInt32(&n, 1) + c.Close() + } + }() + port = ln.Addr().(*net.TCPAddr).Port + ProtectLoopbackPort(port) + return port, &n +} + +// TestGuardedDialer_RefusesResolvedLoopback covers the dial-time check that +// catches names decide() can't see are loopback (DNS, rebinding). +func TestGuardedDialer_RefusesResolvedLoopback(t *testing.T) { + port, accepted := listenProtected(t) + for _, addr := range []string{ + net.JoinHostPort("127.0.0.1", strconv.Itoa(port)), + net.JoinHostPort("localhost", strconv.Itoa(port)), // resolved by the dialer + } { + _, err := GuardedDialer(2*time.Second).Dial("tcp", addr) + if !errors.Is(err, ErrProtectedLoopback) { + t.Errorf("dial %s: err = %v, want ErrProtectedLoopback", addr, err) + } + } + if n := atomic.LoadInt32(accepted); n != 0 { + t.Fatalf("protected listener accepted %d connections", n) + } + for _, addr := range []string{"[::ffff:127.0.0.1]:6200", "0.0.0.0:6200", "[::]:6200", "[::1]:6200"} { + if err := chеckDialAddrеss(addr); !errors.Is(err, ErrProtectedLoopback) { + t.Errorf("checkDialAddress(%s) = %v, want ErrProtectedLoopback", addr, err) + } + } + if err := chеckDialAddrеss("93.184.216.34:6200"); err != nil { + t.Errorf("remote host on browser-range port refused: %v", err) + } +} + +func TestGuardedDialer_AllowsUnprotectedLoopback(t *testing.T) { + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer ln.Close() + go func() { + if c, err := ln.Accept(); err == nil { + c.Close() + } + }() + c, err := GuardedDialer(2*time.Second).Dial("tcp", ln.Addr().String()) + if err != nil { + t.Fatalf("dial unprotected loopback port: %v", err) + } + c.Close() +} + +// TestProxy_PlainHTTPRefusesProtectedLoopback reproduces +// `curl -x http://127.0.0.1:8083 http://127.0.0.1:6217/json/version`. +func TestProxy_PlainHTTPRefusesProtectedLoopback(t *testing.T) { + var hits int32 + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + atomic.AddInt32(&hits, 1) + io.WriteString(w, `{"webSocketDebuggerUrl":"ws://..."}`) + })) + defer backend.Close() + port := backend.Listener.Addr().(*net.TCPAddr).Port + + proxy := NewEgressProxy(0, NewAllowlist()) + defer proxy.Stop() + front := httptest.NewServer(proxy) + defer front.Close() + proxyURL, _ := url.Parse(front.URL) + client := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(proxyURL)}, Timeout: 5 * time.Second} + + // Unprotected loopback port: localhost still bypasses (allowed). + resp, err := client.Get(backend.URL + "/json/version") + if err != nil { + t.Fatal(err) + } + resp.Body.Close() + if resp.StatusCode != http.StatusOK || atomic.LoadInt32(&hits) != 1 { + t.Fatalf("unprotected loopback: status %d hits %d", resp.StatusCode, hits) + } + + ProtectLoopbackPort(port) + for _, host := range []string{"127.0.0.1", "localhost", "2130706433", "0.0.0.0"} { + resp, err := client.Get(fmt.Sprintf("http://%s/json/version", net.JoinHostPort(host, strconv.Itoa(port)))) + if err != nil { + t.Fatalf("%s: %v", host, err) + } + resp.Body.Close() + if resp.StatusCode != http.StatusForbidden { + t.Errorf("%s: status %d, want 403", host, resp.StatusCode) + } + } + if n := atomic.LoadInt32(&hits); n != 1 { + t.Fatalf("protected backend reached %d times through the proxy", n-1) + } + + // The transport itself refuses even when decide() is bypassed (drop the + // keep-alive conn from before the port was protected first). + proxy.transport.CloseIdleConnections() + req, _ := http.NewRequestWithContext(context.Background(), "GET", backend.URL, nil) + if _, err := proxy.transport.RoundTrip(req); !errors.Is(err, ErrProtectedLoopback) { + t.Fatalf("transport RoundTrip err = %v, want ErrProtectedLoopback", err) + } +} + +func TestProxy_ConnectRefusesProtectedLoopback(t *testing.T) { + protectedPort, accepted := listenProtected(t) + + open, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer open.Close() + go func() { + for { + c, err := open.Accept() + if err != nil { + return + } + c.Close() + } + }() + + proxy := NewEgressProxy(0, NewAllowlist()) + defer proxy.Stop() + front := httptest.NewServer(proxy) + defer front.Close() + + connect := func(target string) int { + c, err := net.Dial("tcp", front.Listener.Addr().String()) + if err != nil { + t.Fatal(err) + } + defer c.Close() + c.SetDeadline(time.Now().Add(5 * time.Second)) + fmt.Fprintf(c, "CONNECT %s HTTP/1.1\r\nHost: %s\r\n\r\n", target, target) + resp, err := http.ReadResponse(bufio.NewReader(c), nil) + if err != nil { + t.Fatalf("CONNECT %s: %v", target, err) + } + return resp.StatusCode + } + + if code := connect(open.Addr().String()); code != http.StatusOK { + t.Fatalf("CONNECT to unprotected loopback port: %d, want 200", code) + } + p := strconv.Itoa(protectedPort) + for _, target := range []string{"127.0.0.1:" + p, "localhost:" + p, "[::ffff:127.0.0.1]:" + p, "0177.0.0.1:" + p} { + if code := connect(target); code != http.StatusForbidden { + t.Errorf("CONNECT %s: %d, want 403", target, code) + } + } + if n := atomic.LoadInt32(accepted); n != 0 { + t.Fatalf("protected listener accepted %d connections via CONNECT", n) + } + if !strings.HasPrefix(proxyRevision, "egress-proxy-v") { + t.Fatalf("unexpected revision %s", proxyRevision) + } +} diff --git a/sandbox/internal/egress/proxy.go b/sandbox/internal/egress/proxy.go index 7b645b6a..93ccc8ae 100644 --- a/sandbox/internal/egress/proxy.go +++ b/sandbox/internal/egress/proxy.go @@ -1,12 +1,13 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: egress-proxy-v11-held-conn-ctx-cancel +// REVISION: egress-proxy-v13-error-codes package egress import ( "context" + "errors" "fmt" "io" "log" @@ -21,7 +22,7 @@ import ( "github.com/Hyper-Int/OrcaBot/sandbox/internal/id" ) -const proxyRevision = "egress-proxy-v11-held-conn-ctx-cancel" +const proxyRevision = "egress-proxy-v13-error-codes" func init() { log.Printf("[egress-proxy] REVISION: %s loaded at %s", proxyRevision, time.Now().Format(time.RFC3339)) @@ -85,20 +86,20 @@ type Pending struct { // EgressProxy is an HTTP/HTTPS forward proxy that checks domains against // an allowlist and holds unknown connections until the user approves. type EgressProxy struct { - port int - listener net.Listener + port int + listener net.Listener transparentListeners []net.Listener - server *http.Server - allowlist *Allowlist - pendingByID map[string]*Pending // requestID -> pending approval - pendingByDomain map[string]*Pending // domain -> pending approval - pendingMu sync.Mutex - onApprovalNeeded func(req ApprovalRequest) // callback to broadcast WS event - onResolved func(res ApprovalResolution) - onAuditEvent func(event AuditEvent) - auditCh chan AuditEvent // buffered channel for async audit delivery - transport *http.Transport // shared transport for plain HTTP forwarding - stopCh chan struct{} + server *http.Server + allowlist *Allowlist + pendingByID map[string]*Pending // requestID -> pending approval + pendingByDomain map[string]*Pending // domain -> pending approval + pendingMu sync.Mutex + onApprovalNeeded func(req ApprovalRequest) // callback to broadcast WS event + onResolved func(res ApprovalResolution) + onAuditEvent func(event AuditEvent) + auditCh chan AuditEvent // buffered channel for async audit delivery + transport *http.Transport // shared transport for plain HTTP forwarding + stopCh chan struct{} } const auditChannelSize = 256 @@ -111,8 +112,10 @@ func NewEgressProxy(port int, allowlist *Allowlist) *EgressProxy { pendingByID: make(map[string]*Pending), pendingByDomain: make(map[string]*Pending), auditCh: make(chan AuditEvent, auditChannelSize), - transport: &http.Transport{}, - stopCh: make(chan struct{}), + // Guarded dial: the proxy runs as root, so it must never connect to a + // loopback service pool UIDs are blocked from (browser ports, MCPLocal). + transport: &http.Transport{DialContext: GuardеdDialContеxt(30 * time.Second)}, + stopCh: make(chan struct{}), } } @@ -273,6 +276,8 @@ func isLocalhost(host string) bool { // browser-navigation CheckAndHold, transparent/kernel proxy) must go through this so // they all honor the same order: // +// protected loopback → deny (browser ports / MCPLocal; pool UIDs are blocked +// from them directly, the root proxy must not relay) // localhost → allow (bypass, no audit) // IsDenied → deny (permanent "deny always", precedence over allowlist) // IsAllowed → allow (default/user allowlist; audited) @@ -282,6 +287,12 @@ func isLocalhost(host string) bool { // Returns DecisionDefault/DecisionAllowOnce/DecisionAllowAlways when permitted, or // DecisionDeny/DecisionTimeout when blocked. Callers permit on the allow set. func (p *EgressProxy) decide(ctx context.Context, host string, port int) string { + // REVISION: egress-proxy-v12-protected-loopback + // Names that only resolve to loopback are refused at dial time (GuardedDialer). + if IsProtectedLoopbackTarget(host, port) { + log.Printf("[egress-proxy] Denied protected loopback target %s", net.JoinHostPort(host, strconv.Itoa(port))) + return DecisionDeny + } if isLocalhost(host) { return DecisionDefault } @@ -314,26 +325,26 @@ func (p *EgressProxy) handleConnect(w http.ResponseWriter, r *http.Request) { p.tunnelConnect(w, host, port) return } - http.Error(w, "Egress denied: domain not approved", http.StatusForbidden) + http.Error(w, "E79117: Egress denied: domain not approved", http.StatusForbidden) } // handleHTTP handles plain HTTP proxy requests. func (p *EgressProxy) handleHTTP(w http.ResponseWriter, r *http.Request) { targetURL, host, port, err := extractHTTPDestination(r) if err != nil { - http.Error(w, "Bad proxy request", http.StatusBadRequest) + http.Error(w, "E79118: Bad proxy request", http.StatusBadRequest) return } if !permitted(p.decide(r.Context(), host, port)) { - http.Error(w, "Egress denied: domain not approved", http.StatusForbidden) + http.Error(w, "E79117: Egress denied: domain not approved", http.StatusForbidden) return } // Forward the request outReq, err := http.NewRequestWithContext(r.Context(), r.Method, targetURL.String(), r.Body) if err != nil { - http.Error(w, "Bad request", http.StatusBadRequest) + http.Error(w, "E79119: Bad request", http.StatusBadRequest) return } outReq.Header = r.Header.Clone() @@ -343,7 +354,11 @@ func (p *EgressProxy) handleHTTP(w http.ResponseWriter, r *http.Request) { resp, err := p.transport.RoundTrip(outReq) if err != nil { - http.Error(w, "Upstream error", http.StatusBadGateway) + if errors.Is(err, ErrProtectedLoopback) { + http.Error(w, "E79120: Egress denied: protected loopback service", http.StatusForbidden) + return + } + http.Error(w, "E79121: Upstream error", http.StatusBadGateway) return } defer resp.Body.Close() @@ -472,9 +487,13 @@ func (p *EgressProxy) waitForDecision(ctx context.Context, pending *Pending) str // tunnelConnect establishes a TCP tunnel for CONNECT requests. func (p *EgressProxy) tunnelConnect(w http.ResponseWriter, host string, port int) { target := net.JoinHostPort(host, fmt.Sprintf("%d", port)) - upstream, err := net.DialTimeout("tcp", target, 10*time.Second) + upstream, err := GuardedDialer(10*time.Second).Dial("tcp", target) if err != nil { - http.Error(w, "Cannot connect to upstream", http.StatusBadGateway) + if errors.Is(err, ErrProtectedLoopback) { + http.Error(w, "E79120: Egress denied: protected loopback service", http.StatusForbidden) + return + } + http.Error(w, "E79122: Cannot connect to upstream", http.StatusBadGateway) return } defer upstream.Close() @@ -482,13 +501,13 @@ func (p *EgressProxy) tunnelConnect(w http.ResponseWriter, host string, port int // Hijack the client connection hijacker, ok := w.(http.Hijacker) if !ok { - http.Error(w, "Hijacking not supported", http.StatusInternalServerError) + http.Error(w, "E79123: Hijacking not supported", http.StatusInternalServerError) return } clientConn, _, err := hijacker.Hijack() if err != nil { - http.Error(w, "Hijack failed", http.StatusInternalServerError) + http.Error(w, "E79124: Hijack failed", http.StatusInternalServerError) return } defer clientConn.Close() diff --git a/sandbox/internal/egress/transparent_linux.go b/sandbox/internal/egress/transparent_linux.go index ef1f78e9..6e70afeb 100644 --- a/sandbox/internal/egress/transparent_linux.go +++ b/sandbox/internal/egress/transparent_linux.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: egress-transparent-v5-drop-no-hostname +// REVISION: egress-transparent-v6-guarded-dial //go:build linux @@ -198,7 +198,9 @@ func (p *EgressProxy) handleTransparent(conn net.Conn) { // By resolving domain ourselves, the allowed name maps to its legitimate IPs. // When no Host/SNI was found, domain == origIP and behaviour is unchanged. target := net.JoinHostPort(domain, fmt.Sprintf("%d", origPort)) - upstream, err := net.DialTimeout("tcp", target, 10*time.Second) + // Guarded: a Host/SNI resolving to loopback must not reach a protected + // loopback service (browser ports, MCPLocal). REVISION: egress-transparent-v6-guarded-dial + upstream, err := GuardedDialer(10*time.Second).Dial("tcp", target) if err != nil { log.Printf("[egress-transparent] Dial %s failed: %v", target, err) return diff --git a/sandbox/internal/geminishim/anthropic.go b/sandbox/internal/geminishim/anthropic.go index d40e58b4..3d71d78a 100644 --- a/sandbox/internal/geminishim/anthropic.go +++ b/sandbox/internal/geminishim/anthropic.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: model-gateway-v1-anthropic +// REVISION: model-gateway-v2-error-codes // // Anthropic Messages (Claude) ↔ OpenAI Chat Completions translation, sharing the // same broker-forwarding core (forwardChat) and OpenAI types as the Gemini path. @@ -119,12 +119,12 @@ func parseAnthPath(path string) (sessionID, provider, model string, ok bool) { func (s *Shim) serveAnthropic(w http.ResponseWriter, r *http.Request) { sessionID, provider, model, ok := parseAnthPath(r.URL.Path) if !ok { - writeAnthropicError(w, http.StatusBadRequest, "invalid path; expected /av1/{session}/{provider}/{model}/v1/messages") + writeAnthropicError(w, http.StatusBadRequest, "E79141: invalid path; expected /av1/{session}/{provider}/{model}/v1/messages") return } body, err := io.ReadAll(r.Body) if err != nil { - writeAnthropicError(w, http.StatusBadRequest, "failed to read request body") + writeAnthropicError(w, http.StatusBadRequest, "E79142: failed to read request body") return } @@ -136,14 +136,14 @@ func (s *Shim) serveAnthropic(w http.ResponseWriter, r *http.Request) { var areq anthReq if err := json.Unmarshal(body, &areq); err != nil { - writeAnthropicError(w, http.StatusBadRequest, "invalid Anthropic request: "+err.Error()) + writeAnthropicError(w, http.StatusBadRequest, "E79143: invalid Anthropic request: "+err.Error()) return } oreq := anthropicToOpenAI(areq, model) oreqBody, err := json.Marshal(oreq) if err != nil { - writeAnthropicError(w, http.StatusInternalServerError, "failed to encode upstream request") + writeAnthropicError(w, http.StatusInternalServerError, "E79144: failed to encode upstream request") return } log.Printf("[model-gateway] anthropic -> session=%s provider=%s model=%s stream=%t msgs=%d tools=%d", @@ -151,7 +151,7 @@ func (s *Shim) serveAnthropic(w http.ResponseWriter, r *http.Request) { resp, err := s.forwardChat(sessionID, provider, oreqBody) if err != nil { - writeAnthropicError(w, http.StatusBadGateway, "broker request failed: "+err.Error()) + writeAnthropicError(w, http.StatusBadGateway, "E79145: broker request failed: "+err.Error()) return } defer resp.Body.Close() @@ -159,7 +159,7 @@ func (s *Shim) serveAnthropic(w http.ResponseWriter, r *http.Request) { if resp.StatusCode != http.StatusOK { errBody, _ := io.ReadAll(resp.Body) log.Printf("[model-gateway] anthropic <- upstream HTTP %d session=%s body=%s", resp.StatusCode, sessionID, truncate(string(errBody), 2000)) - writeAnthropicError(w, resp.StatusCode, "upstream error: "+strings.TrimSpace(string(errBody))) + writeAnthropicError(w, resp.StatusCode, "E79146: upstream error: "+strings.TrimSpace(string(errBody))) return } @@ -171,11 +171,11 @@ func (s *Shim) serveAnthropic(w http.ResponseWriter, r *http.Request) { respBody, _ := io.ReadAll(resp.Body) var oresp oaiResp if err := json.Unmarshal(respBody, &oresp); err != nil { - writeAnthropicError(w, http.StatusBadGateway, "failed to parse upstream response") + writeAnthropicError(w, http.StatusBadGateway, "E79147: failed to parse upstream response") return } if oresp.Error != nil { - writeAnthropicError(w, http.StatusBadGateway, "upstream error: "+oresp.Error.String()) + writeAnthropicError(w, http.StatusBadGateway, "E79148: upstream error: "+oresp.Error.String()) return } w.Header().Set("Content-Type", "application/json") diff --git a/sandbox/internal/geminishim/shim.go b/sandbox/internal/geminishim/shim.go index 575adf6c..a88701d8 100644 --- a/sandbox/internal/geminishim/shim.go +++ b/sandbox/internal/geminishim/shim.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: gemini-shim-v1-openrouter-bridge +// REVISION: gemini-shim-v2-error-codes // // Package geminishim is a localhost translation proxy that lets the *official* // Google Gemini CLI talk to OpenRouter. @@ -51,7 +51,7 @@ func truncate(s string, n int) string { return s[:n] + "…(" + fmt.Sprintf("%d", len(s)) + "b)" } -const shimRevision = "gemini-shim-v1-openrouter-bridge" +const shimRevision = "gemini-shim-v2-error-codes" func init() { log.Printf("[gemini-shim] REVISION: %s loaded at %s", shimRevision, time.Now().Format(time.RFC3339)) @@ -108,13 +108,13 @@ func (s *Shim) ServeHTTP(w http.ResponseWriter, r *http.Request) { sessionID, provider, model, method, ok := parsePath(r.URL.Path) if !ok { - writeGeminiError(w, http.StatusBadRequest, "invalid shim path; expected /gv1/{session}/{provider}/{model}/v1.../models/{m}:{method}") + writeGeminiError(w, http.StatusBadRequest, "E79131: invalid shim path; expected /gv1/{session}/{provider}/{model}/v1.../models/{m}:{method}") return } body, err := io.ReadAll(r.Body) if err != nil { - writeGeminiError(w, http.StatusBadRequest, "failed to read request body") + writeGeminiError(w, http.StatusBadRequest, "E79132: failed to read request body") return } @@ -126,9 +126,9 @@ func (s *Shim) ServeHTTP(w http.ResponseWriter, r *http.Request) { case "countTokens": handleCountTokens(w, body) case "embedContent", "batchEmbedContents": - writeGeminiError(w, http.StatusNotImplemented, "embeddings are not supported by the OpenRouter bridge") + writeGeminiError(w, http.StatusNotImplemented, "E79133: embeddings are not supported by the OpenRouter bridge") default: - writeGeminiError(w, http.StatusNotFound, "unsupported method: "+method) + writeGeminiError(w, http.StatusNotFound, "E79134: unsupported method: "+method) } } @@ -191,14 +191,14 @@ func (s *Shim) forwardChat(sessionID, provider string, oreqBody []byte) (*http.R func (s *Shim) handleGenerate(w http.ResponseWriter, sessionID, provider, model string, body []byte, stream bool) { var greq genReq if err := json.Unmarshal(body, &greq); err != nil { - writeGeminiError(w, http.StatusBadRequest, "invalid Gemini request: "+err.Error()) + writeGeminiError(w, http.StatusBadRequest, "E79135: invalid Gemini request: "+err.Error()) return } oreq := geminiToOpenAI(greq, model, stream) oreqBody, err := json.Marshal(oreq) if err != nil { - writeGeminiError(w, http.StatusInternalServerError, "failed to encode upstream request") + writeGeminiError(w, http.StatusInternalServerError, "E79136: failed to encode upstream request") return } @@ -210,7 +210,7 @@ func (s *Shim) handleGenerate(w http.ResponseWriter, sessionID, provider, model resp, err := s.forwardChat(sessionID, provider, oreqBody) if err != nil { - writeGeminiError(w, http.StatusBadGateway, "broker request failed: "+err.Error()) + writeGeminiError(w, http.StatusBadGateway, "E79137: broker request failed: "+err.Error()) return } defer resp.Body.Close() @@ -219,7 +219,7 @@ func (s *Shim) handleGenerate(w http.ResponseWriter, sessionID, provider, model errBody, _ := io.ReadAll(resp.Body) log.Printf("[gemini-shim] <- upstream HTTP %d session=%s model=%s body=%s", resp.StatusCode, sessionID, model, truncate(string(errBody), 2000)) - writeGeminiError(w, resp.StatusCode, "openrouter error: "+strings.TrimSpace(string(errBody))) + writeGeminiError(w, resp.StatusCode, "E79138: openrouter error: "+strings.TrimSpace(string(errBody))) return } @@ -232,13 +232,13 @@ func (s *Shim) handleGenerate(w http.ResponseWriter, sessionID, provider, model respBody, _ := io.ReadAll(resp.Body) if err := json.Unmarshal(respBody, &oresp); err != nil { log.Printf("[gemini-shim] <- unparseable response session=%s body=%s", sessionID, truncate(string(respBody), 2000)) - writeGeminiError(w, http.StatusBadGateway, "failed to parse openrouter response") + writeGeminiError(w, http.StatusBadGateway, "E79139: failed to parse openrouter response") return } // OpenRouter can return 200 with an error envelope (e.g. provider error). if oresp.Error != nil { log.Printf("[gemini-shim] <- upstream error envelope session=%s model=%s err=%s", sessionID, model, oresp.Error.String()) - writeGeminiError(w, http.StatusBadGateway, "openrouter error: "+oresp.Error.String()) + writeGeminiError(w, http.StatusBadGateway, "E79140: openrouter error: "+oresp.Error.String()) return } if debugEnabled() { diff --git a/sandbox/internal/mcp/gateway_client.go b/sandbox/internal/mcp/gateway_client.go index 3325a066..5739f156 100644 --- a/sandbox/internal/mcp/gateway_client.go +++ b/sandbox/internal/mcp/gateway_client.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: gateway-client-v5-batch-integrations +// REVISION: gateway-client-v6-batch-dashboard-token package mcp import ( @@ -12,10 +12,12 @@ import ( "net/http" "os" "time" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" ) func init() { - log.Printf("[gateway-client] REVISION: gateway-client-v5-batch-integrations loaded at %s", time.Now().Format(time.RFC3339)) + log.Printf("[gateway-client] REVISION: gateway-client-v6-batch-dashboard-token loaded at %s", time.Now().Format(time.RFC3339)) } // AuthType indicates how to authenticate with the control plane @@ -124,20 +126,17 @@ type BatchIntegrationsResponse struct { } // BatchListIntegrations returns integrations for ALL terminals in a dashboard in a single call. -// Uses X-Internal-Token auth (not per-PTY token) since it's a dashboard-wide query. +// Uses the dashboard-scoped token (not a per-PTY token) since it's a dashboard-wide query. func (c *GatewayClient) BatchListIntegrations(dashboardID string) (*BatchIntegrationsResponse, error) { - internalToken := os.Getenv("INTERNAL_API_TOKEN") - if internalToken == "" { - return nil, fmt.Errorf("INTERNAL_API_TOKEN not set") - } - url := fmt.Sprintf("%s/internal/dashboards/%s/terminal-integrations", c.controlPlaneURL, dashboardID) req, err := http.NewRequest("GET", url, nil) if err != nil { return nil, fmt.Errorf("failed to create request: %w", err) } - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, dashboardID); err != nil { + return nil, err + } resp, err := c.httpClient.Do(req) if err != nil { diff --git a/sandbox/internal/mirror/mirror.go b/sandbox/internal/mirror/mirror.go index e5e227b9..e5d75c15 100644 --- a/sandbox/internal/mirror/mirror.go +++ b/sandbox/internal/mirror/mirror.go @@ -1,6 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: mirror-v2-dashboard-token + package mirror import ( @@ -12,6 +14,8 @@ import ( "net/http" "os" "strings" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" ) type ManifestEntry struct { @@ -38,24 +42,21 @@ type Manifest struct { type Mirror struct { baseURL string - token string provider string client *http.Client } func NewMirrorFromEnv(provider string) *Mirror { baseURL := strings.TrimSuffix(os.Getenv("CONTROLPLANE_URL"), "/") - token := os.Getenv("INTERNAL_API_TOKEN") return &Mirror{ baseURL: baseURL, - token: token, provider: provider, client: &http.Client{}, } } func (m *Mirror) Enabled() bool { - return m.baseURL != "" && m.token != "" && m.provider != "" + return m.baseURL != "" && m.provider != "" } func (m *Mirror) FetchManifest(ctx context.Context, dashboardID string) (*Manifest, error) { @@ -66,7 +67,9 @@ func (m *Mirror) FetchManifest(ctx context.Context, dashboardID string) (*Manife if err != nil { return nil, err } - req.Header.Set("X-Internal-Token", m.token) + if err := cpauth.Apply(req, dashboardID); err != nil { + return nil, err + } resp, err := m.client.Do(req) if err != nil { @@ -92,7 +95,9 @@ func (m *Mirror) FetchFile(ctx context.Context, dashboardID, fileID string) (io. if err != nil { return nil, nil, err } - req.Header.Set("X-Internal-Token", m.token) + if err := cpauth.Apply(req, dashboardID); err != nil { + return nil, nil, err + } resp, err := m.client.Do(req) if err != nil { @@ -119,7 +124,10 @@ func (m *Mirror) ReportProgress(ctx context.Context, payload map[string]interfac if err != nil { return err } - req.Header.Set("X-Internal-Token", m.token) + dashboardID, _ := payload["dashboardId"].(string) + if err := cpauth.Apply(req, dashboardID); err != nil { + return err + } req.Header.Set("Content-Type", "application/json") resp, err := m.client.Do(req) diff --git a/sandbox/internal/pty/pool.go b/sandbox/internal/pty/pool.go index 232d94ea..d90f259b 100644 --- a/sandbox/internal/pty/pool.go +++ b/sandbox/internal/pty/pool.go @@ -1,11 +1,12 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: pty-pool-v2-two-phase-register +// REVISION: pty-pool-v4-error-codes package pty import ( + "errors" "fmt" "log" "sync" @@ -14,7 +15,7 @@ import ( "time" ) -const poolRevision = "pty-pool-v2-two-phase-register" +const poolRevision = "pty-pool-v4-error-codes" func init() { log.Printf("[pty-pool] REVISION: %s loaded at %s", poolRevision, time.Now().Format(time.RFC3339)) @@ -66,6 +67,36 @@ var ( poolOnce sync.Once ) +// ErrSpawnRefused is returned by every PTY spawn after RefuseSpawns: egress +// control was requested but UID isolation could not be set up, so a PTY would +// run as root outside the kernel egress rules. REVISION: pty-pool-v3-fail-closed-spawn +var ErrSpawnRefused = errors.New("E79770: PTY creation refused: egress isolation failed to initialise (see server log)") + +var spawnRefusal atomic.Pointer[string] + +// RefuseSpawns makes all future PTY spawns fail with ErrSpawnRefused. Called at +// startup when EGRESS_PROXY_ENABLED=true but kernel enforcement failed: running +// PTYs as root would silently bypass egress control (fail closed instead). +// An empty reason lifts the refusal (tests only). +func RefuseSpawns(reason string) { + spawnRefusal.Store(&reason) +} + +// SpawnRefusal returns why PTY spawns are refused, or "" if they are allowed. +func SpawnRefusal() string { + if r := spawnRefusal.Load(); r != nil { + return *r + } + return "" +} + +func chеckSpawnAllowed() error { + if r := SpawnRefusal(); r != "" { + return fmt.Errorf("%w: %s", ErrSpawnRefused, r) + } + return nil +} + // InitPool initializes the global pool. Must be called once at server startup. // sandboxGID is the GID of the "sandbox" group that all pty-NNN users belong to. func InitPool(sandboxGID uint32) { diff --git a/sandbox/internal/pty/pty.go b/sandbox/internal/pty/pty.go index de9c1622..21b0fe11 100644 --- a/sandbox/internal/pty/pty.go +++ b/sandbox/internal/pty/pty.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: pty-v2-release-after-wait +// REVISION: pty-v4-error-codes package pty @@ -170,6 +170,9 @@ func newWithCmd(cmd *exec.Cmd, cols, rows uint16) (*PTY, error) { } func newWithCmdID(ptyID string, cmd *exec.Cmd, cols, rows uint16) (*PTY, error) { + if err := chеckSpawnAllowed(); err != nil { + return nil, err + } // Pre-generate ID if not provided (so it can be used in env vars before process starts) if ptyID == "" { var err error @@ -194,6 +197,9 @@ func newWithCmdID(ptyID string, cmd *exec.Cmd, cols, rows uint16) (*PTY, error) } func newWithCmdIDSlot(slot *SlotEntry, sessionID string, cmd *exec.Cmd, cols, rows uint16) (*PTY, error) { + if err := chеckSpawnAllowed(); err != nil { + return nil, err + } ptyID := slot.PTYID if ptyID == "" { var err error diff --git a/sandbox/internal/sessions/dashboard_token_test.go b/sandbox/internal/sessions/dashboard_token_test.go new file mode 100644 index 00000000..c174d2ec --- /dev/null +++ b/sandbox/internal/sessions/dashboard_token_test.go @@ -0,0 +1,56 @@ +// Copyright 2026 Rob Macrae. All rights reserved. +// SPDX-License-Identifier: LicenseRef-Proprietary + +// REVISION: session-v26-no-dropped-token-fallback + +package sessions + +import ( + "encoding/base64" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" +) + +func testDashboardToken(dashboardID string, exp time.Time, gen int64) string { + enc := base64.RawURLEncoding + payload, _ := json.Marshal(map[string]any{"dashboard_id": dashboardID, "exp": exp.Unix(), "sbx_gen": gen}) + return enc.EncodeToString([]byte(`{"alg":"HS256"}`)) + "." + enc.EncodeToString(payload) + ".sig" +} + +// After the control plane refuses a session's token as permanently invalid +// (rotated PTY_TOKEN_SECRET), DashboardToken must not fall back to the +// session's original copy of it; a fresh token from a PTY creation restores it. +func TestDashboardTokenNoFallbackAfterDrop(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Error(w, `{"error":"E79395: A valid dashboard token is required to refresh"}`, http.StatusUnauthorized) + })) + defer srv.Close() + t.Setenv("CONTROLPLANE_URL", srv.URL) + + old := testDashboardToken("dash-drop", time.Now().Add(-time.Minute), 7) + cpauth.SetSessionToken("sess-drop", "dash-drop", old) + defer cpauth.ForgetSession("sess-drop") + s := &Session{ID: "sess-drop", DashboardID: "dash-drop", MCPToken: old} + + if got := s.DashboardToken(); got != "" { + t.Fatalf("fell back to a token cpauth dropped: %q", got) + } + fresh := testDashboardToken("dash-drop", time.Now().Add(7*24*time.Hour), 7) + s.SetDashboardToken(fresh) + if got := s.DashboardToken(); got != fresh { + t.Fatalf("fresh token not used: %q", got) + } +} + +// When cpauth never took the session's token, the session's own copy is used. +func TestDashboardTokenFallbackWhenNeverTaken(t *testing.T) { + s := &Session{ID: "sess-none", DashboardID: "dash-none", MCPToken: "opaque-token"} + if got := s.DashboardToken(); got != "opaque-token" { + t.Fatalf("got %q, want the session's own token", got) + } +} diff --git a/sandbox/internal/sessions/manager.go b/sandbox/internal/sessions/manager.go index db5d9e3d..13a25248 100644 --- a/sandbox/internal/sessions/manager.go +++ b/sandbox/internal/sessions/manager.go @@ -1,6 +1,8 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary +// REVISION: manager-v4-forget-session-token + package sessions import ( @@ -15,10 +17,17 @@ import ( "time" "github.com/Hyper-Int/OrcaBot/sandbox/internal/broker" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" "github.com/Hyper-Int/OrcaBot/sandbox/internal/geminishim" "github.com/Hyper-Int/OrcaBot/sandbox/internal/id" ) +const managerRevision = "manager-v4-forget-session-token" + +func init() { + log.Printf("[sessions] REVISION: %s loaded at %s", managerRevision, time.Now().Format(time.RFC3339)) +} + var ( ErrSessionNotFound = errors.New("session not found") ) @@ -217,6 +226,8 @@ func (m *Manager) Delete(id string) error { } delete(m.sessions, id) m.mu.Unlock() + sessionDashboards.Delete(id) + cpauth.ForgetSession(id) // REVISION: manager-v2-evict-logged-routes // Release this session's per-session state on the shared broker so it doesn't diff --git a/sandbox/internal/sessions/session.go b/sandbox/internal/sessions/session.go index 5af7b4bf..960d66c5 100644 --- a/sandbox/internal/sessions/session.go +++ b/sandbox/internal/sessions/session.go @@ -1,7 +1,7 @@ // Copyright 2026 Rob Macrae. All rights reserved. // SPDX-License-Identifier: LicenseRef-Proprietary -// REVISION: session-v23-browser-close-race +// REVISION: session-v26-no-dropped-token-fallback // Package sessions manages session lifecycle. // @@ -32,6 +32,7 @@ import ( "github.com/Hyper-Int/OrcaBot/sandbox/internal/agenthooks" "github.com/Hyper-Int/OrcaBot/sandbox/internal/broker" "github.com/Hyper-Int/OrcaBot/sandbox/internal/browser" + "github.com/Hyper-Int/OrcaBot/sandbox/internal/cpauth" "github.com/Hyper-Int/OrcaBot/sandbox/internal/drivesync" "github.com/Hyper-Int/OrcaBot/sandbox/internal/fs" "github.com/Hyper-Int/OrcaBot/sandbox/internal/id" @@ -40,7 +41,7 @@ import ( "github.com/Hyper-Int/OrcaBot/sandbox/internal/statecache" ) -const sessionRevision = "session-v23-browser-close-race" +const sessionRevision = "session-v26-no-dropped-token-fallback" // Allow UUID-style IDs and internal random IDs while rejecting shell metacharacters. // This protects shell-interpolated call sites (e.g. Claude apiKeyHelper command). @@ -183,6 +184,9 @@ type Session struct { // through that port so redirects/subresources are subject to egress approval. // REVISION: browser-v7-proxy-server func NewSessiоn(id string, dashboardID string, mcpToken string, workspaceRoot string, sharedBroker *broker.SecretsBroker, brokerPort int, egressProxyPort int) *Session { + cpauth.SetSessionToken(id, dashboardID, mcpToken) + sessionDashboards.Store(id, dashboardID) + s := &Session{ ID: id, DashboardID: dashboardID, @@ -230,14 +234,50 @@ func NewSessiоn(id string, dashboardID string, mcpToken string, workspaceRoot s return s } +// sessionDashboards maps session ID → dashboard ID. The broker is shared by all +// sessions and its approval callback only carries the session ID, so this is how +// the callback finds the dashboard whose credential to use. +var sessionDashboards sync.Map + +// DashboardToken returns the freshest dashboard-scoped control plane token for +// this session's dashboard (refreshed by cpauth), falling back to the token the +// session was created with only when cpauth never took it (e.g. it could not be +// parsed). If the control plane refused that token as permanently invalid, +// cpauth dropped it and there is no fallback: "" until a fresh token arrives +// (SetDashboardToken). REVISION: session-v26-no-dropped-token-fallback +func (s *Session) DashboardToken() string { + if token := cpauth.DashboardToken(s.DashboardID); token != "" { + return token + } + if cpauth.SessionTokenDropped(s.ID) { + return "" + } + return s.MCPToken +} + +// SetDashboardToken hands cpauth a fresh dashboard token the control plane sent +// for this session (with a PTY creation). It replaces the session's previous +// token, so a VM whose token was dropped (rotated PTY_TOKEN_SECRET, replaced +// sandbox) recovers on the next terminal create. The dashboard is always the +// session's own, never taken from the request. +func (s *Session) SetDashboardToken(token string) { + if token == "" || s.DashboardID == "" { + return + } + cpauth.SetSessionToken(s.ID, s.DashboardID, token) +} + // notifyApprovalNeeded sends a pending approval request to the control plane. func notifyApprovalNeeded(sessionID, secretName, domain string) { controlplaneURL := os.Getenv("CONTROLPLANE_URL") - internalToken := os.Getenv("INTERNAL_API_TOKEN") - if controlplaneURL == "" || internalToken == "" { - fmt.Fprintf(os.Stderr, "Warning: cannot notify approval needed - missing CONTROLPLANE_URL or INTERNAL_API_TOKEN\n") + if controlplaneURL == "" { + fmt.Fprintf(os.Stderr, "Warning: cannot notify approval needed - missing CONTROLPLANE_URL\n") return } + dashboardID := "" + if v, ok := sessionDashboards.Load(sessionID); ok { + dashboardID = v.(string) + } url := fmt.Sprintf("%s/internal/sessions/%s/approval-request", controlplaneURL, sessionID) @@ -261,7 +301,10 @@ func notifyApprovalNeeded(sessionID, secretName, domain string) { return } req.Header.Set("Content-Type", "application/json") - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, dashboardID); err != nil { + fmt.Fprintf(os.Stderr, "Warning: cannot notify approval needed: %v\n", err) + return + } client := &http.Client{Timeout: 10 * time.Second} resp, err := client.Do(req) @@ -496,8 +539,7 @@ func (s *Session) fetchUserMCPTools() []mcp.MCPTool { } controlplaneURL := os.Getenv("CONTROLPLANE_URL") - internalToken := os.Getenv("INTERNAL_API_TOKEN") - if controlplaneURL == "" || internalToken == "" { + if controlplaneURL == "" { return nil } @@ -506,7 +548,9 @@ func (s *Session) fetchUserMCPTools() []mcp.MCPTool { if err != nil { return nil } - req.Header.Set("X-Internal-Token", internalToken) + if err := cpauth.Apply(req, s.DashboardID); err != nil { + return nil + } client := &http.Client{Timeout: 10 * time.Second} resp, err := client.Do(req)